Redis 3.2.5.

Compiling Redis worked as a side effect of jemalloc target specifying
-ldl as needed linker options, otherwise it is not provided during
linking and dlopen() API will remain unresolved symbols.

.gitignore

@@ -27,4 +27,3 @@ deps/lua/src/liblua.a
 .make-*
 .prerequisites
 *.dSYM
-Makefile.dep

CONTRIBUTING

@@ -12,7 +12,7 @@ each source file that you contribute.
 
   PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
   bugs in the Github issues system. We'll be very happy to help you and provide
-  all the support at the Reddit sub:
+  all the support Reddit sub:
 
       http://reddit.com/r/redis
 
@@ -24,7 +24,7 @@ each source file that you contribute.
 
 1. If it is a major feature or a semantical change, please post it as a new submission in r/redis on Reddit at http://reddit.com/r/redis. Try to be passionate about why the feature is needed, make users upvote your proposal to gain traction and so forth. Read feedbacks about the community. But in this first step **please don't write code yet**.
 
-2. If in step 1 you get an acknowledgment from the project leaders, use the
+2. If in step 1 you get an acknowledge from the project leaders, use the
    following procedure to submit a patch:
 
     a. Fork Redis on github ( http://help.github.com/fork-a-repo/ )

README.md

@@ -39,7 +39,7 @@ You can run a 32 bit Redis binary using:
 
     % make 32bit
 
-After building Redis, it is a good idea to test it using:
+After building Redis is a good idea to test it, using:
 
     % make test
 
@@ -47,8 +47,8 @@ Fixing build problems with dependencies or cached build options
 ---------
 
 Redis has some dependencies which are included into the `deps` directory.
-`make` does not automatically rebuild dependencies even if something in
-the source code of dependencies changes.
+`make` does not rebuild dependencies automatically, even if something in the
+source code of dependencies is changed.
 
 When you update the source code with `git pull` or when code inside the
 dependencies tree is modified in any other way, make sure to use the following
@@ -109,14 +109,14 @@ To run Redis with the default configuration just type:
 
     % cd src
     % ./redis-server
-
+    
 If you want to provide your redis.conf, you have to run it using an additional
 parameter (the path of the configuration file):
 
     % cd src
     % ./redis-server /path/to/redis.conf
 
-It is possible to alter the Redis configuration by passing parameters directly
+It is possible to alter the Redis configuration passing parameters directly
 as options using the command line. Examples:
 
     % ./redis-server --port 9999 --slaveof 127.0.0.1 6379
@@ -174,7 +174,7 @@ You'll be able to stop and start Redis using the script named
 `/etc/init.d/redis_<portnumber>`, for instance `/etc/init.d/redis_6379`.
 
 Code contributions
------------------
+---
 
 Note: by contributing code to the Redis project in any form, including sending
 a pull request via Github, a code fragment or patch via private email or
@@ -185,262 +185,7 @@ source distribution.
 Please see the [CONTRIBUTING][2] file in this source distribution for more
 information.
 
+Enjoy!
+
 [1]: https://github.com/antirez/redis/blob/unstable/COPYING
 [2]: https://github.com/antirez/redis/blob/unstable/CONTRIBUTING
-
-Redis internals
-===
-
-If you are reading this README you are likely in front of a Github page
-or you just untarred the Redis distribution tar ball. In both the cases
-you are basically one step away from the source code, so here we explain
-the Redis source code layout, what is in each file as a general idea, the
-most important functions and structures inside the Redis server and so forth.
-We keep all the discussion at a high level without digging into the details
-since this document would be huge otherwise and our code base changes
-continuously, but a general idea should be a good starting point to
-understand more. Moreover most of the code is heavily commented and easy
-to follow.
-
-Source code layout
----
-
-The Redis root directory just contains this README, the Makefile which
-calls the real Makefile inside the `src` directory and an example
-configuration for Redis and Sentinel. You can find a few shell
-scripts that are used in order to execute the Redis, Redis Cluster and
-Redis Sentinel unit tests, which are implemented inside the `tests`
-directory.
-
-Inside the root are the following important directories:
-
-* `src`: contains the Redis implementation, written in C.
-* `tests`: contains the unit tests, implemented in Tcl.
-* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `anitrez/redis`. An exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
-
-There are a few more directories but they are not very important for our goals
-here. We'll focus mostly on `src`, where the Redis implementation is contained,
-exploring what there is inside each file. The order in which files are
-exposed is the logical one to follow in order to disclose different layers
-of complexity incrementally.
-
-Note: lately Redis was refactored quite a bit. Function names and file
-names have been changed, so you may find that this documentation reflects the
-`unstable` branch more closely. For instance in Redis 3.0 the `server.c`
-and `server.h` files were named to `redis.c` and `redis.h`. However the overall
-structure is the same. Keep in mind that all the new developments and pull
-requests should be performed against the `unstable` branch.
-
-server.h
----
-
-The simplest way to understand how a program works is to understand the
-data structures it uses. So we'll start from the main header file of
-Redis, which is `server.h`.
-
-All the server configuration and in general all the shared state is
-defined in a global structure called `server`, of type `struct redisServer`.
-A few important fields in this structure are:
-
-* `server.db` is an array of Redis databases, where data is stored.
-* `server.commands` is the command table.
-* `server.clients` is a linked list of clients connected to the server.
-* `server.master` is a special client, the master, if the instance is a slave.
-
-There are tons of other fields. Most fields are commented directly inside
-the structure definition.
-
-Another important Redis data structure is the one defining a client.
-In the past it was called `redisClient`, now just `client`. The structure
-has many fields, here we'll just show the main ones:
-
-    struct client {
-        int fd;
-        sds querybuf;
-        int argc;
-        robj **argv;
-        redisDb *db;
-        int flags;
-        list *reply;
-        char buf[PROTO_REPLY_CHUNK_BYTES];
-        ... many other fields ...
-    }
-
-The client structure defines a *connected client*:
-
-* The `fd` field is the client socket file descriptor.
-* `argc` and `argv` are populated with the command the client is executing, so that functions implementing a given Redis command can read the arguments.
-* `querybuf` accumulates the requests from the client, which are parsed by the Redis server according to the Redis protocol and executed by calling the implementations of the commands the client is executing.
-* `reply` and `buf` are dynamic and static buffers that accumulate the replies the server sends to the client. These buffers are incrementally written to the socket as soon as the file descriptor is writable.
-
-As you can see in the client structure above, arguments in a command
-are described as `robj` structures. The following is the full `robj`
-structure, which defines a *Redis object*:
-
-    typedef struct redisObject {
-        unsigned type:4;
-        unsigned encoding:4;
-        unsigned lru:LRU_BITS; /* lru time (relative to server.lruclock) */
-        int refcount;
-        void *ptr;
-    } robj;
-
-Basically this structure can represent all the basic Redis data types like
-strings, lists, sets, sorted sets and so forth. The interesting thing is that
-it has a `type` field, so that it is possible to know what type a given
-object has, and a `refcount`, so that the same object can be referenced
-in multiple places without allocating it multiple times. Finally the `ptr`
-field points to the actual representation of the object, which might vary
-even for the same type, depending on the `encoding` used.
-
-Redis objects are used extensively in the Redis internals, however in order
-to avoid the overhead of indirect accesses, recently in many places
-we just use plain dynamic strings not wrapped inside a Redis object.
-
-server.c
----
-
-This is the entry point of the Redis server, where the `main()` function
-is defined. The following are the most important steps in order to startup
-the Redis server.
-
-* `initServerConfig()` setups the default values of the `server` structure.
-* `initServer()` allocates the data structures needed to operate, setup the listening socket, and so forth.
-* `aeMain()` starts the event loop which listens for new connections.
-
-There are two special functions called periodically by the event loop:
-
-1. `serverCron()` is called periodically (according to `server.hz` frequency), and performs tasks that must be performed from time to time, like checking for timedout clients.
-2. `beforeSleep()` is called every time the event loop fired, Redis served a few requests, and is returning back into the event loop.
-
-Inside server.c you can find code that handles other vital things of the Redis server:
-
-* `call()` is used in order to call a given command in the context of a given client.
-* `activeExpireCycle()` handles eviciton of keys with a time to live set via the `EXPIRE` command.
-* `freeMemoryIfNeeded()` is called when a new write command should be performed but Redis is out of memory according to the `maxmemory` directive.
-* The global variable `redisCommandTable` defines all the Redis commands, specifying the name of the command, the function implementing the command, the number of arguments required, and other properties of each command.
-
-networking.c
----
-
-This file defines all the I/O functions with clients, masters and slaves
-(which in Redis are just special clients):
-
-* `createClient()` allocates and initializes a new client.
-* the `addReply*()` family of functions are used by commands implementations in order to append data to the client structure, that will be transmitted to the client as a reply for a given command executed.
-* `writeToClient()` transmits the data pending in the output buffers to the client and is called by the *writable event handler* `sendReplyToClient()`.
-* `readQueryFromClient()` is the *readable event handler* and accumulates data from read from the client into the query buffer.
-* `processInputBuffer()` is the entry point in order to parse the client query buffer according to the Redis protocol. Once commands are ready to be processed, it calls `processCommand()` which is defined inside `server.c` in order to actually execute the command.
-* `freeClient()` deallocates, disconnects and removes a client.
-
-aof.c and rdb.c
----
-
-As you can guess from the names these files implement the RDB and AOF
-persistence for Redis. Redis uses a persistence model based on the `fork()`
-system call in order to create a thread with the same (shared) memory
-content of the main Redis thread. This secondary thread dumps the content
-of the memory on disk. This is used by `rdb.c` to create the snapshots
-on disk and by `aof.c` in order to perform the AOF rewrite when the
-append only file gets too big.
-
-The implementation inside `aof.c` has additional functions in order to
-implement an API that allows commands to append new commands into the AOF
-file as clients execute them.
-
-The `call()` function defined inside `server.c` is responsible to call
-the functions that in turn will write the commands into the AOF.
-
-db.c
----
-
-Certain Redis commands operate on specific data types, others are general.
-Examples of generic commands are `DEL` and `EXPIRE`. They operate on keys
-and not on their values specifically. All those generic commands are
-defined inside `db.c`.
-
-Moreover `db.c` implements an API in order to perform certain operations
-on the Redis dataset without directly accessing the internal data structures.
-
-The most important functions inside `db.c` which are used in many commands
-implementations are the following:
-
-* `lookupKeyRead()` and `lookupKeyWrite()` are used in order to get a pointer to the value associated to a given key, or `NULL` if the key does not exist.
-* `dbAdd()` and its higher level counterpart `setKey()` create a new key in a Redis database.
-* `dbDelete()` removes a key and its associated value.
-* `emptyDb()` removes an entire single database or all the databases defined.
-
-The rest of the file implements the generic commands exposed to the client.
-
-object.c
----
-
-The `robj` structure defining Redis objects was already described. Inside
-`object.c` there are all the functions that operate with Redis objects at
-a basic level, like functions to allocate new objects, handle the reference
-counting and so forth. Notable functions inside this file:
-
-* `incrRefcount()` and `decrRefCount()` are used in order to increment or decrement an object reference count. When it drops to 0 the object is finally freed.
-* `createObject()` allocates a new object. There are also specialized functions to allocate string objects having a specific content, like `createStringObjectFromLongLong()` and similar functions.
-
-This file also implements the `OBJECT` command.
-
-replication.c
----
-
-This is one of the most complex files inside Redis, it is recommended to
-approach it only after getting a bit familiar with the rest of the code base.
-In this file there is the implementation of both the master and slave role
-of Redis.
-
-One of the most important functions inside this file is `replicationFeedSlaves()` that writes commands to the clients representing slave instances connected
-to our master, so that the slaves can get the writes performed by the clients:
-this way their data set will remain synchronized with the one in the master.
-
-This file also implements both the `SYNC` and `PSYNC` commands that are
-used in order to perform the first synchronization between masters and
-slaves, or to continue the replication after a disconnection.
-
-Other C files
----
-
-* `t_hash.c`, `t_list.c`, `t_set.c`, `t_string.c` and `t_zset.c` contains the implementation of the Redis data types. They implement both an API to access a given data type, and the client commands implementations for these data types.
-* `ae.c` implements the Redis event loop, it's a self contained library which is simple to read and understand.
-* `sds.c` is the Redis string library, check http://github.com/antirez/sds for more information.
-* `anet.c` is a library to use POSIX networking in a simpler way compared to the raw interface exposed by the kernel.
-* `dict.c` is an implementation of a non-blocking hash table which rehashes incrementally.
-* `scripting.c` implements Lua scripting. It is completely self contained from the rest of the Redis implementation and is simple enough to understand if you are familar with the Lua API.
-* `cluster.c` implements the Redis Cluster. Probably a good read only after being very familiar with the rest of the Redis code base. If you want to read `cluster.c` make sure to read the [Redis Cluster specification][3].
-
-[3]: http://redis.io/topics/cluster-spec
-
-Anatomy of a Redis command
----
-
-All the Redis commands are defined in the following way:
-
-    void foobarCommand(client *c) {
-        printf("%s",c->argv[1]->ptr); /* Do something with the argument. */
-        addReply(c,shared.ok); /* Reply something to the client. */
-    }
-
-The command is then referenced inside `server.c` in the command table:
-
-    {"foobar",foobarCommand,2,"rtF",0,NULL,0,0,0,0,0},
-
-In the above example `2` is the number of arguments the command takes,
-while `"rtF"` are the command flags, as documented in the command table
-top comment inside `server.c`.
-
-After the command operates in some way, it returns a reply to the client,
-usually using `addReply()` or a similar function defined inside `networking.c`.
-
-There are tons of commands implementations inside th Redis source code
-that can serve as examples of actual commands implementations. To write
-a few toy commands can be a good exercise to familiarize with the code base.
-
-There are also many other files not described here, but it is useless to
-cover everything. We want to just help you with the first steps.
-Eventually you'll find your way inside the Redis code base :-)
-
-Enjoy!

deps/Makefile

@@ -36,6 +36,7 @@ distclean:
 	-(cd hiredis && $(MAKE) clean) > /dev/null || true
 	-(cd linenoise && $(MAKE) clean) > /dev/null || true
 	-(cd lua && $(MAKE) clean) > /dev/null || true
+	-(cd geohash-int && $(MAKE) clean) > /dev/null || true
 	-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
 	-(rm -f .make-*)
 
@@ -81,3 +82,9 @@ jemalloc: .make-prerequisites
 	cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
 
 .PHONY: jemalloc
+
+geohash-int: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd geohash-int && $(MAKE)
+
+.PHONY: geohash-int

deps/geohash-int/Makefile

@@ -0,0 +1,23 @@
+STD=
+WARN= -Wall
+OPT= -O2
+
+R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
+R_LDFLAGS= $(LDFLAGS)
+DEBUG= -g
+
+R_CC=$(CC) $(R_CFLAGS)
+R_LD=$(CC) $(R_LDFLAGS)
+
+all: geohash.o geohash_helper.o
+
+.PHONY: all
+
+geohash.o: geohash.h geohash.c
+geohash_helper.o: geohash.h geohash_helper.h geohash_helper.c
+
+.c.o:
+	$(R_CC) -c $<
+
+clean:
+	rm -f *.o

deps/geohash-int/geohash.c

@@ -151,7 +151,7 @@ int geohashEncode(const GeoHashRange *long_range, const GeoHashRange *lat_range,
 }
 
 int geohashEncodeType(double longitude, double latitude, uint8_t step, GeoHashBits *hash) {
-    GeoHashRange r[2] = {{0}};
+    GeoHashRange r[2] = { { 0 } };
     geohashGetCoordRange(&r[0], &r[1]);
     return geohashEncode(&r[0], &r[1], longitude, latitude, step, hash);
 }
@@ -194,7 +194,7 @@ int geohashDecode(const GeoHashRange long_range, const GeoHashRange lat_range,
 }
 
 int geohashDecodeType(const GeoHashBits hash, GeoHashArea *area) {
-    GeoHashRange r[2] = {{0}};
+    GeoHashRange r[2] = { { 0 } };
     geohashGetCoordRange(&r[0], &r[1]);
     return geohashDecode(r[0], r[1], hash, area);
 }
@@ -211,7 +211,7 @@ int geohashDecodeAreaToLongLat(const GeoHashArea *area, double *xy) {
 }
 
 int geohashDecodeToLongLatType(const GeoHashBits hash, double *xy) {
-    GeoHashArea area = {{0}};
+    GeoHashArea area = { { 0 } };
     if (!xy || !geohashDecodeType(hash, &area))
         return 0;
     return geohashDecodeAreaToLongLat(&area, xy);

deps/geohash-int/geohash_helper.c

@@ -34,9 +34,7 @@
  * https://github.com/yinqiwen/ardb/blob/d42503/src/geo/geohash_helper.cpp
  */
 
-#include "fmacros.h"
 #include "geohash_helper.h"
-#include "debugmacro.h"
 #include <math.h>
 
 #define D_R (M_PI / 180.0)
@@ -82,32 +80,35 @@ uint8_t geohashEstimateStepsByRadius(double range_meters, double lat) {
     return step;
 }
 
-/* Return the bounding box of the search area centered at latitude,longitude
- * having a radius of radius_meter. bounds[0] - bounds[2] is the minimum
- * and maxium longitude, while bounds[1] - bounds[3] is the minimum and
- * maximum latitude.
- *
- * This function does not behave correctly with very large radius values, for
- * instance for the coordinates 81.634948934258375 30.561509253718668 and a
- * radius of 7083 kilometers, it reports as bounding boxes:
- *
- * min_lon 7.680495, min_lat -33.119473, max_lon 155.589402, max_lat 94.242491
- *
- * However, for instance, a min_lon of 7.680495 is not correct, because the
- * point -1.27579540014266968 61.33421815228281559 is at less than 7000
- * kilometers away.
- *
- * Since this function is currently only used as an optimization, the
- * optimization is not used for very big radiuses, however the function
- * should be fixed. */
+int geohashBitsComparator(const GeoHashBits *a, const GeoHashBits *b) {
+    /* If step not equal, compare on step.  Else, compare on bits. */
+    return a->step != b->step ? a->step - b->step : a->bits - b->bits;
+}
+
 int geohashBoundingBox(double longitude, double latitude, double radius_meters,
                        double *bounds) {
     if (!bounds) return 0;
 
-    bounds[0] = longitude - rad_deg(radius_meters/EARTH_RADIUS_IN_METERS/cos(deg_rad(latitude)));
-    bounds[2] = longitude + rad_deg(radius_meters/EARTH_RADIUS_IN_METERS/cos(deg_rad(latitude)));
-    bounds[1] = latitude - rad_deg(radius_meters/EARTH_RADIUS_IN_METERS);
-    bounds[3] = latitude + rad_deg(radius_meters/EARTH_RADIUS_IN_METERS);
+    double lonr, latr;
+    lonr = deg_rad(longitude);
+    latr = deg_rad(latitude);
+
+    if (radius_meters > EARTH_RADIUS_IN_METERS)
+        radius_meters = EARTH_RADIUS_IN_METERS;
+    double distance = radius_meters / EARTH_RADIUS_IN_METERS;
+    double min_latitude = latr - distance;
+    double max_latitude = latr + distance;
+
+    /* Note: we're being lazy and not accounting for coordinates near poles */
+    double min_longitude, max_longitude;
+    double difference_longitude = asin(sin(distance) / cos(latr));
+    min_longitude = lonr - difference_longitude;
+    max_longitude = lonr + difference_longitude;
+
+    bounds[0] = rad_deg(min_longitude);
+    bounds[1] = rad_deg(min_latitude);
+    bounds[2] = rad_deg(max_longitude);
+    bounds[3] = rad_deg(max_latitude);
     return 1;
 }
 
@@ -160,7 +161,7 @@ GeoHashRadius geohashGetAreasByRadius(double longitude, double latitude, double
             < radius_meters) decrease_step = 1;
     }
 
-    if (steps > 1 && decrease_step) {
+    if (decrease_step) {
         steps--;
         geohashEncode(&long_range,&lat_range,longitude,latitude,steps,&hash);
         geohashNeighbors(&hash,&neighbors);
@@ -168,27 +169,25 @@ GeoHashRadius geohashGetAreasByRadius(double longitude, double latitude, double
     }
 
     /* Exclude the search areas that are useless. */
-    if (steps >= 2) {
-        if (area.latitude.min < min_lat) {
-            GZERO(neighbors.south);
-            GZERO(neighbors.south_west);
-            GZERO(neighbors.south_east);
-        }
-        if (area.latitude.max > max_lat) {
-            GZERO(neighbors.north);
-            GZERO(neighbors.north_east);
-            GZERO(neighbors.north_west);
-        }
-        if (area.longitude.min < min_lon) {
-            GZERO(neighbors.west);
-            GZERO(neighbors.south_west);
-            GZERO(neighbors.north_west);
-        }
-        if (area.longitude.max > max_lon) {
-            GZERO(neighbors.east);
-            GZERO(neighbors.south_east);
-            GZERO(neighbors.north_east);
-        }
+    if (area.latitude.min < min_lat) {
+        GZERO(neighbors.south);
+        GZERO(neighbors.south_west);
+        GZERO(neighbors.south_east);
+    }
+    if (area.latitude.max > max_lat) {
+        GZERO(neighbors.north);
+        GZERO(neighbors.north_east);
+        GZERO(neighbors.north_west);
+    }
+    if (area.longitude.min < min_lon) {
+        GZERO(neighbors.west);
+        GZERO(neighbors.south_west);
+        GZERO(neighbors.north_west);
+    }
+    if (area.longitude.max > max_lon) {
+        GZERO(neighbors.east);
+        GZERO(neighbors.south_east);
+        GZERO(neighbors.north_east);
     }
     radius.hash = hash;
     radius.neighbors = neighbors;

deps/geohash-int/geohash_helper.h

@@ -32,6 +32,7 @@
 #ifndef GEOHASH_HELPER_HPP_
 #define GEOHASH_HELPER_HPP_
 
+#include <math.h>
 #include "geohash.h"
 
 #define GZERO(s) s.bits = s.step = 0;

deps/hiredis/.gitignore

@@ -4,4 +4,3 @@
 /*.so
 /*.dylib
 /*.a
-/*.pc

deps/hiredis/.travis.yml

@@ -1,39 +1,6 @@
 language: c
-sudo: false
 compiler:
   - gcc
   - clang
 
-os:
-  - linux
-  - osx
-
-before_script:
-    - if [ "$TRAVIS_OS_NAME" == "osx" ] ; then brew update; brew install redis; fi
-
-addons:
-  apt:
-    packages:
-    - libc6-dbg
-    - libc6-dev
-    - libc6:i386
-    - libc6-dev-i386
-    - libc6-dbg:i386
-    - gcc-multilib
-    - valgrind
-
-env:
-    - CFLAGS="-Werror"
-    - PRE="valgrind --track-origins=yes --leak-check=full"
-    - TARGET="32bit" TARGET_VARS="32bit-vars" CFLAGS="-Werror"
-    - TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
-
-matrix:
-  exclude:
-    - os: osx
-      env: PRE="valgrind --track-origins=yes --leak-check=full"
-
-    - os: osx
-      env: TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
-
-script: make $TARGET CFLAGS="$CFLAGS" && make check PRE="$PRE" && make $TARGET_VARS hiredis-example
+script: make && make check

deps/hiredis/CHANGELOG.md

@@ -1,128 +1,3 @@
-### 1.0.0 (unreleased)
-
-**Fixes**:
-
-* Catch a buffer overflow when formatting the error message
-* Import latest upstream sds. This breaks applications that are linked against the old hiredis v0.13
-* Fix warnings, when compiled with -Wshadow
-* Make hiredis compile in Cygwin on Windows, now CI-tested
-
-**BREAKING CHANGES**:
-
-* Change `redisReply.len` to `size_t`, as it denotes the the size of a string
-
-User code should compare this to `size_t` values as well.
-If it was used to compare to other values, casting might be necessary or can be removed, if casting was applied before.
-
-* Remove backwards compatibility macro's
-
-This removes the following old function aliases, use the new name now:
-
-| Old                         | New                    |
-| --------------------------- | ---------------------- |
-| redisReplyReaderCreate      | redisReaderCreate      |
-| redisReplyReaderCreate      | redisReaderCreate      |
-| redisReplyReaderFree        | redisReaderFree        |
-| redisReplyReaderFeed        | redisReaderFeed        |
-| redisReplyReaderGetReply    | redisReaderGetReply    |
-| redisReplyReaderSetPrivdata | redisReaderSetPrivdata |
-| redisReplyReaderGetObject   | redisReaderGetObject   |
-| redisReplyReaderGetError    | redisReaderGetError    |
-
-* The `DEBUG` variable in the Makefile was renamed to `DEBUG_FLAGS`
-
-Previously it broke some builds for people that had `DEBUG` set to some arbitrary value,
-due to debugging other software.
-By renaming we avoid unintentional name clashes.
-
-Simply rename `DEBUG` to `DEBUG_FLAGS` in your environment to make it working again.
-
-### 0.13.3 (2015-09-16)
-
-* Revert "Clear `REDIS_CONNECTED` flag when connection is closed".
-* Make tests pass on FreeBSD (Thanks, Giacomo Olgeni)
-
-
-If the `REDIS_CONNECTED` flag is cleared,
-the async onDisconnect callback function will never be called.
-This causes problems as the disconnect is never reported back to the user.
-
-### 0.13.2 (2015-08-25)
-
-* Prevent crash on pending replies in async code (Thanks, @switch-st)
-* Clear `REDIS_CONNECTED` flag when connection is closed (Thanks, Jerry Jacobs)
-* Add MacOS X addapter (Thanks, @dizzus)
-* Add Qt adapter (Thanks, Pietro Cerutti)
-* Add Ivykis adapter (Thanks, Gergely Nagy)
-
-All adapters are provided as is and are only tested where possible.
-
-### 0.13.1 (2015-05-03)
-
-This is a bug fix release.
-The new `reconnect` method introduced new struct members, which clashed with pre-defined names in pre-C99 code.
-Another commit forced C99 compilation just to make it work, but of course this is not desirable for outside projects.
-Other non-C99 code can now use hiredis as usual again.
-Sorry for the inconvenience.
-
-* Fix memory leak in async reply handling (Salvatore Sanfilippo)
-* Rename struct member to avoid name clash with pre-c99 code (Alex Balashov, ncopa)
-
-### 0.13.0 (2015-04-16)
-
-This release adds a minimal Windows compatibility layer.
-The parser, standalone since v0.12.0, can now be compiled on Windows
-(and thus used in other client libraries as well)
-
-* Windows compatibility layer for parser code (tzickel)
-* Properly escape data printed to PKGCONF file (Dan Skorupski)
-* Fix tests when assert() undefined (Keith Bennett, Matt Stancliff)
-* Implement a reconnect method for the client context, this changes the structure of `redisContext` (Aaron Bedra)
-
-### 0.12.1 (2015-01-26)
-
-* Fix `make install`: DESTDIR support, install all required files, install PKGCONF in proper location
-* Fix `make test` as 32 bit build on 64 bit platform
-
-### 0.12.0 (2015-01-22)
-
-* Add optional KeepAlive support
-
-* Try again on EINTR errors
-
-* Add libuv adapter
-
-* Add IPv6 support
-
-* Remove possiblity of multiple close on same fd
-
-* Add ability to bind source address on connect
-
-* Add redisConnectFd() and redisFreeKeepFd()
-
-* Fix getaddrinfo() memory leak
-
-* Free string if it is unused (fixes memory leak)
-
-* Improve redisAppendCommandArgv performance 2.5x
-
-* Add support for SO_REUSEADDR
-
-* Fix redisvFormatCommand format parsing
-
-* Add GLib 2.0 adapter
-
-* Refactor reading code into read.c
-
-* Fix errno error buffers to not clobber errors
-
-* Generate pkgconf during build
-
-* Silence _BSD_SOURCE warnings
-
-* Improve digit counting for multibulk creation
-
-
 ### 0.11.0
 
 * Increase the maximum multi-bulk reply depth to 7.

deps/hiredis/Makefile

@@ -3,25 +3,13 @@
 # Copyright (C) 2010-2011 Pieter Noordhuis <pcnoordhuis at gmail dot com>
 # This file is released under the BSD license, see the COPYING file
 
-OBJ=net.o hiredis.o sds.o async.o read.o
-EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev hiredis-example-glib
+OBJ=net.o hiredis.o sds.o async.o
+EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev
 TESTS=hiredis-test
 LIBNAME=libhiredis
-PKGCONFNAME=hiredis.pc
 
-HIREDIS_MAJOR=$(shell grep HIREDIS_MAJOR hiredis.h | awk '{print $$3}')
-HIREDIS_MINOR=$(shell grep HIREDIS_MINOR hiredis.h | awk '{print $$3}')
-HIREDIS_PATCH=$(shell grep HIREDIS_PATCH hiredis.h | awk '{print $$3}')
-HIREDIS_SONAME=$(shell grep HIREDIS_SONAME hiredis.h | awk '{print $$3}')
-
-# Installation related variables and target
-PREFIX?=/usr/local
-INCLUDE_PATH?=include/hiredis
-LIBRARY_PATH?=lib
-PKGCONF_PATH?=pkgconfig
-INSTALL_INCLUDE_PATH= $(DESTDIR)$(PREFIX)/$(INCLUDE_PATH)
-INSTALL_LIBRARY_PATH= $(DESTDIR)$(PREFIX)/$(LIBRARY_PATH)
-INSTALL_PKGCONF_PATH= $(INSTALL_LIBRARY_PATH)/$(PKGCONF_PATH)
+HIREDIS_MAJOR=0
+HIREDIS_MINOR=11
 
 # redis-server configuration used for testing
 REDIS_PORT=56379
@@ -37,16 +25,15 @@ export REDIS_TEST_CONFIG
 
 # Fallback to gcc when $CC is not in $PATH.
 CC:=$(shell sh -c 'type $(CC) >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
-CXX:=$(shell sh -c 'type $(CXX) >/dev/null 2>/dev/null && echo $(CXX) || echo g++')
 OPTIMIZATION?=-O3
 WARNINGS=-Wall -W -Wstrict-prototypes -Wwrite-strings
-DEBUG_FLAGS?= -g -ggdb
-REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(ARCH)
+DEBUG?= -g -ggdb
+REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CFLAGS) $(WARNINGS) $(DEBUG) $(ARCH)
 REAL_LDFLAGS=$(LDFLAGS) $(ARCH)
 
 DYLIBSUFFIX=so
 STLIBSUFFIX=a
-DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
+DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR).$(HIREDIS_MINOR)
 DYLIB_MAJOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
 DYLIBNAME=$(LIBNAME).$(DYLIBSUFFIX)
 DYLIB_MAKE_CMD=$(CC) -shared -Wl,-soname,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
@@ -62,20 +49,19 @@ ifeq ($(uname_S),SunOS)
 endif
 ifeq ($(uname_S),Darwin)
   DYLIBSUFFIX=dylib
-  DYLIB_MINOR_NAME=$(LIBNAME).$(HIREDIS_SONAME).$(DYLIBSUFFIX)
+  DYLIB_MINOR_NAME=$(LIBNAME).$(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(DYLIBSUFFIX)
+  DYLIB_MAJOR_NAME=$(LIBNAME).$(HIREDIS_MAJOR).$(DYLIBSUFFIX)
   DYLIB_MAKE_CMD=$(CC) -shared -Wl,-install_name,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
 endif
 
-all: $(DYLIBNAME) $(STLIBNAME) hiredis-test $(PKGCONFNAME)
+all: $(DYLIBNAME)
 
 # Deps (use make dep to generate this)
-async.o: async.c fmacros.h async.h hiredis.h read.h sds.h net.h dict.c dict.h
-dict.o: dict.c fmacros.h dict.h
-hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h net.h
-net.o: net.c fmacros.h net.h hiredis.h read.h sds.h
-read.o: read.c fmacros.h read.h sds.h
+net.o: net.c fmacros.h net.h hiredis.h
+async.o: async.c async.h hiredis.h sds.h dict.c dict.h
+hiredis.o: hiredis.c fmacros.h hiredis.h net.h sds.h
 sds.o: sds.c sds.h
-test.o: test.c fmacros.h hiredis.h read.h sds.h
+test.o: test.c hiredis.h
 
 $(DYLIBNAME): $(OBJ)
 	$(DYLIB_MAKE_CMD) $(OBJ)
@@ -93,15 +79,6 @@ hiredis-example-libevent: examples/example-libevent.c adapters/libevent.h $(STLI
 hiredis-example-libev: examples/example-libev.c adapters/libev.h $(STLIBNAME)
 	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -lev $(STLIBNAME)
 
-hiredis-example-glib: examples/example-glib.c adapters/glib.h $(STLIBNAME)
-	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) $(shell pkg-config --cflags --libs glib-2.0) -I. $< $(STLIBNAME)
-
-hiredis-example-ivykis: examples/example-ivykis.c adapters/ivykis.h $(STLIBNAME)
-	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -livykis $(STLIBNAME)
-
-hiredis-example-macosx: examples/example-macosx.c adapters/macosx.h $(STLIBNAME)
-	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME)
-
 ifndef AE_DIR
 hiredis-example-ae:
 	@echo "Please specify AE_DIR (e.g. <redis repository>/src)"
@@ -117,20 +94,7 @@ hiredis-example-libuv:
 	@false
 else
 hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
-	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME)
-endif
-
-ifeq ($(and $(QT_MOC),$(QT_INCLUDE_DIR),$(QT_LIBRARY_DIR)),)
-hiredis-example-qt:
-	@echo "Please specify QT_MOC, QT_INCLUDE_DIR AND QT_LIBRARY_DIR"
-	@false
-else
-hiredis-example-qt: examples/example-qt.cpp adapters/qt.h $(STLIBNAME)
-	$(QT_MOC) adapters/qt.h -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore | \
-	    $(CXX) -x c++ -o qt-adapter-moc.o -c - $(REAL_CFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore
-	$(QT_MOC) examples/example-qt.h -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore | \
-	    $(CXX) -x c++ -o qt-example-moc.o -c - $(REAL_CFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore
-	$(CXX) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore -L$(QT_LIBRARY_DIR) qt-adapter-moc.o qt-example-moc.o $< -pthread $(STLIBNAME) -lQtCore
+	$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread $(STLIBNAME)
 endif
 
 hiredis-example: examples/example.c $(STLIBNAME)
@@ -139,16 +103,14 @@ hiredis-example: examples/example.c $(STLIBNAME)
 examples: $(EXAMPLES)
 
 hiredis-test: test.o $(STLIBNAME)
-
-hiredis-%: %.o $(STLIBNAME)
-	$(CC) $(REAL_CFLAGS) -o $@ $(REAL_LDFLAGS) $< $(STLIBNAME)
+	$(CC) -o $@ $(REAL_LDFLAGS) $< $(STLIBNAME)
 
 test: hiredis-test
 	./hiredis-test
 
 check: hiredis-test
 	@echo "$$REDIS_TEST_CONFIG" | $(REDIS_SERVER) -
-	$(PRE) ./hiredis-test -h 127.0.0.1 -p $(REDIS_PORT) -s /tmp/hiredis-test-redis.sock || \
+	./hiredis-test -h 127.0.0.1 -p $(REDIS_PORT) -s /tmp/hiredis-test-redis.sock || \
 			( kill `cat /tmp/hiredis-test-redis.pid` && false )
 	kill `cat /tmp/hiredis-test-redis.pid`
 
@@ -156,38 +118,29 @@ check: hiredis-test
 	$(CC) -std=c99 -pedantic -c $(REAL_CFLAGS) $<
 
 clean:
-	rm -rf $(DYLIBNAME) $(STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
+	rm -rf $(DYLIBNAME) $(STLIBNAME) $(TESTS) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
 
 dep:
 	$(CC) -MM *.c
 
+# Installation related variables and target
+PREFIX?=/usr/local
+INSTALL_INCLUDE_PATH= $(PREFIX)/include/hiredis
+INSTALL_LIBRARY_PATH= $(PREFIX)/lib
+
 ifeq ($(uname_S),SunOS)
   INSTALL?= cp -r
 endif
 
 INSTALL?= cp -a
 
-$(PKGCONFNAME): hiredis.h
-	@echo "Generating $@ for pkgconfig..."
-	@echo prefix=$(PREFIX) > $@
-	@echo exec_prefix=\$${prefix} >> $@
-	@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
-	@echo includedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
-	@echo >> $@
-	@echo Name: hiredis >> $@
-	@echo Description: Minimalistic C client library for Redis. >> $@
-	@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
-	@echo Libs: -L\$${libdir} -lhiredis >> $@
-	@echo Cflags: -I\$${includedir} -D_FILE_OFFSET_BITS=64 >> $@
-
-install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME)
+install: $(DYLIBNAME) $(STLIBNAME)
 	mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_LIBRARY_PATH)
-	$(INSTALL) hiredis.h async.h read.h sds.h adapters $(INSTALL_INCLUDE_PATH)
+	$(INSTALL) hiredis.h async.h adapters $(INSTALL_INCLUDE_PATH)
 	$(INSTALL) $(DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(DYLIB_MINOR_NAME)
-	cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIBNAME)
+	cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIB_MAJOR_NAME)
+	cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MAJOR_NAME) $(DYLIBNAME)
 	$(INSTALL) $(STLIBNAME) $(INSTALL_LIBRARY_PATH)
-	mkdir -p $(INSTALL_PKGCONF_PATH)
-	$(INSTALL) $(PKGCONFNAME) $(INSTALL_PKGCONF_PATH)
 
 32bit:
 	@echo ""
@@ -195,10 +148,6 @@ install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME)
 	@echo ""
 	$(MAKE) CFLAGS="-m32" LDFLAGS="-m32"
 
-32bit-vars:
-	$(eval CFLAGS=-m32)
-	$(eval LDFLAGS=-m32)
-
 gprof:
 	$(MAKE) CFLAGS="-pg" LDFLAGS="-pg"
 
@@ -214,4 +163,4 @@ coverage: gcov
 noopt:
 	$(MAKE) OPTIMIZATION=""
 
-.PHONY: all test check clean dep install 32bit 32bit-vars gprof gcov noopt
+.PHONY: all test check clean dep install 32bit gprof gcov noopt

deps/hiredis/README.md

@@ -1,13 +1,11 @@
 [![Build Status](https://travis-ci.org/redis/hiredis.png)](https://travis-ci.org/redis/hiredis)
 
-**This Readme reflects the latest changed in the master branch. See [v0.13.3](https://github.com/redis/hiredis/tree/v0.13.3) for the Readme and documentation for the latest release.**
-
 # HIREDIS
 
 Hiredis is a minimalistic C client library for the [Redis](http://redis.io/) database.
 
 It is minimalistic because it just adds minimal support for the protocol, but
-at the same time it uses a high level printf-alike API in order to make it
+at the same time it uses an high level printf-alike API in order to make it
 much higher level than otherwise suggested by its minimal code base and the
 lack of explicit bindings for every Redis command.
 
@@ -22,15 +20,7 @@ Redis version >= 1.2.0.
 The library comes with multiple APIs. There is the
 *synchronous API*, the *asynchronous API* and the *reply parsing API*.
 
-## Upgrading to `1.0.0`
-
-Version 1.0.0 marks a stable release of hiredis.
-It includes some minor breaking changes, mostly to make the exposed API more uniform and self-explanatory.
-It also bundles the updated `sds` library, to sync up with upstream and Redis.
-For most applications a recompile against the new hiredis should be enough.
-For code changes see the [Changelog](CHANGELOG.md).
-
-## Upgrading from `<0.9.0`
+## UPGRADING
 
 Version 0.9.0 is a major overhaul of hiredis in every aspect. However, upgrading existing
 code using hiredis should not be a big pain. The key thing to keep in mind when
@@ -41,62 +31,51 @@ the stateless 0.0.1 that only has a file descriptor to work with.
 
 To consume the synchronous API, there are only a few function calls that need to be introduced:
 
-```c
-redisContext *redisConnect(const char *ip, int port);
-void *redisCommand(redisContext *c, const char *format, ...);
-void freeReplyObject(void *reply);
-```
+    redisContext *redisConnect(const char *ip, int port);
+    void *redisCommand(redisContext *c, const char *format, ...);
+    void freeReplyObject(void *reply);
 
 ### Connecting
 
 The function `redisConnect` is used to create a so-called `redisContext`. The
 context is where Hiredis holds state for a connection. The `redisContext`
-struct has an integer `err` field that is non-zero when the connection is in
+struct has an integer `err` field that is non-zero when an the connection is in
 an error state. The field `errstr` will contain a string with a description of
 the error. More information on errors can be found in the **Errors** section.
 After trying to connect to Redis using `redisConnect` you should
 check the `err` field to see if establishing the connection was successful:
-```c
-redisContext *c = redisConnect("127.0.0.1", 6379);
-if (c == NULL || c->err) {
-    if (c) {
+
+    redisContext *c = redisConnect("127.0.0.1", 6379);
+    if (c != NULL && c->err) {
         printf("Error: %s\n", c->errstr);
         // handle error
-    } else {
-        printf("Can't allocate redis context\n");
     }
-}
-```
-
-*Note: A `redisContext` is not thread-safe.*
 
 ### Sending commands
 
 There are several ways to issue commands to Redis. The first that will be introduced is
 `redisCommand`. This function takes a format similar to printf. In the simplest form,
 it is used like this:
-```c
-reply = redisCommand(context, "SET foo bar");
-```
+
+    reply = redisCommand(context, "SET foo bar");
 
 The specifier `%s` interpolates a string in the command, and uses `strlen` to
 determine the length of the string:
-```c
-reply = redisCommand(context, "SET foo %s", value);
-```
+
+    reply = redisCommand(context, "SET foo %s", value);
+
 When you need to pass binary safe strings in a command, the `%b` specifier can be
 used. Together with a pointer to the string, it requires a `size_t` length argument
 of the string:
-```c
-reply = redisCommand(context, "SET foo %b", value, (size_t) valuelen);
-```
+
+    reply = redisCommand(context, "SET foo %b", value, (size_t) valuelen);
+
 Internally, Hiredis splits the command in different arguments and will
 convert it to the protocol used to communicate with Redis.
 One or more spaces separates arguments, so you can use the specifiers
 anywhere in an argument:
-```c
-reply = redisCommand(context, "SET key:%s %s", myid, value);
-```
+
+    reply = redisCommand(context, "SET key:%s %s", myid, value);
 
 ### Using replies
 
@@ -135,11 +114,11 @@ was received:
       Redis may reply with nested arrays but this is fully supported.
 
 Replies should be freed using the `freeReplyObject()` function.
-Note that this function will take care of freeing sub-reply objects
+Note that this function will take care of freeing sub-replies objects
 contained in arrays and nested arrays, so there is no need for the user to
 free the sub replies (it is actually harmful and will corrupt the memory).
 
-**Important:** the current version of hiredis (0.10.0) frees replies when the
+**Important:** the current version of hiredis (0.10.0) free's replies when the
 asynchronous API is used. This means you should not call `freeReplyObject` when
 you use this API. The reply is cleaned up by hiredis _after_ the callback
 returns. This behavior will probably change in future releases, so make sure to
@@ -148,19 +127,19 @@ keep an eye on the changelog when upgrading (see issue #39).
 ### Cleaning up
 
 To disconnect and free the context the following function can be used:
-```c
-void redisFree(redisContext *c);
-```
-This function immediately closes the socket and then frees the allocations done in
+
+    void redisFree(redisContext *c);
+
+This function immediately closes the socket and then free's the allocations done in
 creating the context.
 
 ### Sending commands (cont'd)
 
 Together with `redisCommand`, the function `redisCommandArgv` can be used to issue commands.
 It has the following prototype:
-```c
-void *redisCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
-```
+
+    void *redisCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
+
 It takes the number of arguments `argc`, an array of strings `argv` and the lengths of the
 arguments `argvlen`. For convenience, `argvlen` may be set to `NULL` and the function will
 use `strlen(3)` on every argument to determine its length. Obviously, when any of the arguments
@@ -190,10 +169,10 @@ The function `redisGetReply` is exported as part of the Hiredis API and can be u
 is expected on the socket. To pipeline commands, the only things that needs to be done is
 filling up the output buffer. For this cause, two commands can be used that are identical
 to the `redisCommand` family, apart from not returning a reply:
-```c
-void redisAppendCommand(redisContext *c, const char *format, ...);
-void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
-```
+
+    void redisAppendCommand(redisContext *c, const char *format, ...);
+    void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
+
 After calling either function one or more times, `redisGetReply` can be used to receive the
 subsequent replies. The return value for this function is either `REDIS_OK` or `REDIS_ERR`, where
 the latter means an error occurred while reading a reply. Just as with the other commands,
@@ -201,24 +180,24 @@ the `err` field in the context can be used to find out what the cause of this er
 
 The following examples shows a simple pipeline (resulting in only a single call to `write(2)` and
 a single call to `read(2)`):
-```c
-redisReply *reply;
-redisAppendCommand(context,"SET foo bar");
-redisAppendCommand(context,"GET foo");
-redisGetReply(context,&reply); // reply for SET
-freeReplyObject(reply);
-redisGetReply(context,&reply); // reply for GET
-freeReplyObject(reply);
-```
-This API can also be used to implement a blocking subscriber:
-```c
-reply = redisCommand(context,"SUBSCRIBE foo");
-freeReplyObject(reply);
-while(redisGetReply(context,&reply) == REDIS_OK) {
-    // consume message
+
+    redisReply *reply;
+    redisAppendCommand(context,"SET foo bar");
+    redisAppendCommand(context,"GET foo");
+    redisGetReply(context,&reply); // reply for SET
     freeReplyObject(reply);
-}
-```
+    redisGetReply(context,&reply); // reply for GET
+    freeReplyObject(reply);
+
+This API can also be used to implement a blocking subscriber:
+
+    reply = redisCommand(context,"SUBSCRIBE foo");
+    freeReplyObject(reply);
+    while(redisGetReply(context,&reply) == REDIS_OK) {
+        // consume message
+        freeReplyObject(reply);
+    }
+
 ### Errors
 
 When a function call is not successful, depending on the function either `NULL` or `REDIS_ERR` is
@@ -258,62 +237,58 @@ should be checked after creation to see if there were errors creating the connec
 Because the connection that will be created is non-blocking, the kernel is not able to
 instantly return if the specified host and port is able to accept a connection.
 
-*Note: A `redisAsyncContext` is not thread-safe.*
-
-```c
-redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
-if (c->err) {
-    printf("Error: %s\n", c->errstr);
-    // handle error
-}
-```
+    redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
+    if (c->err) {
+        printf("Error: %s\n", c->errstr);
+        // handle error
+    }
 
 The asynchronous context can hold a disconnect callback function that is called when the
 connection is disconnected (either because of an error or per user request). This function should
 have the following prototype:
-```c
-void(const redisAsyncContext *c, int status);
-```
+
+    void(const redisAsyncContext *c, int status);
+
 On a disconnect, the `status` argument is set to `REDIS_OK` when disconnection was initiated by the
 user, or `REDIS_ERR` when the disconnection was caused by an error. When it is `REDIS_ERR`, the `err`
 field in the context can be accessed to find out the cause of the error.
 
-The context object is always freed after the disconnect callback fired. When a reconnect is needed,
+The context object is always free'd after the disconnect callback fired. When a reconnect is needed,
 the disconnect callback is a good point to do so.
 
 Setting the disconnect callback can only be done once per context. For subsequent calls it will
 return `REDIS_ERR`. The function to set the disconnect callback has the following prototype:
-```c
-int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
-```
+
+    int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
+
 ### Sending commands and their callbacks
 
 In an asynchronous context, commands are automatically pipelined due to the nature of an event loop.
 Therefore, unlike the synchronous API, there is only a single way to send commands.
 Because commands are sent to Redis asynchronously, issuing a command requires a callback function
 that is called when the reply is received. Reply callbacks should have the following prototype:
-```c
-void(redisAsyncContext *c, void *reply, void *privdata);
-```
+
+    void(redisAsyncContext *c, void *reply, void *privdata);
+
 The `privdata` argument can be used to curry arbitrary data to the callback from the point where
 the command is initially queued for execution.
 
 The functions that can be used to issue commands in an asynchronous context are:
-```c
-int redisAsyncCommand(
-  redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
-  const char *format, ...);
-int redisAsyncCommandArgv(
-  redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
-  int argc, const char **argv, const size_t *argvlen);
-```
+
+    int redisAsyncCommand(
+      redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
+      const char *format, ...);
+    int redisAsyncCommandArgv(
+      redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
+      int argc, const char **argv, const size_t *argvlen);
+
 Both functions work like their blocking counterparts. The return value is `REDIS_OK` when the command
 was successfully added to the output buffer and `REDIS_ERR` otherwise. Example: when the connection
 is being disconnected per user-request, no new commands may be added to the output buffer and `REDIS_ERR` is
 returned on calls to the `redisAsyncCommand` family.
 
-If the reply for a command with a `NULL` callback is read, it is immediately freed. When the callback
-for a command is non-`NULL`, the memory is freed immediately following the callback: the reply is only
+If the reply for a command with a `NULL` callback is read, it is immediately free'd. When the callback
+for a command is non-`NULL`, the memory is free'd immediately following the callback: the reply is only
 valid for the duration of the callback.
 
 All pending callbacks are called with a `NULL` reply when the context encountered an error.
@@ -321,14 +296,14 @@ All pending callbacks are called with a `NULL` reply when the context encountere
 ### Disconnecting
 
 An asynchronous connection can be terminated using:
-```c
-void redisAsyncDisconnect(redisAsyncContext *ac);
-```
+
+    void redisAsyncDisconnect(redisAsyncContext *ac);
+
 When this function is called, the connection is **not** immediately terminated. Instead, new
 commands are no longer accepted and the connection is only terminated when all pending commands
 have been written to the socket, their respective replies have been read and their respective
 callbacks have been executed. After this, the disconnection callback is executed with the
-`REDIS_OK` status and the context object is freed.
+`REDIS_OK` status and the context object is free'd.
 
 ### Hooking it up to event library *X*
 
@@ -341,12 +316,12 @@ Hiredis comes with a reply parsing API that makes it easy for writing higher
 level language bindings.
 
 The reply parsing API consists of the following functions:
-```c
-redisReader *redisReaderCreate(void);
-void redisReaderFree(redisReader *reader);
-int redisReaderFeed(redisReader *reader, const char *buf, size_t len);
-int redisReaderGetReply(redisReader *reader, void **reply);
-```
+
+    redisReader *redisReaderCreate(void);
+    void redisReaderFree(redisReader *reader);
+    int redisReaderFeed(redisReader *reader, const char *buf, size_t len);
+    int redisReaderGetReply(redisReader *reader, void **reply);
+
 The same set of functions are used internally by hiredis when creating a
 normal Redis context, the above API just exposes it to the user for a direct
 usage.
@@ -386,7 +361,7 @@ Both when using the Reader API directly or when using it indirectly via a
 normal Redis context, the redisReader structure uses a buffer in order to
 accumulate data from the server.
 Usually this buffer is destroyed when it is empty and is larger than 16
-KiB in order to avoid wasting memory in unused buffers
+kb in order to avoid wasting memory in unused buffers
 
 However when working with very big payloads destroying the buffer may slow
 down performances considerably, so it is possible to modify the max size of
@@ -396,9 +371,9 @@ value for an idle buffer, so the buffer will never get freed.
 
 For instance if you have a normal Redis context you can set the maximum idle
 buffer to zero (unlimited) just with:
-```c
-context->reader->maxbuf = 0;
-```
+
+    context->reader->maxbuf = 0;
+
 This should be done only in order to maximize performances when working with
 large payloads. The context should be set back to `REDIS_READER_MAX_BUF` again
 as soon as possible in order to prevent allocation of useless memory.
@@ -406,6 +381,4 @@ as soon as possible in order to prevent allocation of useless memory.
 ## AUTHORS
 
 Hiredis was written by Salvatore Sanfilippo (antirez at gmail) and
-Pieter Noordhuis (pcnoordhuis at gmail) and is released under the BSD license.  
-Hiredis is currently maintained by Matt Stancliff (matt at genges dot com) and
-Jan-Erik Rediger (janerik at fnordig dot com)
+Pieter Noordhuis (pcnoordhuis at gmail) and is released under the BSD license.

deps/hiredis/adapters/glib.h

@@ -1,153 +0,0 @@
-#ifndef __HIREDIS_GLIB_H__
-#define __HIREDIS_GLIB_H__
-
-#include <glib.h>
-
-#include "../hiredis.h"
-#include "../async.h"
-
-typedef struct
-{
-    GSource source;
-    redisAsyncContext *ac;
-    GPollFD poll_fd;
-} RedisSource;
-
-static void
-redis_source_add_read (gpointer data)
-{
-    RedisSource *source = (RedisSource *)data;
-    g_return_if_fail(source);
-    source->poll_fd.events |= G_IO_IN;
-    g_main_context_wakeup(g_source_get_context((GSource *)data));
-}
-
-static void
-redis_source_del_read (gpointer data)
-{
-    RedisSource *source = (RedisSource *)data;
-    g_return_if_fail(source);
-    source->poll_fd.events &= ~G_IO_IN;
-    g_main_context_wakeup(g_source_get_context((GSource *)data));
-}
-
-static void
-redis_source_add_write (gpointer data)
-{
-    RedisSource *source = (RedisSource *)data;
-    g_return_if_fail(source);
-    source->poll_fd.events |= G_IO_OUT;
-    g_main_context_wakeup(g_source_get_context((GSource *)data));
-}
-
-static void
-redis_source_del_write (gpointer data)
-{
-    RedisSource *source = (RedisSource *)data;
-    g_return_if_fail(source);
-    source->poll_fd.events &= ~G_IO_OUT;
-    g_main_context_wakeup(g_source_get_context((GSource *)data));
-}
-
-static void
-redis_source_cleanup (gpointer data)
-{
-    RedisSource *source = (RedisSource *)data;
-
-    g_return_if_fail(source);
-
-    redis_source_del_read(source);
-    redis_source_del_write(source);
-    /*
-     * It is not our responsibility to remove ourself from the
-     * current main loop. However, we will remove the GPollFD.
-     */
-    if (source->poll_fd.fd >= 0) {
-        g_source_remove_poll((GSource *)data, &source->poll_fd);
-        source->poll_fd.fd = -1;
-    }
-}
-
-static gboolean
-redis_source_prepare (GSource *source,
-                      gint    *timeout_)
-{
-    RedisSource *redis = (RedisSource *)source;
-    *timeout_ = -1;
-    return !!(redis->poll_fd.events & redis->poll_fd.revents);
-}
-
-static gboolean
-redis_source_check (GSource *source)
-{
-    RedisSource *redis = (RedisSource *)source;
-    return !!(redis->poll_fd.events & redis->poll_fd.revents);
-}
-
-static gboolean
-redis_source_dispatch (GSource      *source,
-                       GSourceFunc   callback,
-                       gpointer      user_data)
-{
-    RedisSource *redis = (RedisSource *)source;
-
-    if ((redis->poll_fd.revents & G_IO_OUT)) {
-        redisAsyncHandleWrite(redis->ac);
-        redis->poll_fd.revents &= ~G_IO_OUT;
-    }
-
-    if ((redis->poll_fd.revents & G_IO_IN)) {
-        redisAsyncHandleRead(redis->ac);
-        redis->poll_fd.revents &= ~G_IO_IN;
-    }
-
-    if (callback) {
-        return callback(user_data);
-    }
-
-    return TRUE;
-}
-
-static void
-redis_source_finalize (GSource *source)
-{
-    RedisSource *redis = (RedisSource *)source;
-
-    if (redis->poll_fd.fd >= 0) {
-        g_source_remove_poll(source, &redis->poll_fd);
-        redis->poll_fd.fd = -1;
-    }
-}
-
-static GSource *
-redis_source_new (redisAsyncContext *ac)
-{
-    static GSourceFuncs source_funcs = {
-        .prepare  = redis_source_prepare,
-        .check     = redis_source_check,
-        .dispatch = redis_source_dispatch,
-        .finalize = redis_source_finalize,
-    };
-    redisContext *c = &ac->c;
-    RedisSource *source;
-
-    g_return_val_if_fail(ac != NULL, NULL);
-
-    source = (RedisSource *)g_source_new(&source_funcs, sizeof *source);
-    source->ac = ac;
-    source->poll_fd.fd = c->fd;
-    source->poll_fd.events = 0;
-    source->poll_fd.revents = 0;
-    g_source_add_poll((GSource *)source, &source->poll_fd);
-
-    ac->ev.addRead = redis_source_add_read;
-    ac->ev.delRead = redis_source_del_read;
-    ac->ev.addWrite = redis_source_add_write;
-    ac->ev.delWrite = redis_source_del_write;
-    ac->ev.cleanup = redis_source_cleanup;
-    ac->ev.data = source;
-
-    return (GSource *)source;
-}
-
-#endif /* __HIREDIS_GLIB_H__ */

deps/hiredis/adapters/ivykis.h

@@ -1,81 +0,0 @@
-#ifndef __HIREDIS_IVYKIS_H__
-#define __HIREDIS_IVYKIS_H__
-#include <iv.h>
-#include "../hiredis.h"
-#include "../async.h"
-
-typedef struct redisIvykisEvents {
-    redisAsyncContext *context;
-    struct iv_fd fd;
-} redisIvykisEvents;
-
-static void redisIvykisReadEvent(void *arg) {
-    redisAsyncContext *context = (redisAsyncContext *)arg;
-    redisAsyncHandleRead(context);
-}
-
-static void redisIvykisWriteEvent(void *arg) {
-    redisAsyncContext *context = (redisAsyncContext *)arg;
-    redisAsyncHandleWrite(context);
-}
-
-static void redisIvykisAddRead(void *privdata) {
-    redisIvykisEvents *e = (redisIvykisEvents*)privdata;
-    iv_fd_set_handler_in(&e->fd, redisIvykisReadEvent);
-}
-
-static void redisIvykisDelRead(void *privdata) {
-    redisIvykisEvents *e = (redisIvykisEvents*)privdata;
-    iv_fd_set_handler_in(&e->fd, NULL);
-}
-
-static void redisIvykisAddWrite(void *privdata) {
-    redisIvykisEvents *e = (redisIvykisEvents*)privdata;
-    iv_fd_set_handler_out(&e->fd, redisIvykisWriteEvent);
-}
-
-static void redisIvykisDelWrite(void *privdata) {
-    redisIvykisEvents *e = (redisIvykisEvents*)privdata;
-    iv_fd_set_handler_out(&e->fd, NULL);
-}
-
-static void redisIvykisCleanup(void *privdata) {
-    redisIvykisEvents *e = (redisIvykisEvents*)privdata;
-
-    iv_fd_unregister(&e->fd);
-    free(e);
-}
-
-static int redisIvykisAttach(redisAsyncContext *ac) {
-    redisContext *c = &(ac->c);
-    redisIvykisEvents *e;
-
-    /* Nothing should be attached when something is already attached */
-    if (ac->ev.data != NULL)
-        return REDIS_ERR;
-
-    /* Create container for context and r/w events */
-    e = (redisIvykisEvents*)malloc(sizeof(*e));
-    e->context = ac;
-
-    /* Register functions to start/stop listening for events */
-    ac->ev.addRead = redisIvykisAddRead;
-    ac->ev.delRead = redisIvykisDelRead;
-    ac->ev.addWrite = redisIvykisAddWrite;
-    ac->ev.delWrite = redisIvykisDelWrite;
-    ac->ev.cleanup = redisIvykisCleanup;
-    ac->ev.data = e;
-
-    /* Initialize and install read/write events */
-    IV_FD_INIT(&e->fd);
-    e->fd.fd = c->fd;
-    e->fd.handler_in = redisIvykisReadEvent;
-    e->fd.handler_out = redisIvykisWriteEvent;
-    e->fd.handler_err = NULL;
-    e->fd.cookie = e->context;
-
-    iv_fd_register(&e->fd);
-
-    return REDIS_OK;
-}
-#endif

deps/hiredis/adapters/libevent.h

@@ -30,13 +30,13 @@
 
 #ifndef __HIREDIS_LIBEVENT_H__
 #define __HIREDIS_LIBEVENT_H__
-#include <event2/event.h>
+#include <event.h>
 #include "../hiredis.h"
 #include "../async.h"
 
 typedef struct redisLibeventEvents {
     redisAsyncContext *context;
-    struct event *rev, *wev;
+    struct event rev, wev;
 } redisLibeventEvents;
 
 static void redisLibeventReadEvent(int fd, short event, void *arg) {
@@ -53,28 +53,28 @@ static void redisLibeventWriteEvent(int fd, short event, void *arg) {
 
 static void redisLibeventAddRead(void *privdata) {
     redisLibeventEvents *e = (redisLibeventEvents*)privdata;
-    event_add(e->rev,NULL);
+    event_add(&e->rev,NULL);
 }
 
 static void redisLibeventDelRead(void *privdata) {
     redisLibeventEvents *e = (redisLibeventEvents*)privdata;
-    event_del(e->rev);
+    event_del(&e->rev);
 }
 
 static void redisLibeventAddWrite(void *privdata) {
     redisLibeventEvents *e = (redisLibeventEvents*)privdata;
-    event_add(e->wev,NULL);
+    event_add(&e->wev,NULL);
 }
 
 static void redisLibeventDelWrite(void *privdata) {
     redisLibeventEvents *e = (redisLibeventEvents*)privdata;
-    event_del(e->wev);
+    event_del(&e->wev);
 }
 
 static void redisLibeventCleanup(void *privdata) {
     redisLibeventEvents *e = (redisLibeventEvents*)privdata;
-    event_del(e->rev);
-    event_del(e->wev);
+    event_del(&e->rev);
+    event_del(&e->wev);
     free(e);
 }
 
@@ -99,10 +99,10 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
     ac->ev.data = e;
 
     /* Initialize and install read/write events */
-    e->rev = event_new(base, c->fd, EV_READ, redisLibeventReadEvent, e);
-    e->wev = event_new(base, c->fd, EV_WRITE, redisLibeventWriteEvent, e);
-    event_add(e->rev, NULL);
-    event_add(e->wev, NULL);
+    event_set(&e->rev,c->fd,EV_READ,redisLibeventReadEvent,e);
+    event_set(&e->wev,c->fd,EV_WRITE,redisLibeventWriteEvent,e);
+    event_base_set(base,&e->rev);
+    event_base_set(base,&e->wev);
     return REDIS_OK;
 }
 #endif

deps/hiredis/adapters/libuv.h

@@ -1,6 +1,5 @@
 #ifndef __HIREDIS_LIBUV_H__
 #define __HIREDIS_LIBUV_H__
-#include <stdlib.h>
 #include <uv.h>
 #include "../hiredis.h"
 #include "../async.h"
@@ -12,6 +11,7 @@ typedef struct redisLibuvEvents {
   int                events;
 } redisLibuvEvents;
 
+int redisLibuvAttach(redisAsyncContext*, uv_loop_t*);
 
 static void redisLibuvPoll(uv_poll_t* handle, int status, int events) {
   redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
@@ -20,10 +20,10 @@ static void redisLibuvPoll(uv_poll_t* handle, int status, int events) {
     return;
   }
 
-  if (p->context != NULL && (events & UV_READABLE)) {
+  if (events & UV_READABLE) {
     redisAsyncHandleRead(p->context);
   }
-  if (p->context != NULL && (events & UV_WRITABLE)) {
+  if (events & UV_WRITABLE) {
     redisAsyncHandleWrite(p->context);
   }
 }
@@ -83,7 +83,6 @@ static void on_close(uv_handle_t* handle) {
 static void redisLibuvCleanup(void *privdata) {
   redisLibuvEvents* p = (redisLibuvEvents*)privdata;
 
-  p->context = NULL; // indicate that context might no longer exist
   uv_close((uv_handle_t*)&p->handle, on_close);
 }
 

deps/hiredis/adapters/macosx.h

@@ -1,114 +0,0 @@
-//
-//  Created by Дмитрий Бахвалов on 13.07.15.
-//  Copyright (c) 2015 Dmitry Bakhvalov. All rights reserved.
-//
-
-#ifndef __HIREDIS_MACOSX_H__
-#define __HIREDIS_MACOSX_H__
-
-#include <CoreFoundation/CoreFoundation.h>
-
-#include "../hiredis.h"
-#include "../async.h"
-
-typedef struct {
-    redisAsyncContext *context;
-    CFSocketRef socketRef;
-    CFRunLoopSourceRef sourceRef;
-} RedisRunLoop;
-
-static int freeRedisRunLoop(RedisRunLoop* redisRunLoop) {
-    if( redisRunLoop != NULL ) {
-        if( redisRunLoop->sourceRef != NULL ) {
-            CFRunLoopSourceInvalidate(redisRunLoop->sourceRef);
-            CFRelease(redisRunLoop->sourceRef);
-        }
-        if( redisRunLoop->socketRef != NULL ) {
-            CFSocketInvalidate(redisRunLoop->socketRef);
-            CFRelease(redisRunLoop->socketRef);
-        }
-        free(redisRunLoop);
-    }
-    return REDIS_ERR;
-}
-
-static void redisMacOSAddRead(void *privdata) {
-    RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
-    CFSocketEnableCallBacks(redisRunLoop->socketRef, kCFSocketReadCallBack);
-}
-
-static void redisMacOSDelRead(void *privdata) {
-    RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
-    CFSocketDisableCallBacks(redisRunLoop->socketRef, kCFSocketReadCallBack);
-}
-
-static void redisMacOSAddWrite(void *privdata) {
-    RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
-    CFSocketEnableCallBacks(redisRunLoop->socketRef, kCFSocketWriteCallBack);
-}
-
-static void redisMacOSDelWrite(void *privdata) {
-    RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
-    CFSocketDisableCallBacks(redisRunLoop->socketRef, kCFSocketWriteCallBack);
-}
-
-static void redisMacOSCleanup(void *privdata) {
-    RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
-    freeRedisRunLoop(redisRunLoop);
-}
-
-static void redisMacOSAsyncCallback(CFSocketRef __unused s, CFSocketCallBackType callbackType, CFDataRef __unused address, const void __unused *data, void *info) {
-    redisAsyncContext* context = (redisAsyncContext*) info;
-
-    switch (callbackType) {
-        case kCFSocketReadCallBack:
-            redisAsyncHandleRead(context);
-            break;
-
-        case kCFSocketWriteCallBack:
-            redisAsyncHandleWrite(context);
-            break;
-
-        default:
-            break;
-    }
-}
-
-static int redisMacOSAttach(redisAsyncContext *redisAsyncCtx, CFRunLoopRef runLoop) {
-    redisContext *redisCtx = &(redisAsyncCtx->c);
-
-    /* Nothing should be attached when something is already attached */
-    if( redisAsyncCtx->ev.data != NULL ) return REDIS_ERR;
-
-    RedisRunLoop* redisRunLoop = (RedisRunLoop*) calloc(1, sizeof(RedisRunLoop));
-    if( !redisRunLoop ) return REDIS_ERR;
-
-    /* Setup redis stuff */
-    redisRunLoop->context = redisAsyncCtx;
-
-    redisAsyncCtx->ev.addRead  = redisMacOSAddRead;
-    redisAsyncCtx->ev.delRead  = redisMacOSDelRead;
-    redisAsyncCtx->ev.addWrite = redisMacOSAddWrite;
-    redisAsyncCtx->ev.delWrite = redisMacOSDelWrite;
-    redisAsyncCtx->ev.cleanup  = redisMacOSCleanup;
-    redisAsyncCtx->ev.data     = redisRunLoop;
-
-    /* Initialize and install read/write events */
-    CFSocketContext socketCtx = { 0, redisAsyncCtx, NULL, NULL, NULL };
-
-    redisRunLoop->socketRef = CFSocketCreateWithNative(NULL, redisCtx->fd,
-                                                       kCFSocketReadCallBack | kCFSocketWriteCallBack,
-                                                       redisMacOSAsyncCallback,
-                                                       &socketCtx);
-    if( !redisRunLoop->socketRef ) return freeRedisRunLoop(redisRunLoop);
-
-    redisRunLoop->sourceRef = CFSocketCreateRunLoopSource(NULL, redisRunLoop->socketRef, 0);
-    if( !redisRunLoop->sourceRef ) return freeRedisRunLoop(redisRunLoop);
-
-    CFRunLoopAddSource(runLoop, redisRunLoop->sourceRef, kCFRunLoopDefaultMode);
-
-    return REDIS_OK;
-}
-
-#endif
-

deps/hiredis/adapters/qt.h

@@ -1,135 +0,0 @@
-/*-
- * Copyright (C) 2014 Pietro Cerutti <gahr@gahr.ch>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#ifndef __HIREDIS_QT_H__
-#define __HIREDIS_QT_H__
-#include <QSocketNotifier>
-#include "../async.h"
-
-static void RedisQtAddRead(void *);
-static void RedisQtDelRead(void *);
-static void RedisQtAddWrite(void *);
-static void RedisQtDelWrite(void *);
-static void RedisQtCleanup(void *);
-
-class RedisQtAdapter : public QObject {
-
-    Q_OBJECT
-
-    friend
-    void RedisQtAddRead(void * adapter) {
-        RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
-        a->addRead();
-    }
-
-    friend
-    void RedisQtDelRead(void * adapter) {
-        RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
-        a->delRead();
-    }
-
-    friend
-    void RedisQtAddWrite(void * adapter) {
-        RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
-        a->addWrite();
-    }
-
-    friend
-    void RedisQtDelWrite(void * adapter) {
-        RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
-        a->delWrite();
-    }
-
-    friend
-    void RedisQtCleanup(void * adapter) {
-        RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
-        a->cleanup();
-    }
-
-    public:
-        RedisQtAdapter(QObject * parent = 0)
-            : QObject(parent), m_ctx(0), m_read(0), m_write(0) { }
-
-        ~RedisQtAdapter() {
-            if (m_ctx != 0) {
-                m_ctx->ev.data = NULL;
-            }
-        }
-
-        int setContext(redisAsyncContext * ac) {
-            if (ac->ev.data != NULL) {
-                return REDIS_ERR;
-            }
-            m_ctx = ac;
-            m_ctx->ev.data = this;
-            m_ctx->ev.addRead = RedisQtAddRead;
-            m_ctx->ev.delRead = RedisQtDelRead;
-            m_ctx->ev.addWrite = RedisQtAddWrite;
-            m_ctx->ev.delWrite = RedisQtDelWrite;
-            m_ctx->ev.cleanup = RedisQtCleanup;
-            return REDIS_OK;
-        }
-
-    private:
-        void addRead() {
-            if (m_read) return;
-            m_read = new QSocketNotifier(m_ctx->c.fd, QSocketNotifier::Read, 0);
-            connect(m_read, SIGNAL(activated(int)), this, SLOT(read()));
-        }
-
-        void delRead() {
-            if (!m_read) return;
-            delete m_read;
-            m_read = 0;
-        }
-
-        void addWrite() {
-            if (m_write) return;
-            m_write = new QSocketNotifier(m_ctx->c.fd, QSocketNotifier::Write, 0);
-            connect(m_write, SIGNAL(activated(int)), this, SLOT(write()));
-        }
-
-        void delWrite() {
-            if (!m_write) return;
-            delete m_write;
-            m_write = 0;
-        }
-
-        void cleanup() {
-            delRead();
-            delWrite();
-        }
-
-    private slots:
-        void read() { redisAsyncHandleRead(m_ctx); }
-        void write() { redisAsyncHandleWrite(m_ctx); }
-
-    private:
-        redisAsyncContext * m_ctx;
-        QSocketNotifier * m_read;
-        QSocketNotifier * m_write;
-};
-
-#endif /* !__HIREDIS_QT_H__ */

deps/hiredis/appveyor.yml

@@ -1,36 +0,0 @@
-# Appveyor configuration file for CI build of hiredis on Windows (under Cygwin)
-environment:
-    matrix:
-        - CYG_ROOT: C:\cygwin64
-          CYG_SETUP: setup-x86_64.exe
-          CYG_MIRROR: http://cygwin.mirror.constant.com
-          CYG_CACHE: C:\cygwin64\var\cache\setup
-          CYG_BASH: C:\cygwin64\bin\bash
-          CC: gcc
-        - CYG_ROOT: C:\cygwin
-          CYG_SETUP: setup-x86.exe
-          CYG_MIRROR: http://cygwin.mirror.constant.com
-          CYG_CACHE: C:\cygwin\var\cache\setup
-          CYG_BASH: C:\cygwin\bin\bash
-          CC: gcc
-          TARGET: 32bit
-          TARGET_VARS: 32bit-vars
-
-# Cache Cygwin files to speed up build
-cache:
-    - '%CYG_CACHE%'
-clone_depth: 1
-
-# Attempt to ensure we don't try to convert line endings to Win32 CRLF as this will cause build to fail
-init:
-    - git config --global core.autocrlf input
-
-# Install needed build dependencies
-install:
-    - ps: 'Start-FileDownload "http://cygwin.com/$env:CYG_SETUP" -FileName "$env:CYG_SETUP"'
-    - '%CYG_SETUP% --quiet-mode --no-shortcuts --only-site --root "%CYG_ROOT%" --site "%CYG_MIRROR%" --local-package-dir "%CYG_CACHE%" --packages automake,bison,gcc-core,libtool,make,gettext-devel,gettext,intltool,pkg-config,clang,llvm > NUL 2>&1'
-    - '%CYG_BASH% -lc "cygcheck -dc cygwin"'
-
-build_script:
-    - 'echo building...'
-    - '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; make LDFLAGS=$LDFLAGS CC=$CC $TARGET CFLAGS=$CFLAGS && make LDFLAGS=$LDFLAGS CC=$CC $TARGET_VARS hiredis-example"'

deps/hiredis/async.c

@@ -58,7 +58,7 @@
     } while(0);
 
 /* Forward declaration of function in hiredis.c */
-int __redisAppendCommand(redisContext *c, const char *cmd, size_t len);
+void __redisAppendCommand(redisContext *c, char *cmd, size_t len);
 
 /* Functions managing dictionary of callbacks for pub/sub. */
 static unsigned int callbackHash(const void *key) {
@@ -142,9 +142,6 @@ static redisAsyncContext *redisAsyncInitialize(redisContext *c) {
 /* We want the error field to be accessible directly instead of requiring
  * an indirection to the redisContext struct. */
 static void __redisAsyncCopyError(redisAsyncContext *ac) {
-    if (!ac)
-        return;
-
     redisContext *c = &(ac->c);
     ac->err = c->err;
     ac->errstr = c->errstr;
@@ -176,14 +173,6 @@ redisAsyncContext *redisAsyncConnectBind(const char *ip, int port,
     return ac;
 }
 
-redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
-                                                  const char *source_addr) {
-    redisContext *c = redisConnectBindNonBlockWithReuse(ip,port,source_addr);
-    redisAsyncContext *ac = redisAsyncInitialize(c);
-    __redisAsyncCopyError(ac);
-    return ac;
-}
-
 redisAsyncContext *redisAsyncConnectUnix(const char *path) {
     redisContext *c;
     redisAsyncContext *ac;
@@ -418,8 +407,7 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
         if (reply == NULL) {
             /* When the connection is being disconnected and there are
              * no more replies, this is the cue to really disconnect. */
-            if (c->flags & REDIS_DISCONNECTING && sdslen(c->obuf) == 0
-                && ac->replies.head == NULL) {
+            if (c->flags & REDIS_DISCONNECTING && sdslen(c->obuf) == 0) {
                 __redisAsyncDisconnect(ac);
                 return;
             }
@@ -489,7 +477,7 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
 }
 
 /* Internal helper function to detect socket status the first time a read or
- * write event fires. When connecting was not successful, the connect callback
+ * write event fires. When connecting was not succesful, the connect callback
  * is called with a REDIS_ERR status and the context is free'd. */
 static int __redisAsyncHandleConnect(redisAsyncContext *ac) {
     redisContext *c = &(ac->c);
@@ -563,8 +551,8 @@ void redisAsyncHandleWrite(redisAsyncContext *ac) {
 
 /* Sets a pointer to the first argument and its length starting at p. Returns
  * the number of bytes to skip to get to the following argument. */
-static const char *nextArgument(const char *start, const char **str, size_t *len) {
-    const char *p = start;
+static char *nextArgument(char *start, char **str, size_t *len) {
+    char *p = start;
     if (p[0] != '$') {
         p = strchr(p,'$');
         if (p == NULL) return NULL;
@@ -580,15 +568,14 @@ static const char *nextArgument(const char *start, const char **str, size_t *len
 /* Helper function for the redisAsyncCommand* family of functions. Writes a
  * formatted command to the output buffer and registers the provided callback
  * function with the context. */
-static int __redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *cmd, size_t len) {
+static int __redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, char *cmd, size_t len) {
     redisContext *c = &(ac->c);
     redisCallback cb;
     int pvariant, hasnext;
-    const char *cstr, *astr;
+    char *cstr, *astr;
     size_t clen, alen;
-    const char *p;
+    char *p;
     sds sname;
-    int ret;
 
     /* Don't accept new commands when the connection is about to be closed. */
     if (c->flags & (REDIS_DISCONNECTING | REDIS_FREEING)) return REDIS_ERR;
@@ -612,11 +599,9 @@ static int __redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void
         while ((p = nextArgument(p,&astr,&alen)) != NULL) {
             sname = sdsnewlen(astr,alen);
             if (pvariant)
-                ret = dictReplace(ac->sub.patterns,sname,&cb);
+                dictReplace(ac->sub.patterns,sname,&cb);
             else
-                ret = dictReplace(ac->sub.channels,sname,&cb);
-
-            if (ret == 0) sdsfree(sname);
+                dictReplace(ac->sub.channels,sname,&cb);
         }
     } else if (strncasecmp(cstr,"unsubscribe\r\n",13) == 0) {
         /* It is only useful to call (P)UNSUBSCRIBE when the context is
@@ -652,11 +637,6 @@ int redisvAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdat
     int len;
     int status;
     len = redisvFormatCommand(&cmd,format,ap);
-
-    /* We don't want to pass -1 or -2 to future functions as a length. */
-    if (len < 0)
-        return REDIS_ERR;
-
     status = __redisAsyncCommand(ac,fn,privdata,cmd,len);
     free(cmd);
     return status;
@@ -672,16 +652,11 @@ int redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata
 }
 
 int redisAsyncCommandArgv(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen) {
-    sds cmd;
+    char *cmd;
     int len;
     int status;
-    len = redisFormatSdsCommandArgv(&cmd,argc,argv,argvlen);
+    len = redisFormatCommandArgv(&cmd,argc,argv,argvlen);
     status = __redisAsyncCommand(ac,fn,privdata,cmd,len);
-    sdsfree(cmd);
-    return status;
-}
-
-int redisAsyncFormattedCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *cmd, size_t len) {
-    int status = __redisAsyncCommand(ac,fn,privdata,cmd,len);
+    free(cmd);
     return status;
 }

deps/hiredis/async.h

@@ -103,8 +103,6 @@ typedef struct redisAsyncContext {
 /* Functions that proxy to hiredis */
 redisAsyncContext *redisAsyncConnect(const char *ip, int port);
 redisAsyncContext *redisAsyncConnectBind(const char *ip, int port, const char *source_addr);
-redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
-                                                  const char *source_addr);
 redisAsyncContext *redisAsyncConnectUnix(const char *path);
 int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
 int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
@@ -120,7 +118,6 @@ void redisAsyncHandleWrite(redisAsyncContext *ac);
 int redisvAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *format, va_list ap);
 int redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *format, ...);
 int redisAsyncCommandArgv(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);
-int redisAsyncFormattedCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *cmd, size_t len);
 
 #ifdef __cplusplus
 }

deps/hiredis/dict.c

@@ -161,7 +161,7 @@ static int dictReplace(dict *ht, void *key, void *val) {
     dictEntry *entry, auxentry;
 
     /* Try to add the element. If the key
-     * does not exists dictAdd will succeed. */
+     * does not exists dictAdd will suceed. */
     if (dictAdd(ht, key, val) == DICT_OK)
         return 1;
     /* It already exists, get the entry */
@@ -293,7 +293,7 @@ static void dictReleaseIterator(dictIterator *iter) {
 
 /* Expand the hash table if needed */
 static int _dictExpandIfNeeded(dict *ht) {
-    /* If the hash table is empty expand it to the initial size,
+    /* If the hash table is empty expand it to the intial size,
      * if the table is "full" dobule its size. */
     if (ht->size == 0)
         return dictExpand(ht, DICT_HT_INITIAL_SIZE);

deps/hiredis/examples/example-glib.c

@@ -1,73 +0,0 @@
-#include <stdlib.h>
-
-#include <hiredis.h>
-#include <async.h>
-#include <adapters/glib.h>
-
-static GMainLoop *mainloop;
-
-static void
-connect_cb (const redisAsyncContext *ac G_GNUC_UNUSED,
-            int status)
-{
-    if (status != REDIS_OK) {
-        g_printerr("Failed to connect: %s\n", ac->errstr);
-        g_main_loop_quit(mainloop);
-    } else {
-        g_printerr("Connected...\n");
-    }
-}
-
-static void
-disconnect_cb (const redisAsyncContext *ac G_GNUC_UNUSED,
-               int status)
-{
-    if (status != REDIS_OK) {
-        g_error("Failed to disconnect: %s", ac->errstr);
-    } else {
-        g_printerr("Disconnected...\n");
-        g_main_loop_quit(mainloop);
-    }
-}
-
-static void
-command_cb(redisAsyncContext *ac,
-           gpointer r,
-           gpointer user_data G_GNUC_UNUSED)
-{
-    redisReply *reply = r;
-
-    if (reply) {
-        g_print("REPLY: %s\n", reply->str);
-    }
-
-    redisAsyncDisconnect(ac);
-}
-
-gint
-main (gint argc     G_GNUC_UNUSED,
-      gchar *argv[] G_GNUC_UNUSED)
-{
-    redisAsyncContext *ac;
-    GMainContext *context = NULL;
-    GSource *source;
-
-    ac = redisAsyncConnect("127.0.0.1", 6379);
-    if (ac->err) {
-        g_printerr("%s\n", ac->errstr);
-        exit(EXIT_FAILURE);
-    }
-
-    source = redis_source_new(ac);
-    mainloop = g_main_loop_new(context, FALSE);
-    g_source_attach(source, context);
-
-    redisAsyncSetConnectCallback(ac, connect_cb);
-    redisAsyncSetDisconnectCallback(ac, disconnect_cb);
-    redisAsyncCommand(ac, command_cb, NULL, "SET key 1234");
-    redisAsyncCommand(ac, command_cb, NULL, "GET key");
-
-    g_main_loop_run(mainloop);
-
-    return EXIT_SUCCESS;
-}

deps/hiredis/examples/example-ivykis.c

@@ -1,58 +0,0 @@
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <signal.h>
-
-#include <hiredis.h>
-#include <async.h>
-#include <adapters/ivykis.h>
-
-void getCallback(redisAsyncContext *c, void *r, void *privdata) {
-    redisReply *reply = r;
-    if (reply == NULL) return;
-    printf("argv[%s]: %s\n", (char*)privdata, reply->str);
-
-    /* Disconnect after receiving the reply to GET */
-    redisAsyncDisconnect(c);
-}
-
-void connectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Connected...\n");
-}
-
-void disconnectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Disconnected...\n");
-}
-
-int main (int argc, char **argv) {
-    signal(SIGPIPE, SIG_IGN);
-
-    iv_init();
-
-    redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
-    if (c->err) {
-        /* Let *c leak for now... */
-        printf("Error: %s\n", c->errstr);
-        return 1;
-    }
-
-    redisIvykisAttach(c);
-    redisAsyncSetConnectCallback(c,connectCallback);
-    redisAsyncSetDisconnectCallback(c,disconnectCallback);
-    redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
-    redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
-
-    iv_main();
-
-    iv_deinit();
-
-    return 0;
-}

deps/hiredis/examples/example-macosx.c

@@ -1,66 +0,0 @@
-//
-//  Created by Дмитрий Бахвалов on 13.07.15.
-//  Copyright (c) 2015 Dmitry Bakhvalov. All rights reserved.
-//
-
-#include <stdio.h>
-
-#include <hiredis.h>
-#include <async.h>
-#include <adapters/macosx.h>
-
-void getCallback(redisAsyncContext *c, void *r, void *privdata) {
-    redisReply *reply = r;
-    if (reply == NULL) return;
-    printf("argv[%s]: %s\n", (char*)privdata, reply->str);
-
-    /* Disconnect after receiving the reply to GET */
-    redisAsyncDisconnect(c);
-}
-
-void connectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Connected...\n");
-}
-
-void disconnectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    CFRunLoopStop(CFRunLoopGetCurrent());
-    printf("Disconnected...\n");
-}
-
-int main (int argc, char **argv) {
-    signal(SIGPIPE, SIG_IGN);
-
-    CFRunLoopRef loop = CFRunLoopGetCurrent();
-    if( !loop ) {
-        printf("Error: Cannot get current run loop\n");
-        return 1;
-    }
-
-    redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
-    if (c->err) {
-        /* Let *c leak for now... */
-        printf("Error: %s\n", c->errstr);
-        return 1;
-    }
-
-    redisMacOSAttach(c, loop);
-
-    redisAsyncSetConnectCallback(c,connectCallback);
-    redisAsyncSetDisconnectCallback(c,disconnectCallback);
-
-    redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
-    redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
-
-    CFRunLoopRun();
-
-    return 0;
-}
-

deps/hiredis/examples/example-qt.cpp

@@ -1,46 +0,0 @@
-#include <iostream>
-using namespace std;
-
-#include <QCoreApplication>
-#include <QTimer>
-
-#include "example-qt.h"
-
-void getCallback(redisAsyncContext *, void * r, void * privdata) {
-
-    redisReply * reply = static_cast<redisReply *>(r);
-    ExampleQt * ex = static_cast<ExampleQt *>(privdata);
-    if (reply == nullptr || ex == nullptr) return;
-
-    cout << "key: " << reply->str << endl;
-
-    ex->finish();
-}
-
-void ExampleQt::run() {
-
-    m_ctx = redisAsyncConnect("localhost", 6379);
-
-    if (m_ctx->err) {
-        cerr << "Error: " << m_ctx->errstr << endl;
-        redisAsyncFree(m_ctx);
-        emit finished();
-    }
-
-    m_adapter.setContext(m_ctx);
-
-    redisAsyncCommand(m_ctx, NULL, NULL, "SET key %s", m_value);
-    redisAsyncCommand(m_ctx, getCallback, this, "GET key");
-}
-
-int main (int argc, char **argv) {
-
-    QCoreApplication app(argc, argv);
-
-    ExampleQt example(argv[argc-1]);
-
-    QObject::connect(&example, SIGNAL(finished()), &app, SLOT(quit()));
-    QTimer::singleShot(0, &example, SLOT(run()));
-
-    return app.exec();
-}

deps/hiredis/examples/example-qt.h

@@ -1,32 +0,0 @@
-#ifndef __HIREDIS_EXAMPLE_QT_H
-#define __HIREDIS_EXAMPLE_QT_H
-
-#include <adapters/qt.h>
-
-class ExampleQt : public QObject {
-
-    Q_OBJECT
-
-    public:
-        ExampleQt(const char * value, QObject * parent = 0)
-            : QObject(parent), m_value(value) {}
-
-    signals:
-        void finished();
-
-    public slots:
-        void run();
-
-    private:
-        void finish() { emit finished(); }
-
-    private:
-        const char * m_value;
-        redisAsyncContext * m_ctx;
-        RedisQtAdapter m_adapter;
-
-    friend
-    void getCallback(redisAsyncContext *, void *, void *);
-};
-
-#endif /* !__HIREDIS_EXAMPLE_QT_H */

deps/hiredis/examples/example.c

@@ -57,7 +57,7 @@ int main(int argc, char **argv) {
     for (j = 0; j < 10; j++) {
         char buf[64];
 
-        snprintf(buf,64,"%u",j);
+        snprintf(buf,64,"%d",j);
         reply = redisCommand(c,"LPUSH mylist element-%s", buf);
         freeReplyObject(reply);
     }

deps/hiredis/fmacros.h

@@ -1,24 +1,23 @@
 #ifndef __HIREDIS_FMACRO_H
 #define __HIREDIS_FMACRO_H
 
-#if defined(__linux__)
+#if !defined(_BSD_SOURCE)
 #define _BSD_SOURCE
-#define _DEFAULT_SOURCE
 #endif
 
-#if defined(__CYGWIN__)
-#include <sys/cdefs.h>
+#if defined(_AIX)
+#define _ALL_SOURCE
 #endif
 
 #if defined(__sun__)
 #define _POSIX_C_SOURCE 200112L
-#else
-#if !(defined(__APPLE__) && defined(__MACH__))
+#elif defined(__linux__) || defined(__OpenBSD__) || defined(__NetBSD__)
 #define _XOPEN_SOURCE 600
-#endif
+#else
+#define _XOPEN_SOURCE
 #endif
 
-#if defined(__APPLE__) && defined(__MACH__)
+#if __APPLE__ && __MACH__
 #define _OSX
 #endif
 

deps/hiredis/hiredis.c

@@ -1,8 +1,6 @@
 /*
  * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2014, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- * Copyright (c) 2015, Matt Stancliff <matt at genges dot com>,
- *                     Jan-Erik Rediger <janerik at fnordig dot com>
+ * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
  *
  * All rights reserved.
  *
@@ -75,9 +73,6 @@ void freeReplyObject(void *reply) {
     redisReply *r = reply;
     size_t j;
 
-    if (r == NULL)
-        return;
-
     switch(r->type) {
     case REDIS_REPLY_INTEGER:
         break; /* Nothing to free */
@@ -188,23 +183,504 @@ static void *createNilObject(const redisReadTask *task) {
     return r;
 }
 
-/* Return the number of digits of 'v' when converted to string in radix 10.
- * Implementation borrowed from link in redis/src/util.c:string2ll(). */
-static uint32_t countDigits(uint64_t v) {
-  uint32_t result = 1;
-  for (;;) {
-    if (v < 10) return result;
-    if (v < 100) return result + 1;
-    if (v < 1000) return result + 2;
-    if (v < 10000) return result + 3;
-    v /= 10000U;
-    result += 4;
-  }
+static void __redisReaderSetError(redisReader *r, int type, const char *str) {
+    size_t len;
+
+    if (r->reply != NULL && r->fn && r->fn->freeObject) {
+        r->fn->freeObject(r->reply);
+        r->reply = NULL;
+    }
+
+    /* Clear input buffer on errors. */
+    if (r->buf != NULL) {
+        sdsfree(r->buf);
+        r->buf = NULL;
+        r->pos = r->len = 0;
+    }
+
+    /* Reset task stack. */
+    r->ridx = -1;
+
+    /* Set error. */
+    r->err = type;
+    len = strlen(str);
+    len = len < (sizeof(r->errstr)-1) ? len : (sizeof(r->errstr)-1);
+    memcpy(r->errstr,str,len);
+    r->errstr[len] = '\0';
+}
+
+static size_t chrtos(char *buf, size_t size, char byte) {
+    size_t len = 0;
+
+    switch(byte) {
+    case '\\':
+    case '"':
+        len = snprintf(buf,size,"\"\\%c\"",byte);
+        break;
+    case '\n': len = snprintf(buf,size,"\"\\n\""); break;
+    case '\r': len = snprintf(buf,size,"\"\\r\""); break;
+    case '\t': len = snprintf(buf,size,"\"\\t\""); break;
+    case '\a': len = snprintf(buf,size,"\"\\a\""); break;
+    case '\b': len = snprintf(buf,size,"\"\\b\""); break;
+    default:
+        if (isprint(byte))
+            len = snprintf(buf,size,"\"%c\"",byte);
+        else
+            len = snprintf(buf,size,"\"\\x%02x\"",(unsigned char)byte);
+        break;
+    }
+
+    return len;
+}
+
+static void __redisReaderSetErrorProtocolByte(redisReader *r, char byte) {
+    char cbuf[8], sbuf[128];
+
+    chrtos(cbuf,sizeof(cbuf),byte);
+    snprintf(sbuf,sizeof(sbuf),
+        "Protocol error, got %s as reply type byte", cbuf);
+    __redisReaderSetError(r,REDIS_ERR_PROTOCOL,sbuf);
+}
+
+static void __redisReaderSetErrorOOM(redisReader *r) {
+    __redisReaderSetError(r,REDIS_ERR_OOM,"Out of memory");
+}
+
+static char *readBytes(redisReader *r, unsigned int bytes) {
+    char *p;
+    if (r->len-r->pos >= bytes) {
+        p = r->buf+r->pos;
+        r->pos += bytes;
+        return p;
+    }
+    return NULL;
+}
+
+/* Find pointer to \r\n. */
+static char *seekNewline(char *s, size_t len) {
+    int pos = 0;
+    int _len = len-1;
+
+    /* Position should be < len-1 because the character at "pos" should be
+     * followed by a \n. Note that strchr cannot be used because it doesn't
+     * allow to search a limited length and the buffer that is being searched
+     * might not have a trailing NULL character. */
+    while (pos < _len) {
+        while(pos < _len && s[pos] != '\r') pos++;
+        if (s[pos] != '\r') {
+            /* Not found. */
+            return NULL;
+        } else {
+            if (s[pos+1] == '\n') {
+                /* Found. */
+                return s+pos;
+            } else {
+                /* Continue searching. */
+                pos++;
+            }
+        }
+    }
+    return NULL;
+}
+
+/* Read a long long value starting at *s, under the assumption that it will be
+ * terminated by \r\n. Ambiguously returns -1 for unexpected input. */
+static long long readLongLong(char *s) {
+    long long v = 0;
+    int dec, mult = 1;
+    char c;
+
+    if (*s == '-') {
+        mult = -1;
+        s++;
+    } else if (*s == '+') {
+        mult = 1;
+        s++;
+    }
+
+    while ((c = *(s++)) != '\r') {
+        dec = c - '0';
+        if (dec >= 0 && dec < 10) {
+            v *= 10;
+            v += dec;
+        } else {
+            /* Should not happen... */
+            return -1;
+        }
+    }
+
+    return mult*v;
+}
+
+static char *readLine(redisReader *r, int *_len) {
+    char *p, *s;
+    int len;
+
+    p = r->buf+r->pos;
+    s = seekNewline(p,(r->len-r->pos));
+    if (s != NULL) {
+        len = s-(r->buf+r->pos);
+        r->pos += len+2; /* skip \r\n */
+        if (_len) *_len = len;
+        return p;
+    }
+    return NULL;
+}
+
+static void moveToNextTask(redisReader *r) {
+    redisReadTask *cur, *prv;
+    while (r->ridx >= 0) {
+        /* Return a.s.a.p. when the stack is now empty. */
+        if (r->ridx == 0) {
+            r->ridx--;
+            return;
+        }
+
+        cur = &(r->rstack[r->ridx]);
+        prv = &(r->rstack[r->ridx-1]);
+        assert(prv->type == REDIS_REPLY_ARRAY);
+        if (cur->idx == prv->elements-1) {
+            r->ridx--;
+        } else {
+            /* Reset the type because the next item can be anything */
+            assert(cur->idx < prv->elements);
+            cur->type = -1;
+            cur->elements = -1;
+            cur->idx++;
+            return;
+        }
+    }
+}
+
+static int processLineItem(redisReader *r) {
+    redisReadTask *cur = &(r->rstack[r->ridx]);
+    void *obj;
+    char *p;
+    int len;
+
+    if ((p = readLine(r,&len)) != NULL) {
+        if (cur->type == REDIS_REPLY_INTEGER) {
+            if (r->fn && r->fn->createInteger)
+                obj = r->fn->createInteger(cur,readLongLong(p));
+            else
+                obj = (void*)REDIS_REPLY_INTEGER;
+        } else {
+            /* Type will be error or status. */
+            if (r->fn && r->fn->createString)
+                obj = r->fn->createString(cur,p,len);
+            else
+                obj = (void*)(size_t)(cur->type);
+        }
+
+        if (obj == NULL) {
+            __redisReaderSetErrorOOM(r);
+            return REDIS_ERR;
+        }
+
+        /* Set reply if this is the root object. */
+        if (r->ridx == 0) r->reply = obj;
+        moveToNextTask(r);
+        return REDIS_OK;
+    }
+
+    return REDIS_ERR;
+}
+
+static int processBulkItem(redisReader *r) {
+    redisReadTask *cur = &(r->rstack[r->ridx]);
+    void *obj = NULL;
+    char *p, *s;
+    long len;
+    unsigned long bytelen;
+    int success = 0;
+
+    p = r->buf+r->pos;
+    s = seekNewline(p,r->len-r->pos);
+    if (s != NULL) {
+        p = r->buf+r->pos;
+        bytelen = s-(r->buf+r->pos)+2; /* include \r\n */
+        len = readLongLong(p);
+
+        if (len < 0) {
+            /* The nil object can always be created. */
+            if (r->fn && r->fn->createNil)
+                obj = r->fn->createNil(cur);
+            else
+                obj = (void*)REDIS_REPLY_NIL;
+            success = 1;
+        } else {
+            /* Only continue when the buffer contains the entire bulk item. */
+            bytelen += len+2; /* include \r\n */
+            if (r->pos+bytelen <= r->len) {
+                if (r->fn && r->fn->createString)
+                    obj = r->fn->createString(cur,s+2,len);
+                else
+                    obj = (void*)REDIS_REPLY_STRING;
+                success = 1;
+            }
+        }
+
+        /* Proceed when obj was created. */
+        if (success) {
+            if (obj == NULL) {
+                __redisReaderSetErrorOOM(r);
+                return REDIS_ERR;
+            }
+
+            r->pos += bytelen;
+
+            /* Set reply if this is the root object. */
+            if (r->ridx == 0) r->reply = obj;
+            moveToNextTask(r);
+            return REDIS_OK;
+        }
+    }
+
+    return REDIS_ERR;
+}
+
+static int processMultiBulkItem(redisReader *r) {
+    redisReadTask *cur = &(r->rstack[r->ridx]);
+    void *obj;
+    char *p;
+    long elements;
+    int root = 0;
+
+    /* Set error for nested multi bulks with depth > 7 */
+    if (r->ridx == 8) {
+        __redisReaderSetError(r,REDIS_ERR_PROTOCOL,
+            "No support for nested multi bulk replies with depth > 7");
+        return REDIS_ERR;
+    }
+
+    if ((p = readLine(r,NULL)) != NULL) {
+        elements = readLongLong(p);
+        root = (r->ridx == 0);
+
+        if (elements == -1) {
+            if (r->fn && r->fn->createNil)
+                obj = r->fn->createNil(cur);
+            else
+                obj = (void*)REDIS_REPLY_NIL;
+
+            if (obj == NULL) {
+                __redisReaderSetErrorOOM(r);
+                return REDIS_ERR;
+            }
+
+            moveToNextTask(r);
+        } else {
+            if (r->fn && r->fn->createArray)
+                obj = r->fn->createArray(cur,elements);
+            else
+                obj = (void*)REDIS_REPLY_ARRAY;
+
+            if (obj == NULL) {
+                __redisReaderSetErrorOOM(r);
+                return REDIS_ERR;
+            }
+
+            /* Modify task stack when there are more than 0 elements. */
+            if (elements > 0) {
+                cur->elements = elements;
+                cur->obj = obj;
+                r->ridx++;
+                r->rstack[r->ridx].type = -1;
+                r->rstack[r->ridx].elements = -1;
+                r->rstack[r->ridx].idx = 0;
+                r->rstack[r->ridx].obj = NULL;
+                r->rstack[r->ridx].parent = cur;
+                r->rstack[r->ridx].privdata = r->privdata;
+            } else {
+                moveToNextTask(r);
+            }
+        }
+
+        /* Set reply if this is the root object. */
+        if (root) r->reply = obj;
+        return REDIS_OK;
+    }
+
+    return REDIS_ERR;
+}
+
+static int processItem(redisReader *r) {
+    redisReadTask *cur = &(r->rstack[r->ridx]);
+    char *p;
+
+    /* check if we need to read type */
+    if (cur->type < 0) {
+        if ((p = readBytes(r,1)) != NULL) {
+            switch (p[0]) {
+            case '-':
+                cur->type = REDIS_REPLY_ERROR;
+                break;
+            case '+':
+                cur->type = REDIS_REPLY_STATUS;
+                break;
+            case ':':
+                cur->type = REDIS_REPLY_INTEGER;
+                break;
+            case '$':
+                cur->type = REDIS_REPLY_STRING;
+                break;
+            case '*':
+                cur->type = REDIS_REPLY_ARRAY;
+                break;
+            default:
+                __redisReaderSetErrorProtocolByte(r,*p);
+                return REDIS_ERR;
+            }
+        } else {
+            /* could not consume 1 byte */
+            return REDIS_ERR;
+        }
+    }
+
+    /* process typed item */
+    switch(cur->type) {
+    case REDIS_REPLY_ERROR:
+    case REDIS_REPLY_STATUS:
+    case REDIS_REPLY_INTEGER:
+        return processLineItem(r);
+    case REDIS_REPLY_STRING:
+        return processBulkItem(r);
+    case REDIS_REPLY_ARRAY:
+        return processMultiBulkItem(r);
+    default:
+        assert(NULL);
+        return REDIS_ERR; /* Avoid warning. */
+    }
+}
+
+redisReader *redisReaderCreate(void) {
+    redisReader *r;
+
+    r = calloc(sizeof(redisReader),1);
+    if (r == NULL)
+        return NULL;
+
+    r->err = 0;
+    r->errstr[0] = '\0';
+    r->fn = &defaultFunctions;
+    r->buf = sdsempty();
+    r->maxbuf = REDIS_READER_MAX_BUF;
+    if (r->buf == NULL) {
+        free(r);
+        return NULL;
+    }
+
+    r->ridx = -1;
+    return r;
+}
+
+void redisReaderFree(redisReader *r) {
+    if (r->reply != NULL && r->fn && r->fn->freeObject)
+        r->fn->freeObject(r->reply);
+    if (r->buf != NULL)
+        sdsfree(r->buf);
+    free(r);
+}
+
+int redisReaderFeed(redisReader *r, const char *buf, size_t len) {
+    sds newbuf;
+
+    /* Return early when this reader is in an erroneous state. */
+    if (r->err)
+        return REDIS_ERR;
+
+    /* Copy the provided buffer. */
+    if (buf != NULL && len >= 1) {
+        /* Destroy internal buffer when it is empty and is quite large. */
+        if (r->len == 0 && r->maxbuf != 0 && sdsavail(r->buf) > r->maxbuf) {
+            sdsfree(r->buf);
+            r->buf = sdsempty();
+            r->pos = 0;
+
+            /* r->buf should not be NULL since we just free'd a larger one. */
+            assert(r->buf != NULL);
+        }
+
+        newbuf = sdscatlen(r->buf,buf,len);
+        if (newbuf == NULL) {
+            __redisReaderSetErrorOOM(r);
+            return REDIS_ERR;
+        }
+
+        r->buf = newbuf;
+        r->len = sdslen(r->buf);
+    }
+
+    return REDIS_OK;
+}
+
+int redisReaderGetReply(redisReader *r, void **reply) {
+    /* Default target pointer to NULL. */
+    if (reply != NULL)
+        *reply = NULL;
+
+    /* Return early when this reader is in an erroneous state. */
+    if (r->err)
+        return REDIS_ERR;
+
+    /* When the buffer is empty, there will never be a reply. */
+    if (r->len == 0)
+        return REDIS_OK;
+
+    /* Set first item to process when the stack is empty. */
+    if (r->ridx == -1) {
+        r->rstack[0].type = -1;
+        r->rstack[0].elements = -1;
+        r->rstack[0].idx = -1;
+        r->rstack[0].obj = NULL;
+        r->rstack[0].parent = NULL;
+        r->rstack[0].privdata = r->privdata;
+        r->ridx = 0;
+    }
+
+    /* Process items in reply. */
+    while (r->ridx >= 0)
+        if (processItem(r) != REDIS_OK)
+            break;
+
+    /* Return ASAP when an error occurred. */
+    if (r->err)
+        return REDIS_ERR;
+
+    /* Discard part of the buffer when we've consumed at least 1k, to avoid
+     * doing unnecessary calls to memmove() in sds.c. */
+    if (r->pos >= 1024) {
+        sdsrange(r->buf,r->pos,-1);
+        r->pos = 0;
+        r->len = sdslen(r->buf);
+    }
+
+    /* Emit a reply when there is one. */
+    if (r->ridx == -1) {
+        if (reply != NULL)
+            *reply = r->reply;
+        r->reply = NULL;
+    }
+    return REDIS_OK;
+}
+
+/* Calculate the number of bytes needed to represent an integer as string. */
+static int intlen(int i) {
+    int len = 0;
+    if (i < 0) {
+        len++;
+        i = -i;
+    }
+    do {
+        len++;
+        i /= 10;
+    } while(i);
+    return len;
 }
 
 /* Helper that calculates the bulk length given a certain string length. */
 static size_t bulklen(size_t len) {
-    return 1+countDigits(len)+2+len+2;
+    return 1+intlen(len)+2+len+2;
 }
 
 int redisvFormatCommand(char **target, const char *format, va_list ap) {
@@ -216,7 +692,6 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
     char **curargv = NULL, **newargv = NULL;
     int argc = 0;
     int totlen = 0;
-    int error_type = 0; /* 0 = no error; -1 = memory error; -2 = format error */
     int j;
 
     /* Abort if there is not target to set */
@@ -233,19 +708,19 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
             if (*c == ' ') {
                 if (touched) {
                     newargv = realloc(curargv,sizeof(char*)*(argc+1));
-                    if (newargv == NULL) goto memory_err;
+                    if (newargv == NULL) goto err;
                     curargv = newargv;
                     curargv[argc++] = curarg;
                     totlen += bulklen(sdslen(curarg));
 
                     /* curarg is put in argv so it can be overwritten. */
                     curarg = sdsempty();
-                    if (curarg == NULL) goto memory_err;
+                    if (curarg == NULL) goto err;
                     touched = 0;
                 }
             } else {
                 newarg = sdscatlen(curarg,c,1);
-                if (newarg == NULL) goto memory_err;
+                if (newarg == NULL) goto err;
                 curarg = newarg;
                 touched = 1;
             }
@@ -276,14 +751,17 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
                 /* Try to detect printf format */
                 {
                     static const char intfmts[] = "diouxX";
-                    static const char flags[] = "#0-+ ";
                     char _format[16];
                     const char *_p = c+1;
                     size_t _l = 0;
                     va_list _cpy;
 
                     /* Flags */
-                    while (*_p != '\0' && strchr(flags,*_p) != NULL) _p++;
+                    if (*_p != '\0' && *_p == '#') _p++;
+                    if (*_p != '\0' && *_p == '0') _p++;
+                    if (*_p != '\0' && *_p == '-') _p++;
+                    if (*_p != '\0' && *_p == ' ') _p++;
+                    if (*_p != '\0' && *_p == '+') _p++;
 
                     /* Field width */
                     while (*_p != '\0' && isdigit(*_p)) _p++;
@@ -351,7 +829,7 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
 
                 fmt_invalid:
                     va_end(_cpy);
-                    goto format_err;
+                    goto err;
 
                 fmt_valid:
                     _l = (_p+1)-c;
@@ -370,7 +848,7 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
                 }
             }
 
-            if (newarg == NULL) goto memory_err;
+            if (newarg == NULL) goto err;
             curarg = newarg;
 
             touched = 1;
@@ -382,7 +860,7 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
     /* Add the last argument if needed */
     if (touched) {
         newargv = realloc(curargv,sizeof(char*)*(argc+1));
-        if (newargv == NULL) goto memory_err;
+        if (newargv == NULL) goto err;
         curargv = newargv;
         curargv[argc++] = curarg;
         totlen += bulklen(sdslen(curarg));
@@ -394,11 +872,11 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
     curarg = NULL;
 
     /* Add bytes needed to hold multi bulk count */
-    totlen += 1+countDigits(argc)+2;
+    totlen += 1+intlen(argc)+2;
 
     /* Build the command at protocol level */
     cmd = malloc(totlen+1);
-    if (cmd == NULL) goto memory_err;
+    if (cmd == NULL) goto err;
 
     pos = sprintf(cmd,"*%d\r\n",argc);
     for (j = 0; j < argc; j++) {
@@ -416,29 +894,20 @@ int redisvFormatCommand(char **target, const char *format, va_list ap) {
     *target = cmd;
     return totlen;
 
-format_err:
-    error_type = -2;
-    goto cleanup;
+err:
+    while(argc--)
+        sdsfree(curargv[argc]);
+    free(curargv);
 
-memory_err:
-    error_type = -1;
-    goto cleanup;
-
-cleanup:
-    if (curargv) {
-        while(argc--)
-            sdsfree(curargv[argc]);
-        free(curargv);
-    }
-
-    sdsfree(curarg);
+    if (curarg != NULL)
+        sdsfree(curarg);
 
     /* No need to check cmd since it is the last statement that can fail,
      * but do it anyway to be as defensive as possible. */
     if (cmd != NULL)
         free(cmd);
 
-    return error_type;
+    return -1;
 }
 
 /* Format a command according to the Redis protocol. This function
@@ -459,69 +928,9 @@ int redisFormatCommand(char **target, const char *format, ...) {
     va_start(ap,format);
     len = redisvFormatCommand(target,format,ap);
     va_end(ap);
-
-    /* The API says "-1" means bad result, but we now also return "-2" in some
-     * cases.  Force the return value to always be -1. */
-    if (len < 0)
-        len = -1;
-
     return len;
 }
 
-/* Format a command according to the Redis protocol using an sds string and
- * sdscatfmt for the processing of arguments. This function takes the
- * number of arguments, an array with arguments and an array with their
- * lengths. If the latter is set to NULL, strlen will be used to compute the
- * argument lengths.
- */
-int redisFormatSdsCommandArgv(sds *target, int argc, const char **argv,
-                              const size_t *argvlen)
-{
-    sds cmd;
-    unsigned long long totlen;
-    int j;
-    size_t len;
-
-    /* Abort on a NULL target */
-    if (target == NULL)
-        return -1;
-
-    /* Calculate our total size */
-    totlen = 1+countDigits(argc)+2;
-    for (j = 0; j < argc; j++) {
-        len = argvlen ? argvlen[j] : strlen(argv[j]);
-        totlen += bulklen(len);
-    }
-
-    /* Use an SDS string for command construction */
-    cmd = sdsempty();
-    if (cmd == NULL)
-        return -1;
-
-    /* We already know how much storage we need */
-    cmd = sdsMakeRoomFor(cmd, totlen);
-    if (cmd == NULL)
-        return -1;
-
-    /* Construct command */
-    cmd = sdscatfmt(cmd, "*%i\r\n", argc);
-    for (j=0; j < argc; j++) {
-        len = argvlen ? argvlen[j] : strlen(argv[j]);
-        cmd = sdscatfmt(cmd, "$%u\r\n", len);
-        cmd = sdscatlen(cmd, argv[j], len);
-        cmd = sdscatlen(cmd, "\r\n", sizeof("\r\n")-1);
-    }
-
-    assert(sdslen(cmd)==totlen);
-
-    *target = cmd;
-    return totlen;
-}
-
-void redisFreeSdsCommand(sds cmd) {
-    sdsfree(cmd);
-}
-
 /* Format a command according to the Redis protocol. This function takes the
  * number of arguments, an array with arguments and an array with their
  * lengths. If the latter is set to NULL, strlen will be used to compute the
@@ -533,12 +942,8 @@ int redisFormatCommandArgv(char **target, int argc, const char **argv, const siz
     size_t len;
     int totlen, j;
 
-    /* Abort on a NULL target */
-    if (target == NULL)
-        return -1;
-
     /* Calculate number of bytes needed for the command */
-    totlen = 1+countDigits(argc)+2;
+    totlen = 1+intlen(argc)+2;
     for (j = 0; j < argc; j++) {
         len = argvlen ? argvlen[j] : strlen(argv[j]);
         totlen += bulklen(len);
@@ -565,10 +970,6 @@ int redisFormatCommandArgv(char **target, int argc, const char **argv, const siz
     return totlen;
 }
 
-void redisFreeCommand(char *cmd) {
-    free(cmd);
-}
-
 void __redisSetError(redisContext *c, int type, const char *str) {
     size_t len;
 
@@ -581,14 +982,10 @@ void __redisSetError(redisContext *c, int type, const char *str) {
     } else {
         /* Only REDIS_ERR_IO may lack a description! */
         assert(type == REDIS_ERR_IO);
-        __redis_strerror_r(errno, c->errstr, sizeof(c->errstr));
+        strerror_r(errno,c->errstr,sizeof(c->errstr));
     }
 }
 
-redisReader *redisReaderCreate(void) {
-    return redisReaderCreateWithFunctions(&defaultFunctions);
-}
-
 static redisContext *redisContextInit(void) {
     redisContext *c;
 
@@ -600,72 +997,24 @@ static redisContext *redisContextInit(void) {
     c->errstr[0] = '\0';
     c->obuf = sdsempty();
     c->reader = redisReaderCreate();
-    c->tcp.host = NULL;
-    c->tcp.source_addr = NULL;
-    c->unix_sock.path = NULL;
-    c->timeout = NULL;
-
-    if (c->obuf == NULL || c->reader == NULL) {
-        redisFree(c);
-        return NULL;
-    }
-
     return c;
 }
 
 void redisFree(redisContext *c) {
-    if (c == NULL)
-        return;
     if (c->fd > 0)
         close(c->fd);
     if (c->obuf != NULL)
         sdsfree(c->obuf);
     if (c->reader != NULL)
         redisReaderFree(c->reader);
-    if (c->tcp.host)
-        free(c->tcp.host);
-    if (c->tcp.source_addr)
-        free(c->tcp.source_addr);
-    if (c->unix_sock.path)
-        free(c->unix_sock.path);
-    if (c->timeout)
-        free(c->timeout);
     free(c);
 }
 
 int redisFreeKeepFd(redisContext *c) {
-    int fd = c->fd;
-    c->fd = -1;
-    redisFree(c);
-    return fd;
-}
-
-int redisReconnect(redisContext *c) {
-    c->err = 0;
-    memset(c->errstr, '\0', strlen(c->errstr));
-
-    if (c->fd > 0) {
-        close(c->fd);
-    }
-
-    sdsfree(c->obuf);
-    redisReaderFree(c->reader);
-
-    c->obuf = sdsempty();
-    c->reader = redisReaderCreate();
-
-    if (c->connection_type == REDIS_CONN_TCP) {
-        return redisContextConnectBindTcp(c, c->tcp.host, c->tcp.port,
-                c->timeout, c->tcp.source_addr);
-    } else if (c->connection_type == REDIS_CONN_UNIX) {
-        return redisContextConnectUnix(c, c->unix_sock.path, c->timeout);
-    } else {
-        /* Something bad happened here and shouldn't have. There isn't
-           enough information in the context to reconnect. */
-        __redisSetError(c,REDIS_ERR_OTHER,"Not enough information to reconnect");
-    }
-
-    return REDIS_ERR;
+	int fd = c->fd;
+	c->fd = -1;
+	redisFree(c);
+	return fd;
 }
 
 /* Connect to a Redis instance. On error the field error in the returned
@@ -715,15 +1064,6 @@ redisContext *redisConnectBindNonBlock(const char *ip, int port,
     return c;
 }
 
-redisContext *redisConnectBindNonBlockWithReuse(const char *ip, int port,
-                                                const char *source_addr) {
-    redisContext *c = redisContextInit();
-    c->flags &= ~REDIS_BLOCK;
-    c->flags |= REDIS_REUSEADDR;
-    redisContextConnectBindTcp(c,ip,port,NULL,source_addr);
-    return c;
-}
-
 redisContext *redisConnectUnix(const char *path) {
     redisContext *c;
 
@@ -822,10 +1162,10 @@ int redisBufferRead(redisContext *c) {
 /* Write the output buffer to the socket.
  *
  * Returns REDIS_OK when the buffer is empty, or (a part of) the buffer was
- * successfully written to the socket. When the buffer is empty after the
+ * succesfully written to the socket. When the buffer is empty after the
  * write operation, "done" is set to 1 (if given).
  *
- * Returns REDIS_ERR if an error occurred trying to write and sets
+ * Returns REDIS_ERR if an error occured trying to write and sets
  * c->errstr to hold the appropriate error string.
  */
 int redisBufferWrite(redisContext *c, int *done) {
@@ -934,9 +1274,6 @@ int redisvAppendCommand(redisContext *c, const char *format, va_list ap) {
     if (len == -1) {
         __redisSetError(c,REDIS_ERR_OOM,"Out of memory");
         return REDIS_ERR;
-    } else if (len == -2) {
-        __redisSetError(c,REDIS_ERR_OTHER,"Invalid format string");
-        return REDIS_ERR;
     }
 
     if (__redisAppendCommand(c,cmd,len) != REDIS_OK) {
@@ -959,21 +1296,21 @@ int redisAppendCommand(redisContext *c, const char *format, ...) {
 }
 
 int redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen) {
-    sds cmd;
+    char *cmd;
     int len;
 
-    len = redisFormatSdsCommandArgv(&cmd,argc,argv,argvlen);
+    len = redisFormatCommandArgv(&cmd,argc,argv,argvlen);
     if (len == -1) {
         __redisSetError(c,REDIS_ERR_OOM,"Out of memory");
         return REDIS_ERR;
     }
 
     if (__redisAppendCommand(c,cmd,len) != REDIS_OK) {
-        sdsfree(cmd);
+        free(cmd);
         return REDIS_ERR;
     }
 
-    sdsfree(cmd);
+    free(cmd);
     return REDIS_OK;
 }
 
@@ -984,7 +1321,7 @@ int redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const s
  * context is non-blocking, the "reply" pointer will not be used and the
  * command is simply appended to the write buffer.
  *
- * Returns the reply when a reply was successfully retrieved. Returns NULL
+ * Returns the reply when a reply was succesfully retrieved. Returns NULL
  * otherwise. When NULL is returned in a blocking context, the error field
  * in the context will be set.
  */

deps/hiredis/hiredis.h

@@ -1,8 +1,6 @@
 /*
  * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2014, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- * Copyright (c) 2015, Matt Stancliff <matt at genges dot com>,
- *                     Jan-Erik Rediger <janerik at fnordig dot com>
+ * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
  *
  * All rights reserved.
  *
@@ -33,16 +31,26 @@
 
 #ifndef __HIREDIS_H
 #define __HIREDIS_H
-#include "read.h"
+#include <stdio.h> /* for size_t */
 #include <stdarg.h> /* for va_list */
 #include <sys/time.h> /* for struct timeval */
-#include <stdint.h> /* uintXX_t, etc */
-#include "sds.h" /* for sds */
 
 #define HIREDIS_MAJOR 0
-#define HIREDIS_MINOR 13
-#define HIREDIS_PATCH 3
-#define HIREDIS_SONAME 0.13
+#define HIREDIS_MINOR 11
+#define HIREDIS_PATCH 0
+
+#define REDIS_ERR -1
+#define REDIS_OK 0
+
+/* When an error occurs, the err flag in a context is set to hold the type of
+ * error that occured. REDIS_ERR_IO means there was an I/O error and you
+ * should use the "errno" variable to find out what is wrong.
+ * For other values, the "errstr" field will hold a description. */
+#define REDIS_ERR_IO 1 /* Error in read or write */
+#define REDIS_ERR_EOF 3 /* End of file */
+#define REDIS_ERR_PROTOCOL 4 /* Protocol error */
+#define REDIS_ERR_OOM 5 /* Out of memory */
+#define REDIS_ERR_OTHER 2 /* Everything else... */
 
 /* Connection type can be blocking or non-blocking and is set in the
  * least significant bit of the flags field in redisContext. */
@@ -71,39 +79,17 @@
 /* Flag that is set when monitor mode is active */
 #define REDIS_MONITORING 0x40
 
-/* Flag that is set when we should set SO_REUSEADDR before calling bind() */
-#define REDIS_REUSEADDR 0x80
+#define REDIS_REPLY_STRING 1
+#define REDIS_REPLY_ARRAY 2
+#define REDIS_REPLY_INTEGER 3
+#define REDIS_REPLY_NIL 4
+#define REDIS_REPLY_STATUS 5
+#define REDIS_REPLY_ERROR 6
+
+#define REDIS_READER_MAX_BUF (1024*16)  /* Default max unused reader buffer. */
 
 #define REDIS_KEEPALIVE_INTERVAL 15 /* seconds */
 
-/* number of times we retry to connect in the case of EADDRNOTAVAIL and
- * SO_REUSEADDR is being used. */
-#define REDIS_CONNECT_RETRIES  10
-
-/* strerror_r has two completely different prototypes and behaviors
- * depending on system issues, so we need to operate on the error buffer
- * differently depending on which strerror_r we're using. */
-#ifndef _GNU_SOURCE
-/* "regular" POSIX strerror_r that does the right thing. */
-#define __redis_strerror_r(errno, buf, len)                                    \
-    do {                                                                       \
-        strerror_r((errno), (buf), (len));                                     \
-    } while (0)
-#else
-/* "bad" GNU strerror_r we need to clean up after. */
-#define __redis_strerror_r(errno, buf, len)                                    \
-    do {                                                                       \
-        char *err_str = strerror_r((errno), (buf), (len));                     \
-        /* If return value _isn't_ the start of the buffer we passed in,       \
-         * then GNU strerror_r returned an internal static buffer and we       \
-         * need to copy the result into our private buffer. */                 \
-        if (err_str != (buf)) {                                                \
-            strncpy((buf), err_str, ((len) - 1));                              \
-            buf[(len)-1] = '\0';                                               \
-        }                                                                      \
-    } while (0)
-#endif
-
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -112,13 +98,61 @@ extern "C" {
 typedef struct redisReply {
     int type; /* REDIS_REPLY_* */
     long long integer; /* The integer when type is REDIS_REPLY_INTEGER */
-    size_t len; /* Length of string */
+    int len; /* Length of string */
     char *str; /* Used for both REDIS_REPLY_ERROR and REDIS_REPLY_STRING */
     size_t elements; /* number of elements, for REDIS_REPLY_ARRAY */
     struct redisReply **element; /* elements vector for REDIS_REPLY_ARRAY */
 } redisReply;
 
+typedef struct redisReadTask {
+    int type;
+    int elements; /* number of elements in multibulk container */
+    int idx; /* index in parent (array) object */
+    void *obj; /* holds user-generated value for a read task */
+    struct redisReadTask *parent; /* parent task */
+    void *privdata; /* user-settable arbitrary field */
+} redisReadTask;
+
+typedef struct redisReplyObjectFunctions {
+    void *(*createString)(const redisReadTask*, char*, size_t);
+    void *(*createArray)(const redisReadTask*, int);
+    void *(*createInteger)(const redisReadTask*, long long);
+    void *(*createNil)(const redisReadTask*);
+    void (*freeObject)(void*);
+} redisReplyObjectFunctions;
+
+/* State for the protocol parser */
+typedef struct redisReader {
+    int err; /* Error flags, 0 when there is no error */
+    char errstr[128]; /* String representation of error when applicable */
+
+    char *buf; /* Read buffer */
+    size_t pos; /* Buffer cursor */
+    size_t len; /* Buffer length */
+    size_t maxbuf; /* Max length of unused buffer */
+
+    redisReadTask rstack[9];
+    int ridx; /* Index of current read task */
+    void *reply; /* Temporary reply pointer */
+
+    redisReplyObjectFunctions *fn;
+    void *privdata;
+} redisReader;
+
+/* Public API for the protocol parser. */
 redisReader *redisReaderCreate(void);
+void redisReaderFree(redisReader *r);
+int redisReaderFeed(redisReader *r, const char *buf, size_t len);
+int redisReaderGetReply(redisReader *r, void **reply);
+
+/* Backwards compatibility, can be removed on big version bump. */
+#define redisReplyReaderCreate redisReaderCreate
+#define redisReplyReaderFree redisReaderFree
+#define redisReplyReaderFeed redisReaderFeed
+#define redisReplyReaderGetReply redisReaderGetReply
+#define redisReplyReaderSetPrivdata(_r, _p) (int)(((redisReader*)(_r))->privdata = (_p))
+#define redisReplyReaderGetObject(_r) (((redisReader*)(_r))->reply)
+#define redisReplyReaderGetError(_r) (((redisReader*)(_r))->errstr)
 
 /* Function to free the reply objects hiredis returns by default. */
 void freeReplyObject(void *reply);
@@ -127,14 +161,6 @@ void freeReplyObject(void *reply);
 int redisvFormatCommand(char **target, const char *format, va_list ap);
 int redisFormatCommand(char **target, const char *format, ...);
 int redisFormatCommandArgv(char **target, int argc, const char **argv, const size_t *argvlen);
-int redisFormatSdsCommandArgv(sds *target, int argc, const char ** argv, const size_t *argvlen);
-void redisFreeCommand(char *cmd);
-void redisFreeSdsCommand(sds cmd);
-
-enum redisConnectionType {
-    REDIS_CONN_TCP,
-    REDIS_CONN_UNIX
-};
 
 /* Context for a connection to Redis */
 typedef struct redisContext {
@@ -144,45 +170,16 @@ typedef struct redisContext {
     int flags;
     char *obuf; /* Write buffer */
     redisReader *reader; /* Protocol reader */
-
-    enum redisConnectionType connection_type;
-    struct timeval *timeout;
-
-    struct {
-        char *host;
-        char *source_addr;
-        int port;
-    } tcp;
-
-    struct {
-        char *path;
-    } unix_sock;
-
 } redisContext;
 
 redisContext *redisConnect(const char *ip, int port);
 redisContext *redisConnectWithTimeout(const char *ip, int port, const struct timeval tv);
 redisContext *redisConnectNonBlock(const char *ip, int port);
-redisContext *redisConnectBindNonBlock(const char *ip, int port,
-                                       const char *source_addr);
-redisContext *redisConnectBindNonBlockWithReuse(const char *ip, int port,
-                                                const char *source_addr);
+redisContext *redisConnectBindNonBlock(const char *ip, int port, const char *source_addr);
 redisContext *redisConnectUnix(const char *path);
 redisContext *redisConnectUnixWithTimeout(const char *path, const struct timeval tv);
 redisContext *redisConnectUnixNonBlock(const char *path);
 redisContext *redisConnectFd(int fd);
-
-/**
- * Reconnect the given context using the saved information.
- *
- * This re-uses the exact same connect options as in the initial connection.
- * host, ip (or path), timeout and bind address are reused,
- * flags are used unmodified from the existing context.
- *
- * Returns REDIS_OK on successful connect or REDIS_ERR otherwise.
- */
-int redisReconnect(redisContext *c);
-
 int redisSetTimeout(redisContext *c, const struct timeval tv);
 int redisEnableKeepAlive(redisContext *c);
 void redisFree(redisContext *c);

deps/hiredis/net.c

@@ -1,9 +1,7 @@
 /* Extracted from anet.c to work properly with Hiredis error reporting.
  *
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2014, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- * Copyright (c) 2015, Matt Stancliff <matt at genges dot com>,
- *                     Jan-Erik Rediger <janerik at fnordig dot com>
+ * Copyright (c) 2006-2011, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
  *
  * All rights reserved.
  *
@@ -49,7 +47,6 @@
 #include <stdio.h>
 #include <poll.h>
 #include <limits.h>
-#include <stdlib.h>
 
 #include "net.h"
 #include "sds.h"
@@ -70,7 +67,7 @@ static void __redisSetErrorFromErrno(redisContext *c, int type, const char *pref
 
     if (prefix != NULL)
         len = snprintf(buf,sizeof(buf),"%s: ",prefix);
-    __redis_strerror_r(errno, (char *)(buf + len), sizeof(buf) - len);
+    strerror_r(errno,buf+len,sizeof(buf)-len);
     __redisSetError(c,type,buf);
 }
 
@@ -141,7 +138,7 @@ int redisKeepAlive(redisContext *c, int interval) {
         return REDIS_ERR;
     }
 #else
-#if defined(__GLIBC__) && !defined(__FreeBSD_kernel__)
+#ifndef __sun
     val = interval;
     if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &val, sizeof(val)) < 0) {
         __redisSetError(c,REDIS_ERR_OTHER,strerror(errno));
@@ -178,15 +175,19 @@ static int redisSetTcpNoDelay(redisContext *c) {
 
 #define __MAX_MSEC (((LONG_MAX) - 999) / 1000)
 
-static int redisContextTimeoutMsec(redisContext *c, long *result)
-{
-    const struct timeval *timeout = c->timeout;
-    long msec = -1;
+static int redisContextWaitReady(redisContext *c, const struct timeval *timeout) {
+    struct pollfd   wfd[1];
+    long msec;
+
+    msec          = -1;
+    wfd[0].fd     = c->fd;
+    wfd[0].events = POLLOUT;
 
     /* Only use timeout when not NULL. */
     if (timeout != NULL) {
         if (timeout->tv_usec > 1000000 || timeout->tv_sec > __MAX_MSEC) {
-            *result = msec;
+            __redisSetErrorFromErrno(c, REDIS_ERR_IO, NULL);
+            redisContextCloseFd(c);
             return REDIS_ERR;
         }
 
@@ -197,16 +198,6 @@ static int redisContextTimeoutMsec(redisContext *c, long *result)
         }
     }
 
-    *result = msec;
-    return REDIS_OK;
-}
-
-static int redisContextWaitReady(redisContext *c, long msec) {
-    struct pollfd   wfd[1];
-
-    wfd[0].fd     = c->fd;
-    wfd[0].events = POLLOUT;
-
     if (errno == EINPROGRESS) {
         int res;
 
@@ -265,57 +256,10 @@ int redisContextSetTimeout(redisContext *c, const struct timeval tv) {
 static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
                                    const struct timeval *timeout,
                                    const char *source_addr) {
-    int s, rv, n;
+    int s, rv;
     char _port[6];  /* strlen("65535"); */
     struct addrinfo hints, *servinfo, *bservinfo, *p, *b;
     int blocking = (c->flags & REDIS_BLOCK);
-    int reuseaddr = (c->flags & REDIS_REUSEADDR);
-    int reuses = 0;
-    long timeout_msec = -1;
-
-    servinfo = NULL;
-    c->connection_type = REDIS_CONN_TCP;
-    c->tcp.port = port;
-
-    /* We need to take possession of the passed parameters
-     * to make them reusable for a reconnect.
-     * We also carefully check we don't free data we already own,
-     * as in the case of the reconnect method.
-     *
-     * This is a bit ugly, but atleast it works and doesn't leak memory.
-     **/
-    if (c->tcp.host != addr) {
-        if (c->tcp.host)
-            free(c->tcp.host);
-
-        c->tcp.host = strdup(addr);
-    }
-
-    if (timeout) {
-        if (c->timeout != timeout) {
-            if (c->timeout == NULL)
-                c->timeout = malloc(sizeof(struct timeval));
-
-            memcpy(c->timeout, timeout, sizeof(struct timeval));
-        }
-    } else {
-        if (c->timeout)
-            free(c->timeout);
-        c->timeout = NULL;
-    }
-
-    if (redisContextTimeoutMsec(c, &timeout_msec) != REDIS_OK) {
-        __redisSetError(c, REDIS_ERR_IO, "Invalid timeout specified");
-        goto error;
-    }
-
-    if (source_addr == NULL) {
-        free(c->tcp.source_addr);
-        c->tcp.source_addr = NULL;
-    } else if (c->tcp.source_addr != source_addr) {
-        free(c->tcp.source_addr);
-        c->tcp.source_addr = strdup(source_addr);
-    }
 
     snprintf(_port, 6, "%d", port);
     memset(&hints,0,sizeof(hints));
@@ -327,7 +271,7 @@ static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
      * as this would add latency to every connect. Otherwise a more sensible
      * route could be: Use IPv6 if both addresses are available and there is IPv6
      * connectivity. */
-    if ((rv = getaddrinfo(c->tcp.host,_port,&hints,&servinfo)) != 0) {
+    if ((rv = getaddrinfo(addr,_port,&hints,&servinfo)) != 0) {
          hints.ai_family = AF_INET6;
          if ((rv = getaddrinfo(addr,_port,&hints,&servinfo)) != 0) {
             __redisSetError(c,REDIS_ERR_OTHER,gai_strerror(rv));
@@ -335,31 +279,21 @@ static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
         }
     }
     for (p = servinfo; p != NULL; p = p->ai_next) {
-addrretry:
         if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
             continue;
 
         c->fd = s;
         if (redisSetBlocking(c,0) != REDIS_OK)
             goto error;
-        if (c->tcp.source_addr) {
+        if (source_addr) {
             int bound = 0;
             /* Using getaddrinfo saves us from self-determining IPv4 vs IPv6 */
-            if ((rv = getaddrinfo(c->tcp.source_addr, NULL, &hints, &bservinfo)) != 0) {
+            if ((rv = getaddrinfo(source_addr, NULL, &hints, &bservinfo)) != 0) {
                 char buf[128];
                 snprintf(buf,sizeof(buf),"Can't get addr: %s",gai_strerror(rv));
                 __redisSetError(c,REDIS_ERR_OTHER,buf);
                 goto error;
             }
-
-            if (reuseaddr) {
-                n = 1;
-                if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char*) &n,
-                               sizeof(n)) < 0) {
-                    goto error;
-                }
-            }
-
             for (b = bservinfo; b != NULL; b = b->ai_next) {
                 if (bind(s,b->ai_addr,b->ai_addrlen) != -1) {
                     bound = 1;
@@ -380,15 +314,8 @@ addrretry:
                 continue;
             } else if (errno == EINPROGRESS && !blocking) {
                 /* This is ok. */
-            } else if (errno == EADDRNOTAVAIL && reuseaddr) {
-                if (++reuses >= REDIS_CONNECT_RETRIES) {
-                    goto error;
-                } else {
-                    redisContextCloseFd(c);
-                    goto addrretry;
-                }
             } else {
-                if (redisContextWaitReady(c,timeout_msec) != REDIS_OK)
+                if (redisContextWaitReady(c,timeout) != REDIS_OK)
                     goto error;
             }
         }
@@ -429,40 +356,19 @@ int redisContextConnectBindTcp(redisContext *c, const char *addr, int port,
 int redisContextConnectUnix(redisContext *c, const char *path, const struct timeval *timeout) {
     int blocking = (c->flags & REDIS_BLOCK);
     struct sockaddr_un sa;
-    long timeout_msec = -1;
 
     if (redisCreateSocket(c,AF_LOCAL) < 0)
         return REDIS_ERR;
     if (redisSetBlocking(c,0) != REDIS_OK)
         return REDIS_ERR;
 
-    c->connection_type = REDIS_CONN_UNIX;
-    if (c->unix_sock.path != path)
-        c->unix_sock.path = strdup(path);
-
-    if (timeout) {
-        if (c->timeout != timeout) {
-            if (c->timeout == NULL)
-                c->timeout = malloc(sizeof(struct timeval));
-
-            memcpy(c->timeout, timeout, sizeof(struct timeval));
-        }
-    } else {
-        if (c->timeout)
-            free(c->timeout);
-        c->timeout = NULL;
-    }
-
-    if (redisContextTimeoutMsec(c,&timeout_msec) != REDIS_OK)
-        return REDIS_ERR;
-
     sa.sun_family = AF_LOCAL;
     strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
     if (connect(c->fd, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
         if (errno == EINPROGRESS && !blocking) {
             /* This is ok. */
         } else {
-            if (redisContextWaitReady(c,timeout_msec) != REDIS_OK)
+            if (redisContextWaitReady(c,timeout) != REDIS_OK)
                 return REDIS_ERR;
         }
     }

deps/hiredis/net.h

@@ -1,9 +1,7 @@
 /* Extracted from anet.c to work properly with Hiredis error reporting.
  *
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2014, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- * Copyright (c) 2015, Matt Stancliff <matt at genges dot com>,
- *                     Jan-Erik Rediger <janerik at fnordig dot com>
+ * Copyright (c) 2006-2011, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
  *
  * All rights reserved.
  *
@@ -37,7 +35,7 @@
 
 #include "hiredis.h"
 
-#if defined(__sun)
+#if defined(__sun) || defined(_AIX)
 #define AF_LOCAL AF_UNIX
 #endif
 

deps/hiredis/read.c

@@ -1,525 +0,0 @@
-/*
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- *
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-
-#include "fmacros.h"
-#include <string.h>
-#include <stdlib.h>
-#ifndef _MSC_VER
-#include <unistd.h>
-#endif
-#include <assert.h>
-#include <errno.h>
-#include <ctype.h>
-
-#include "read.h"
-#include "sds.h"
-
-static void __redisReaderSetError(redisReader *r, int type, const char *str) {
-    size_t len;
-
-    if (r->reply != NULL && r->fn && r->fn->freeObject) {
-        r->fn->freeObject(r->reply);
-        r->reply = NULL;
-    }
-
-    /* Clear input buffer on errors. */
-    if (r->buf != NULL) {
-        sdsfree(r->buf);
-        r->buf = NULL;
-        r->pos = r->len = 0;
-    }
-
-    /* Reset task stack. */
-    r->ridx = -1;
-
-    /* Set error. */
-    r->err = type;
-    len = strlen(str);
-    len = len < (sizeof(r->errstr)-1) ? len : (sizeof(r->errstr)-1);
-    memcpy(r->errstr,str,len);
-    r->errstr[len] = '\0';
-}
-
-static size_t chrtos(char *buf, size_t size, char byte) {
-    size_t len = 0;
-
-    switch(byte) {
-    case '\\':
-    case '"':
-        len = snprintf(buf,size,"\"\\%c\"",byte);
-        break;
-    case '\n': len = snprintf(buf,size,"\"\\n\""); break;
-    case '\r': len = snprintf(buf,size,"\"\\r\""); break;
-    case '\t': len = snprintf(buf,size,"\"\\t\""); break;
-    case '\a': len = snprintf(buf,size,"\"\\a\""); break;
-    case '\b': len = snprintf(buf,size,"\"\\b\""); break;
-    default:
-        if (isprint(byte))
-            len = snprintf(buf,size,"\"%c\"",byte);
-        else
-            len = snprintf(buf,size,"\"\\x%02x\"",(unsigned char)byte);
-        break;
-    }
-
-    return len;
-}
-
-static void __redisReaderSetErrorProtocolByte(redisReader *r, char byte) {
-    char cbuf[8], sbuf[128];
-
-    chrtos(cbuf,sizeof(cbuf),byte);
-    snprintf(sbuf,sizeof(sbuf),
-        "Protocol error, got %s as reply type byte", cbuf);
-    __redisReaderSetError(r,REDIS_ERR_PROTOCOL,sbuf);
-}
-
-static void __redisReaderSetErrorOOM(redisReader *r) {
-    __redisReaderSetError(r,REDIS_ERR_OOM,"Out of memory");
-}
-
-static char *readBytes(redisReader *r, unsigned int bytes) {
-    char *p;
-    if (r->len-r->pos >= bytes) {
-        p = r->buf+r->pos;
-        r->pos += bytes;
-        return p;
-    }
-    return NULL;
-}
-
-/* Find pointer to \r\n. */
-static char *seekNewline(char *s, size_t len) {
-    int pos = 0;
-    int _len = len-1;
-
-    /* Position should be < len-1 because the character at "pos" should be
-     * followed by a \n. Note that strchr cannot be used because it doesn't
-     * allow to search a limited length and the buffer that is being searched
-     * might not have a trailing NULL character. */
-    while (pos < _len) {
-        while(pos < _len && s[pos] != '\r') pos++;
-        if (pos==_len) {
-            /* Not found. */
-            return NULL;
-        } else {
-            if (s[pos+1] == '\n') {
-                /* Found. */
-                return s+pos;
-            } else {
-                /* Continue searching. */
-                pos++;
-            }
-        }
-    }
-    return NULL;
-}
-
-/* Read a long long value starting at *s, under the assumption that it will be
- * terminated by \r\n. Ambiguously returns -1 for unexpected input. */
-static long long readLongLong(char *s) {
-    long long v = 0;
-    int dec, mult = 1;
-    char c;
-
-    if (*s == '-') {
-        mult = -1;
-        s++;
-    } else if (*s == '+') {
-        mult = 1;
-        s++;
-    }
-
-    while ((c = *(s++)) != '\r') {
-        dec = c - '0';
-        if (dec >= 0 && dec < 10) {
-            v *= 10;
-            v += dec;
-        } else {
-            /* Should not happen... */
-            return -1;
-        }
-    }
-
-    return mult*v;
-}
-
-static char *readLine(redisReader *r, int *_len) {
-    char *p, *s;
-    int len;
-
-    p = r->buf+r->pos;
-    s = seekNewline(p,(r->len-r->pos));
-    if (s != NULL) {
-        len = s-(r->buf+r->pos);
-        r->pos += len+2; /* skip \r\n */
-        if (_len) *_len = len;
-        return p;
-    }
-    return NULL;
-}
-
-static void moveToNextTask(redisReader *r) {
-    redisReadTask *cur, *prv;
-    while (r->ridx >= 0) {
-        /* Return a.s.a.p. when the stack is now empty. */
-        if (r->ridx == 0) {
-            r->ridx--;
-            return;
-        }
-
-        cur = &(r->rstack[r->ridx]);
-        prv = &(r->rstack[r->ridx-1]);
-        assert(prv->type == REDIS_REPLY_ARRAY);
-        if (cur->idx == prv->elements-1) {
-            r->ridx--;
-        } else {
-            /* Reset the type because the next item can be anything */
-            assert(cur->idx < prv->elements);
-            cur->type = -1;
-            cur->elements = -1;
-            cur->idx++;
-            return;
-        }
-    }
-}
-
-static int processLineItem(redisReader *r) {
-    redisReadTask *cur = &(r->rstack[r->ridx]);
-    void *obj;
-    char *p;
-    int len;
-
-    if ((p = readLine(r,&len)) != NULL) {
-        if (cur->type == REDIS_REPLY_INTEGER) {
-            if (r->fn && r->fn->createInteger)
-                obj = r->fn->createInteger(cur,readLongLong(p));
-            else
-                obj = (void*)REDIS_REPLY_INTEGER;
-        } else {
-            /* Type will be error or status. */
-            if (r->fn && r->fn->createString)
-                obj = r->fn->createString(cur,p,len);
-            else
-                obj = (void*)(size_t)(cur->type);
-        }
-
-        if (obj == NULL) {
-            __redisReaderSetErrorOOM(r);
-            return REDIS_ERR;
-        }
-
-        /* Set reply if this is the root object. */
-        if (r->ridx == 0) r->reply = obj;
-        moveToNextTask(r);
-        return REDIS_OK;
-    }
-
-    return REDIS_ERR;
-}
-
-static int processBulkItem(redisReader *r) {
-    redisReadTask *cur = &(r->rstack[r->ridx]);
-    void *obj = NULL;
-    char *p, *s;
-    long len;
-    unsigned long bytelen;
-    int success = 0;
-
-    p = r->buf+r->pos;
-    s = seekNewline(p,r->len-r->pos);
-    if (s != NULL) {
-        p = r->buf+r->pos;
-        bytelen = s-(r->buf+r->pos)+2; /* include \r\n */
-        len = readLongLong(p);
-
-        if (len < 0) {
-            /* The nil object can always be created. */
-            if (r->fn && r->fn->createNil)
-                obj = r->fn->createNil(cur);
-            else
-                obj = (void*)REDIS_REPLY_NIL;
-            success = 1;
-        } else {
-            /* Only continue when the buffer contains the entire bulk item. */
-            bytelen += len+2; /* include \r\n */
-            if (r->pos+bytelen <= r->len) {
-                if (r->fn && r->fn->createString)
-                    obj = r->fn->createString(cur,s+2,len);
-                else
-                    obj = (void*)REDIS_REPLY_STRING;
-                success = 1;
-            }
-        }
-
-        /* Proceed when obj was created. */
-        if (success) {
-            if (obj == NULL) {
-                __redisReaderSetErrorOOM(r);
-                return REDIS_ERR;
-            }
-
-            r->pos += bytelen;
-
-            /* Set reply if this is the root object. */
-            if (r->ridx == 0) r->reply = obj;
-            moveToNextTask(r);
-            return REDIS_OK;
-        }
-    }
-
-    return REDIS_ERR;
-}
-
-static int processMultiBulkItem(redisReader *r) {
-    redisReadTask *cur = &(r->rstack[r->ridx]);
-    void *obj;
-    char *p;
-    long elements;
-    int root = 0;
-
-    /* Set error for nested multi bulks with depth > 7 */
-    if (r->ridx == 8) {
-        __redisReaderSetError(r,REDIS_ERR_PROTOCOL,
-            "No support for nested multi bulk replies with depth > 7");
-        return REDIS_ERR;
-    }
-
-    if ((p = readLine(r,NULL)) != NULL) {
-        elements = readLongLong(p);
-        root = (r->ridx == 0);
-
-        if (elements == -1) {
-            if (r->fn && r->fn->createNil)
-                obj = r->fn->createNil(cur);
-            else
-                obj = (void*)REDIS_REPLY_NIL;
-
-            if (obj == NULL) {
-                __redisReaderSetErrorOOM(r);
-                return REDIS_ERR;
-            }
-
-            moveToNextTask(r);
-        } else {
-            if (r->fn && r->fn->createArray)
-                obj = r->fn->createArray(cur,elements);
-            else
-                obj = (void*)REDIS_REPLY_ARRAY;
-
-            if (obj == NULL) {
-                __redisReaderSetErrorOOM(r);
-                return REDIS_ERR;
-            }
-
-            /* Modify task stack when there are more than 0 elements. */
-            if (elements > 0) {
-                cur->elements = elements;
-                cur->obj = obj;
-                r->ridx++;
-                r->rstack[r->ridx].type = -1;
-                r->rstack[r->ridx].elements = -1;
-                r->rstack[r->ridx].idx = 0;
-                r->rstack[r->ridx].obj = NULL;
-                r->rstack[r->ridx].parent = cur;
-                r->rstack[r->ridx].privdata = r->privdata;
-            } else {
-                moveToNextTask(r);
-            }
-        }
-
-        /* Set reply if this is the root object. */
-        if (root) r->reply = obj;
-        return REDIS_OK;
-    }
-
-    return REDIS_ERR;
-}
-
-static int processItem(redisReader *r) {
-    redisReadTask *cur = &(r->rstack[r->ridx]);
-    char *p;
-
-    /* check if we need to read type */
-    if (cur->type < 0) {
-        if ((p = readBytes(r,1)) != NULL) {
-            switch (p[0]) {
-            case '-':
-                cur->type = REDIS_REPLY_ERROR;
-                break;
-            case '+':
-                cur->type = REDIS_REPLY_STATUS;
-                break;
-            case ':':
-                cur->type = REDIS_REPLY_INTEGER;
-                break;
-            case '$':
-                cur->type = REDIS_REPLY_STRING;
-                break;
-            case '*':
-                cur->type = REDIS_REPLY_ARRAY;
-                break;
-            default:
-                __redisReaderSetErrorProtocolByte(r,*p);
-                return REDIS_ERR;
-            }
-        } else {
-            /* could not consume 1 byte */
-            return REDIS_ERR;
-        }
-    }
-
-    /* process typed item */
-    switch(cur->type) {
-    case REDIS_REPLY_ERROR:
-    case REDIS_REPLY_STATUS:
-    case REDIS_REPLY_INTEGER:
-        return processLineItem(r);
-    case REDIS_REPLY_STRING:
-        return processBulkItem(r);
-    case REDIS_REPLY_ARRAY:
-        return processMultiBulkItem(r);
-    default:
-        assert(NULL);
-        return REDIS_ERR; /* Avoid warning. */
-    }
-}
-
-redisReader *redisReaderCreateWithFunctions(redisReplyObjectFunctions *fn) {
-    redisReader *r;
-
-    r = calloc(sizeof(redisReader),1);
-    if (r == NULL)
-        return NULL;
-
-    r->err = 0;
-    r->errstr[0] = '\0';
-    r->fn = fn;
-    r->buf = sdsempty();
-    r->maxbuf = REDIS_READER_MAX_BUF;
-    if (r->buf == NULL) {
-        free(r);
-        return NULL;
-    }
-
-    r->ridx = -1;
-    return r;
-}
-
-void redisReaderFree(redisReader *r) {
-    if (r->reply != NULL && r->fn && r->fn->freeObject)
-        r->fn->freeObject(r->reply);
-    if (r->buf != NULL)
-        sdsfree(r->buf);
-    free(r);
-}
-
-int redisReaderFeed(redisReader *r, const char *buf, size_t len) {
-    sds newbuf;
-
-    /* Return early when this reader is in an erroneous state. */
-    if (r->err)
-        return REDIS_ERR;
-
-    /* Copy the provided buffer. */
-    if (buf != NULL && len >= 1) {
-        /* Destroy internal buffer when it is empty and is quite large. */
-        if (r->len == 0 && r->maxbuf != 0 && sdsavail(r->buf) > r->maxbuf) {
-            sdsfree(r->buf);
-            r->buf = sdsempty();
-            r->pos = 0;
-
-            /* r->buf should not be NULL since we just free'd a larger one. */
-            assert(r->buf != NULL);
-        }
-
-        newbuf = sdscatlen(r->buf,buf,len);
-        if (newbuf == NULL) {
-            __redisReaderSetErrorOOM(r);
-            return REDIS_ERR;
-        }
-
-        r->buf = newbuf;
-        r->len = sdslen(r->buf);
-    }
-
-    return REDIS_OK;
-}
-
-int redisReaderGetReply(redisReader *r, void **reply) {
-    /* Default target pointer to NULL. */
-    if (reply != NULL)
-        *reply = NULL;
-
-    /* Return early when this reader is in an erroneous state. */
-    if (r->err)
-        return REDIS_ERR;
-
-    /* When the buffer is empty, there will never be a reply. */
-    if (r->len == 0)
-        return REDIS_OK;
-
-    /* Set first item to process when the stack is empty. */
-    if (r->ridx == -1) {
-        r->rstack[0].type = -1;
-        r->rstack[0].elements = -1;
-        r->rstack[0].idx = -1;
-        r->rstack[0].obj = NULL;
-        r->rstack[0].parent = NULL;
-        r->rstack[0].privdata = r->privdata;
-        r->ridx = 0;
-    }
-
-    /* Process items in reply. */
-    while (r->ridx >= 0)
-        if (processItem(r) != REDIS_OK)
-            break;
-
-    /* Return ASAP when an error occurred. */
-    if (r->err)
-        return REDIS_ERR;
-
-    /* Discard part of the buffer when we've consumed at least 1k, to avoid
-     * doing unnecessary calls to memmove() in sds.c. */
-    if (r->pos >= 1024) {
-        sdsrange(r->buf,r->pos,-1);
-        r->pos = 0;
-        r->len = sdslen(r->buf);
-    }
-
-    /* Emit a reply when there is one. */
-    if (r->ridx == -1) {
-        if (reply != NULL)
-            *reply = r->reply;
-        r->reply = NULL;
-    }
-    return REDIS_OK;
-}

deps/hiredis/read.h

@@ -1,111 +0,0 @@
-/*
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- *
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-
-#ifndef __HIREDIS_READ_H
-#define __HIREDIS_READ_H
-#include <stdio.h> /* for size_t */
-
-#define REDIS_ERR -1
-#define REDIS_OK 0
-
-/* When an error occurs, the err flag in a context is set to hold the type of
- * error that occurred. REDIS_ERR_IO means there was an I/O error and you
- * should use the "errno" variable to find out what is wrong.
- * For other values, the "errstr" field will hold a description. */
-#define REDIS_ERR_IO 1 /* Error in read or write */
-#define REDIS_ERR_EOF 3 /* End of file */
-#define REDIS_ERR_PROTOCOL 4 /* Protocol error */
-#define REDIS_ERR_OOM 5 /* Out of memory */
-#define REDIS_ERR_OTHER 2 /* Everything else... */
-
-#define REDIS_REPLY_STRING 1
-#define REDIS_REPLY_ARRAY 2
-#define REDIS_REPLY_INTEGER 3
-#define REDIS_REPLY_NIL 4
-#define REDIS_REPLY_STATUS 5
-#define REDIS_REPLY_ERROR 6
-
-#define REDIS_READER_MAX_BUF (1024*16)  /* Default max unused reader buffer. */
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct redisReadTask {
-    int type;
-    int elements; /* number of elements in multibulk container */
-    int idx; /* index in parent (array) object */
-    void *obj; /* holds user-generated value for a read task */
-    struct redisReadTask *parent; /* parent task */
-    void *privdata; /* user-settable arbitrary field */
-} redisReadTask;
-
-typedef struct redisReplyObjectFunctions {
-    void *(*createString)(const redisReadTask*, char*, size_t);
-    void *(*createArray)(const redisReadTask*, int);
-    void *(*createInteger)(const redisReadTask*, long long);
-    void *(*createNil)(const redisReadTask*);
-    void (*freeObject)(void*);
-} redisReplyObjectFunctions;
-
-typedef struct redisReader {
-    int err; /* Error flags, 0 when there is no error */
-    char errstr[128]; /* String representation of error when applicable */
-
-    char *buf; /* Read buffer */
-    size_t pos; /* Buffer cursor */
-    size_t len; /* Buffer length */
-    size_t maxbuf; /* Max length of unused buffer */
-
-    redisReadTask rstack[9];
-    int ridx; /* Index of current read task */
-    void *reply; /* Temporary reply pointer */
-
-    redisReplyObjectFunctions *fn;
-    void *privdata;
-} redisReader;
-
-/* Public API for the protocol parser. */
-redisReader *redisReaderCreateWithFunctions(redisReplyObjectFunctions *fn);
-void redisReaderFree(redisReader *r);
-int redisReaderFeed(redisReader *r, const char *buf, size_t len);
-int redisReaderGetReply(redisReader *r, void **reply);
-
-#define redisReaderSetPrivdata(_r, _p) (int)(((redisReader*)(_r))->privdata = (_p))
-#define redisReaderGetObject(_r) (((redisReader*)(_r))->reply)
-#define redisReaderGetError(_r) (((redisReader*)(_r))->errstr)
-
-#ifdef __cplusplus
-}
-#endif
-
-#endif

deps/hiredis/sds.c

@@ -89,9 +89,9 @@ sds sdsnewlen(const void *init, size_t initlen) {
     unsigned char *fp; /* flags pointer. */
 
     sh = s_malloc(hdrlen+initlen+1);
-    if (sh == NULL) return NULL;
     if (!init)
         memset(sh, 0, hdrlen+initlen+1);
+    if (sh == NULL) return NULL;
     s = (char*)sh+hdrlen;
     fp = ((unsigned char*)s)-1;
     switch(type) {
@@ -577,12 +577,14 @@ sds sdscatprintf(sds s, const char *fmt, ...) {
  * %% - Verbatim "%" character.
  */
 sds sdscatfmt(sds s, char const *fmt, ...) {
+    size_t initlen = sdslen(s);
     const char *f = fmt;
     int i;
     va_list ap;
 
     va_start(ap,fmt);
-    i = sdslen(s); /* Position of the next byte to write to dest str. */
+    f = fmt;    /* Next format specifier byte to process. */
+    i = initlen; /* Position of the next byte to write to dest str. */
     while(*f) {
         char next, *str;
         size_t l;

deps/hiredis/sds.h

@@ -79,7 +79,7 @@ struct __attribute__ ((__packed__)) sdshdr64 {
 #define SDS_TYPE_64 4
 #define SDS_TYPE_MASK 7
 #define SDS_TYPE_BITS 3
-#define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (struct sdshdr##T *)((s)-(sizeof(struct sdshdr##T)));
+#define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (void*)((s)-(sizeof(struct sdshdr##T)));
 #define SDS_HDR(T,s) ((struct sdshdr##T *)((s)-(sizeof(struct sdshdr##T))))
 #define SDS_TYPE_5_LEN(f) ((f)>>SDS_TYPE_BITS)
 

deps/hiredis/sdsalloc.h

@@ -1,7 +1,6 @@
 /* SDSLib 2.0 -- A C dynamic strings library
  *
  * Copyright (c) 2006-2015, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2015, Oran Agra
  * Copyright (c) 2015, Redis Labs, Inc
  * All rights reserved.
  *
@@ -37,6 +36,7 @@
  * the include of your alternate allocator if needed (not needed in order
  * to use the default libc allocator). */
 
-#define s_malloc malloc
-#define s_realloc realloc
-#define s_free free
+#include "zmalloc.h"
+#define s_malloc zmalloc
+#define s_realloc zrealloc
+#define s_free zfree

deps/hiredis/test.c

@@ -11,7 +11,6 @@
 #include <limits.h>
 
 #include "hiredis.h"
-#include "net.h"
 
 enum connection_type {
     CONN_TCP,
@@ -30,7 +29,7 @@ struct config {
 
     struct {
         const char *path;
-    } unix_sock;
+    } unix;
 };
 
 /* The following lines make up our testing "framework" :) */
@@ -44,13 +43,6 @@ static long long usec(void) {
     return (((long long)tv.tv_sec)*1000000)+tv.tv_usec;
 }
 
-/* The assert() calls below have side effects, so we need assert()
- * even if we are compiling without asserts (-DNDEBUG). */
-#ifdef NDEBUG
-#undef assert
-#define assert(e) (void)(e)
-#endif
-
 static redisContext *select_database(redisContext *c) {
     redisReply *reply;
 
@@ -59,7 +51,7 @@ static redisContext *select_database(redisContext *c) {
     assert(reply != NULL);
     freeReplyObject(reply);
 
-    /* Make sure the DB is emtpy */
+    /* Make sure the DB is empty */
     reply = redisCommand(c,"DBSIZE");
     assert(reply != NULL);
     if (reply->type == REDIS_REPLY_INTEGER && reply->integer == 0) {
@@ -97,10 +89,10 @@ static redisContext *connect(struct config config) {
     if (config.type == CONN_TCP) {
         c = redisConnect(config.tcp.host, config.tcp.port);
     } else if (config.type == CONN_UNIX) {
-        c = redisConnectUnix(config.unix_sock.path);
+        c = redisConnectUnix(config.unix.path);
     } else if (config.type == CONN_FD) {
         /* Create a dummy connection just to get an fd to inherit */
-        redisContext *dummy_ctx = redisConnectUnix(config.unix_sock.path);
+        redisContext *dummy_ctx = redisConnectUnix(config.unix.path);
         if (dummy_ctx) {
             int fd = disconnect(dummy_ctx, 1);
             printf("Connecting to inherited fd %d\n", fd);
@@ -115,7 +107,6 @@ static redisContext *connect(struct config config) {
         exit(1);
     } else if (c->err) {
         printf("Connection error: %s\n", c->errstr);
-        redisFree(c);
         exit(1);
     }
 
@@ -224,22 +215,6 @@ static void test_format_commands(void) {
     test_cond(strncmp(cmd,"*3\r\n$3\r\nSET\r\n$7\r\nfoo\0xxx\r\n$3\r\nbar\r\n",len) == 0 &&
         len == 4+4+(3+2)+4+(7+2)+4+(3+2));
     free(cmd);
-
-    sds sds_cmd;
-
-    sds_cmd = sdsempty();
-    test("Format command into sds by passing argc/argv without lengths: ");
-    len = redisFormatSdsCommandArgv(&sds_cmd,argc,argv,NULL);
-    test_cond(strncmp(sds_cmd,"*3\r\n$3\r\nSET\r\n$3\r\nfoo\r\n$3\r\nbar\r\n",len) == 0 &&
-        len == 4+4+(3+2)+4+(3+2)+4+(3+2));
-    sdsfree(sds_cmd);
-
-    sds_cmd = sdsempty();
-    test("Format command into sds by passing argc/argv with lengths: ");
-    len = redisFormatSdsCommandArgv(&sds_cmd,argc,argv,lens);
-    test_cond(strncmp(sds_cmd,"*3\r\n$3\r\nSET\r\n$7\r\nfoo\0xxx\r\n$3\r\nbar\r\n",len) == 0 &&
-        len == 4+4+(3+2)+4+(7+2)+4+(3+2));
-    sdsfree(sds_cmd);
 }
 
 static void test_append_formatted_commands(struct config config) {
@@ -343,31 +318,16 @@ static void test_reply_reader(void) {
     redisReaderFree(reader);
 }
 
-static void test_free_null(void) {
-    void *redisCtx = NULL;
-    void *reply = NULL;
-
-    test("Don't fail when redisFree is passed a NULL value: ");
-    redisFree(redisCtx);
-    test_cond(redisCtx == NULL);
-
-    test("Don't fail when freeReplyObject is passed a NULL value: ");
-    freeReplyObject(reply);
-    test_cond(reply == NULL);
-}
-
 static void test_blocking_connection_errors(void) {
     redisContext *c;
 
     test("Returns error when host cannot be resolved: ");
-    c = redisConnect((char*)"idontexist.test", 6379);
+    c = redisConnect((char*)"idontexist.local", 6379);
     test_cond(c->err == REDIS_ERR_OTHER &&
         (strcmp(c->errstr,"Name or service not known") == 0 ||
-         strcmp(c->errstr,"Can't resolve: idontexist.test") == 0 ||
+         strcmp(c->errstr,"Can't resolve: idontexist.local") == 0 ||
          strcmp(c->errstr,"nodename nor servname provided, or not known") == 0 ||
          strcmp(c->errstr,"No address associated with hostname") == 0 ||
-         strcmp(c->errstr,"Temporary failure in name resolution") == 0 ||
-         strcmp(c->errstr,"hostname nor servname provided, or not known") == 0 ||
          strcmp(c->errstr,"no address associated with name") == 0));
     redisFree(c);
 
@@ -377,7 +337,7 @@ static void test_blocking_connection_errors(void) {
         strcmp(c->errstr,"Connection refused") == 0);
     redisFree(c);
 
-    test("Returns error when the unix_sock socket path doesn't accept connections: ");
+    test("Returns error when the unix socket path doesn't accept connections: ");
     c = redisConnectUnix((char*)"/tmp/idontexist.sock");
     test_cond(c->err == REDIS_ERR_IO); /* Don't care about the message... */
     redisFree(c);
@@ -461,52 +421,6 @@ static void test_blocking_connection(struct config config) {
     disconnect(c, 0);
 }
 
-static void test_blocking_connection_timeouts(struct config config) {
-    redisContext *c;
-    redisReply *reply;
-    ssize_t s;
-    const char *cmd = "DEBUG SLEEP 3\r\n";
-    struct timeval tv;
-
-    c = connect(config);
-    test("Successfully completes a command when the timeout is not exceeded: ");
-    reply = redisCommand(c,"SET foo fast");
-    freeReplyObject(reply);
-    tv.tv_sec = 0;
-    tv.tv_usec = 10000;
-    redisSetTimeout(c, tv);
-    reply = redisCommand(c, "GET foo");
-    test_cond(reply != NULL && reply->type == REDIS_REPLY_STRING && memcmp(reply->str, "fast", 4) == 0);
-    freeReplyObject(reply);
-    disconnect(c, 0);
-
-    c = connect(config);
-    test("Does not return a reply when the command times out: ");
-    s = write(c->fd, cmd, strlen(cmd));
-    tv.tv_sec = 0;
-    tv.tv_usec = 10000;
-    redisSetTimeout(c, tv);
-    reply = redisCommand(c, "GET foo");
-    test_cond(s > 0 && reply == NULL && c->err == REDIS_ERR_IO && strcmp(c->errstr, "Resource temporarily unavailable") == 0);
-    freeReplyObject(reply);
-
-    test("Reconnect properly reconnects after a timeout: ");
-    redisReconnect(c);
-    reply = redisCommand(c, "PING");
-    test_cond(reply != NULL && reply->type == REDIS_REPLY_STATUS && strcmp(reply->str, "PONG") == 0);
-    freeReplyObject(reply);
-
-    test("Reconnect properly uses owned parameters: ");
-    config.tcp.host = "foo";
-    config.unix_sock.path = "foo";
-    redisReconnect(c);
-    reply = redisCommand(c, "PING");
-    test_cond(reply != NULL && reply->type == REDIS_REPLY_STATUS && strcmp(reply->str, "PONG") == 0);
-    freeReplyObject(reply);
-
-    disconnect(c, 0);
-}
-
 static void test_blocking_io_errors(struct config config) {
     redisContext *c;
     redisReply *reply;
@@ -530,7 +444,7 @@ static void test_blocking_io_errors(struct config config) {
 
     test("Returns I/O error when the connection is lost: ");
     reply = redisCommand(c,"QUIT");
-    if (major > 2 || (major == 2 && minor > 0)) {
+    if (major >= 2 && minor > 0) {
         /* > 2.0 returns OK on QUIT and read() should be issued once more
          * to know the descriptor is at EOF. */
         test_cond(strcasecmp(reply->str,"OK") == 0 &&
@@ -568,8 +482,7 @@ static void test_invalid_timeout_errors(struct config config) {
 
     c = redisConnectWithTimeout(config.tcp.host, config.tcp.port, config.tcp.timeout);
 
-    test_cond(c->err == REDIS_ERR_IO && strcmp(c->errstr, "Invalid timeout specified") == 0);
-    redisFree(c);
+    test_cond(c->err == REDIS_ERR_IO);
 
     test("Set error when an invalid timeout sec value is given to redisConnectWithTimeout: ");
 
@@ -578,7 +491,8 @@ static void test_invalid_timeout_errors(struct config config) {
 
     c = redisConnectWithTimeout(config.tcp.host, config.tcp.port, config.tcp.timeout);
 
-    test_cond(c->err == REDIS_ERR_IO && strcmp(c->errstr, "Invalid timeout specified") == 0);
+    test_cond(c->err == REDIS_ERR_IO);
+
     redisFree(c);
 }
 
@@ -752,7 +666,7 @@ int main(int argc, char **argv) {
             .host = "127.0.0.1",
             .port = 6379
         },
-        .unix_sock = {
+        .unix = {
             .path = "/tmp/redis.sock"
         }
     };
@@ -773,7 +687,7 @@ int main(int argc, char **argv) {
             cfg.tcp.port = atoi(argv[0]);
         } else if (argc >= 2 && !strcmp(argv[0],"-s")) {
             argv++; argc--;
-            cfg.unix_sock.path = argv[0];
+            cfg.unix.path = argv[0];
         } else if (argc >= 1 && !strcmp(argv[0],"--skip-throughput")) {
             throughput = 0;
         } else if (argc >= 1 && !strcmp(argv[0],"--skip-inherit-fd")) {
@@ -788,31 +702,27 @@ int main(int argc, char **argv) {
     test_format_commands();
     test_reply_reader();
     test_blocking_connection_errors();
-    test_free_null();
 
     printf("\nTesting against TCP connection (%s:%d):\n", cfg.tcp.host, cfg.tcp.port);
     cfg.type = CONN_TCP;
     test_blocking_connection(cfg);
-    test_blocking_connection_timeouts(cfg);
     test_blocking_io_errors(cfg);
     test_invalid_timeout_errors(cfg);
     test_append_formatted_commands(cfg);
     if (throughput) test_throughput(cfg);
 
-    printf("\nTesting against Unix socket connection (%s):\n", cfg.unix_sock.path);
+    printf("\nTesting against Unix socket connection (%s):\n", cfg.unix.path);
     cfg.type = CONN_UNIX;
     test_blocking_connection(cfg);
-    test_blocking_connection_timeouts(cfg);
     test_blocking_io_errors(cfg);
     if (throughput) test_throughput(cfg);
 
     if (test_inherit_fd) {
-        printf("\nTesting against inherited fd (%s):\n", cfg.unix_sock.path);
+        printf("\nTesting against inherited fd (%s):\n", cfg.unix.path);
         cfg.type = CONN_FD;
         test_blocking_connection(cfg);
     }
 
-
     if (fails) {
         printf("*** %d TESTS FAILED ***\n", fails);
         return 1;

deps/hiredis/win32.h

@@ -1,42 +0,0 @@
-#ifndef _WIN32_HELPER_INCLUDE
-#define _WIN32_HELPER_INCLUDE
-#ifdef _MSC_VER
-
-#ifndef inline
-#define inline __inline
-#endif
-
-#ifndef va_copy
-#define va_copy(d,s) ((d) = (s))
-#endif
-
-#ifndef snprintf
-#define snprintf c99_snprintf
-
-__inline int c99_vsnprintf(char* str, size_t size, const char* format, va_list ap)
-{
-    int count = -1;
-
-    if (size != 0)
-        count = _vsnprintf_s(str, size, _TRUNCATE, format, ap);
-    if (count == -1)
-        count = _vscprintf(format, ap);
-
-    return count;
-}
-
-__inline int c99_snprintf(char* str, size_t size, const char* format, ...)
-{
-    int count;
-    va_list ap;
-
-    va_start(ap, format);
-    count = c99_vsnprintf(str, size, format, ap);
-    va_end(ap);
-
-    return count;
-}
-#endif
-
-#endif
-#endif

deps/hiredis/zmalloc.h

@@ -0,0 +1,13 @@
+/* Drop in replacement for zmalloc.h in order to just use libc malloc without
+ * any wrappering. */
+
+#ifndef ZMALLOC_H
+#define ZMALLOC_H
+
+#define zmalloc malloc
+#define zrealloc realloc
+#define zcalloc(x) calloc(x,1)
+#define zfree free
+#define zstrdup strdup
+
+#endif

deps/jemalloc/include/jemalloc/jemalloc_macros.h.in

@@ -100,7 +100,3 @@
 #  define JEMALLOC_RESTRICT_RETURN
 #  define JEMALLOC_ALLOCATOR
 #endif
-
-/* This version of Jemalloc, modified for Redis, has the je_get_defrag_hint()
- * function. */
-#define JEMALLOC_FRAG_HINT

deps/jemalloc/src/jemalloc.c

@@ -2591,35 +2591,3 @@ jemalloc_postfork_child(void)
 }
 
 /******************************************************************************/
-
-/* Helps the application decide if a pointer is worth re-allocating in order to reduce fragmentation.
- * returns 0 if the allocation is in the currently active run,
- * or when it is not causing any frag issue (large or huge bin)
- * returns the bin utilization and run utilization both in fixed point 16:16.
- * If the application decides to re-allocate it should use MALLOCX_TCACHE_NONE when doing so. */
-JEMALLOC_EXPORT int JEMALLOC_NOTHROW
-je_get_defrag_hint(void* ptr, int *bin_util, int *run_util) {
-    int defrag = 0;
-    arena_chunk_t *chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
-    if (likely(chunk != ptr)) { /* indication that this is not a HUGE alloc */
-        size_t pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> LG_PAGE;
-        size_t mapbits = arena_mapbits_get(chunk, pageind);
-        if (likely((mapbits & CHUNK_MAP_LARGE) == 0)) { /* indication that this is not a LARGE alloc */
-            arena_t *arena = extent_node_arena_get(&chunk->node);
-            size_t rpages_ind = pageind - arena_mapbits_small_runind_get(chunk, pageind);
-            arena_run_t *run = &arena_miscelm_get(chunk, rpages_ind)->run;
-            arena_bin_t *bin = &arena->bins[run->binind];
-            malloc_mutex_lock(&bin->lock);
-            /* runs that are in the same chunk in as the current chunk, are likely to be the next currun */
-            if (chunk != (arena_chunk_t *)CHUNK_ADDR2BASE(bin->runcur)) {
-                arena_bin_info_t *bin_info = &arena_bin_info[run->binind];
-                size_t availregs = bin_info->nregs * bin->stats.curruns;
-                *bin_util = (bin->stats.curregs<<16) / availregs;
-                *run_util = ((bin_info->nregs - run->nfree)<<16) / bin_info->nregs;
-                defrag = 1;
-            }
-            malloc_mutex_unlock(&bin->lock);
-        }
-    }
-    return defrag;
-}

redis.conf

@@ -35,14 +35,6 @@
 # include /path/to/local.conf
 # include /path/to/other.conf
 
-################################## MODULES #####################################
-
-# Load modules at startup. If the server is not able to load modules
-# it will abort. It is possible to use multiple loadmodule directives.
-#
-# loadmodule /path/to/my_module.so
-# loadmodule /path/to/other_module.so
-
 ################################## NETWORK #####################################
 
 # By default, if no "bind" configuration directive is specified, Redis listens
@@ -185,14 +177,6 @@ logfile ""
 # dbid is a number between 0 and 'databases'-1
 databases 16
 
-# By default Redis shows an ASCII art logo only when started to log to the
-# standard output and if the standard output is a TTY. Basically this means
-# that normally a logo is displayed only in interactive sessions.
-#
-# However it is possible to force the pre-4.0 behavior and always show a
-# ASCII art logo in startup logs by setting the following option to yes.
-always-show-logo yes
-
 ################################ SNAPSHOTTING  ################################
 #
 # Save the DB on disk:
@@ -410,10 +394,6 @@ repl-disable-tcp-nodelay no
 # need to elapse, starting from the time the last slave disconnected, for
 # the backlog buffer to be freed.
 #
-# Note that slaves never free the backlog for timeout, since they may be
-# promoted to masters later, and should be able to correctly "partially
-# resynchronize" with the slaves: hence they should always accumulate backlog.
-#
 # A value of 0 means to never release the backlog.
 #
 # repl-backlog-ttl 3600
@@ -460,7 +440,7 @@ slave-priority 100
 # offers this information, which is used, among other tools, by
 # Redis Sentinel in order to discover slave instances.
 # Another place where this info is available is in the output of the
-# "ROLE" command of a master.
+# "ROLE" command of a masteer.
 #
 # The listed IP and address normally reported by a slave is obtained
 # in the following way:
@@ -518,7 +498,7 @@ slave-priority 100
 # Please note that changing the name of commands that are logged into the
 # AOF file or transmitted to slaves may cause problems.
 
-################################### CLIENTS ####################################
+################################### LIMITS ####################################
 
 # Set the max number of connected clients at the same time. By default
 # this limit is set to 10000 clients, however if the Redis server is not
@@ -531,9 +511,7 @@ slave-priority 100
 #
 # maxclients 10000
 
-############################## MEMORY MANAGEMENT ################################
-
-# Set a memory usage limit to the specified amount of bytes.
+# Don't use more memory than the specified amount of bytes.
 # When the memory limit is reached Redis will try to remove keys
 # according to the eviction policy selected (see maxmemory-policy).
 #
@@ -542,8 +520,8 @@ slave-priority 100
 # that would use more memory, like SET, LPUSH, and so on, and will continue
 # to reply to read-only commands like GET.
 #
-# This option is usually useful when using Redis as an LRU or LFU cache, or to
-# set a hard memory limit for an instance (using the 'noeviction' policy).
+# This option is usually useful when using Redis as an LRU cache, or to set
+# a hard memory limit for an instance (using the 'noeviction' policy).
 #
 # WARNING: If you have slaves attached to an instance with maxmemory on,
 # the size of the output buffers needed to feed the slaves are subtracted
@@ -561,20 +539,12 @@ slave-priority 100
 # MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
 # is reached. You can select among five behaviors:
 #
-# volatile-lru -> Evict using approximated LRU among the keys with an expire set.
-# allkeys-lru -> Evict any key using approximated LRU.
-# volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
-# allkeys-lfu -> Evict any key using approximated LFU.
-# volatile-random -> Remove a random key among the ones with an expire set.
-# allkeys-random -> Remove a random key, any key.
-# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
-# noeviction -> Don't evict anything, just return an error on write operations.
-#
-# LRU means Least Recently Used
-# LFU means Least Frequently Used
-#
-# Both LRU, LFU and volatile-ttl are implemented using approximated
-# randomized algorithms.
+# volatile-lru -> remove the key with an expire set using an LRU algorithm
+# allkeys-lru -> remove any key according to the LRU algorithm
+# volatile-random -> remove a random key with an expire set
+# allkeys-random -> remove a random key, any key
+# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
+# noeviction -> don't expire at all, just return an error on write operations
 #
 # Note: with any of the above policies, Redis will return an error on write
 #       operations, when there are no suitable keys for eviction.
@@ -589,66 +559,17 @@ slave-priority 100
 #
 # maxmemory-policy noeviction
 
-# LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated
+# LRU and minimal TTL algorithms are not precise algorithms but approximated
 # algorithms (in order to save memory), so you can tune it for speed or
 # accuracy. For default Redis will check five keys and pick the one that was
 # used less recently, you can change the sample size using the following
 # configuration directive.
 #
 # The default of 5 produces good enough results. 10 Approximates very closely
-# true LRU but costs more CPU. 3 is faster but not very accurate.
+# true LRU but costs a bit more CPU. 3 is very fast but not very accurate.
 #
 # maxmemory-samples 5
 
-############################# LAZY FREEING ####################################
-
-# Redis has two primitives to delete keys. One is called DEL and is a blocking
-# deletion of the object. It means that the server stops processing new commands
-# in order to reclaim all the memory associated with an object in a synchronous
-# way. If the key deleted is associated with a small object, the time needed
-# in order to execute th DEL command is very small and comparable to most other
-# O(1) or O(log_N) commands in Redis. However if the key is associated with an
-# aggregated value containing millions of elements, the server can block for
-# a long time (even seconds) in order to complete the operation.
-#
-# For the above reasons Redis also offers non blocking deletion primitives
-# such as UNLINK (non blocking DEL) and the ASYNC option of FLUSHALL and
-# FLUSHDB commands, in order to reclaim memory in background. Those commands
-# are executed in constant time. Another thread will incrementally free the
-# object in the background as fast as possible.
-#
-# DEL, UNLINK and ASYNC option of FLUSHALL and FLUSHDB are user-controlled.
-# It's up to the design of the application to understand when it is a good
-# idea to use one or the other. However the Redis server sometimes has to
-# delete keys or flush the whole database as a side effect of other operations.
-# Specifically Redis deletes objects independently of an user call in the
-# following scenarios:
-#
-# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
-#    in order to make room for new data, without going over the specified
-#    memory limit.
-# 2) Because of expire: when a key with an associated time to live (see the
-#    EXPIRE command) must be deleted from memory.
-# 3) Because of a side effect of a command that stores data on a key that may
-#    already exist. For example the RENAME command may delete the old key
-#    content when it is replaced with another one. Similarly SUNIONSTORE
-#    or SORT with STORE option may delete existing keys. The SET command
-#    itself removes any old content of the specified key in order to replace
-#    it with the specified string.
-# 4) During replication, when a slave performs a full resynchronization with
-#    its master, the content of the whole database is removed in order to
-#    load the RDB file just transfered.
-#
-# In all the above cases the default is to delete objects in a blocking way,
-# like if DEL was called. However you can configure each case specifically
-# in order to instead release memory in a non-blocking way like if UNLINK
-# was called, using the following configuration directives:
-
-lazyfree-lazy-eviction no
-lazyfree-lazy-expire no
-lazyfree-lazy-server-del no
-slave-lazy-flush no
-
 ############################## APPEND ONLY MODE ###############################
 
 # By default Redis asynchronously dumps the dataset on disk. This mode is
@@ -767,20 +688,6 @@ auto-aof-rewrite-min-size 64mb
 # will be found.
 aof-load-truncated yes
 
-# When rewriting the AOF file, Redis is able to use an RDB preamble in the
-# AOF file for faster rewrites and recoveries. When this option is turned
-# on the rewritten AOF file is composed of two different stanzas:
-#
-#   [RDB file][AOF tail]
-#
-# When loading Redis recognizes that the AOF file starts with the "REDIS"
-# string and loads the prefixed RDB file, and continues loading the AOF
-# tail.
-#
-# This is currently turned off by default in order to avoid the surprise
-# of a format change, but will at some point be used as the default.
-aof-use-rdb-preamble no
-
 ################################ LUA SCRIPTING  ###############################
 
 # Max execution time of a Lua script in milliseconds.
@@ -830,7 +737,7 @@ lua-time-limit 5000
 # A slave of a failing master will avoid to start a failover if its data
 # looks too old.
 #
-# There is no simple way for a slave to actually have an exact measure of
+# There is no simple way for a slave to actually have a exact measure of
 # its "data age", so the following two checks are performed:
 #
 # 1) If there are multiple slaves able to failover, they exchange messages
@@ -907,39 +814,6 @@ lua-time-limit 5000
 # In order to setup your cluster make sure to read the documentation
 # available at http://redis.io web site.
 
-########################## CLUSTER DOCKER/NAT support  ########################
-
-# In certain deployments, Redis Cluster nodes address discovery fails, because
-# addresses are NAT-ted or because ports are forwarded (the typical case is
-# Docker and other containers).
-#
-# In order to make Redis Cluster working in such environments, a static
-# configuration where each node known its public address is needed. The
-# following two options are used for this scope, and are:
-#
-# * cluster-announce-ip
-# * cluster-announce-port
-# * cluster-announce-bus-port
-#
-# Each instruct the node about its address, client port, and cluster message
-# bus port. The information is then published in the header of the bus packets
-# so that other nodes will be able to correctly map the address of the node
-# publishing the information.
-#
-# If the above options are not used, the normal Redis Cluster auto-detection
-# will be used instead.
-#
-# Note that when remapped, the bus port may not be at the fixed offset of
-# clients port + 10000, so you can specify any port and bus-port depending
-# on how they get remapped. If the bus-port is not set, a fixed offset of
-# 10000 will be used as usually.
-#
-# Example:
-#
-# cluster-announce-ip 10.1.1.5
-# cluster-announce-port 6379
-# cluster-announce-bus-port 6380
-
 ################################## SLOW LOG ###################################
 
 # The Redis Slow Log is a system to log queries that exceeded a specified
@@ -1176,118 +1050,3 @@ hz 10
 # in order to commit the file to the disk more incrementally and avoid
 # big latency spikes.
 aof-rewrite-incremental-fsync yes
-
-# Redis LFU eviction (see maxmemory setting) can be tuned. However it is a good
-# idea to start with the default settings and only change them after investigating
-# how to improve the performances and how the keys LFU change over time, which
-# is possible to inspect via the OBJECT FREQ command.
-#
-# There are two tunable parameters in the Redis LFU implementation: the
-# counter logarithm factor and the counter decay time. It is important to
-# understand what the two parameters mean before changing them.
-#
-# The LFU counter is just 8 bits per key, it's maximum value is 255, so Redis
-# uses a probabilistic increment with logarithmic behavior. Given the value
-# of the old counter, when a key is accessed, the counter is incremented in
-# this way:
-#
-# 1. A random number R between 0 and 1 is extracted.
-# 2. A probability P is calculated as 1/(old_value*lfu_log_factor+1).
-# 3. The counter is incremented only if R < P.
-#
-# The default lfu-log-factor is 10. This is a table of how the frequency
-# counter changes with a different number of accesses with different
-# logarithmic factors:
-#
-# +--------+------------+------------+------------+------------+------------+
-# | factor | 100 hits   | 1000 hits  | 100K hits  | 1M hits    | 10M hits   |
-# +--------+------------+------------+------------+------------+------------+
-# | 0      | 104        | 255        | 255        | 255        | 255        |
-# +--------+------------+------------+------------+------------+------------+
-# | 1      | 18         | 49         | 255        | 255        | 255        |
-# +--------+------------+------------+------------+------------+------------+
-# | 10     | 10         | 18         | 142        | 255        | 255        |
-# +--------+------------+------------+------------+------------+------------+
-# | 100    | 8          | 11         | 49         | 143        | 255        |
-# +--------+------------+------------+------------+------------+------------+
-#
-# NOTE: The above table was obtained by running the following commands:
-#
-#   redis-benchmark -n 1000000 incr foo
-#   redis-cli object freq foo
-#
-# NOTE 2: The counter initial value is 5 in order to give new objects a chance
-# to accumulate hits.
-#
-# The counter decay time is the time, in minutes, that must elapse in order
-# for the key counter to be divided by two (or decremented if it has a value
-# less <= 10).
-#
-# The default value for the lfu-decay-time is 1. A Special value of 0 means to
-# decay the counter every time it happens to be scanned.
-#
-# lfu-log-factor 10
-# lfu-decay-time 1
-
-########################### ACTIVE DEFRAGMENTATION #######################
-#
-# WARNING THIS FEATURE IS EXPERIMENTAL. However it was stress tested
-# even in production and manually tested by multiple engineers for some
-# time.
-#
-# What is active defragmentation?
-# -------------------------------
-#
-# Active (online) defragmentation allows a Redis server to compact the
-# spaces left between small allocations and deallocations of data in memory,
-# thus allowing to reclaim back memory.
-#
-# Fragmentation is a natural process that happens with every allocator (but
-# less so with Jemalloc, fortunately) and certain workloads. Normally a server
-# restart is needed in order to lower the fragmentation, or at least to flush
-# away all the data and create it again. However thanks to this feature
-# implemented by Oran Agra for Redis 4.0 this process can happen at runtime
-# in an "hot" way, while the server is running.
-#
-# Basically when the fragmentation is over a certain level (see the
-# configuration options below) Redis will start to create new copies of the
-# values in contiguous memory regions by exploiting certain specific Jemalloc
-# features (in order to understand if an allocation is causing fragmentation
-# and to allocate it in a better place), and at the same time, will release the
-# old copies of the data. This process, repeated incrementally for all the keys
-# will cause the fragmentation to drop back to normal values.
-#
-# Important things to understand:
-#
-# 1. This feature is disabled by default, and only works if you compiled Redis
-#    to use the copy of Jemalloc we ship with the source code of Redis.
-#    This is the default with Linux builds.
-#
-# 2. You never need to enable this feature if you don't have fragmentation
-#    issues.
-#
-# 3. Once you experience fragmentation, you can enable this feature when
-#    needed with the command "CONFIG SET activedefrag yes".
-#
-# The configuration parameters are able to fine tune the behavior of the
-# defragmentation process. If you are not sure about what they mean it is
-# a good idea to leave the defaults untouched.
-
-# Enabled active defragmentation
-# activedefrag yes
-
-# Minimum amount of fragmentation waste to start active defrag
-# active-defrag-ignore-bytes 100mb
-
-# Minimum percentage of fragmentation to start active defrag
-# active-defrag-threshold-lower 10
-
-# Maximum percentage of fragmentation at which we use maximum effort
-# active-defrag-threshold-upper 100
-
-# Minimal effort for defrag in CPU percentage
-# active-defrag-cycle-min 25
-
-# Maximal effort for defrag in CPU percentage
-# active-defrag-cycle-max 75
-

src/Makefile

@@ -14,33 +14,23 @@
 
 release_hdr := $(shell sh -c './mkreleasehdr.sh')
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
-uname_M := $(shell sh -c 'uname -m 2>/dev/null || echo not')
 OPTIMIZATION?=-O2
-DEPENDENCY_TARGETS=hiredis linenoise lua
-NODEPS:=clean distclean
+DEPENDENCY_TARGETS=hiredis linenoise lua geohash-int
 
 # Default settings
 STD=-std=c99 -pedantic -DREDIS_STATIC=''
-WARN=-Wall -W -Wno-missing-field-initializers
+WARN=-Wall -W
 OPT=$(OPTIMIZATION)
 
 PREFIX?=/usr/local
 INSTALL_BIN=$(PREFIX)/bin
 INSTALL=install
 
-# Default allocator defaults to Jemalloc if it's not an ARM
-MALLOC=libc
-ifneq ($(uname_M),armv6l)
-ifneq ($(uname_M),armv7l)
+# Default allocator
 ifeq ($(uname_S),Linux)
 	MALLOC=jemalloc
-endif
-endif
-endif
-
-# To get ARM stack traces if Redis crashes we need a special C flag.
-ifneq (,$(findstring armv,$(uname_M)))
-        CFLAGS+=-funwind-tables
+else
+	MALLOC=libc
 endif
 
 # Backwards compatibility for selecting an allocator
@@ -63,20 +53,13 @@ endif
 # Override default settings if possible
 -include .make-settings
 
-FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
+FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS) -I../deps/geohash-int
 FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
 FINAL_LIBS=-lm
 DEBUG=-g -ggdb
 
 ifeq ($(uname_S),SunOS)
 	# SunOS
-        ifneq ($(@@),32bit)
-		CFLAGS+= -m64
-		LDFLAGS+= -m64
-	endif
-	DEBUG=-g
-	DEBUG_FLAGS=-g
-	export CFLAGS LDFLAGS DEBUG DEBUG_FLAGS
 	INSTALL=cp -pf
 	FINAL_CFLAGS+= -D__EXTENSIONS__ -D_XPG6
 	FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
@@ -144,28 +127,31 @@ endif
 
 REDIS_SERVER_NAME=redis-server
 REDIS_SENTINEL_NAME=redis-sentinel
-REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o
+REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o geo.o
+REDIS_GEOHASH_OBJ=../deps/geohash-int/geohash.o ../deps/geohash-int/geohash_helper.o
 REDIS_CLI_NAME=redis-cli
 REDIS_CLI_OBJ=anet.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o
 REDIS_BENCHMARK_NAME=redis-benchmark
 REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o zmalloc.o redis-benchmark.o
 REDIS_CHECK_RDB_NAME=redis-check-rdb
 REDIS_CHECK_AOF_NAME=redis-check-aof
+REDIS_CHECK_AOF_OBJ=redis-check-aof.o
 
 all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
 	@echo ""
 	@echo "Hint: It's a good idea to run 'make test' ;)"
 	@echo ""
 
-Makefile.dep:
-	-$(REDIS_CC) -MM *.c > Makefile.dep 2> /dev/null || true
-
-ifeq (0, $(words $(findstring $(MAKECMDGOALS), $(NODEPS))))
--include Makefile.dep
-endif
-
 .PHONY: all
 
+# Deps (use make dep to generate this)
+include Makefile.dep
+
+dep:
+	$(REDIS_CC) -MM *.c > Makefile.dep
+
+.PHONY: dep
+
 persist-settings: distclean
 	echo STD=$(STD) >> .make-settings
 	echo WARN=$(WARN) >> .make-settings
@@ -196,7 +182,7 @@ endif
 
 # redis-server
 $(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
-	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(FINAL_LIBS)
+	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(REDIS_GEOHASH_OBJ) $(FINAL_LIBS)
 
 # redis-sentinel
 $(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)
@@ -206,10 +192,6 @@ $(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)
 $(REDIS_CHECK_RDB_NAME): $(REDIS_SERVER_NAME)
 	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_RDB_NAME)
 
-# redis-check-aof
-$(REDIS_CHECK_AOF_NAME): $(REDIS_SERVER_NAME)
-	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME)
-
 # redis-cli
 $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
 	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(FINAL_LIBS)
@@ -218,8 +200,9 @@ $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
 $(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
 	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS)
 
-dict-benchmark: dict.c zmalloc.c sds.c siphash.c
-	$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS)
+# redis-check-aof
+$(REDIS_CHECK_AOF_NAME): $(REDIS_CHECK_AOF_OBJ)
+	$(REDIS_LD) -o $@ $^ $(FINAL_LIBS)
 
 # Because the jemalloc.h header is generated as a part of the jemalloc build,
 # building it should complete before building any other object. Instead of
@@ -228,7 +211,7 @@ dict-benchmark: dict.c zmalloc.c sds.c siphash.c
 	$(REDIS_CC) -c $<
 
 clean:
-	rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep dict-benchmark
+	rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html
 
 .PHONY: clean
 
@@ -253,7 +236,7 @@ lcov:
 	@genhtml --legend -o lcov-html redis.info
 
 test-sds: sds.c sds.h
-	$(REDIS_CC) sds.c zmalloc.c -DSDS_TEST_MAIN $(FINAL_LIBS) -o /tmp/sds_test
+	$(REDIS_CC) sds.c zmalloc.c -DSDS_TEST_MAIN -o /tmp/sds_test
 	/tmp/sds_test
 
 .PHONY: lcov
@@ -276,9 +259,6 @@ noopt:
 valgrind:
 	$(MAKE) OPTIMIZATION="-O0" MALLOC="libc"
 
-helgrind:
-	$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" CFLAGS="-D__ATOMIC_VAR_FORCE_SYNC_MACROS"
-
 src/help.h:
 	@../utils/generate-command-help.rb > help.h
 

src/Makefile.dep

@@ -0,0 +1,181 @@
+adlist.o: adlist.c adlist.h zmalloc.h
+ae.o: ae.c ae.h zmalloc.h config.h ae_kqueue.c ae_epoll.c ae_select.c ae_evport.c
+ae_epoll.o: ae_epoll.c
+ae_evport.o: ae_evport.c
+ae_kqueue.o: ae_kqueue.c
+ae_select.o: ae_select.c
+anet.o: anet.c fmacros.h anet.h
+aof.o: aof.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ bio.h
+bio.o: bio.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ bio.h
+bitops.o: bitops.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+blocked.o: blocked.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+cluster.o: cluster.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ cluster.h
+config.o: config.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ cluster.h
+crc16.o: crc16.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+crc64.o: crc64.c
+db.o: db.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ cluster.h
+debug.o: debug.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ bio.h
+dict.o: dict.c fmacros.h dict.h zmalloc.h redisassert.h
+endianconv.o: endianconv.c
+geo.o: geo.c geo.h server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ ../deps/geohash-int/geohash_helper.h ../deps/geohash-int/geohash.h
+hyperloglog.o: hyperloglog.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+intset.o: intset.c intset.h zmalloc.h endianconv.h config.h
+latency.o: latency.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+lzf_c.o: lzf_c.c lzfP.h
+lzf_d.o: lzf_d.c lzfP.h
+memtest.o: memtest.c config.h
+multi.o: multi.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+networking.o: networking.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+notify.o: notify.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+object.o: object.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+pqsort.o: pqsort.c
+pubsub.o: pubsub.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+quicklist.o: quicklist.c quicklist.h zmalloc.h ziplist.h util.h sds.h \
+ lzf.h
+rand.o: rand.c
+rdb.o: rdb.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ lzf.h
+redis-benchmark.o: redis-benchmark.c fmacros.h ../deps/hiredis/sds.h ae.h \
+ ../deps/hiredis/hiredis.h adlist.h zmalloc.h
+redis-check-aof.o: redis-check-aof.c fmacros.h config.h
+redis-check-rdb.o: redis-check-rdb.c server.h fmacros.h config.h \
+ solarisfixes.h ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h \
+ sds.h dict.h adlist.h zmalloc.h anet.h ziplist.h intset.h version.h \
+ util.h latency.h sparkline.h quicklist.h zipmap.h sha1.h endianconv.h \
+ crc64.h rdb.h rio.h lzf.h
+redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
+ ../deps/hiredis/sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h \
+ anet.h ae.h
+release.o: release.c release.h version.h crc64.h
+replication.o: replication.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+rio.o: rio.c fmacros.h rio.h sds.h util.h crc64.h config.h server.h \
+ solarisfixes.h ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h \
+ dict.h adlist.h zmalloc.h anet.h ziplist.h intset.h version.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h rdb.h
+scripting.o: scripting.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ rand.h cluster.h ../deps/lua/src/lauxlib.h ../deps/lua/src/lua.h \
+ ../deps/lua/src/lualib.h
+sds.o: sds.c sds.h sdsalloc.h zmalloc.h
+sentinel.o: sentinel.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ ../deps/hiredis/hiredis.h ../deps/hiredis/async.h \
+ ../deps/hiredis/hiredis.h
+server.o: server.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ cluster.h slowlog.h bio.h asciilogo.h
+setproctitle.o: setproctitle.c
+sha1.o: sha1.c solarisfixes.h sha1.h config.h
+slowlog.o: slowlog.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ slowlog.h
+sort.o: sort.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h \
+ pqsort.h
+sparkline.o: sparkline.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+syncio.o: syncio.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+t_hash.o: t_hash.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+t_list.o: t_list.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+t_set.o: t_set.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+t_string.o: t_string.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+t_zset.o: t_zset.c server.h fmacros.h config.h solarisfixes.h \
+ ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+ adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h latency.h \
+ sparkline.h quicklist.h zipmap.h sha1.h endianconv.h crc64.h rdb.h rio.h
+util.o: util.c fmacros.h util.h sds.h sha1.h
+ziplist.o: ziplist.c zmalloc.h util.h sds.h ziplist.h endianconv.h \
+ config.h redisassert.h
+zipmap.o: zipmap.c zmalloc.h endianconv.h config.h
+zmalloc.o: zmalloc.c config.h zmalloc.h

src/adlist.c

@@ -52,8 +52,10 @@ list *listCreate(void)
     return list;
 }
 
-/* Remove all the elements from the list without destroying the list itself. */
-void listEmpty(list *list)
+/* Free the whole list.
+ *
+ * This function can't fail. */
+void listRelease(list *list)
 {
     unsigned long len;
     listNode *current, *next;
@@ -66,16 +68,6 @@ void listEmpty(list *list)
         zfree(current);
         current = next;
     }
-    list->head = list->tail = NULL;
-    list->len = 0;
-}
-
-/* Free the whole list.
- *
- * This function can't fail. */
-void listRelease(list *list)
-{
-    listEmpty(list);
     zfree(list);
 }
 
@@ -341,22 +333,3 @@ void listRotate(list *list) {
     tail->next = list->head;
     list->head = tail;
 }
-
-/* Add all the elements of the list 'o' at the end of the
- * list 'l'. The list 'other' remains empty but otherwise valid. */
-void listJoin(list *l, list *o) {
-    if (o->head)
-        o->head->prev = l->tail;
-
-    if (l->tail)
-        l->tail->next = o->head;
-    else
-        l->head = o->head;
-
-    l->tail = o->tail;
-    l->len += o->len;
-
-    /* Setup other as an empty list. */
-    o->head = o->tail = NULL;
-    o->len = 0;
-}

src/adlist.h

@@ -72,7 +72,6 @@ typedef struct list {
 /* Prototypes */
 list *listCreate(void);
 void listRelease(list *list);
-void listEmpty(list *list);
 list *listAddNodeHead(list *list, void *value);
 list *listAddNodeTail(list *list, void *value);
 list *listInsertNode(list *list, listNode *old_node, void *value, int after);
@@ -86,7 +85,6 @@ listNode *listIndex(list *list, long index);
 void listRewind(list *list, listIter *li);
 void listRewindTail(list *list, listIter *li);
 void listRotate(list *list);
-void listJoin(list *l, list *o);
 
 /* Directions for iterators */
 #define AL_START_HEAD 0

src/ae.c

@@ -75,7 +75,6 @@ aeEventLoop *aeCreateEventLoop(int setsize) {
     eventLoop->stop = 0;
     eventLoop->maxfd = -1;
     eventLoop->beforesleep = NULL;
-    eventLoop->aftersleep = NULL;
     if (aeApiCreate(eventLoop) == -1) goto err;
     /* Events with mask == AE_NONE are not set. So let's initialize the
      * vector with it. */
@@ -344,7 +343,6 @@ static int processTimeEvents(aeEventLoop *eventLoop) {
  * if flags has AE_FILE_EVENTS set, file events are processed.
  * if flags has AE_TIME_EVENTS set, time events are processed.
  * if flags has AE_DONT_WAIT set the function returns ASAP until all
- * if flags has AE_CALL_AFTER_SLEEP set, the aftersleep callback is called.
  * the events that's possible to process without to wait are processed.
  *
  * The function returns the number of events processed. */
@@ -399,14 +397,7 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
             }
         }
 
-        /* Call the multiplexing API, will return only on timeout or when
-         * some event fires. */
         numevents = aeApiPoll(eventLoop, tvp);
-
-        /* After sleep callback. */
-        if (eventLoop->aftersleep != NULL && flags & AE_CALL_AFTER_SLEEP)
-            eventLoop->aftersleep(eventLoop);
-
         for (j = 0; j < numevents; j++) {
             aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
             int mask = eventLoop->fired[j].mask;
@@ -461,7 +452,7 @@ void aeMain(aeEventLoop *eventLoop) {
     while (!eventLoop->stop) {
         if (eventLoop->beforesleep != NULL)
             eventLoop->beforesleep(eventLoop);
-        aeProcessEvents(eventLoop, AE_ALL_EVENTS|AE_CALL_AFTER_SLEEP);
+        aeProcessEvents(eventLoop, AE_ALL_EVENTS);
     }
 }
 
@@ -472,7 +463,3 @@ char *aeGetApiName(void) {
 void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep) {
     eventLoop->beforesleep = beforesleep;
 }
-
-void aeSetAfterSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *aftersleep) {
-    eventLoop->aftersleep = aftersleep;
-}

src/ae.h

@@ -46,7 +46,6 @@
 #define AE_TIME_EVENTS 2
 #define AE_ALL_EVENTS (AE_FILE_EVENTS|AE_TIME_EVENTS)
 #define AE_DONT_WAIT 4
-#define AE_CALL_AFTER_SLEEP 8
 
 #define AE_NOMORE -1
 #define AE_DELETED_EVENT_ID -1
@@ -99,7 +98,6 @@ typedef struct aeEventLoop {
     int stop;
     void *apidata; /* This is used for polling API specific data */
     aeBeforeSleepProc *beforesleep;
-    aeBeforeSleepProc *aftersleep;
 } aeEventLoop;
 
 /* Prototypes */
@@ -119,7 +117,6 @@ int aeWait(int fd, int mask, long long milliseconds);
 void aeMain(aeEventLoop *eventLoop);
 char *aeGetApiName(void);
 void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep);
-void aeSetAfterSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *aftersleep);
 int aeGetSetSize(aeEventLoop *eventLoop);
 int aeResizeSetSize(aeEventLoop *eventLoop, int setsize);
 

src/anet.c

@@ -380,10 +380,8 @@ int anetUnixGenericConnect(char *err, char *path, int flags)
     sa.sun_family = AF_LOCAL;
     strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
     if (flags & ANET_CONNECT_NONBLOCK) {
-        if (anetNonBlock(err,s) != ANET_OK) {
-            close(s);
+        if (anetNonBlock(err,s) != ANET_OK)
             return ANET_ERR;
-        }
     }
     if (connect(s,(struct sockaddr*)&sa,sizeof(sa)) == -1) {
         if (errno == EINPROGRESS &&
@@ -464,7 +462,7 @@ static int anetV6Only(char *err, int s) {
 
 static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backlog)
 {
-    int s = -1, rv;
+    int s, rv;
     char _port[6];  /* strlen("65535") */
     struct addrinfo hints, *servinfo, *p;
 
@@ -493,7 +491,6 @@ static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backl
     }
 
 error:
-    if (s != -1) close(s);
     s = ANET_ERR;
 end:
     freeaddrinfo(servinfo);

src/aof.c

@@ -115,7 +115,6 @@ void aofChildWriteDiffData(aeEventLoop *el, int fd, void *privdata, int mask) {
             if (nwritten <= 0) return;
             memmove(block->buf,block->buf+nwritten,block->used-nwritten);
             block->used -= nwritten;
-            block->free += nwritten;
         }
         if (block->used == 0) listDelNode(server.aof_rewrite_buf_blocks,ln);
     }
@@ -536,22 +535,6 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a
         buf = catAppendOnlyGenericCommand(buf,3,tmpargv);
         decrRefCount(tmpargv[0]);
         buf = catAppendOnlyExpireAtCommand(buf,cmd,argv[1],argv[2]);
-    } else if (cmd->proc == setCommand && argc > 3) {
-        int i;
-        robj *exarg = NULL, *pxarg = NULL;
-        /* Translate SET [EX seconds][PX milliseconds] to SET and PEXPIREAT */
-        buf = catAppendOnlyGenericCommand(buf,3,argv);
-        for (i = 3; i < argc; i ++) {
-            if (!strcasecmp(argv[i]->ptr, "ex")) exarg = argv[i+1];
-            if (!strcasecmp(argv[i]->ptr, "px")) pxarg = argv[i+1];
-        }
-        serverAssert(!(exarg && pxarg));
-        if (exarg)
-            buf = catAppendOnlyExpireAtCommand(buf,server.expireCommand,argv[1],
-                                               exarg);
-        if (pxarg)
-            buf = catAppendOnlyExpireAtCommand(buf,server.pexpireCommand,argv[1],
-                                               pxarg);
     } else {
         /* All the other commands don't need translation or need the
          * same translation already operated in the command vector
@@ -633,23 +616,19 @@ int loadAppendOnlyFile(char *filename) {
     struct redis_stat sb;
     int old_aof_state = server.aof_state;
     long loops = 0;
-    off_t valid_up_to = 0; /* Offset of latest well-formed command loaded. */
+    off_t valid_up_to = 0; /* Offset of the latest well-formed command loaded. */
 
-    if (fp == NULL) {
-        serverLog(LL_WARNING,"Fatal error: can't open the append log file for reading: %s",strerror(errno));
-        exit(1);
-    }
-
-    /* Handle a zero-length AOF file as a special case. An emtpy AOF file
-     * is a valid AOF because an empty server with AOF enabled will create
-     * a zero length file at startup, that will remain like that if no write
-     * operation is received. */
     if (fp && redis_fstat(fileno(fp),&sb) != -1 && sb.st_size == 0) {
         server.aof_current_size = 0;
         fclose(fp);
         return C_ERR;
     }
 
+    if (fp == NULL) {
+        serverLog(LL_WARNING,"Fatal error: can't open the append log file for reading: %s",strerror(errno));
+        exit(1);
+    }
+
     /* Temporarily disable AOF, to prevent EXEC from feeding a MULTI
      * to the same file we're about to read. */
     server.aof_state = AOF_OFF;
@@ -657,28 +636,6 @@ int loadAppendOnlyFile(char *filename) {
     fakeClient = createFakeClient();
     startLoading(fp);
 
-    /* Check if this AOF file has an RDB preamble. In that case we need to
-     * load the RDB file and later continue loading the AOF tail. */
-    char sig[5]; /* "REDIS" */
-    if (fread(sig,1,5,fp) != 5 || memcmp(sig,"REDIS",5) != 0) {
-        /* No RDB preamble, seek back at 0 offset. */
-        if (fseek(fp,0,SEEK_SET) == -1) goto readerr;
-    } else {
-        /* RDB preamble. Pass loading the RDB functions. */
-        rio rdb;
-
-        serverLog(LL_NOTICE,"Reading RDB preamble from AOF file...");
-        if (fseek(fp,0,SEEK_SET) == -1) goto readerr;
-        rioInitWithFile(&rdb,fp);
-        if (rdbLoadRio(&rdb,NULL) != C_OK) {
-            serverLog(LL_WARNING,"Error reading the RDB preamble of the AOF file, AOF loading aborted");
-            goto readerr;
-        } else {
-            serverLog(LL_NOTICE,"Reading the remaining AOF tail...");
-        }
-    }
-
-    /* Read the actual AOF file, in REPL format, command by command. */
     while(1) {
         int argc, j;
         unsigned long len;
@@ -739,7 +696,6 @@ int loadAppendOnlyFile(char *filename) {
         }
 
         /* Run the command in the context of a fake client */
-        fakeClient->cmd = cmd;
         cmd->proc(fakeClient);
 
         /* The fake client should not have a reply */
@@ -750,7 +706,6 @@ int loadAppendOnlyFile(char *filename) {
         /* Clean up. Command code may have changed argv/argc so we use the
          * argv/argc of the client instead of the local variables. */
         freeFakeClientArgv(fakeClient);
-        fakeClient->cmd = NULL;
         if (server.aof_load_truncated) valid_up_to = ftello(fp);
     }
 
@@ -888,7 +843,7 @@ int rewriteSetObject(rio *r, robj *key, robj *o) {
         dictEntry *de;
 
         while((de = dictNext(di)) != NULL) {
-            sds ele = dictGetKey(de);
+            robj *eleobj = dictGetKey(de);
             if (count == 0) {
                 int cmd_items = (items > AOF_REWRITE_ITEMS_PER_CMD) ?
                     AOF_REWRITE_ITEMS_PER_CMD : items;
@@ -897,7 +852,7 @@ int rewriteSetObject(rio *r, robj *key, robj *o) {
                 if (rioWriteBulkString(r,"SADD",4) == 0) return 0;
                 if (rioWriteBulkObject(r,key) == 0) return 0;
             }
-            if (rioWriteBulkString(r,ele,sdslen(ele)) == 0) return 0;
+            if (rioWriteBulkObject(r,eleobj) == 0) return 0;
             if (++count == AOF_REWRITE_ITEMS_PER_CMD) count = 0;
             items--;
         }
@@ -954,7 +909,7 @@ int rewriteSortedSetObject(rio *r, robj *key, robj *o) {
         dictEntry *de;
 
         while((de = dictNext(di)) != NULL) {
-            sds ele = dictGetKey(de);
+            robj *eleobj = dictGetKey(de);
             double *score = dictGetVal(de);
 
             if (count == 0) {
@@ -966,7 +921,7 @@ int rewriteSortedSetObject(rio *r, robj *key, robj *o) {
                 if (rioWriteBulkObject(r,key) == 0) return 0;
             }
             if (rioWriteBulkDouble(r,*score) == 0) return 0;
-            if (rioWriteBulkString(r,ele,sdslen(ele)) == 0) return 0;
+            if (rioWriteBulkObject(r,eleobj) == 0) return 0;
             if (++count == AOF_REWRITE_ITEMS_PER_CMD) count = 0;
             items--;
         }
@@ -990,13 +945,17 @@ static int rioWriteHashIteratorCursor(rio *r, hashTypeIterator *hi, int what) {
         long long vll = LLONG_MAX;
 
         hashTypeCurrentFromZiplist(hi, what, &vstr, &vlen, &vll);
-        if (vstr)
+        if (vstr) {
             return rioWriteBulkString(r, (char*)vstr, vlen);
-        else
+        } else {
             return rioWriteBulkLongLong(r, vll);
+        }
+
     } else if (hi->encoding == OBJ_ENCODING_HT) {
-        sds value = hashTypeCurrentFromHashTable(hi, what);
-        return rioWriteBulkString(r, value, sdslen(value));
+        robj *value;
+
+        hashTypeCurrentFromHashTable(hi, what, &value);
+        return rioWriteBulkObject(r, value);
     }
 
     serverPanic("Unknown hash encoding");
@@ -1031,22 +990,6 @@ int rewriteHashObject(rio *r, robj *key, robj *o) {
     return 1;
 }
 
-/* Call the module type callback in order to rewrite a data type
- * that is exported by a module and is not handled by Redis itself.
- * The function returns 0 on error, 1 on success. */
-int rewriteModuleObject(rio *r, robj *key, robj *o) {
-    RedisModuleIO io;
-    moduleValue *mv = o->ptr;
-    moduleType *mt = mv->type;
-    moduleInitIOContext(io,mt,r);
-    mt->aof_rewrite(&io,key,mv->value);
-    if (io.ctx) {
-        moduleFreeContext(io.ctx);
-        zfree(io.ctx);
-    }
-    return io.error ? 0 : 1;
-}
-
 /* This function is called by the child rewriting the AOF file to read
  * the difference accumulated from the parent into a buffer, that is
  * concatenated at the end of the rewrite. */
@@ -1062,23 +1005,51 @@ ssize_t aofReadDiffFromParent(void) {
     return total;
 }
 
-int rewriteAppendOnlyFileRio(rio *aof) {
+/* Write a sequence of commands able to fully rebuild the dataset into
+ * "filename". Used both by REWRITEAOF and BGREWRITEAOF.
+ *
+ * In order to minimize the number of commands needed in the rewritten
+ * log Redis uses variadic commands when possible, such as RPUSH, SADD
+ * and ZADD. However at max AOF_REWRITE_ITEMS_PER_CMD items per time
+ * are inserted using a single command. */
+int rewriteAppendOnlyFile(char *filename) {
     dictIterator *di = NULL;
     dictEntry *de;
-    size_t processed = 0;
-    long long now = mstime();
+    rio aof;
+    FILE *fp;
+    char tmpfile[256];
     int j;
+    long long now = mstime();
+    char byte;
+    size_t processed = 0;
 
+    /* Note that we have to use a different temp name here compared to the
+     * one used by rewriteAppendOnlyFileBackground() function. */
+    snprintf(tmpfile,256,"temp-rewriteaof-%d.aof", (int) getpid());
+    fp = fopen(tmpfile,"w");
+    if (!fp) {
+        serverLog(LL_WARNING, "Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): %s", strerror(errno));
+        return C_ERR;
+    }
+
+    server.aof_child_diff = sdsempty();
+    rioInitWithFile(&aof,fp);
+    if (server.aof_rewrite_incremental_fsync)
+        rioSetAutoSync(&aof,AOF_AUTOSYNC_BYTES);
     for (j = 0; j < server.dbnum; j++) {
         char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n";
         redisDb *db = server.db+j;
         dict *d = db->dict;
         if (dictSize(d) == 0) continue;
         di = dictGetSafeIterator(d);
+        if (!di) {
+            fclose(fp);
+            return C_ERR;
+        }
 
         /* SELECT the new DB */
-        if (rioWrite(aof,selectcmd,sizeof(selectcmd)-1) == 0) goto werr;
-        if (rioWriteBulkLongLong(aof,j) == 0) goto werr;
+        if (rioWrite(&aof,selectcmd,sizeof(selectcmd)-1) == 0) goto werr;
+        if (rioWriteBulkLongLong(&aof,j) == 0) goto werr;
 
         /* Iterate this DB writing every entry */
         while((de = dictNext(di)) != NULL) {
@@ -1099,83 +1070,37 @@ int rewriteAppendOnlyFileRio(rio *aof) {
             if (o->type == OBJ_STRING) {
                 /* Emit a SET command */
                 char cmd[]="*3\r\n$3\r\nSET\r\n";
-                if (rioWrite(aof,cmd,sizeof(cmd)-1) == 0) goto werr;
+                if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr;
                 /* Key and value */
-                if (rioWriteBulkObject(aof,&key) == 0) goto werr;
-                if (rioWriteBulkObject(aof,o) == 0) goto werr;
+                if (rioWriteBulkObject(&aof,&key) == 0) goto werr;
+                if (rioWriteBulkObject(&aof,o) == 0) goto werr;
             } else if (o->type == OBJ_LIST) {
-                if (rewriteListObject(aof,&key,o) == 0) goto werr;
+                if (rewriteListObject(&aof,&key,o) == 0) goto werr;
             } else if (o->type == OBJ_SET) {
-                if (rewriteSetObject(aof,&key,o) == 0) goto werr;
+                if (rewriteSetObject(&aof,&key,o) == 0) goto werr;
             } else if (o->type == OBJ_ZSET) {
-                if (rewriteSortedSetObject(aof,&key,o) == 0) goto werr;
+                if (rewriteSortedSetObject(&aof,&key,o) == 0) goto werr;
             } else if (o->type == OBJ_HASH) {
-                if (rewriteHashObject(aof,&key,o) == 0) goto werr;
-            } else if (o->type == OBJ_MODULE) {
-                if (rewriteModuleObject(aof,&key,o) == 0) goto werr;
+                if (rewriteHashObject(&aof,&key,o) == 0) goto werr;
             } else {
                 serverPanic("Unknown object type");
             }
             /* Save the expire time */
             if (expiretime != -1) {
                 char cmd[]="*3\r\n$9\r\nPEXPIREAT\r\n";
-                if (rioWrite(aof,cmd,sizeof(cmd)-1) == 0) goto werr;
-                if (rioWriteBulkObject(aof,&key) == 0) goto werr;
-                if (rioWriteBulkLongLong(aof,expiretime) == 0) goto werr;
+                if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr;
+                if (rioWriteBulkObject(&aof,&key) == 0) goto werr;
+                if (rioWriteBulkLongLong(&aof,expiretime) == 0) goto werr;
             }
             /* Read some diff from the parent process from time to time. */
-            if (aof->processed_bytes > processed+AOF_READ_DIFF_INTERVAL_BYTES) {
-                processed = aof->processed_bytes;
+            if (aof.processed_bytes > processed+1024*10) {
+                processed = aof.processed_bytes;
                 aofReadDiffFromParent();
             }
         }
         dictReleaseIterator(di);
         di = NULL;
     }
-    return C_OK;
-
-werr:
-    if (di) dictReleaseIterator(di);
-    return C_ERR;
-}
-
-/* Write a sequence of commands able to fully rebuild the dataset into
- * "filename". Used both by REWRITEAOF and BGREWRITEAOF.
- *
- * In order to minimize the number of commands needed in the rewritten
- * log Redis uses variadic commands when possible, such as RPUSH, SADD
- * and ZADD. However at max AOF_REWRITE_ITEMS_PER_CMD items per time
- * are inserted using a single command. */
-int rewriteAppendOnlyFile(char *filename) {
-    rio aof;
-    FILE *fp;
-    char tmpfile[256];
-    char byte;
-
-    /* Note that we have to use a different temp name here compared to the
-     * one used by rewriteAppendOnlyFileBackground() function. */
-    snprintf(tmpfile,256,"temp-rewriteaof-%d.aof", (int) getpid());
-    fp = fopen(tmpfile,"w");
-    if (!fp) {
-        serverLog(LL_WARNING, "Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): %s", strerror(errno));
-        return C_ERR;
-    }
-
-    server.aof_child_diff = sdsempty();
-    rioInitWithFile(&aof,fp);
-
-    if (server.aof_rewrite_incremental_fsync)
-        rioSetAutoSync(&aof,AOF_AUTOSYNC_BYTES);
-
-    if (server.aof_use_rdb_preamble) {
-        int error;
-        if (rdbSaveRio(&aof,&error,RDB_SAVE_AOF_PREAMBLE,NULL) == C_ERR) {
-            errno = error;
-            goto werr;
-        }
-    } else {
-        if (rewriteAppendOnlyFileRio(&aof) == C_ERR) goto werr;
-    }
 
     /* Do an initial slow fsync here while the parent is still sending
      * data, in order to make the next final fsync faster. */
@@ -1241,6 +1166,7 @@ werr:
     serverLog(LL_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
     fclose(fp);
     unlink(tmpfile);
+    if (di) dictReleaseIterator(di);
     return C_ERR;
 }
 
@@ -1340,7 +1266,6 @@ int rewriteAppendOnlyFileBackground(void) {
 
     if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
     if (aofCreatePipes() != C_OK) return C_ERR;
-    openChildInfoPipe();
     start = ustime();
     if ((childpid = fork()) == 0) {
         char tmpfile[256];
@@ -1350,16 +1275,13 @@ int rewriteAppendOnlyFileBackground(void) {
         redisSetProcTitle("redis-aof-rewrite");
         snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
         if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
-            size_t private_dirty = zmalloc_get_private_dirty(-1);
+            size_t private_dirty = zmalloc_get_private_dirty();
 
             if (private_dirty) {
                 serverLog(LL_NOTICE,
                     "AOF rewrite: %zu MB of memory used by copy-on-write",
                     private_dirty/(1024*1024));
             }
-
-            server.child_info_data.cow_size = private_dirty;
-            sendChildInfo(CHILD_INFO_TYPE_AOF);
             exitFromChild(0);
         } else {
             exitFromChild(1);
@@ -1370,11 +1292,9 @@ int rewriteAppendOnlyFileBackground(void) {
         server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
         latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
         if (childpid == -1) {
-            closeChildInfoPipe();
             serverLog(LL_WARNING,
                 "Can't rewrite append only file in background: fork: %s",
                 strerror(errno));
-            aofClosePipes();
             return C_ERR;
         }
         serverLog(LL_NOTICE,

src/atomicvar.h

@@ -1,133 +0,0 @@
-/* This file implements atomic counters using __atomic or __sync macros if
- * available, otherwise synchronizing different threads using a mutex.
- *
- * The exported interaface is composed of three macros:
- *
- * atomicIncr(var,count) -- Increment the atomic counter
- * atomicGetIncr(var,oldvalue_var,count) -- Get and increment the atomic counter
- * atomicDecr(var,count) -- Decrement the atomic counter
- * atomicGet(var,dstvar) -- Fetch the atomic counter value
- * atomicSet(var,value)  -- Set the atomic counter value
- *
- * The variable 'var' should also have a declared mutex with the same
- * name and the "_mutex" postfix, for instance:
- *
- *  long myvar;
- *  pthread_mutex_t myvar_mutex;
- *  atomicSet(myvar,12345);
- *
- * If atomic primitives are availble (tested in config.h) the mutex
- * is not used.
- *
- * Never use return value from the macros, instead use the AtomicGetIncr()
- * if you need to get the current value and increment it atomically, like
- * in the followign example:
- *
- *  long oldvalue;
- *  atomicGetIncr(myvar,oldvalue,1);
- *  doSomethingWith(oldvalue);
- *
- * ----------------------------------------------------------------------------
- *
- * Copyright (c) 2015, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include <pthread.h>
-
-#ifndef __ATOMIC_VAR_H
-#define __ATOMIC_VAR_H
-
-/* To test Redis with Helgrind (a Valgrind tool) it is useful to define
- * the following macro, so that __sync macros are used: those can be detected
- * by Helgrind (even if they are less efficient) so that no false positive
- * is reported. */
-// #define __ATOMIC_VAR_FORCE_SYNC_MACROS
-
-#if !defined(__ATOMIC_VAR_FORCE_SYNC_MACROS) && defined(__ATOMIC_RELAXED) && !defined(__sun) && (!defined(__clang__) || !defined(__APPLE__) || __apple_build_version__ > 4210057)
-/* Implementation using __atomic macros. */
-
-#define atomicIncr(var,count) __atomic_add_fetch(&var,(count),__ATOMIC_RELAXED)
-#define atomicGetIncr(var,oldvalue_var,count) do { \
-    oldvalue_var = __atomic_fetch_add(&var,(count),__ATOMIC_RELAXED); \
-} while(0)
-#define atomicDecr(var,count) __atomic_sub_fetch(&var,(count),__ATOMIC_RELAXED)
-#define atomicGet(var,dstvar) do { \
-    dstvar = __atomic_load_n(&var,__ATOMIC_RELAXED); \
-} while(0)
-#define atomicSet(var,value) __atomic_store_n(&var,value,__ATOMIC_RELAXED)
-#define REDIS_ATOMIC_API "atomic-builtin"
-
-#elif defined(HAVE_ATOMIC)
-/* Implementation using __sync macros. */
-
-#define atomicIncr(var,count) __sync_add_and_fetch(&var,(count))
-#define atomicGetIncr(var,oldvalue_var,count) do { \
-    oldvalue_var = __sync_fetch_and_add(&var,(count)); \
-} while(0)
-#define atomicDecr(var,count) __sync_sub_and_fetch(&var,(count))
-#define atomicGet(var,dstvar) do { \
-    dstvar = __sync_sub_and_fetch(&var,0); \
-} while(0)
-#define atomicSet(var,value) do { \
-    while(!__sync_bool_compare_and_swap(&var,var,value)); \
-} while(0)
-#define REDIS_ATOMIC_API "sync-builtin"
-
-#else
-/* Implementation using pthread mutex. */
-
-#define atomicIncr(var,count) do { \
-    pthread_mutex_lock(&var ## _mutex); \
-    var += (count); \
-    pthread_mutex_unlock(&var ## _mutex); \
-} while(0)
-#define atomicGetIncr(var,oldvalue_var,count) do { \
-    pthread_mutex_lock(&var ## _mutex); \
-    oldvalue_var = var; \
-    var += (count); \
-    pthread_mutex_unlock(&var ## _mutex); \
-} while(0)
-#define atomicDecr(var,count) do { \
-    pthread_mutex_lock(&var ## _mutex); \
-    var -= (count); \
-    pthread_mutex_unlock(&var ## _mutex); \
-} while(0)
-#define atomicGet(var,dstvar) do { \
-    pthread_mutex_lock(&var ## _mutex); \
-    dstvar = var; \
-    pthread_mutex_unlock(&var ## _mutex); \
-} while(0)
-#define atomicSet(var,value) do { \
-    pthread_mutex_lock(&var ## _mutex); \
-    var = value; \
-    pthread_mutex_unlock(&var ## _mutex); \
-} while(0)
-#define REDIS_ATOMIC_API "pthread-mutex"
-
-#endif
-#endif /* __ATOMIC_VAR_H */

src/bio.c

@@ -63,8 +63,7 @@
 
 static pthread_t bio_threads[BIO_NUM_OPS];
 static pthread_mutex_t bio_mutex[BIO_NUM_OPS];
-static pthread_cond_t bio_newjob_cond[BIO_NUM_OPS];
-static pthread_cond_t bio_step_cond[BIO_NUM_OPS];
+static pthread_cond_t bio_condvar[BIO_NUM_OPS];
 static list *bio_jobs[BIO_NUM_OPS];
 /* The following array is used to hold the number of pending jobs for every
  * OP type. This allows us to export the bioPendingJobsOfType() API that is
@@ -84,9 +83,6 @@ struct bio_job {
 };
 
 void *bioProcessBackgroundJobs(void *arg);
-void lazyfreeFreeObjectFromBioThread(robj *o);
-void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2);
-void lazyfreeFreeSlotsMapFromBioThread(zskiplist *sl);
 
 /* Make sure we have enough stack to perform all the things we do in the
  * main thread. */
@@ -102,8 +98,7 @@ void bioInit(void) {
     /* Initialization of state vars and objects */
     for (j = 0; j < BIO_NUM_OPS; j++) {
         pthread_mutex_init(&bio_mutex[j],NULL);
-        pthread_cond_init(&bio_newjob_cond[j],NULL);
-        pthread_cond_init(&bio_step_cond[j],NULL);
+        pthread_cond_init(&bio_condvar[j],NULL);
         bio_jobs[j] = listCreate();
         bio_pending[j] = 0;
     }
@@ -138,7 +133,7 @@ void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) {
     pthread_mutex_lock(&bio_mutex[type]);
     listAddNodeTail(bio_jobs[type],job);
     bio_pending[type]++;
-    pthread_cond_signal(&bio_newjob_cond[type]);
+    pthread_cond_signal(&bio_condvar[type]);
     pthread_mutex_unlock(&bio_mutex[type]);
 }
 
@@ -173,7 +168,7 @@ void *bioProcessBackgroundJobs(void *arg) {
 
         /* The loop always starts with the lock hold. */
         if (listLength(bio_jobs[type]) == 0) {
-            pthread_cond_wait(&bio_newjob_cond[type],&bio_mutex[type]);
+            pthread_cond_wait(&bio_condvar[type],&bio_mutex[type]);
             continue;
         }
         /* Pop the job from the queue. */
@@ -188,25 +183,11 @@ void *bioProcessBackgroundJobs(void *arg) {
             close((long)job->arg1);
         } else if (type == BIO_AOF_FSYNC) {
             aof_fsync((long)job->arg1);
-        } else if (type == BIO_LAZY_FREE) {
-            /* What we free changes depending on what arguments are set:
-             * arg1 -> free the object at pointer.
-             * arg2 & arg3 -> free two dictionaries (a Redis DB).
-             * only arg3 -> free the skiplist. */
-            if (job->arg1)
-                lazyfreeFreeObjectFromBioThread(job->arg1);
-            else if (job->arg2 && job->arg3)
-                lazyfreeFreeDatabaseFromBioThread(job->arg2,job->arg3);
-            else if (job->arg3)
-                lazyfreeFreeSlotsMapFromBioThread(job->arg3);
         } else {
             serverPanic("Wrong job type in bioProcessBackgroundJobs().");
         }
         zfree(job);
 
-        /* Unblock threads blocked on bioWaitStepOfType() if any. */
-        pthread_cond_broadcast(&bio_step_cond[type]);
-
         /* Lock again before reiterating the loop, if there are no longer
          * jobs to process we'll block again in pthread_cond_wait(). */
         pthread_mutex_lock(&bio_mutex[type]);
@@ -224,28 +205,6 @@ unsigned long long bioPendingJobsOfType(int type) {
     return val;
 }
 
-/* If there are pending jobs for the specified type, the function blocks
- * and waits that the next job was processed. Otherwise the function
- * does not block and returns ASAP.
- *
- * The function returns the number of jobs still to process of the
- * requested type.
- *
- * This function is useful when from another thread, we want to wait
- * a bio.c thread to do more work in a blocking way.
- */
-unsigned long long bioWaitStepOfType(int type) {
-    unsigned long long val;
-    pthread_mutex_lock(&bio_mutex[type]);
-    val = bio_pending[type];
-    if (val != 0) {
-        pthread_cond_wait(&bio_step_cond[type],&bio_mutex[type]);
-        val = bio_pending[type];
-    }
-    pthread_mutex_unlock(&bio_mutex[type]);
-    return val;
-}
-
 /* Kill the running bio threads in an unclean way. This function should be
  * used only when it's critical to stop the threads for some reason.
  * Currently Redis does this only on crash (for instance on SIGSEGV) in order

src/bio.h

@@ -31,12 +31,11 @@
 void bioInit(void);
 void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
 unsigned long long bioPendingJobsOfType(int type);
-unsigned long long bioWaitStepOfType(int type);
+void bioWaitPendingJobsLE(int type, unsigned long long num);
 time_t bioOlderJobOfType(int type);
 void bioKillThreads(void);
 
 /* Background job opcodes */
 #define BIO_CLOSE_FILE    0 /* Deferred close(2) syscall. */
 #define BIO_AOF_FSYNC     1 /* Deferred AOF fsync. */
-#define BIO_LAZY_FREE     2 /* Deferred objects freeing. */
-#define BIO_NUM_OPS       3
+#define BIO_NUM_OPS       2

src/bitops.c

@@ -104,7 +104,6 @@ long redisBitpos(void *s, unsigned long count, int bit) {
     unsigned long skipval, word = 0, one;
     long pos = 0; /* Position of bit, to return to the caller. */
     unsigned long j;
-    int found;
 
     /* Process whole words first, seeking for first word that is not
      * all ones or all zeros respectively if we are lookig for zeros
@@ -118,27 +117,21 @@ long redisBitpos(void *s, unsigned long count, int bit) {
     /* Skip initial bits not aligned to sizeof(unsigned long) byte by byte. */
     skipval = bit ? 0 : UCHAR_MAX;
     c = (unsigned char*) s;
-    found = 0;
     while((unsigned long)c & (sizeof(*l)-1) && count) {
-        if (*c != skipval) {
-            found = 1;
-            break;
-        }
+        if (*c != skipval) break;
         c++;
         count--;
         pos += 8;
     }
 
     /* Skip bits with full word step. */
+    skipval = bit ? 0 : ULONG_MAX;
     l = (unsigned long*) c;
-    if (!found) {
-        skipval = bit ? 0 : ULONG_MAX;
-        while (count >= sizeof(*l)) {
-            if (*l != skipval) break;
-            l++;
-            count -= sizeof(*l);
-            pos += sizeof(*l)*8;
-        }
+    while (count >= sizeof(*l)) {
+        if (*l != skipval) break;
+        l++;
+        count -= sizeof(*l);
+        pos += sizeof(*l)*8;
     }
 
     /* Load bytes into "word" considering the first byte as the most significant
@@ -661,11 +654,8 @@ void bitopCommand(client *c) {
 
         /* Fast path: as far as we have data for all the input bitmaps we
          * can take a fast path that performs much better than the
-         * vanilla algorithm. On ARM we skip the fast path since it will
-         * result in GCC compiling the code using multiple-words load/store
-         * operations that are not supported even in ARM >= v6. */
+         * vanilla algorithm. */
         j = 0;
-        #ifndef USE_ALIGNED_ACCESS
         if (minlen >= sizeof(unsigned long)*4 && numkeys <= 16) {
             unsigned long *lp[16];
             unsigned long *lres = (unsigned long*) res;
@@ -726,7 +716,6 @@ void bitopCommand(client *c) {
                 }
             }
         }
-        #endif
 
         /* j is set to the next byte to process by the previous loop. */
         for (; j < maxlen; j++) {
@@ -918,7 +907,7 @@ void bitfieldCommand(client *c) {
     struct bitfieldOp *ops = NULL; /* Array of ops to execute at end. */
     int owtype = BFOVERFLOW_WRAP; /* Overflow type. */
     int readonly = 1;
-    size_t higest_write_offset = 0;
+    long higest_write_offset = 0;
 
     for (j = 2; j < c->argc; j++) {
         int remargs = c->argc-j-1; /* Remaining args other than current. */
@@ -968,8 +957,7 @@ void bitfieldCommand(client *c) {
 
         if (opcode != BITFIELDOP_GET) {
             readonly = 0;
-            if (higest_write_offset < bitoffset + bits - 1)
-                higest_write_offset = bitoffset + bits - 1;
+            higest_write_offset = bitoffset + bits - 1;
             /* INCRBY and SET require another argument. */
             if (getLongLongFromObjectOrReply(c,c->argv[j+3],&i64,NULL) != C_OK){
                 zfree(ops);

src/blocked.c

@@ -136,8 +136,6 @@ void unblockClient(client *c) {
         unblockClientWaitingData(c);
     } else if (c->btype == BLOCKED_WAIT) {
         unblockClientWaitingReplicas(c);
-    } else if (c->btype == BLOCKED_MODULE) {
-        unblockClientFromModule(c);
     } else {
         serverPanic("Unknown btype in unblockClient().");
     }
@@ -155,15 +153,12 @@ void unblockClient(client *c) {
 }
 
 /* This function gets called when a blocked client timed out in order to
- * send it a reply of some kind. After this function is called,
- * unblockClient() will be called with the same client as argument. */
+ * send it a reply of some kind. */
 void replyToBlockedClientTimedOut(client *c) {
     if (c->btype == BLOCKED_LIST) {
         addReply(c,shared.nullmultibulk);
     } else if (c->btype == BLOCKED_WAIT) {
         addReplyLongLong(c,replicationCountAcksByOffset(c->bpop.reploffset));
-    } else if (c->btype == BLOCKED_MODULE) {
-        moduleBlockedClientTimedOut(c);
     } else {
         serverPanic("Unknown btype in replyToBlockedClientTimedOut().");
     }

src/childinfo.c

@@ -1,85 +0,0 @@
-/*
- * Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "server.h"
-#include <unistd.h>
-
-/* Open a child-parent channel used in order to move information about the
- * RDB / AOF saving process from the child to the parent (for instance
- * the amount of copy on write memory used) */
-void openChildInfoPipe(void) {
-    if (pipe(server.child_info_pipe) == -1) {
-        /* On error our two file descriptors should be still set to -1,
-         * but we call anyway cloesChildInfoPipe() since can't hurt. */
-        closeChildInfoPipe();
-    } else if (anetNonBlock(NULL,server.child_info_pipe[0]) != ANET_OK) {
-        closeChildInfoPipe();
-    } else {
-        memset(&server.child_info_data,0,sizeof(server.child_info_data));
-    }
-}
-
-/* Close the pipes opened with openChildInfoPipe(). */
-void closeChildInfoPipe(void) {
-    if (server.child_info_pipe[0] != -1 ||
-        server.child_info_pipe[1] != -1)
-    {
-        close(server.child_info_pipe[0]);
-        close(server.child_info_pipe[1]);
-        server.child_info_pipe[0] = -1;
-        server.child_info_pipe[1] = -1;
-    }
-}
-
-/* Send COW data to parent. The child should call this function after populating
- * the corresponding fields it want to sent (according to the process type). */
-void sendChildInfo(int ptype) {
-    if (server.child_info_pipe[1] == -1) return;
-    server.child_info_data.magic = CHILD_INFO_MAGIC;
-    server.child_info_data.process_type = ptype;
-    ssize_t wlen = sizeof(server.child_info_data);
-    if (write(server.child_info_pipe[1],&server.child_info_data,wlen) != wlen) {
-        /* Nothing to do on error, this will be detected by the other side. */
-    }
-}
-
-/* Receive COW data from parent. */
-void receiveChildInfo(void) {
-    if (server.child_info_pipe[0] == -1) return;
-    ssize_t wlen = sizeof(server.child_info_data);
-    if (read(server.child_info_pipe[0],&server.child_info_data,wlen) == wlen &&
-        server.child_info_data.magic == CHILD_INFO_MAGIC)
-    {
-        if (server.child_info_data.process_type == CHILD_INFO_TYPE_RDB) {
-            server.stat_rdb_cow_bytes = server.child_info_data.cow_size;
-        } else if (server.child_info_data.process_type == CHILD_INFO_TYPE_AOF) {
-            server.stat_aof_cow_bytes = server.child_info_data.cow_size;
-        }
-    }
-}

src/cluster.c

@@ -37,6 +37,7 @@
 #include <arpa/inet.h>
 #include <fcntl.h>
 #include <unistd.h>
+#include <sys/socket.h>
 #include <sys/stat.h>
 #include <sys/file.h>
 #include <math.h>
@@ -128,7 +129,7 @@ int clusterLoadConfig(char *filename) {
         /* Skip blank lines, they can be created either by users manually
          * editing nodes.conf or by the config writing process if stopped
          * before the truncate() call. */
-        if (line[0] == '\n' || line[0] == '\0') continue;
+        if (line[0] == '\n') continue;
 
         /* Split the line into arguments for processing. */
         argv = sdssplitargs(line,&argc);
@@ -167,17 +168,7 @@ int clusterLoadConfig(char *filename) {
         if ((p = strrchr(argv[1],':')) == NULL) goto fmterr;
         *p = '\0';
         memcpy(n->ip,argv[1],strlen(argv[1])+1);
-        char *port = p+1;
-        char *busp = strchr(port,'@');
-        if (busp) {
-            *busp = '\0';
-            busp++;
-        }
-        n->port = atoi(port);
-        /* In older versions of nodes.conf the "@busport" part is missing.
-         * In this case we set it to the default offset of 10000 from the
-         * base port. */
-        n->cport = busp ? atoi(busp) : n->port + CLUSTER_PORT_INCR;
+        n->port = atoi(p+1);
 
         /* Parse flags */
         p = s = argv[2];
@@ -243,7 +234,6 @@ int clusterLoadConfig(char *filename) {
                 *p = '\0';
                 direction = p[1]; /* Either '>' or '<' */
                 slot = atoi(argv[j]+1);
-                if (slot < 0 || slot >= CLUSTER_SLOTS) goto fmterr;
                 p += 3;
                 cn = clusterLookupNode(p);
                 if (!cn) {
@@ -263,8 +253,6 @@ int clusterLoadConfig(char *filename) {
             } else {
                 start = stop = atoi(argv[j]);
             }
-            if (start < 0 || start >= CLUSTER_SLOTS) goto fmterr;
-            if (stop < 0 || stop >= CLUSTER_SLOTS) goto fmterr;
             while(start <= stop) clusterAddSlot(n, start++);
         }
 
@@ -425,11 +413,8 @@ void clusterInit(void) {
     server.cluster->failover_auth_epoch = 0;
     server.cluster->cant_failover_reason = CLUSTER_CANT_FAILOVER_NONE;
     server.cluster->lastVoteEpoch = 0;
-    for (int i = 0; i < CLUSTERMSG_TYPE_COUNT; i++) {
-        server.cluster->stats_bus_messages_sent[i] = 0;
-        server.cluster->stats_bus_messages_received[i] = 0;
-    }
-    server.cluster->stats_pfail_nodes = 0;
+    server.cluster->stats_bus_messages_sent = 0;
+    server.cluster->stats_bus_messages_received = 0;
     memset(server.cluster->slots,0, sizeof(server.cluster->slots));
     clusterCloseAllSlots();
 
@@ -481,19 +466,12 @@ void clusterInit(void) {
         }
     }
 
-    /* The slots -> keys map is a radix tree. Initialize it here. */
-    server.cluster->slots_to_keys = raxNew();
-    memset(server.cluster->slots_keys_count,0,
-           sizeof(server.cluster->slots_keys_count));
+    /* The slots -> keys map is a sorted set. Init it. */
+    server.cluster->slots_to_keys = zslCreate();
 
-    /* Set myself->port / cport to my listening ports, we'll just need to
-     * discover the IP address via MEET messages. */
+    /* Set myself->port to my listening port, we'll just need to discover
+     * the IP address via MEET messages. */
     myself->port = server.port;
-    myself->cport = server.port+CLUSTER_PORT_INCR;
-    if (server.cluster_announce_port)
-        myself->port = server.cluster_announce_port;
-    if (server.cluster_announce_bus_port)
-        myself->cport = server.cluster_announce_bus_port;
 
     server.cluster->mf_end = 0;
     resetManualFailover();
@@ -517,7 +495,7 @@ void clusterReset(int hard) {
     if (nodeIsSlave(myself)) {
         clusterSetNodeAsMaster(myself);
         replicationUnsetMaster();
-        emptyDb(-1,EMPTYDB_NO_FLAGS,NULL);
+        emptyDb(NULL);
     }
 
     /* Close slots, reset manual failover state. */
@@ -692,7 +670,6 @@ clusterNode *createClusterNode(char *nodename, int flags) {
     node->link = NULL;
     memset(node->ip,0,sizeof(node->ip));
     node->port = 0;
-    node->cport = 0;
     node->fail_reports = listCreate();
     node->voted_time = 0;
     node->orphaned_time = 0;
@@ -1235,7 +1212,7 @@ void clearNodeFailureIfNeeded(clusterNode *node) {
 /* Return true if we already have a node in HANDSHAKE state matching the
  * specified ip address and port number. This function is used in order to
  * avoid adding a new handshake node for the same address multiple times. */
-int clusterHandshakeInProgress(char *ip, int port, int cport) {
+int clusterHandshakeInProgress(char *ip, int port) {
     dictIterator *di;
     dictEntry *de;
 
@@ -1244,9 +1221,7 @@ int clusterHandshakeInProgress(char *ip, int port, int cport) {
         clusterNode *node = dictGetVal(de);
 
         if (!nodeInHandshake(node)) continue;
-        if (!strcasecmp(node->ip,ip) &&
-            node->port == port &&
-            node->cport == cport) break;
+        if (!strcasecmp(node->ip,ip) && node->port == port) break;
     }
     dictReleaseIterator(di);
     return de != NULL;
@@ -1259,7 +1234,7 @@ int clusterHandshakeInProgress(char *ip, int port, int cport) {
  *
  * EAGAIN - There is already an handshake in progress for this address.
  * EINVAL - IP or port are not valid. */
-int clusterStartHandshake(char *ip, int port, int cport) {
+int clusterStartHandshake(char *ip, int port) {
     clusterNode *n;
     char norm_ip[NET_IP_STR_LEN];
     struct sockaddr_storage sa;
@@ -1279,7 +1254,7 @@ int clusterStartHandshake(char *ip, int port, int cport) {
     }
 
     /* Port sanity check */
-    if (port <= 0 || port > 65535 || cport <= 0 || cport > 65535) {
+    if (port <= 0 || port > (65535-CLUSTER_PORT_INCR)) {
         errno = EINVAL;
         return 0;
     }
@@ -1296,7 +1271,7 @@ int clusterStartHandshake(char *ip, int port, int cport) {
             (void*)&(((struct sockaddr_in6 *)&sa)->sin6_addr),
             norm_ip,NET_IP_STR_LEN);
 
-    if (clusterHandshakeInProgress(norm_ip,port,cport)) {
+    if (clusterHandshakeInProgress(norm_ip,port)) {
         errno = EAGAIN;
         return 0;
     }
@@ -1307,7 +1282,6 @@ int clusterStartHandshake(char *ip, int port, int cport) {
     n = createClusterNode(NULL,CLUSTER_NODE_HANDSHAKE|CLUSTER_NODE_MEET);
     memcpy(n->ip,norm_ip,sizeof(n->ip));
     n->port = port;
-    n->cport = cport;
     clusterAddNode(n);
     return 1;
 }
@@ -1326,16 +1300,13 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
         clusterNode *node;
         sds ci;
 
-        if (server.verbosity == LL_DEBUG) {
-            ci = representClusterNodeFlags(sdsempty(), flags);
-            serverLog(LL_DEBUG,"GOSSIP %.40s %s:%d@%d %s",
-                g->nodename,
-                g->ip,
-                ntohs(g->port),
-                ntohs(g->cport),
-                ci);
-            sdsfree(ci);
-        }
+        ci = representClusterNodeFlags(sdsempty(), flags);
+        serverLog(LL_DEBUG,"GOSSIP %.40s %s:%d %s",
+            g->nodename,
+            g->ip,
+            ntohs(g->port),
+            ci);
+        sdsfree(ci);
 
         /* Update our state accordingly to the gossip sections */
         node = clusterLookupNode(g->nodename);
@@ -1359,28 +1330,6 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
                 }
             }
 
-            /* If from our POV the node is up (no failure flags are set),
-             * we have no pending ping for the node, nor we have failure
-             * reports for this node, update the last pong time with the
-             * one we see from the other nodes. */
-            if (!(flags & (CLUSTER_NODE_FAIL|CLUSTER_NODE_PFAIL)) &&
-                node->ping_sent == 0 &&
-                clusterNodeFailureReportsCount(node) == 0)
-            {
-                mstime_t pongtime = ntohl(g->pong_received);
-                pongtime *= 1000; /* Convert back to milliseconds. */
-
-                /* Replace the pong time with the received one only if
-                 * it's greater than our view but is not in the future
-                 * (with 500 milliseconds tolerance) from the POV of our
-                 * clock. */
-                if (pongtime <= (server.mstime+500) &&
-                    pongtime > node->pong_received)
-                {
-                    node->pong_received = pongtime;
-                }
-            }
-
             /* If we already know this node, but it is not reachable, and
              * we see a different address in the gossip section of a node that
              * can talk with this other node, update the address, disconnect
@@ -1389,14 +1338,11 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
             if (node->flags & (CLUSTER_NODE_FAIL|CLUSTER_NODE_PFAIL) &&
                 !(flags & CLUSTER_NODE_NOADDR) &&
                 !(flags & (CLUSTER_NODE_FAIL|CLUSTER_NODE_PFAIL)) &&
-                (strcasecmp(node->ip,g->ip) ||
-                 node->port != ntohs(g->port) ||
-                 node->cport != ntohs(g->cport)))
+                (strcasecmp(node->ip,g->ip) || node->port != ntohs(g->port)))
             {
                 if (node->link) freeClusterLink(node->link);
                 memcpy(node->ip,g->ip,NET_IP_STR_LEN);
                 node->port = ntohs(g->port);
-                node->cport = ntohs(g->cport);
                 node->flags &= ~CLUSTER_NODE_NOADDR;
             }
         } else {
@@ -1410,7 +1356,7 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
                 !(flags & CLUSTER_NODE_NOADDR) &&
                 !clusterBlacklistExists(g->nodename))
             {
-                clusterStartHandshake(g->ip,ntohs(g->port),ntohs(g->cport));
+                clusterStartHandshake(g->ip,ntohs(g->port));
             }
         }
 
@@ -1419,36 +1365,23 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
     }
 }
 
-/* IP -> string conversion. 'buf' is supposed to at least be 46 bytes.
- * If 'announced_ip' length is non-zero, it is used instead of extracting
- * the IP from the socket peer address. */
-void nodeIp2String(char *buf, clusterLink *link, char *announced_ip) {
-    if (announced_ip[0] != '\0') {
-        memcpy(buf,announced_ip,NET_IP_STR_LEN);
-        buf[NET_IP_STR_LEN-1] = '\0'; /* We are not sure the input is sane. */
-    } else {
-        anetPeerToString(link->fd, buf, NET_IP_STR_LEN, NULL);
-    }
+/* IP -> string conversion. 'buf' is supposed to at least be 46 bytes. */
+void nodeIp2String(char *buf, clusterLink *link) {
+    anetPeerToString(link->fd, buf, NET_IP_STR_LEN, NULL);
 }
 
 /* Update the node address to the IP address that can be extracted
- * from link->fd, or if hdr->myip is non empty, to the address the node
- * is announcing us. The port is taken from the packet header as well.
- *
- * If the address or port changed, disconnect the node link so that we'll
- * connect again to the new address.
+ * from link->fd, and at the specified port.
+ * Also disconnect the node link so that we'll connect again to the new
+ * address.
  *
  * If the ip/port pair are already correct no operation is performed at
  * all.
  *
  * The function returns 0 if the node address is still the same,
  * otherwise 1 is returned. */
-int nodeUpdateAddressIfNeeded(clusterNode *node, clusterLink *link,
-                              clusterMsg *hdr)
-{
+int nodeUpdateAddressIfNeeded(clusterNode *node, clusterLink *link, int port) {
     char ip[NET_IP_STR_LEN] = {0};
-    int port = ntohs(hdr->port);
-    int cport = ntohs(hdr->cport);
 
     /* We don't proceed if the link is the same as the sender link, as this
      * function is designed to see if the node link is consistent with the
@@ -1458,14 +1391,12 @@ int nodeUpdateAddressIfNeeded(clusterNode *node, clusterLink *link,
      * it is safe to call during packet processing. */
     if (link == node->link) return 0;
 
-    nodeIp2String(ip,link,hdr->myip);
-    if (node->port == port && node->cport == cport &&
-        strcmp(ip,node->ip) == 0) return 0;
+    nodeIp2String(ip,link);
+    if (node->port == port && strcmp(ip,node->ip) == 0) return 0;
 
     /* IP / port is different, update it. */
     memcpy(node->ip,ip,sizeof(ip));
     node->port = port;
-    node->cport = cport;
     if (node->link) freeClusterLink(node->link);
     node->flags &= ~CLUSTER_NODE_NOADDR;
     serverLog(LL_WARNING,"Address updated for node %.40s, now %s:%d",
@@ -1612,8 +1543,7 @@ int clusterProcessPacket(clusterLink *link) {
     uint32_t totlen = ntohl(hdr->totlen);
     uint16_t type = ntohs(hdr->type);
 
-    if (type < CLUSTERMSG_TYPE_COUNT)
-        server.cluster->stats_bus_messages_received[type]++;
+    server.cluster->stats_bus_messages_received++;
     serverLog(LL_DEBUG,"--- Processing packet of type %d, %lu bytes",
         type, (unsigned long) totlen);
 
@@ -1705,7 +1635,7 @@ int clusterProcessPacket(clusterLink *link) {
 
         /* We use incoming MEET messages in order to set the address
          * for 'myself', since only other cluster nodes will send us
-         * MEET messages on handshakes, when the cluster joins, or
+         * MEET messagses on handshakes, when the cluster joins, or
          * later if we changed address, and those nodes will use our
          * official address to connect to us. So by obtaining this address
          * from the socket is a simple way to discover / update our own
@@ -1714,9 +1644,7 @@ int clusterProcessPacket(clusterLink *link) {
          * However if we don't have an address at all, we update the address
          * even with a normal PING packet. If it's wrong it will be fixed
          * by MEET later. */
-        if ((type == CLUSTERMSG_TYPE_MEET || myself->ip[0] == '\0') &&
-            server.cluster_announce_ip == NULL)
-        {
+        if (type == CLUSTERMSG_TYPE_MEET || myself->ip[0] == '\0') {
             char ip[NET_IP_STR_LEN];
 
             if (anetSockName(link->fd,ip,sizeof(ip),NULL) != -1 &&
@@ -1737,9 +1665,8 @@ int clusterProcessPacket(clusterLink *link) {
             clusterNode *node;
 
             node = createClusterNode(NULL,CLUSTER_NODE_HANDSHAKE);
-            nodeIp2String(node->ip,link,hdr->myip);
+            nodeIp2String(node->ip,link);
             node->port = ntohs(hdr->port);
-            node->cport = ntohs(hdr->cport);
             clusterAddNode(node);
             clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
         }
@@ -1769,7 +1696,7 @@ int clusterProcessPacket(clusterLink *link) {
                     serverLog(LL_VERBOSE,
                         "Handshake: we already know node %.40s, "
                         "updating the address if needed.", sender->name);
-                    if (nodeUpdateAddressIfNeeded(sender,link,hdr))
+                    if (nodeUpdateAddressIfNeeded(sender,link,ntohs(hdr->port)))
                     {
                         clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|
                                              CLUSTER_TODO_UPDATE_STATE);
@@ -1801,7 +1728,6 @@ int clusterProcessPacket(clusterLink *link) {
                 link->node->flags |= CLUSTER_NODE_NOADDR;
                 link->node->ip[0] = '\0';
                 link->node->port = 0;
-                link->node->cport = 0;
                 freeClusterLink(link);
                 clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
                 return 0;
@@ -1811,7 +1737,7 @@ int clusterProcessPacket(clusterLink *link) {
         /* Update the node address if it changed. */
         if (sender && type == CLUSTERMSG_TYPE_PING &&
             !nodeInHandshake(sender) &&
-            nodeUpdateAddressIfNeeded(sender,link,hdr))
+            nodeUpdateAddressIfNeeded(sender,link,ntohs(hdr->port)))
         {
             clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|
                                  CLUSTER_TODO_UPDATE_STATE);
@@ -2160,12 +2086,7 @@ void clusterSendMessage(clusterLink *link, unsigned char *msg, size_t msglen) {
                     clusterWriteHandler,link);
 
     link->sndbuf = sdscatlen(link->sndbuf, msg, msglen);
-
-    /* Populate sent messages stats. */
-    clusterMsg *hdr = (clusterMsg*) msg;
-    uint16_t type = ntohs(hdr->type);
-    if (type < CLUSTERMSG_TYPE_COUNT)
-        server.cluster->stats_bus_messages_sent[type]++;
+    server.cluster->stats_bus_messages_sent++;
 }
 
 /* Send a message to all the nodes that are part of the cluster having
@@ -2213,28 +2134,11 @@ void clusterBuildMessageHdr(clusterMsg *hdr, int type) {
     hdr->type = htons(type);
     memcpy(hdr->sender,myself->name,CLUSTER_NAMELEN);
 
-    /* If cluster-announce-ip option is enabled, force the receivers of our
-     * packets to use the specified address for this node. Otherwise if the
-     * first byte is zero, they'll do auto discovery. */
-    memset(hdr->myip,0,NET_IP_STR_LEN);
-    if (server.cluster_announce_ip) {
-        strncpy(hdr->myip,server.cluster_announce_ip,NET_IP_STR_LEN);
-        hdr->myip[NET_IP_STR_LEN-1] = '\0';
-    }
-
-    /* Handle cluster-announce-port as well. */
-    int announced_port = server.cluster_announce_port ?
-                         server.cluster_announce_port : server.port;
-    int announced_cport = server.cluster_announce_bus_port ?
-                          server.cluster_announce_bus_port :
-                          (server.port + CLUSTER_PORT_INCR);
-
     memcpy(hdr->myslots,master->slots,sizeof(hdr->myslots));
     memset(hdr->slaveof,0,CLUSTER_NAMELEN);
     if (myself->slaveof != NULL)
         memcpy(hdr->slaveof,myself->slaveof->name, CLUSTER_NAMELEN);
-    hdr->port = htons(announced_port);
-    hdr->cport = htons(announced_cport);
+    hdr->port = htons(server.port);
     hdr->flags = htons(myself->flags);
     hdr->state = server.cluster->state;
 
@@ -2266,33 +2170,6 @@ void clusterBuildMessageHdr(clusterMsg *hdr, int type) {
     /* For PING, PONG, and MEET, fixing the totlen field is up to the caller. */
 }
 
-/* Return non zero if the node is already present in the gossip section of the
- * message pointed by 'hdr' and having 'count' gossip entries. Otherwise
- * zero is returned. Helper for clusterSendPing(). */
-int clusterNodeIsInGossipSection(clusterMsg *hdr, int count, clusterNode *n) {
-    int j;
-    for (j = 0; j < count; j++) {
-        if (memcmp(hdr->data.ping.gossip[j].nodename,n->name,
-                CLUSTER_NAMELEN) == 0) break;
-    }
-    return j != count;
-}
-
-/* Set the i-th entry of the gossip section in the message pointed by 'hdr'
- * to the info of the specified node 'n'. */
-void clusterSetGossipEntry(clusterMsg *hdr, int i, clusterNode *n) {
-    clusterMsgDataGossip *gossip;
-    gossip = &(hdr->data.ping.gossip[i]);
-    memcpy(gossip->nodename,n->name,CLUSTER_NAMELEN);
-    gossip->ping_sent = htonl(n->ping_sent/1000);
-    gossip->pong_received = htonl(n->pong_received/1000);
-    memcpy(gossip->ip,n->ip,sizeof(n->ip));
-    gossip->port = htons(n->port);
-    gossip->cport = htons(n->cport);
-    gossip->flags = htons(n->flags);
-    gossip->notused1 = 0;
-}
-
 /* Send a PING or PONG packet to the specified node, making sure to add enough
  * gossip informations. */
 void clusterSendPing(clusterLink *link, int type) {
@@ -2337,15 +2214,11 @@ void clusterSendPing(clusterLink *link, int type) {
     if (wanted < 3) wanted = 3;
     if (wanted > freshnodes) wanted = freshnodes;
 
-    /* Include all the nodes in PFAIL state, so that failure reports are
-     * faster to propagate to go from PFAIL to FAIL state. */
-    int pfail_wanted = server.cluster->stats_pfail_nodes;
-
     /* Compute the maxium totlen to allocate our buffer. We'll fix the totlen
      * later according to the number of gossip sections we really were able
      * to put inside the packet. */
     totlen = sizeof(clusterMsg)-sizeof(union clusterMsgData);
-    totlen += (sizeof(clusterMsgDataGossip)*(wanted+pfail_wanted));
+    totlen += (sizeof(clusterMsgDataGossip)*wanted);
     /* Note: clusterBuildMessageHdr() expects the buffer to be always at least
      * sizeof(clusterMsg) or more. */
     if (totlen < (int)sizeof(clusterMsg)) totlen = sizeof(clusterMsg);
@@ -2362,13 +2235,17 @@ void clusterSendPing(clusterLink *link, int type) {
     while(freshnodes > 0 && gossipcount < wanted && maxiterations--) {
         dictEntry *de = dictGetRandomKey(server.cluster->nodes);
         clusterNode *this = dictGetVal(de);
+        clusterMsgDataGossip *gossip;
+        int j;
 
         /* Don't include this node: the whole packet header is about us
          * already, so we just gossip about other nodes. */
         if (this == myself) continue;
 
-        /* PFAIL nodes will be added later. */
-        if (this->flags & CLUSTER_NODE_PFAIL) continue;
+        /* Give a bias to FAIL/PFAIL nodes. */
+        if (maxiterations > wanted*2 &&
+            !(this->flags & (CLUSTER_NODE_PFAIL|CLUSTER_NODE_FAIL)))
+            continue;
 
         /* In the gossip section don't include:
          * 1) Nodes in HANDSHAKE state.
@@ -2382,37 +2259,27 @@ void clusterSendPing(clusterLink *link, int type) {
             continue;
         }
 
-        /* Do not add a node we already have. */
-        if (clusterNodeIsInGossipSection(hdr,gossipcount,this)) continue;
+        /* Check if we already added this node */
+        for (j = 0; j < gossipcount; j++) {
+            if (memcmp(hdr->data.ping.gossip[j].nodename,this->name,
+                    CLUSTER_NAMELEN) == 0) break;
+        }
+        if (j != gossipcount) continue;
 
         /* Add it */
-        clusterSetGossipEntry(hdr,gossipcount,this);
         freshnodes--;
+        gossip = &(hdr->data.ping.gossip[gossipcount]);
+        memcpy(gossip->nodename,this->name,CLUSTER_NAMELEN);
+        gossip->ping_sent = htonl(this->ping_sent);
+        gossip->pong_received = htonl(this->pong_received);
+        memcpy(gossip->ip,this->ip,sizeof(this->ip));
+        gossip->port = htons(this->port);
+        gossip->flags = htons(this->flags);
+        gossip->notused1 = 0;
+        gossip->notused2 = 0;
         gossipcount++;
     }
 
-    /* If there are PFAIL nodes, add them at the end. */
-    if (pfail_wanted) {
-        dictIterator *di;
-        dictEntry *de;
-
-        di = dictGetSafeIterator(server.cluster->nodes);
-        while((de = dictNext(di)) != NULL && pfail_wanted > 0) {
-            clusterNode *node = dictGetVal(de);
-            if (node->flags & CLUSTER_NODE_HANDSHAKE) continue;
-            if (node->flags & CLUSTER_NODE_NOADDR) continue;
-            if (!(node->flags & CLUSTER_NODE_PFAIL)) continue;
-            clusterSetGossipEntry(hdr,gossipcount,node);
-            freshnodes--;
-            gossipcount++;
-            /* We take the count of the slots we allocated, since the
-             * PFAIL stats may not match perfectly with the current number
-             * of PFAIL nodes. */
-            pfail_wanted--;
-        }
-        dictReleaseIterator(di);
-    }
-
     /* Ready to send... fix the totlen fiend and queue the message in the
      * output buffer. */
     totlen = sizeof(clusterMsg)-sizeof(union clusterMsgData);
@@ -2848,7 +2715,7 @@ void clusterHandleSlaveFailover(void) {
      * and wait for replies), and the failover retry time (the time to wait
      * before trying to get voted again).
      *
-     * Timeout is MAX(NODE_TIMEOUT*2,2000) milliseconds.
+     * Timeout is MIN(NODE_TIMEOUT*2,2000) milliseconds.
      * Retry is two times the Timeout.
      */
     auth_timeout = server.cluster_node_timeout*2;
@@ -3206,31 +3073,6 @@ void clusterCron(void) {
 
     iteration++; /* Number of times this function was called so far. */
 
-    /* We want to take myself->ip in sync with the cluster-announce-ip option.
-     * The option can be set at runtime via CONFIG SET, so we periodically check
-     * if the option changed to reflect this into myself->ip. */
-    {
-        static char *prev_ip = NULL;
-        char *curr_ip = server.cluster_announce_ip;
-        int changed = 0;
-
-        if (prev_ip == NULL && curr_ip != NULL) changed = 1;
-        if (prev_ip != NULL && curr_ip == NULL) changed = 1;
-        if (prev_ip && curr_ip && strcmp(prev_ip,curr_ip)) changed = 1;
-
-        if (changed) {
-            prev_ip = curr_ip;
-            if (prev_ip) prev_ip = zstrdup(prev_ip);
-
-            if (curr_ip) {
-                strncpy(myself->ip,server.cluster_announce_ip,NET_IP_STR_LEN);
-                myself->ip[NET_IP_STR_LEN-1] = '\0';
-            } else {
-                myself->ip[0] = '\0'; /* Force autodetection. */
-            }
-        }
-    }
-
     /* The handshake timeout is the time after which a handshake node that was
      * not turned into a normal node is removed from the nodes. Usually it is
      * just the NODE_TIMEOUT value, but when NODE_TIMEOUT is too small we use
@@ -3238,21 +3080,13 @@ void clusterCron(void) {
     handshake_timeout = server.cluster_node_timeout;
     if (handshake_timeout < 1000) handshake_timeout = 1000;
 
-    /* Check if we have disconnected nodes and re-establish the connection.
-     * Also update a few stats while we are here, that can be used to make
-     * better decisions in other part of the code. */
+    /* Check if we have disconnected nodes and re-establish the connection. */
     di = dictGetSafeIterator(server.cluster->nodes);
-    server.cluster->stats_pfail_nodes = 0;
     while((de = dictNext(di)) != NULL) {
         clusterNode *node = dictGetVal(de);
 
-        /* Not interested in reconnecting the link with myself or nodes
-         * for which we have no address. */
         if (node->flags & (CLUSTER_NODE_MYSELF|CLUSTER_NODE_NOADDR)) continue;
 
-        if (node->flags & CLUSTER_NODE_PFAIL)
-            server.cluster->stats_pfail_nodes++;
-
         /* A Node in HANDSHAKE state has a limited lifespan equal to the
          * configured node timeout. */
         if (nodeInHandshake(node) && now - node->ctime > handshake_timeout) {
@@ -3266,7 +3100,7 @@ void clusterCron(void) {
             clusterLink *link;
 
             fd = anetTcpNonBlockBindConnect(server.neterr, node->ip,
-                node->cport, NET_FIRST_BIND_ADDR);
+                node->port+CLUSTER_PORT_INCR, NET_FIRST_BIND_ADDR);
             if (fd == -1) {
                 /* We got a synchronous error from connect before
                  * clusterSendPing() had a chance to be called.
@@ -3276,7 +3110,8 @@ void clusterCron(void) {
                 if (node->ping_sent == 0) node->ping_sent = mstime();
                 serverLog(LL_DEBUG, "Unable to connect to "
                     "Cluster Node [%s]:%d -> %s", node->ip,
-                    node->cport, server.neterr);
+                    node->port+CLUSTER_PORT_INCR,
+                    server.neterr);
                 continue;
             }
             link = createClusterLink(node);
@@ -3307,7 +3142,7 @@ void clusterCron(void) {
             node->flags &= ~CLUSTER_NODE_MEET;
 
             serverLog(LL_DEBUG,"Connecting with Node %.40s at %s:%d",
-                    node->name, node->ip, node->cport);
+                    node->name, node->ip, node->port+CLUSTER_PORT_INCR);
         }
     }
     dictReleaseIterator(di);
@@ -3605,10 +3440,8 @@ int clusterDelNodeSlots(clusterNode *node) {
     int deleted = 0, j;
 
     for (j = 0; j < CLUSTER_SLOTS; j++) {
-        if (clusterNodeGetSlotBit(node,j)) {
-            clusterDelSlot(j);
-            deleted++;
-        }
+        if (clusterNodeGetSlotBit(node,j)) clusterDelSlot(j);
+        deleted++;
     }
     return deleted;
 }
@@ -3842,14 +3675,15 @@ static struct redisNodeFlags redisNodeFlagsTable[] = {
 /* Concatenate the comma separated list of node flags to the given SDS
  * string 'ci'. */
 sds representClusterNodeFlags(sds ci, uint16_t flags) {
-    size_t orig_len = sdslen(ci);
-    int i, size = sizeof(redisNodeFlagsTable)/sizeof(struct redisNodeFlags);
-    for (i = 0; i < size; i++) {
-        struct redisNodeFlags *nodeflag = redisNodeFlagsTable + i;
-        if (flags & nodeflag->flag) ci = sdscat(ci, nodeflag->name);
+    if (flags == 0) {
+        ci = sdscat(ci,"noflags,");
+    } else {
+        int i, size = sizeof(redisNodeFlagsTable)/sizeof(struct redisNodeFlags);
+        for (i = 0; i < size; i++) {
+            struct redisNodeFlags *nodeflag = redisNodeFlagsTable + i;
+            if (flags & nodeflag->flag) ci = sdscat(ci, nodeflag->name);
+        }
     }
-    /* If no flag was added, add the "noflags" special flag. */
-    if (sdslen(ci) == orig_len) ci = sdscat(ci,"noflags,");
     sdsIncrLen(ci,-1); /* Remove trailing comma. */
     return ci;
 }
@@ -3863,11 +3697,10 @@ sds clusterGenNodeDescription(clusterNode *node) {
     sds ci;
 
     /* Node coordinates */
-    ci = sdscatprintf(sdsempty(),"%.40s %s:%d@%d ",
+    ci = sdscatprintf(sdsempty(),"%.40s %s:%d ",
         node->name,
         node->ip,
-        node->port,
-        node->cport);
+        node->port);
 
     /* Flags */
     ci = representClusterNodeFlags(ci, node->flags);
@@ -3958,21 +3791,6 @@ sds clusterGenNodesDescription(int filter) {
  * CLUSTER command
  * -------------------------------------------------------------------------- */
 
-const char *clusterGetMessageTypeString(int type) {
-    switch(type) {
-    case CLUSTERMSG_TYPE_PING: return "ping";
-    case CLUSTERMSG_TYPE_PONG: return "pong";
-    case CLUSTERMSG_TYPE_MEET: return "meet";
-    case CLUSTERMSG_TYPE_FAIL: return "fail";
-    case CLUSTERMSG_TYPE_PUBLISH: return "publish";
-    case CLUSTERMSG_TYPE_FAILOVER_AUTH_REQUEST: return "auth-req";
-    case CLUSTERMSG_TYPE_FAILOVER_AUTH_ACK: return "auth-ack";
-    case CLUSTERMSG_TYPE_UPDATE: return "update";
-    case CLUSTERMSG_TYPE_MFSTART: return "mfstart";
-    }
-    return "unknown";
-}
-
 int getSlotOrReply(client *c, robj *o) {
     long long slot;
 
@@ -4065,27 +3883,16 @@ void clusterCommand(client *c) {
         return;
     }
 
-    if (!strcasecmp(c->argv[1]->ptr,"meet") && (c->argc == 4 || c->argc == 5)) {
-        /* CLUSTER MEET <ip> <port> [cport] */
-        long long port, cport;
+    if (!strcasecmp(c->argv[1]->ptr,"meet") && c->argc == 4) {
+        long long port;
 
         if (getLongLongFromObject(c->argv[3], &port) != C_OK) {
-            addReplyErrorFormat(c,"Invalid TCP base port specified: %s",
+            addReplyErrorFormat(c,"Invalid TCP port specified: %s",
                                 (char*)c->argv[3]->ptr);
             return;
         }
 
-        if (c->argc == 5) {
-            if (getLongLongFromObject(c->argv[4], &cport) != C_OK) {
-                addReplyErrorFormat(c,"Invalid TCP bus port specified: %s",
-                                    (char*)c->argv[4]->ptr);
-                return;
-            }
-        } else {
-            cport = port + CLUSTER_PORT_INCR;
-        }
-
-        if (clusterStartHandshake(c->argv[2]->ptr,port,cport) == 0 &&
+        if (clusterStartHandshake(c->argv[2]->ptr,port) == 0 &&
             errno == EINVAL)
         {
             addReplyErrorFormat(c,"Invalid node address specified: %s:%s",
@@ -4200,7 +4007,7 @@ void clusterCommand(client *c) {
             }
             if ((n = clusterLookupNode(c->argv[4]->ptr)) == NULL) {
                 addReplyErrorFormat(c,"I don't know about node %s",
-                    (char*)c->argv[4]->ptr);
+                    (char*)c->argv[3]->ptr);
                 return;
             }
             server.cluster->importing_slots_from[slot] = n;
@@ -4304,6 +4111,8 @@ void clusterCommand(client *c) {
             "cluster_size:%d\r\n"
             "cluster_current_epoch:%llu\r\n"
             "cluster_my_epoch:%llu\r\n"
+            "cluster_stats_messages_sent:%lld\r\n"
+            "cluster_stats_messages_received:%lld\r\n"
             , statestr[server.cluster->state],
             slots_assigned,
             slots_ok,
@@ -4312,36 +4121,10 @@ void clusterCommand(client *c) {
             dictSize(server.cluster->nodes),
             server.cluster->size,
             (unsigned long long) server.cluster->currentEpoch,
-            (unsigned long long) myepoch
+            (unsigned long long) myepoch,
+            server.cluster->stats_bus_messages_sent,
+            server.cluster->stats_bus_messages_received
         );
-
-        /* Show stats about messages sent and received. */
-        long long tot_msg_sent = 0;
-        long long tot_msg_received = 0;
-
-        for (int i = 0; i < CLUSTERMSG_TYPE_COUNT; i++) {
-            if (server.cluster->stats_bus_messages_sent[i] == 0) continue;
-            tot_msg_sent += server.cluster->stats_bus_messages_sent[i];
-            info = sdscatprintf(info,
-                "cluster_stats_messages_%s_sent:%lld\r\n",
-                clusterGetMessageTypeString(i),
-                server.cluster->stats_bus_messages_sent[i]);
-        }
-        info = sdscatprintf(info,
-            "cluster_stats_messages_sent:%lld\r\n", tot_msg_sent);
-
-        for (int i = 0; i < CLUSTERMSG_TYPE_COUNT; i++) {
-            if (server.cluster->stats_bus_messages_received[i] == 0) continue;
-            tot_msg_received += server.cluster->stats_bus_messages_received[i];
-            info = sdscatprintf(info,
-                "cluster_stats_messages_%s_received:%lld\r\n",
-                clusterGetMessageTypeString(i),
-                server.cluster->stats_bus_messages_received[i]);
-        }
-        info = sdscatprintf(info,
-            "cluster_stats_messages_received:%lld\r\n", tot_msg_received);
-
-        /* Produce the reply protocol. */
         addReplySds(c,sdscatprintf(sdsempty(),"$%lu\r\n",
             (unsigned long)sdslen(info)));
         addReplySds(c,info);
@@ -4386,18 +4169,10 @@ void clusterCommand(client *c) {
             return;
         }
 
-        /* Avoid allocating more than needed in case of large COUNT argument
-         * and smaller actual number of keys. */
-        unsigned int keys_in_slot = countKeysInSlot(slot);
-        if (maxkeys > keys_in_slot) maxkeys = keys_in_slot;
-
         keys = zmalloc(sizeof(robj*)*maxkeys);
         numkeys = getKeysInSlot(slot, keys, maxkeys);
         addReplyMultiBulkLen(c,numkeys);
-        for (j = 0; j < numkeys; j++) {
-            addReplyBulk(c,keys[j]);
-            decrRefCount(keys[j]);
-        }
+        for (j = 0; j < numkeys; j++) addReplyBulk(c,keys[j]);
         zfree(keys);
     } else if (!strcasecmp(c->argv[1]->ptr,"forget") && c->argc == 3) {
         /* CLUSTER FORGET <NODE ID> */
@@ -4744,7 +4519,7 @@ void restoreCommand(client *c) {
 
     /* Create the key and set the TTL if any */
     dbAdd(c->db,c->argv[1],obj);
-    if (ttl) setExpire(c,c->db,c->argv[1],mstime()+ttl);
+    if (ttl) setExpire(c->db,c->argv[1],mstime()+ttl);
     signalModifiedKey(c->db,c->argv[1]);
     addReply(c,shared.ok);
     server.dirty++;
@@ -4877,13 +4652,13 @@ void migrateCommand(client *c) {
     int copy, replace, j;
     long timeout;
     long dbid;
+    long long ttl, expireat;
     robj **ov = NULL; /* Objects to migrate. */
     robj **kv = NULL; /* Key names. */
     robj **newargv = NULL; /* Used to rewrite the command as DEL ... keys ... */
     rio cmd, payload;
     int may_retry = 1;
     int write_error = 0;
-    int argv_rewritten = 0;
 
     /* To support the KEYS option we need the following additional state. */
     int first_key = 3; /* Argument index of the first key. */
@@ -4892,6 +4667,7 @@ void migrateCommand(client *c) {
     /* Initialization */
     copy = 0;
     replace = 0;
+    ttl = 0;
 
     /* Parse additional options */
     for (j = 6; j < c->argc; j++) {
@@ -4967,9 +4743,7 @@ try_again:
 
     /* Create RESTORE payload and generate the protocol to call the command. */
     for (j = 0; j < num_keys; j++) {
-        long long ttl = 0;
-        long long expireat = getExpire(c->db,kv[j]);
-
+        expireat = getExpire(c->db,kv[j]);
         if (expireat != -1) {
             ttl = expireat-mstime();
             if (ttl < 1) ttl = 1;
@@ -5067,20 +4841,12 @@ try_again:
         goto socket_err; /* A retry is guaranteed because of tested conditions.*/
     }
 
-    /* On socket errors, close the migration socket now that we still have
-     * the original host/port in the ARGV. Later the original command may be
-     * rewritten to DEL and will be too later. */
-    if (socket_error) migrateCloseSocket(c->argv[1],c->argv[2]);
-
     if (!copy) {
-        /* Translate MIGRATE as DEL for replication/AOF. Note that we do
-         * this only for the keys for which we received an acknowledgement
-         * from the receiving Redis server, by using the del_idx index. */
+        /* Translate MIGRATE as DEL for replication/AOF. */
         if (del_idx > 1) {
             newargv[0] = createStringObject("DEL",3);
             /* Note that the following call takes ownership of newargv. */
             replaceClientCommandVector(c,del_idx,newargv);
-            argv_rewritten = 1;
         } else {
             /* No key transfer acknowledged, no need to rewrite as DEL. */
             zfree(newargv);
@@ -5089,8 +4855,8 @@ try_again:
     }
 
     /* If we are here and a socket error happened, we don't want to retry.
-     * Just signal the problem to the client, but only do it if we did not
-     * already queue a different error reported by the destination server. */
+     * Just signal the problem to the client, but only do it if we don't
+     * already queued a different error reported by the destination server. */
     if (!error_from_target && socket_error) {
         may_retry = 0;
         goto socket_err;
@@ -5098,11 +4864,7 @@ try_again:
 
     if (!error_from_target) {
         /* Success! Update the last_dbid in migrateCachedSocket, so that we can
-         * avoid SELECT the next time if the target DB is the same. Reply +OK.
-         *
-         * Note: If we reached this point, even if socket_error is true
-         * still the SELECT command succeeded (otherwise the code jumps to
-         * socket_err label. */
+         * avoid SELECT the next time if the target DB is the same. Reply +OK. */
         cs->last_dbid = dbid;
         addReply(c,shared.ok);
     } else {
@@ -5112,6 +4874,7 @@ try_again:
 
     sdsfree(cmd.io.buffer.ptr);
     zfree(ov); zfree(kv); zfree(newargv);
+    if (socket_error) migrateCloseSocket(c->argv[1],c->argv[2]);
     return;
 
 /* On socket errors we try to close the cached socket and try again.
@@ -5121,12 +4884,7 @@ socket_err:
     /* Cleanup we want to perform in both the retry and no retry case.
      * Note: Closing the migrate socket will also force SELECT next time. */
     sdsfree(cmd.io.buffer.ptr);
-
-    /* If the command was rewritten as DEL and there was a socket error,
-     * we already closed the socket earlier. While migrateCloseSocket()
-     * is idempotent, the host/port arguments are now gone, so don't do it
-     * again. */
-    if (!argv_rewritten) migrateCloseSocket(c->argv[1],c->argv[2]);
+    migrateCloseSocket(c->argv[1],c->argv[2]);
     zfree(newargv);
     newargv = NULL; /* This will get reallocated on retry. */
 
@@ -5429,9 +5187,8 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
             return 1;
         }
 
-        /* All keys must belong to the same slot, so check first key only. */
         di = dictGetIterator(c->bpop.keys);
-        if ((de = dictNext(di)) != NULL) {
+        while((de = dictNext(di)) != NULL) {
             robj *key = dictGetKey(de);
             int slot = keyHashSlot((char*)key->ptr, sdslen(key->ptr));
             clusterNode *node = server.cluster->slots[slot];
@@ -5449,7 +5206,6 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
                     clusterRedirectClient(c,node,slot,
                         CLUSTER_REDIR_MOVED);
                 }
-                dictReleaseIterator(di);
                 return 1;
             }
         }

src/cluster.h

@@ -73,29 +73,6 @@ typedef struct clusterLink {
 #define CLUSTER_CANT_FAILOVER_WAITING_VOTES 4
 #define CLUSTER_CANT_FAILOVER_RELOG_PERIOD (60*5) /* seconds. */
 
-/* clusterState todo_before_sleep flags. */
-#define CLUSTER_TODO_HANDLE_FAILOVER (1<<0)
-#define CLUSTER_TODO_UPDATE_STATE (1<<1)
-#define CLUSTER_TODO_SAVE_CONFIG (1<<2)
-#define CLUSTER_TODO_FSYNC_CONFIG (1<<3)
-
-/* Message types.
- *
- * Note that the PING, PONG and MEET messages are actually the same exact
- * kind of packet. PONG is the reply to ping, in the exact format as a PING,
- * while MEET is a special PING that forces the receiver to add the sender
- * as a node (if it is not already in the list). */
-#define CLUSTERMSG_TYPE_PING 0          /* Ping */
-#define CLUSTERMSG_TYPE_PONG 1          /* Pong (reply to Ping) */
-#define CLUSTERMSG_TYPE_MEET 2          /* Meet "let's join" message */
-#define CLUSTERMSG_TYPE_FAIL 3          /* Mark node xxx as failing */
-#define CLUSTERMSG_TYPE_PUBLISH 4       /* Pub/Sub Publish propagation */
-#define CLUSTERMSG_TYPE_FAILOVER_AUTH_REQUEST 5 /* May I failover? */
-#define CLUSTERMSG_TYPE_FAILOVER_AUTH_ACK 6     /* Yes, you have my vote */
-#define CLUSTERMSG_TYPE_UPDATE 7        /* Another node slots configuration */
-#define CLUSTERMSG_TYPE_MFSTART 8       /* Pause clients for manual failover */
-#define CLUSTERMSG_TYPE_COUNT 9         /* Total number of message types. */
-
 /* This structure represent elements of node->fail_reports. */
 typedef struct clusterNodeFailReport {
     struct clusterNode *node;  /* Node reporting the failure condition. */
@@ -123,8 +100,7 @@ typedef struct clusterNode {
     mstime_t orphaned_time;     /* Starting time of orphaned master condition */
     long long repl_offset;      /* Last known repl offset for this node. */
     char ip[NET_IP_STR_LEN];  /* Latest known IP address of this node */
-    int port;                   /* Latest known clients port of this node */
-    int cport;                  /* Latest known cluster port of this node. */
+    int port;                   /* Latest known port of this node */
     clusterLink *link;          /* TCP/IP link with this node */
     list *fail_reports;         /* List of nodes signaling this as failing */
 } clusterNode;
@@ -139,8 +115,7 @@ typedef struct clusterState {
     clusterNode *migrating_slots_to[CLUSTER_SLOTS];
     clusterNode *importing_slots_from[CLUSTER_SLOTS];
     clusterNode *slots[CLUSTER_SLOTS];
-    uint64_t slots_keys_count[CLUSTER_SLOTS];
-    rax *slots_to_keys;
+    zskiplist *slots_to_keys;
     /* The following fields are used to take the slave state on elections. */
     mstime_t failover_auth_time; /* Time of previous or next election. */
     int failover_auth_count;    /* Number of votes received so far. */
@@ -162,15 +137,32 @@ typedef struct clusterState {
     /* The followign fields are used by masters to take state on elections. */
     uint64_t lastVoteEpoch;     /* Epoch of the last vote granted. */
     int todo_before_sleep; /* Things to do in clusterBeforeSleep(). */
-    /* Messages received and sent by type. */
-    long long stats_bus_messages_sent[CLUSTERMSG_TYPE_COUNT];
-    long long stats_bus_messages_received[CLUSTERMSG_TYPE_COUNT];
-    long long stats_pfail_nodes;    /* Number of nodes in PFAIL status,
-                                       excluding nodes without address. */
+    long long stats_bus_messages_sent;  /* Num of msg sent via cluster bus. */
+    long long stats_bus_messages_received; /* Num of msg rcvd via cluster bus.*/
 } clusterState;
 
+/* clusterState todo_before_sleep flags. */
+#define CLUSTER_TODO_HANDLE_FAILOVER (1<<0)
+#define CLUSTER_TODO_UPDATE_STATE (1<<1)
+#define CLUSTER_TODO_SAVE_CONFIG (1<<2)
+#define CLUSTER_TODO_FSYNC_CONFIG (1<<3)
+
 /* Redis cluster messages header */
 
+/* Note that the PING, PONG and MEET messages are actually the same exact
+ * kind of packet. PONG is the reply to ping, in the exact format as a PING,
+ * while MEET is a special PING that forces the receiver to add the sender
+ * as a node (if it is not already in the list). */
+#define CLUSTERMSG_TYPE_PING 0          /* Ping */
+#define CLUSTERMSG_TYPE_PONG 1          /* Pong (reply to Ping) */
+#define CLUSTERMSG_TYPE_MEET 2          /* Meet "let's join" message */
+#define CLUSTERMSG_TYPE_FAIL 3          /* Mark node xxx as failing */
+#define CLUSTERMSG_TYPE_PUBLISH 4       /* Pub/Sub Publish propagation */
+#define CLUSTERMSG_TYPE_FAILOVER_AUTH_REQUEST 5 /* May I failover? */
+#define CLUSTERMSG_TYPE_FAILOVER_AUTH_ACK 6     /* Yes, you have my vote */
+#define CLUSTERMSG_TYPE_UPDATE 7        /* Another node slots configuration */
+#define CLUSTERMSG_TYPE_MFSTART 8       /* Pause clients for manual failover */
+
 /* Initially we don't know our "name", but we'll find it once we connect
  * to the first node, using the getsockname() function. Then we'll use this
  * address for all the next messages. */
@@ -179,10 +171,10 @@ typedef struct {
     uint32_t ping_sent;
     uint32_t pong_received;
     char ip[NET_IP_STR_LEN];  /* IP address last time it was seen */
-    uint16_t port;              /* base port last time it was seen */
-    uint16_t cport;             /* cluster port last time it was seen */
+    uint16_t port;              /* port last time it was seen */
     uint16_t flags;             /* node->flags copy */
-    uint32_t notused1;
+    uint16_t notused1;          /* Some room for future improvements. */
+    uint32_t notused2;
 } clusterMsgDataGossip;
 
 typedef struct {
@@ -227,13 +219,13 @@ union clusterMsgData {
     } update;
 };
 
-#define CLUSTER_PROTO_VER 1 /* Cluster bus protocol version. */
+#define CLUSTER_PROTO_VER 0 /* Cluster bus protocol version. */
 
 typedef struct {
     char sig[4];        /* Siganture "RCmb" (Redis Cluster message bus). */
     uint32_t totlen;    /* Total length of this message */
-    uint16_t ver;       /* Protocol version, currently set to 1. */
-    uint16_t port;      /* TCP base port number. */
+    uint16_t ver;       /* Protocol version, currently set to 0. */
+    uint16_t notused0;  /* 2 bytes not used. */
     uint16_t type;      /* Message type */
     uint16_t count;     /* Only used for some kind of messages. */
     uint64_t currentEpoch;  /* The epoch accordingly to the sending node. */
@@ -245,10 +237,9 @@ typedef struct {
     char sender[CLUSTER_NAMELEN]; /* Name of the sender node */
     unsigned char myslots[CLUSTER_SLOTS/8];
     char slaveof[CLUSTER_NAMELEN];
-    char myip[NET_IP_STR_LEN];    /* Sender IP, if not all zeroed. */
-    char notused1[34];  /* 34 bytes reserved for future usage. */
-    uint16_t cport;      /* Sender TCP cluster bus port */
-    uint16_t flags;      /* Sender node flags */
+    char notused1[32];  /* 32 bytes reserved for future usage. */
+    uint16_t port;      /* Sender TCP base port */
+    uint16_t flags;     /* Sender node flags */
     unsigned char state; /* Cluster state from the POV of the sender */
     unsigned char mflags[3]; /* Message flags: CLUSTERMSG_FLAG[012]_... */
     union clusterMsgData data;

src/config.c

@@ -45,11 +45,9 @@ typedef struct configEnum {
 
 configEnum maxmemory_policy_enum[] = {
     {"volatile-lru", MAXMEMORY_VOLATILE_LRU},
-    {"volatile-lfu", MAXMEMORY_VOLATILE_LFU},
     {"volatile-random",MAXMEMORY_VOLATILE_RANDOM},
     {"volatile-ttl",MAXMEMORY_VOLATILE_TTL},
     {"allkeys-lru",MAXMEMORY_ALLKEYS_LRU},
-    {"allkeys-lfu",MAXMEMORY_ALLKEYS_LFU},
     {"allkeys-random",MAXMEMORY_ALLKEYS_RANDOM},
     {"noeviction",MAXMEMORY_NO_EVICTION},
     {NULL, 0}
@@ -155,20 +153,6 @@ void resetServerSaveParams(void) {
     server.saveparamslen = 0;
 }
 
-void queueLoadModule(sds path, sds *argv, int argc) {
-    int i;
-    struct moduleLoadQueueEntry *loadmod;
-
-    loadmod = zmalloc(sizeof(struct moduleLoadQueueEntry));
-    loadmod->argv = zmalloc(sizeof(robj*)*argc);
-    loadmod->path = sdsnew(path);
-    loadmod->argc = argc;
-    for (i = 0; i < argc; i++) {
-        loadmod->argv[i] = createRawStringObject(argv[i],sdslen(argv[i]));
-    }
-    listAddNodeTail(server.loadmodule_queue,loadmod);
-}
-
 void loadServerConfigFromString(char *config) {
     char *err = NULL;
     int linenum = 0, totlines, i;
@@ -283,10 +267,6 @@ void loadServerConfigFromString(char *config) {
                 }
                 fclose(logfp);
             }
-        } else if (!strcasecmp(argv[0],"always-show-logo") && argc == 2) {
-            if ((server.always_show_logo = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"syslog-enabled") && argc == 2) {
             if ((server.syslog_enabled = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
@@ -328,18 +308,6 @@ void loadServerConfigFromString(char *config) {
                 err = "maxmemory-samples must be 1 or greater";
                 goto loaderr;
             }
-        } else if (!strcasecmp(argv[0],"lfu-log-factor") && argc == 2) {
-            server.lfu_log_factor = atoi(argv[1]);
-            if (server.maxmemory_samples < 0) {
-                err = "lfu-log-factor must be 0 or greater";
-                goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"lfu-decay-time") && argc == 2) {
-            server.lfu_decay_time = atoi(argv[1]);
-            if (server.maxmemory_samples < 1) {
-                err = "lfu-decay-time must be 0 or greater";
-                goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"slaveof") && argc == 3) {
             slaveof_linenum = linenum;
             server.masterhost = sdsnew(argv[1]);
@@ -407,26 +375,6 @@ void loadServerConfigFromString(char *config) {
             if ((server.activerehashing = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
-        } else if (!strcasecmp(argv[0],"lazyfree-lazy-eviction") && argc == 2) {
-            if ((server.lazyfree_lazy_eviction = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"lazyfree-lazy-expire") && argc == 2) {
-            if ((server.lazyfree_lazy_expire = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"lazyfree-lazy-server-del") && argc == 2){
-            if ((server.lazyfree_lazy_server_del = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"slave-lazy-flush") && argc == 2) {
-            if ((server.repl_slave_lazy_flush = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"activedefrag") && argc == 2) {
-            if ((server.active_defrag_enabled = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"daemonize") && argc == 2) {
             if ((server.daemonize = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
@@ -483,10 +431,6 @@ void loadServerConfigFromString(char *config) {
             if ((server.aof_load_truncated = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
-        } else if (!strcasecmp(argv[0],"aof-use-rdb-preamble") && argc == 2) {
-            if ((server.aof_use_rdb_preamble = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"requirepass") && argc == 2) {
             if (strlen(argv[1]) > CONFIG_AUTHPASS_MAX_LEN) {
                 err = "Password is longer than CONFIG_AUTHPASS_MAX_LEN";
@@ -503,36 +447,6 @@ void loadServerConfigFromString(char *config) {
             }
             zfree(server.rdb_filename);
             server.rdb_filename = zstrdup(argv[1]);
-        } else if (!strcasecmp(argv[0],"active-defrag-threshold-lower") && argc == 2) {
-            server.active_defrag_threshold_lower = atoi(argv[1]);
-            if (server.active_defrag_threshold_lower < 0) {
-                err = "active-defrag-threshold-lower must be 0 or greater";
-                goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"active-defrag-threshold-upper") && argc == 2) {
-            server.active_defrag_threshold_upper = atoi(argv[1]);
-            if (server.active_defrag_threshold_upper < 0) {
-                err = "active-defrag-threshold-upper must be 0 or greater";
-                goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"active-defrag-ignore-bytes") && argc == 2) {
-            server.active_defrag_ignore_bytes = memtoll(argv[1], NULL);
-            if (server.active_defrag_ignore_bytes <= 0) {
-                err = "active-defrag-ignore-bytes must above 0";
-                goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"active-defrag-cycle-min") && argc == 2) {
-            server.active_defrag_cycle_min = atoi(argv[1]);
-            if (server.active_defrag_cycle_min < 1 || server.active_defrag_cycle_min > 99) {
-                err = "active-defrag-cycle-min must be between 1 and 99";
-                goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"active-defrag-cycle-max") && argc == 2) {
-            server.active_defrag_cycle_max = atoi(argv[1]);
-            if (server.active_defrag_cycle_max < 1 || server.active_defrag_cycle_max > 99) {
-                err = "active-defrag-cycle-max must be between 1 and 99";
-                goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"hash-max-ziplist-entries") && argc == 2) {
             server.hash_max_ziplist_entries = memtoll(argv[1], NULL);
         } else if (!strcasecmp(argv[0],"hash-max-ziplist-value") && argc == 2) {
@@ -584,25 +498,6 @@ void loadServerConfigFromString(char *config) {
         } else if (!strcasecmp(argv[0],"cluster-config-file") && argc == 2) {
             zfree(server.cluster_configfile);
             server.cluster_configfile = zstrdup(argv[1]);
-        } else if (!strcasecmp(argv[0],"cluster-announce-ip") && argc == 2) {
-            zfree(server.cluster_announce_ip);
-            server.cluster_announce_ip = zstrdup(argv[1]);
-        } else if (!strcasecmp(argv[0],"cluster-announce-port") && argc == 2) {
-            server.cluster_announce_port = atoi(argv[1]);
-            if (server.cluster_announce_port < 0 ||
-                server.cluster_announce_port > 65535)
-            {
-                err = "Invalid port"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"cluster-announce-bus-port") &&
-                   argc == 2)
-        {
-            server.cluster_announce_bus_port = atoi(argv[1]);
-            if (server.cluster_announce_bus_port < 0 ||
-                server.cluster_announce_bus_port > 65535)
-            {
-                err = "Invalid port"; goto loaderr;
-            }
         } else if (!strcasecmp(argv[0],"cluster-require-full-coverage") &&
                     argc == 2)
         {
@@ -713,8 +608,6 @@ void loadServerConfigFromString(char *config) {
                     "Allowed values: 'upstart', 'systemd', 'auto', or 'no'";
                 goto loaderr;
             }
-        } else if (!strcasecmp(argv[0],"loadmodule") && argc >= 2) {
-            queueLoadModule(argv[1],&argv[2],argc-2);
         } else if (!strcasecmp(argv[0],"sentinel")) {
             /* argc == 1 is handled by main() as we need to enter the sentinel
              * mode ASAP. */
@@ -847,9 +740,6 @@ void configSetCommand(client *c) {
     } config_set_special_field("masterauth") {
         zfree(server.masterauth);
         server.masterauth = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
-    } config_set_special_field("cluster-announce-ip") {
-        zfree(server.cluster_announce_ip);
-        server.cluster_announce_ip = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
     } config_set_special_field("maxclients") {
         int orig_value = server.maxclients;
 
@@ -997,38 +887,16 @@ void configSetCommand(client *c) {
       "aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync) {
     } config_set_bool_field(
       "aof-load-truncated",server.aof_load_truncated) {
-    } config_set_bool_field(
-      "aof-use-rdb-preamble",server.aof_use_rdb_preamble) {
     } config_set_bool_field(
       "slave-serve-stale-data",server.repl_serve_stale_data) {
     } config_set_bool_field(
       "slave-read-only",server.repl_slave_ro) {
     } config_set_bool_field(
       "activerehashing",server.activerehashing) {
-    } config_set_bool_field(
-      "activedefrag",server.active_defrag_enabled) {
-#ifndef HAVE_DEFRAG
-        if (server.active_defrag_enabled) {
-            server.active_defrag_enabled = 0;
-            addReplyError(c,
-                "Active defragmentation cannot be enabled: it requires a "
-                "Redis server compiled with a modified Jemalloc like the "
-                "one shipped by default with the Redis source distribution");
-            return;
-        }
-#endif
     } config_set_bool_field(
       "protected-mode",server.protected_mode) {
     } config_set_bool_field(
       "stop-writes-on-bgsave-error",server.stop_writes_on_bgsave_err) {
-    } config_set_bool_field(
-      "lazyfree-lazy-eviction",server.lazyfree_lazy_eviction) {
-    } config_set_bool_field(
-      "lazyfree-lazy-expire",server.lazyfree_lazy_expire) {
-    } config_set_bool_field(
-      "lazyfree-lazy-server-del",server.lazyfree_lazy_server_del) {
-    } config_set_bool_field(
-      "slave-lazy-flush",server.repl_slave_lazy_flush) {
     } config_set_bool_field(
       "no-appendfsync-on-rewrite",server.aof_no_fsync_on_rewrite) {
 
@@ -1038,24 +906,12 @@ void configSetCommand(client *c) {
       "tcp-keepalive",server.tcpkeepalive,0,LLONG_MAX) {
     } config_set_numerical_field(
       "maxmemory-samples",server.maxmemory_samples,1,LLONG_MAX) {
-    } config_set_numerical_field(
-      "lfu-log-factor",server.lfu_log_factor,0,LLONG_MAX) {
-    } config_set_numerical_field(
-      "lfu-decay-time",server.lfu_decay_time,0,LLONG_MAX) {
     } config_set_numerical_field(
       "timeout",server.maxidletime,0,LONG_MAX) {
-    } config_set_numerical_field(
-      "active-defrag-threshold-lower",server.active_defrag_threshold_lower,0,1000) {
-    } config_set_numerical_field(
-      "active-defrag-threshold-upper",server.active_defrag_threshold_upper,0,1000) {
-    } config_set_memory_field(
-      "active-defrag-ignore-bytes",server.active_defrag_ignore_bytes) {
-    } config_set_numerical_field(
-      "active-defrag-cycle-min",server.active_defrag_cycle_min,1,99) {
-    } config_set_numerical_field(
-      "active-defrag-cycle-max",server.active_defrag_cycle_max,1,99) {
     } config_set_numerical_field(
       "auto-aof-rewrite-percentage",server.aof_rewrite_perc,0,LLONG_MAX){
+    } config_set_numerical_field(
+      "auto-aof-rewrite-min-size",server.aof_rewrite_min_size,0,LLONG_MAX) {
     } config_set_numerical_field(
       "hash-max-ziplist-entries",server.hash_max_ziplist_entries,0,LLONG_MAX) {
     } config_set_numerical_field(
@@ -1102,10 +958,6 @@ void configSetCommand(client *c) {
         refreshGoodSlavesCount();
     } config_set_numerical_field(
       "cluster-node-timeout",server.cluster_node_timeout,0,LLONG_MAX) {
-    } config_set_numerical_field(
-      "cluster-announce-port",server.cluster_announce_port,0,65535) {
-    } config_set_numerical_field(
-      "cluster-announce-bus-port",server.cluster_announce_bus_port,0,65535) {
     } config_set_numerical_field(
       "cluster-migration-barrier",server.cluster_migration_barrier,0,LLONG_MAX){
     } config_set_numerical_field(
@@ -1134,8 +986,6 @@ void configSetCommand(client *c) {
         }
     } config_set_memory_field("repl-backlog-size",ll) {
         resizeReplicationBacklog(ll);
-    } config_set_memory_field("auto-aof-rewrite-min-size",ll) {
-        server.aof_rewrite_min_size = ll;
 
     /* Enumeration fields.
      * config_set_enum_field(name,var,enum_var) */
@@ -1212,7 +1062,6 @@ void configGetCommand(client *c) {
     config_get_string_field("dbfilename",server.rdb_filename);
     config_get_string_field("requirepass",server.requirepass);
     config_get_string_field("masterauth",server.masterauth);
-    config_get_string_field("cluster-announce-ip",server.cluster_announce_ip);
     config_get_string_field("unixsocket",server.unixsocket);
     config_get_string_field("logfile",server.logfile);
     config_get_string_field("pidfile",server.pidfile);
@@ -1222,11 +1071,6 @@ void configGetCommand(client *c) {
     config_get_numerical_field("maxmemory",server.maxmemory);
     config_get_numerical_field("maxmemory-samples",server.maxmemory_samples);
     config_get_numerical_field("timeout",server.maxidletime);
-    config_get_numerical_field("active-defrag-threshold-lower",server.active_defrag_threshold_lower);
-    config_get_numerical_field("active-defrag-threshold-upper",server.active_defrag_threshold_upper);
-    config_get_numerical_field("active-defrag-ignore-bytes",server.active_defrag_ignore_bytes);
-    config_get_numerical_field("active-defrag-cycle-min",server.active_defrag_cycle_min);
-    config_get_numerical_field("active-defrag-cycle-max",server.active_defrag_cycle_max);
     config_get_numerical_field("auto-aof-rewrite-percentage",
             server.aof_rewrite_perc);
     config_get_numerical_field("auto-aof-rewrite-min-size",
@@ -1255,8 +1099,6 @@ void configGetCommand(client *c) {
     config_get_numerical_field("slowlog-max-len",
             server.slowlog_max_len);
     config_get_numerical_field("port",server.port);
-    config_get_numerical_field("cluster-announce-port",server.cluster_announce_port);
-    config_get_numerical_field("cluster-announce-bus-port",server.cluster_announce_bus_port);
     config_get_numerical_field("tcp-backlog",server.tcp_backlog);
     config_get_numerical_field("databases",server.dbnum);
     config_get_numerical_field("repl-ping-slave-period",server.repl_ping_slave_period);
@@ -1291,7 +1133,6 @@ void configGetCommand(client *c) {
     config_get_bool_field("rdbcompression", server.rdb_compression);
     config_get_bool_field("rdbchecksum", server.rdb_checksum);
     config_get_bool_field("activerehashing", server.activerehashing);
-    config_get_bool_field("activedefrag", server.active_defrag_enabled);
     config_get_bool_field("protected-mode", server.protected_mode);
     config_get_bool_field("repl-disable-tcp-nodelay",
             server.repl_disable_tcp_nodelay);
@@ -1301,16 +1142,6 @@ void configGetCommand(client *c) {
             server.aof_rewrite_incremental_fsync);
     config_get_bool_field("aof-load-truncated",
             server.aof_load_truncated);
-    config_get_bool_field("aof-use-rdb-preamble",
-            server.aof_use_rdb_preamble);
-    config_get_bool_field("lazyfree-lazy-eviction",
-            server.lazyfree_lazy_eviction);
-    config_get_bool_field("lazyfree-lazy-expire",
-            server.lazyfree_lazy_expire);
-    config_get_bool_field("lazyfree-lazy-server-del",
-            server.lazyfree_lazy_server_del);
-    config_get_bool_field("slave-lazy-flush",
-            server.repl_slave_lazy_flush);
 
     /* Enum values */
     config_get_enum_field("maxmemory-policy",
@@ -1423,7 +1254,7 @@ void configGetCommand(client *c) {
 /* We use the following dictionary type to store where a configuration
  * option is mentioned in the old configuration file, so it's
  * like "maxmemory" -> list of line numbers (first line is zero). */
-uint64_t dictSdsCaseHash(const void *key);
+unsigned int dictSdsCaseHash(const void *key);
 int dictSdsKeyCaseCompare(void *privdata, const void *key1, const void *key2);
 void dictSdsDestructor(void *privdata, void *val);
 void dictListDestructor(void *privdata, void *val);
@@ -1950,8 +1781,6 @@ int rewriteConfig(char *path) {
     rewriteConfigYesNoOption(state,"daemonize",server.daemonize,0);
     rewriteConfigStringOption(state,"pidfile",server.pidfile,CONFIG_DEFAULT_PID_FILE);
     rewriteConfigNumericalOption(state,"port",server.port,CONFIG_DEFAULT_SERVER_PORT);
-    rewriteConfigNumericalOption(state,"cluster-announce-port",server.cluster_announce_port,CONFIG_DEFAULT_CLUSTER_ANNOUNCE_PORT);
-    rewriteConfigNumericalOption(state,"cluster-announce-bus-port",server.cluster_announce_bus_port,CONFIG_DEFAULT_CLUSTER_ANNOUNCE_BUS_PORT);
     rewriteConfigNumericalOption(state,"tcp-backlog",server.tcp_backlog,CONFIG_DEFAULT_TCP_BACKLOG);
     rewriteConfigBindOption(state);
     rewriteConfigStringOption(state,"unixsocket",server.unixsocket,NULL);
@@ -1974,7 +1803,6 @@ int rewriteConfig(char *path) {
     rewriteConfigSlaveofOption(state);
     rewriteConfigStringOption(state,"slave-announce-ip",server.slave_announce_ip,CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP);
     rewriteConfigStringOption(state,"masterauth",server.masterauth,NULL);
-    rewriteConfigStringOption(state,"cluster-announce-ip",server.cluster_announce_ip,NULL);
     rewriteConfigYesNoOption(state,"slave-serve-stale-data",server.repl_serve_stale_data,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA);
     rewriteConfigYesNoOption(state,"slave-read-only",server.repl_slave_ro,CONFIG_DEFAULT_SLAVE_READ_ONLY);
     rewriteConfigNumericalOption(state,"repl-ping-slave-period",server.repl_ping_slave_period,CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD);
@@ -1992,11 +1820,6 @@ int rewriteConfig(char *path) {
     rewriteConfigBytesOption(state,"maxmemory",server.maxmemory,CONFIG_DEFAULT_MAXMEMORY);
     rewriteConfigEnumOption(state,"maxmemory-policy",server.maxmemory_policy,maxmemory_policy_enum,CONFIG_DEFAULT_MAXMEMORY_POLICY);
     rewriteConfigNumericalOption(state,"maxmemory-samples",server.maxmemory_samples,CONFIG_DEFAULT_MAXMEMORY_SAMPLES);
-    rewriteConfigNumericalOption(state,"active-defrag-threshold-lower",server.active_defrag_threshold_lower,CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER);
-    rewriteConfigNumericalOption(state,"active-defrag-threshold-upper",server.active_defrag_threshold_upper,CONFIG_DEFAULT_DEFRAG_THRESHOLD_UPPER);
-    rewriteConfigBytesOption(state,"active-defrag-ignore-bytes",server.active_defrag_ignore_bytes,CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES);
-    rewriteConfigNumericalOption(state,"active-defrag-cycle-min",server.active_defrag_cycle_min,CONFIG_DEFAULT_DEFRAG_CYCLE_MIN);
-    rewriteConfigNumericalOption(state,"active-defrag-cycle-max",server.active_defrag_cycle_max,CONFIG_DEFAULT_DEFRAG_CYCLE_MAX);
     rewriteConfigYesNoOption(state,"appendonly",server.aof_state != AOF_OFF,0);
     rewriteConfigStringOption(state,"appendfilename",server.aof_filename,CONFIG_DEFAULT_AOF_FILENAME);
     rewriteConfigEnumOption(state,"appendfsync",server.aof_fsync,aof_fsync_enum,CONFIG_DEFAULT_AOF_FSYNC);
@@ -2023,18 +1846,12 @@ int rewriteConfig(char *path) {
     rewriteConfigNumericalOption(state,"zset-max-ziplist-value",server.zset_max_ziplist_value,OBJ_ZSET_MAX_ZIPLIST_VALUE);
     rewriteConfigNumericalOption(state,"hll-sparse-max-bytes",server.hll_sparse_max_bytes,CONFIG_DEFAULT_HLL_SPARSE_MAX_BYTES);
     rewriteConfigYesNoOption(state,"activerehashing",server.activerehashing,CONFIG_DEFAULT_ACTIVE_REHASHING);
-    rewriteConfigYesNoOption(state,"activedefrag",server.active_defrag_enabled,CONFIG_DEFAULT_ACTIVE_DEFRAG);
     rewriteConfigYesNoOption(state,"protected-mode",server.protected_mode,CONFIG_DEFAULT_PROTECTED_MODE);
     rewriteConfigClientoutputbufferlimitOption(state);
     rewriteConfigNumericalOption(state,"hz",server.hz,CONFIG_DEFAULT_HZ);
     rewriteConfigYesNoOption(state,"aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync,CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC);
     rewriteConfigYesNoOption(state,"aof-load-truncated",server.aof_load_truncated,CONFIG_DEFAULT_AOF_LOAD_TRUNCATED);
-    rewriteConfigYesNoOption(state,"aof-use-rdb-preamble",server.aof_use_rdb_preamble,CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE);
     rewriteConfigEnumOption(state,"supervised",server.supervised_mode,supervised_mode_enum,SUPERVISED_NONE);
-    rewriteConfigYesNoOption(state,"lazyfree-lazy-eviction",server.lazyfree_lazy_eviction,CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION);
-    rewriteConfigYesNoOption(state,"lazyfree-lazy-expire",server.lazyfree_lazy_expire,CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE);
-    rewriteConfigYesNoOption(state,"lazyfree-lazy-server-del",server.lazyfree_lazy_server_del,CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL);
-    rewriteConfigYesNoOption(state,"slave-lazy-flush",server.repl_slave_lazy_flush,CONFIG_DEFAULT_SLAVE_LAZY_FLUSH);
 
     /* Rewrite Sentinel config if in Sentinel mode. */
     if (server.sentinel_mode) rewriteConfigSentinelOption(state);

src/config.h

@@ -206,22 +206,4 @@ void setproctitle(const char *fmt, ...);
 #endif
 #endif
 
-/* Make sure we can test for ARM just checking for __arm__, since sometimes
- * __arm is defined but __arm__ is not. */
-#if defined(__arm) && !defined(__arm__)
-#define __arm__
-#endif
-#if defined (__aarch64__) && !defined(__arm64__)
-#define __arm64__
-#endif
-
-/* Make sure we can test for SPARC just checking for __sparc__. */
-#if defined(__sparc) && !defined(__sparc__)
-#define __sparc__
-#endif
-
-#if defined(__sparc__) || defined(__arm__)
-#define USE_ALIGNED_ACCESS
-#endif
-
 #endif

src/db.c

@@ -29,11 +29,14 @@
 
 #include "server.h"
 #include "cluster.h"
-#include "atomicvar.h"
 
 #include <signal.h>
 #include <ctype.h>
 
+void slotToKeyAdd(robj *key);
+void slotToKeyDel(robj *key);
+void slotToKeyFlush(void);
+
 /*-----------------------------------------------------------------------------
  * C-level DB API
  *----------------------------------------------------------------------------*/
@@ -53,13 +56,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
             server.aof_child_pid == -1 &&
             !(flags & LOOKUP_NOTOUCH))
         {
-            if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-                unsigned long ldt = val->lru >> 8;
-                unsigned long counter = LFULogIncr(val->lru & 255);
-                val->lru = (ldt << 8) | counter;
-            } else {
-                val->lru = LRU_CLOCK();
-            }
+            val->lru = LRU_CLOCK();
         }
         return val;
     } else {
@@ -93,7 +90,7 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
 
     if (expireIfNeeded(db,key) == 1) {
         /* Key expired. If we are in the context of a master, expireIfNeeded()
-         * returns 0 only when the key does not exist at all, so it's safe
+         * returns 0 only when the key does not exist at all, so it's save
          * to return NULL ASAP. */
         if (server.masterhost == NULL) return NULL;
 
@@ -175,14 +172,7 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
     dictEntry *de = dictFind(db->dict,key->ptr);
 
     serverAssertWithInfo(NULL,key,de != NULL);
-    if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-        robj *old = dictGetVal(de);
-        int saved_lru = old->lru;
-        dictReplace(db->dict, key->ptr, val);
-        val->lru = saved_lru;
-    } else {
-        dictReplace(db->dict, key->ptr, val);
-    }
+    dictReplace(db->dict, key->ptr, val);
 }
 
 /* High level Set operation. This function can be used in order to set
@@ -190,9 +180,7 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
  *
  * 1) The ref count of the value object is incremented.
  * 2) clients WATCHing for the destination key notified.
- * 3) The expire time of the key is reset (the key is made persistent).
- *
- * All the new keys in the database should be craeted via this interface. */
+ * 3) The expire time of the key is reset (the key is made persistent). */
 void setKey(redisDb *db, robj *key, robj *val) {
     if (lookupKeyWrite(db,key) == NULL) {
         dbAdd(db,key,val);
@@ -235,7 +223,7 @@ robj *dbRandomKey(redisDb *db) {
 }
 
 /* Delete a key, value, and associated expiration entry if any, from the DB */
-int dbSyncDelete(redisDb *db, robj *key) {
+int dbDelete(redisDb *db, robj *key) {
     /* Deleting an entry from the expires dict will not free the sds of
      * the key, because it is shared with the main dictionary. */
     if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
@@ -247,13 +235,6 @@ int dbSyncDelete(redisDb *db, robj *key) {
     }
 }
 
-/* This is a wrapper whose behavior depends on the Redis lazy free
- * configuration. Deletes the key synchronously or asynchronously. */
-int dbDelete(redisDb *db, robj *key) {
-    return server.lazyfree_lazy_server_del ? dbAsyncDelete(db,key) :
-                                             dbSyncDelete(db,key);
-}
-
 /* Prepare the string object stored at 'key' to be modified destructively
  * to implement commands like SETBIT or APPEND.
  *
@@ -292,47 +273,16 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o) {
     return o;
 }
 
-/* Remove all keys from all the databases in a Redis server.
- * If callback is given the function is called from time to time to
- * signal that work is in progress.
- *
- * The dbnum can be -1 if all teh DBs should be flushed, or the specified
- * DB number if we want to flush only a single Redis database number.
- *
- * Flags are be EMPTYDB_NO_FLAGS if no special flags are specified or
- * EMPTYDB_ASYNC if we want the memory to be freed in a different thread
- * and the function to return ASAP.
- *
- * On success the fuction returns the number of keys removed from the
- * database(s). Otherwise -1 is returned in the specific case the
- * DB number is out of range, and errno is set to EINVAL. */
-long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
-    int j, async = (flags & EMPTYDB_ASYNC);
+long long emptyDb(void(callback)(void*)) {
+    int j;
     long long removed = 0;
 
-    if (dbnum < -1 || dbnum >= server.dbnum) {
-        errno = EINVAL;
-        return -1;
-    }
-
     for (j = 0; j < server.dbnum; j++) {
-        if (dbnum != -1 && dbnum != j) continue;
         removed += dictSize(server.db[j].dict);
-        if (async) {
-            emptyDbAsync(&server.db[j]);
-        } else {
-            dictEmpty(server.db[j].dict,callback);
-            dictEmpty(server.db[j].expires,callback);
-        }
+        dictEmpty(server.db[j].dict,callback);
+        dictEmpty(server.db[j].expires,callback);
     }
-    if (server.cluster_enabled) {
-        if (async) {
-            slotToKeyFlushAsync();
-        } else {
-            slotToKeyFlush();
-        }
-    }
-    if (dbnum == -1) flushSlaveKeysWithExpireList();
+    if (server.cluster_enabled) slotToKeyFlush();
     return removed;
 }
 
@@ -364,49 +314,18 @@ void signalFlushedDb(int dbid) {
  * Type agnostic commands operating on the key space
  *----------------------------------------------------------------------------*/
 
-/* Return the set of flags to use for the emptyDb() call for FLUSHALL
- * and FLUSHDB commands.
- *
- * Currently the command just attempts to parse the "ASYNC" option. It
- * also checks if the command arity is wrong.
- *
- * On success C_OK is returned and the flags are stored in *flags, otherwise
- * C_ERR is returned and the function sends an error to the client. */
-int getFlushCommandFlags(client *c, int *flags) {
-    /* Parse the optional ASYNC option. */
-    if (c->argc > 1) {
-        if (c->argc > 2 || strcasecmp(c->argv[1]->ptr,"async")) {
-            addReply(c,shared.syntaxerr);
-            return C_ERR;
-        }
-        *flags = EMPTYDB_ASYNC;
-    } else {
-        *flags = EMPTYDB_NO_FLAGS;
-    }
-    return C_OK;
-}
-
-/* FLUSHDB [ASYNC]
- *
- * Flushes the currently SELECTed Redis DB. */
 void flushdbCommand(client *c) {
-    int flags;
-
-    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
+    server.dirty += dictSize(c->db->dict);
     signalFlushedDb(c->db->id);
-    server.dirty += emptyDb(c->db->id,flags,NULL);
+    dictEmpty(c->db->dict,NULL);
+    dictEmpty(c->db->expires,NULL);
+    if (server.cluster_enabled) slotToKeyFlush();
     addReply(c,shared.ok);
 }
 
-/* FLUSHALL [ASYNC]
- *
- * Flushes the whole server data set. */
 void flushallCommand(client *c) {
-    int flags;
-
-    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
     signalFlushedDb(-1);
-    server.dirty += emptyDb(-1,flags,NULL);
+    server.dirty += emptyDb(NULL);
     addReply(c,shared.ok);
     if (server.rdb_child_pid != -1) {
         kill(server.rdb_child_pid,SIGUSR1);
@@ -416,39 +335,26 @@ void flushallCommand(client *c) {
         /* Normally rdbSave() will reset dirty, but we don't want this here
          * as otherwise FLUSHALL will not be replicated nor put into the AOF. */
         int saved_dirty = server.dirty;
-        rdbSaveInfo rsi, *rsiptr;
-        rsiptr = rdbPopulateSaveInfo(&rsi);
-        rdbSave(server.rdb_filename,rsiptr);
+        rdbSave(server.rdb_filename);
         server.dirty = saved_dirty;
     }
     server.dirty++;
 }
 
-/* This command implements DEL and LAZYDEL. */
-void delGenericCommand(client *c, int lazy) {
-    int numdel = 0, j;
+void delCommand(client *c) {
+    int deleted = 0, j;
 
     for (j = 1; j < c->argc; j++) {
         expireIfNeeded(c->db,c->argv[j]);
-        int deleted  = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
-                              dbSyncDelete(c->db,c->argv[j]);
-        if (deleted) {
+        if (dbDelete(c->db,c->argv[j])) {
             signalModifiedKey(c->db,c->argv[j]);
             notifyKeyspaceEvent(NOTIFY_GENERIC,
                 "del",c->argv[j],c->db->id);
             server.dirty++;
-            numdel++;
+            deleted++;
         }
     }
-    addReplyLongLong(c,numdel);
-}
-
-void delCommand(client *c) {
-    delGenericCommand(c,0);
-}
-
-void unlinkCommand(client *c) {
-    delGenericCommand(c,1);
+    addReplyLongLong(c,deleted);
 }
 
 /* EXISTS key1 key2 ... key_N.
@@ -476,7 +382,7 @@ void selectCommand(client *c) {
         return;
     }
     if (selectDb(c,id) == C_ERR) {
-        addReplyError(c,"DB index is out of range");
+        addReplyError(c,"invalid DB index");
     } else {
         addReply(c,shared.ok);
     }
@@ -533,16 +439,16 @@ void scanCallback(void *privdata, const dictEntry *de) {
         sds sdskey = dictGetKey(de);
         key = createStringObject(sdskey, sdslen(sdskey));
     } else if (o->type == OBJ_SET) {
-        sds keysds = dictGetKey(de);
-        key = createStringObject(keysds,sdslen(keysds));
+        key = dictGetKey(de);
+        incrRefCount(key);
     } else if (o->type == OBJ_HASH) {
-        sds sdskey = dictGetKey(de);
-        sds sdsval = dictGetVal(de);
-        key = createStringObject(sdskey,sdslen(sdskey));
-        val = createStringObject(sdsval,sdslen(sdsval));
+        key = dictGetKey(de);
+        incrRefCount(key);
+        val = dictGetVal(de);
+        incrRefCount(val);
     } else if (o->type == OBJ_ZSET) {
-        sds sdskey = dictGetKey(de);
-        key = createStringObject(sdskey,sdslen(sdskey));
+        key = dictGetKey(de);
+        incrRefCount(key);
         val = createStringObjectFromLongDouble(*(double*)dictGetVal(de),0);
     } else {
         serverPanic("Type not handled in SCAN callback.");
@@ -667,7 +573,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
         privdata[0] = keys;
         privdata[1] = o;
         do {
-            cursor = dictScan(ht, cursor, scanCallback, NULL, privdata);
+            cursor = dictScan(ht, cursor, scanCallback, privdata);
         } while (cursor &&
               maxiterations-- &&
               listLength(keys) < (unsigned long)count);
@@ -788,10 +694,6 @@ void typeCommand(client *c) {
         case OBJ_SET: type = "set"; break;
         case OBJ_ZSET: type = "zset"; break;
         case OBJ_HASH: type = "hash"; break;
-        case OBJ_MODULE: {
-            moduleValue *mv = o->ptr;
-            type = mv->type->name;
-        }; break;
         default: type = "unknown"; break;
         }
     }
@@ -856,7 +758,7 @@ void renameGenericCommand(client *c, int nx) {
         dbDelete(c->db,c->argv[2]);
     }
     dbAdd(c->db,c->argv[2],o);
-    if (expire != -1) setExpire(c,c->db,c->argv[2],expire);
+    if (expire != -1) setExpire(c->db,c->argv[2],expire);
     dbDelete(c->db,c->argv[1]);
     signalModifiedKey(c->db,c->argv[1]);
     signalModifiedKey(c->db,c->argv[2]);
@@ -922,7 +824,7 @@ void moveCommand(client *c) {
         return;
     }
     dbAdd(dst,c->argv[1],o);
-    if (expire != -1) setExpire(c,dst,c->argv[1],expire);
+    if (expire != -1) setExpire(dst,c->argv[1],expire);
     incrRefCount(o);
 
     /* OK! key moved, free the entry in the source DB */
@@ -931,91 +833,6 @@ void moveCommand(client *c) {
     addReply(c,shared.cone);
 }
 
-/* Helper function for dbSwapDatabases(): scans the list of keys that have
- * one or more blocked clients for B[LR]POP or other list blocking commands
- * and signal the keys are ready if they are lists. See the comment where
- * the function is used for more info. */
-void scanDatabaseForReadyLists(redisDb *db) {
-    dictEntry *de;
-    dictIterator *di = dictGetSafeIterator(db->blocking_keys);
-    while((de = dictNext(di)) != NULL) {
-        robj *key = dictGetKey(de);
-        robj *value = lookupKey(db,key,LOOKUP_NOTOUCH);
-        if (value && value->type == OBJ_LIST)
-            signalListAsReady(db, key);
-    }
-    dictReleaseIterator(di);
-}
-
-/* Swap two databases at runtime so that all clients will magically see
- * the new database even if already connected. Note that the client
- * structure c->db points to a given DB, so we need to be smarter and
- * swap the underlying referenced structures, otherwise we would need
- * to fix all the references to the Redis DB structure.
- *
- * Returns C_ERR if at least one of the DB ids are out of range, otherwise
- * C_OK is returned. */
-int dbSwapDatabases(int id1, int id2) {
-    if (id1 < 0 || id1 >= server.dbnum ||
-        id2 < 0 || id2 >= server.dbnum) return C_ERR;
-    if (id1 == id2) return C_OK;
-    redisDb aux = server.db[id1];
-    redisDb *db1 = &server.db[id1], *db2 = &server.db[id2];
-
-    /* Swap hash tables. Note that we don't swap blocking_keys,
-     * ready_keys and watched_keys, since we want clients to
-     * remain in the same DB they were. */
-    db1->dict = db2->dict;
-    db1->expires = db2->expires;
-    db1->avg_ttl = db2->avg_ttl;
-
-    db2->dict = aux.dict;
-    db2->expires = aux.expires;
-    db2->avg_ttl = aux.avg_ttl;
-
-    /* Now we need to handle clients blocked on lists: as an effect
-     * of swapping the two DBs, a client that was waiting for list
-     * X in a given DB, may now actually be unblocked if X happens
-     * to exist in the new version of the DB, after the swap.
-     *
-     * However normally we only do this check for efficiency reasons
-     * in dbAdd() when a list is created. So here we need to rescan
-     * the list of clients blocked on lists and signal lists as ready
-     * if needed. */
-    scanDatabaseForReadyLists(db1);
-    scanDatabaseForReadyLists(db2);
-    return C_OK;
-}
-
-/* SWAPDB db1 db2 */
-void swapdbCommand(client *c) {
-    long id1, id2;
-
-    /* Not allowed in cluster mode: we have just DB 0 there. */
-    if (server.cluster_enabled) {
-        addReplyError(c,"SWAPDB is not allowed in cluster mode");
-        return;
-    }
-
-    /* Get the two DBs indexes. */
-    if (getLongFromObjectOrReply(c, c->argv[1], &id1,
-        "invalid first DB index") != C_OK)
-        return;
-
-    if (getLongFromObjectOrReply(c, c->argv[2], &id2,
-        "invalid second DB index") != C_OK)
-        return;
-
-    /* Swap... */
-    if (dbSwapDatabases(id1,id2) == C_ERR) {
-        addReplyError(c,"DB index is out of range");
-        return;
-    } else {
-        server.dirty++;
-        addReply(c,shared.ok);
-    }
-}
-
 /*-----------------------------------------------------------------------------
  * Expires API
  *----------------------------------------------------------------------------*/
@@ -1027,22 +844,14 @@ int removeExpire(redisDb *db, robj *key) {
     return dictDelete(db->expires,key->ptr) == DICT_OK;
 }
 
-/* Set an expire to the specified key. If the expire is set in the context
- * of an user calling a command 'c' is the client, otherwise 'c' is set
- * to NULL. The 'when' parameter is the absolute unix time in milliseconds
- * after which the key will no longer be considered valid. */
-void setExpire(client *c, redisDb *db, robj *key, long long when) {
+void setExpire(redisDb *db, robj *key, long long when) {
     dictEntry *kde, *de;
 
     /* Reuse the sds from the main dict in the expire dict */
     kde = dictFind(db->dict,key->ptr);
     serverAssertWithInfo(NULL,key,kde != NULL);
-    de = dictAddOrFind(db->expires,dictGetKey(kde));
+    de = dictReplaceRaw(db->expires,dictGetKey(kde));
     dictSetSignedIntegerVal(de,when);
-
-    int writable_slave = server.masterhost && server.repl_slave_ro == 0;
-    if (c && writable_slave && !(c->flags & CLIENT_MASTER))
-        rememberSlaveKeyWithExpire(db,key);
 }
 
 /* Return the expire time of the specified key, or -1 if no expire
@@ -1068,10 +877,10 @@ long long getExpire(redisDb *db, robj *key) {
  * AOF and the master->slave link guarantee operation ordering, everything
  * will be consistent even if we allow write operations against expiring
  * keys. */
-void propagateExpire(redisDb *db, robj *key, int lazy) {
+void propagateExpire(redisDb *db, robj *key) {
     robj *argv[2];
 
-    argv[0] = lazy ? shared.unlink : shared.del;
+    argv[0] = shared.del;
     argv[1] = key;
     incrRefCount(argv[0]);
     incrRefCount(argv[1]);
@@ -1114,11 +923,137 @@ int expireIfNeeded(redisDb *db, robj *key) {
 
     /* Delete the key */
     server.stat_expiredkeys++;
-    propagateExpire(db,key,server.lazyfree_lazy_expire);
+    propagateExpire(db,key);
     notifyKeyspaceEvent(NOTIFY_EXPIRED,
         "expired",key,db->id);
-    return server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
-                                         dbSyncDelete(db,key);
+    return dbDelete(db,key);
+}
+
+/*-----------------------------------------------------------------------------
+ * Expires Commands
+ *----------------------------------------------------------------------------*/
+
+/* This is the generic command implementation for EXPIRE, PEXPIRE, EXPIREAT
+ * and PEXPIREAT. Because the commad second argument may be relative or absolute
+ * the "basetime" argument is used to signal what the base time is (either 0
+ * for *AT variants of the command, or the current time for relative expires).
+ *
+ * unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for
+ * the argv[2] parameter. The basetime is always specified in milliseconds. */
+void expireGenericCommand(client *c, long long basetime, int unit) {
+    robj *key = c->argv[1], *param = c->argv[2];
+    long long when; /* unix time in milliseconds when the key will expire. */
+
+    if (getLongLongFromObjectOrReply(c, param, &when, NULL) != C_OK)
+        return;
+
+    if (unit == UNIT_SECONDS) when *= 1000;
+    when += basetime;
+
+    /* No key, return zero. */
+    if (lookupKeyWrite(c->db,key) == NULL) {
+        addReply(c,shared.czero);
+        return;
+    }
+
+    /* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past
+     * should never be executed as a DEL when load the AOF or in the context
+     * of a slave instance.
+     *
+     * Instead we take the other branch of the IF statement setting an expire
+     * (possibly in the past) and wait for an explicit DEL from the master. */
+    if (when <= mstime() && !server.loading && !server.masterhost) {
+        robj *aux;
+
+        serverAssertWithInfo(c,key,dbDelete(c->db,key));
+        server.dirty++;
+
+        /* Replicate/AOF this as an explicit DEL. */
+        aux = createStringObject("DEL",3);
+        rewriteClientCommandVector(c,2,aux,key);
+        decrRefCount(aux);
+        signalModifiedKey(c->db,key);
+        notifyKeyspaceEvent(NOTIFY_GENERIC,"del",key,c->db->id);
+        addReply(c, shared.cone);
+        return;
+    } else {
+        setExpire(c->db,key,when);
+        addReply(c,shared.cone);
+        signalModifiedKey(c->db,key);
+        notifyKeyspaceEvent(NOTIFY_GENERIC,"expire",key,c->db->id);
+        server.dirty++;
+        return;
+    }
+}
+
+void expireCommand(client *c) {
+    expireGenericCommand(c,mstime(),UNIT_SECONDS);
+}
+
+void expireatCommand(client *c) {
+    expireGenericCommand(c,0,UNIT_SECONDS);
+}
+
+void pexpireCommand(client *c) {
+    expireGenericCommand(c,mstime(),UNIT_MILLISECONDS);
+}
+
+void pexpireatCommand(client *c) {
+    expireGenericCommand(c,0,UNIT_MILLISECONDS);
+}
+
+void ttlGenericCommand(client *c, int output_ms) {
+    long long expire, ttl = -1;
+
+    /* If the key does not exist at all, return -2 */
+    if (lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH) == NULL) {
+        addReplyLongLong(c,-2);
+        return;
+    }
+    /* The key exists. Return -1 if it has no expire, or the actual
+     * TTL value otherwise. */
+    expire = getExpire(c->db,c->argv[1]);
+    if (expire != -1) {
+        ttl = expire-mstime();
+        if (ttl < 0) ttl = 0;
+    }
+    if (ttl == -1) {
+        addReplyLongLong(c,-1);
+    } else {
+        addReplyLongLong(c,output_ms ? ttl : ((ttl+500)/1000));
+    }
+}
+
+void ttlCommand(client *c) {
+    ttlGenericCommand(c, 0);
+}
+
+void pttlCommand(client *c) {
+    ttlGenericCommand(c, 1);
+}
+
+void persistCommand(client *c) {
+    dictEntry *de;
+
+    de = dictFind(c->db->dict,c->argv[1]->ptr);
+    if (de == NULL) {
+        addReply(c,shared.czero);
+    } else {
+        if (removeExpire(c->db,c->argv[1])) {
+            addReply(c,shared.cone);
+            server.dirty++;
+        } else {
+            addReply(c,shared.czero);
+        }
+    }
+}
+
+/* TOUCH key1 [key2 key3 ... keyN] */
+void touchCommand(client *c) {
+    int touched = 0;
+    for (int j = 1; j < c->argc; j++)
+        if (lookupKeyRead(c->db,c->argv[j]) != NULL) touched++;
+    addReplyLongLong(c,touched);
 }
 
 /* -----------------------------------------------------------------------------
@@ -1135,24 +1070,11 @@ int *getKeysUsingCommandTable(struct redisCommand *cmd,robj **argv, int argc, in
         *numkeys = 0;
         return NULL;
     }
-
     last = cmd->lastkey;
     if (last < 0) last = argc+last;
     keys = zmalloc(sizeof(int)*((last - cmd->firstkey)+1));
     for (j = cmd->firstkey; j <= last; j += cmd->keystep) {
-        if (j >= argc) {
-            /* Modules command do not have dispatch time arity checks, so
-             * we need to handle the case where the user passed an invalid
-             * number of arguments here. In this case we return no keys
-             * and expect the module command to report an arity error. */
-            if (cmd->flags & CMD_MODULE) {
-                zfree(keys);
-                *numkeys = 0;
-                return NULL;
-            } else {
-                serverPanic("Redis built-in command declared keys positions not matching the arity requirements.");
-            }
-        }
+        serverAssert(j < argc);
         keys[i++] = j;
     }
     *numkeys = i;
@@ -1171,9 +1093,7 @@ int *getKeysUsingCommandTable(struct redisCommand *cmd,robj **argv, int argc, in
  * This function uses the command table if a command-specific helper function
  * is not required, otherwise it calls the command-specific function. */
 int *getKeysFromCommand(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
-    if (cmd->flags & CMD_MODULE_GETKEYS) {
-        return moduleGetCommandKeysViaAPI(cmd,argv,argc,numkeys);
-    } else if (!(cmd->flags & CMD_MODULE) && cmd->getkeys_proc) {
+    if (cmd->getkeys_proc) {
         return cmd->getkeys_proc(cmd,argv,argc,numkeys);
     } else {
         return getKeysUsingCommandTable(cmd,argv,argc,numkeys);
@@ -1314,125 +1234,90 @@ int *migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkey
     return keys;
 }
 
-/* Helper function to extract keys from following commands:
- * GEORADIUS key x y radius unit [WITHDIST] [WITHHASH] [WITHCOORD] [ASC|DESC]
- *                             [COUNT count] [STORE key] [STOREDIST key]
- * GEORADIUSBYMEMBER key member radius unit ... options ... */
-int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
-    int i, num, *keys;
-    UNUSED(cmd);
-
-    /* Check for the presence of the stored key in the command */
-    int stored_key = -1;
-    for (i = 5; i < argc; i++) {
-        char *arg = argv[i]->ptr;
-        /* For the case when user specifies both "store" and "storedist" options, the
-         * second key specified would override the first key. This behavior is kept 
-         * the same as in georadiusCommand method.
-         */
-        if ((!strcasecmp(arg, "store") || !strcasecmp(arg, "storedist")) && ((i+1) < argc)) {
-            stored_key = i+1;
-            i++;
-        }
-    }
-    num = 1 + (stored_key == -1 ? 0 : 1);
-
-    /* Keys in the command come from two places:
-     * argv[1] = key,
-     * argv[5...n] = stored key if present
-     */
-    keys = zmalloc(sizeof(int) * num);
-
-    /* Add all key positions to keys[] */
-    keys[0] = 1;
-    if(num > 1) {
-         keys[1] = stored_key;
-    }
-    *numkeys = num; 
-    return keys;
-}
-
 /* Slot to Key API. This is used by Redis Cluster in order to obtain in
  * a fast way a key that belongs to a specified hash slot. This is useful
- * while rehashing the cluster and in other conditions when we need to
- * understand if we have keys for a given hash slot. */
-void slotToKeyUpdateKey(robj *key, int add) {
-    unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
-    unsigned char buf[64];
-    unsigned char *indexed = buf;
-    size_t keylen = sdslen(key->ptr);
-
-    server.cluster->slots_keys_count[hashslot] += add ? 1 : -1;
-    if (keylen+2 > 64) indexed = zmalloc(keylen+2);
-    indexed[0] = (hashslot >> 8) & 0xff;
-    indexed[1] = hashslot & 0xff;
-    memcpy(indexed+2,key->ptr,keylen);
-    if (add) {
-        raxInsert(server.cluster->slots_to_keys,indexed,keylen+2,NULL,NULL);
-    } else {
-        raxRemove(server.cluster->slots_to_keys,indexed,keylen+2,NULL);
-    }
-    if (indexed != buf) zfree(indexed);
-}
-
+ * while rehashing the cluster. */
 void slotToKeyAdd(robj *key) {
-    slotToKeyUpdateKey(key,1);
+    unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
+
+    zslInsert(server.cluster->slots_to_keys,hashslot,key);
+    incrRefCount(key);
 }
 
 void slotToKeyDel(robj *key) {
-    slotToKeyUpdateKey(key,0);
+    unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
+
+    zslDelete(server.cluster->slots_to_keys,hashslot,key);
 }
 
 void slotToKeyFlush(void) {
-    raxFree(server.cluster->slots_to_keys);
-    server.cluster->slots_to_keys = raxNew();
-    memset(server.cluster->slots_keys_count,0,
-           sizeof(server.cluster->slots_keys_count));
+    zslFree(server.cluster->slots_to_keys);
+    server.cluster->slots_to_keys = zslCreate();
 }
 
-/* Pupulate the specified array of objects with keys in the specified slot.
- * New objects are returned to represent keys, it's up to the caller to
- * decrement the reference count to release the keys names. */
 unsigned int getKeysInSlot(unsigned int hashslot, robj **keys, unsigned int count) {
-    raxIterator iter;
+    zskiplistNode *n;
+    zrangespec range;
     int j = 0;
-    unsigned char indexed[2];
 
-    indexed[0] = (hashslot >> 8) & 0xff;
-    indexed[1] = hashslot & 0xff;
-    raxStart(&iter,server.cluster->slots_to_keys);
-    raxSeek(&iter,">=",indexed,2);
-    while(count-- && raxNext(&iter)) {
-        if (iter.key[0] != indexed[0] || iter.key[1] != indexed[1]) break;
-        keys[j++] = createStringObject((char*)iter.key+2,iter.key_len-2);
+    range.min = range.max = hashslot;
+    range.minex = range.maxex = 0;
+
+    n = zslFirstInRange(server.cluster->slots_to_keys, &range);
+    while(n && n->score == hashslot && count--) {
+        keys[j++] = n->obj;
+        n = n->level[0].forward;
     }
-    raxStop(&iter);
     return j;
 }
 
 /* Remove all the keys in the specified hash slot.
  * The number of removed items is returned. */
 unsigned int delKeysInSlot(unsigned int hashslot) {
-    raxIterator iter;
+    zskiplistNode *n;
+    zrangespec range;
     int j = 0;
-    unsigned char indexed[2];
 
-    indexed[0] = (hashslot >> 8) & 0xff;
-    indexed[1] = hashslot & 0xff;
-    raxStart(&iter,server.cluster->slots_to_keys);
-    while(server.cluster->slots_keys_count[hashslot]) {
-        raxSeek(&iter,">=",indexed,2);
-        raxNext(&iter);
+    range.min = range.max = hashslot;
+    range.minex = range.maxex = 0;
 
-        robj *key = createStringObject((char*)iter.key+2,iter.key_len-2);
+    n = zslFirstInRange(server.cluster->slots_to_keys, &range);
+    while(n && n->score == hashslot) {
+        robj *key = n->obj;
+        n = n->level[0].forward; /* Go to the next item before freeing it. */
+        incrRefCount(key); /* Protect the object while freeing it. */
         dbDelete(&server.db[0],key);
         decrRefCount(key);
         j++;
     }
-    raxStop(&iter);
     return j;
 }
 
 unsigned int countKeysInSlot(unsigned int hashslot) {
-    return server.cluster->slots_keys_count[hashslot];
+    zskiplist *zsl = server.cluster->slots_to_keys;
+    zskiplistNode *zn;
+    zrangespec range;
+    int rank, count = 0;
+
+    range.min = range.max = hashslot;
+    range.minex = range.maxex = 0;
+
+    /* Find first element in range */
+    zn = zslFirstInRange(zsl, &range);
+
+    /* Use rank of first element, if any, to determine preliminary count */
+    if (zn != NULL) {
+        rank = zslGetRank(zsl, zn->score, zn->obj);
+        count = (zsl->length - (rank - 1));
+
+        /* Find last element in range */
+        zn = zslLastInRange(zsl, &range);
+
+        /* Use rank of last element, if any, to determine the actual count */
+        if (zn != NULL) {
+            rank = zslGetRank(zsl, zn->score, zn->obj);
+            count -= (zsl->length - rank);
+        }
+    }
+    return count;
 }

src/debug.c

@@ -33,7 +33,6 @@
 
 #include <arpa/inet.h>
 #include <signal.h>
-#include <dlfcn.h>
 
 #ifdef HAVE_BACKTRACE
 #include <execinfo.h>
@@ -41,6 +40,7 @@
 #include <fcntl.h>
 #include "bio.h"
 #include <unistd.h>
+#include <dlfcn.h>
 #endif /* HAVE_BACKTRACE */
 
 #ifdef __CYGWIN__
@@ -126,7 +126,7 @@ void computeDatasetDigest(unsigned char *final) {
         redisDb *db = server.db+j;
 
         if (dictSize(db->dict) == 0) continue;
-        di = dictGetSafeIterator(db->dict);
+        di = dictGetIterator(db->dict);
 
         /* hash the DB id, so the same dataset moved in a different
          * DB will lead to a different digest */
@@ -165,10 +165,10 @@ void computeDatasetDigest(unsigned char *final) {
                 listTypeReleaseIterator(li);
             } else if (o->type == OBJ_SET) {
                 setTypeIterator *si = setTypeInitIterator(o);
-                sds sdsele;
-                while((sdsele = setTypeNextObject(si)) != NULL) {
-                    xorDigest(digest,sdsele,sdslen(sdsele));
-                    sdsfree(sdsele);
+                robj *ele;
+                while((ele = setTypeNextObject(si)) != NULL) {
+                    xorObjectDigest(digest,ele);
+                    decrRefCount(ele);
                 }
                 setTypeReleaseIterator(si);
             } else if (o->type == OBJ_ZSET) {
@@ -210,12 +210,12 @@ void computeDatasetDigest(unsigned char *final) {
                     dictEntry *de;
 
                     while((de = dictNext(di)) != NULL) {
-                        sds sdsele = dictGetKey(de);
+                        robj *eleobj = dictGetKey(de);
                         double *score = dictGetVal(de);
 
                         snprintf(buf,sizeof(buf),"%.17g",*score);
                         memset(eledigest,0,20);
-                        mixDigest(eledigest,sdsele,sdslen(sdsele));
+                        mixObjectDigest(eledigest,eleobj);
                         mixDigest(eledigest,buf,strlen(buf));
                         xorDigest(digest,eledigest,20);
                     }
@@ -224,30 +224,23 @@ void computeDatasetDigest(unsigned char *final) {
                     serverPanic("Unknown sorted set encoding");
                 }
             } else if (o->type == OBJ_HASH) {
-                hashTypeIterator *hi = hashTypeInitIterator(o);
+                hashTypeIterator *hi;
+                robj *obj;
+
+                hi = hashTypeInitIterator(o);
                 while (hashTypeNext(hi) != C_ERR) {
                     unsigned char eledigest[20];
-                    sds sdsele;
 
                     memset(eledigest,0,20);
-                    sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_KEY);
-                    mixDigest(eledigest,sdsele,sdslen(sdsele));
-                    sdsfree(sdsele);
-                    sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_VALUE);
-                    mixDigest(eledigest,sdsele,sdslen(sdsele));
-                    sdsfree(sdsele);
+                    obj = hashTypeCurrentObject(hi,OBJ_HASH_KEY);
+                    mixObjectDigest(eledigest,obj);
+                    decrRefCount(obj);
+                    obj = hashTypeCurrentObject(hi,OBJ_HASH_VALUE);
+                    mixObjectDigest(eledigest,obj);
+                    decrRefCount(obj);
                     xorDigest(digest,eledigest,20);
                 }
                 hashTypeReleaseIterator(hi);
-            } else if (o->type == OBJ_MODULE) {
-                RedisModuleDigest md;
-                moduleValue *mv = o->ptr;
-                moduleType *mt = mv->type;
-                moduleInitDigestContext(md);
-                if (mt->digest) {
-                    mt->digest(&md,mv->value);
-                    xorDigest(digest,md.x,sizeof(md.x));
-                }
             } else {
                 serverPanic("Unknown object type");
             }
@@ -261,6 +254,14 @@ void computeDatasetDigest(unsigned char *final) {
     }
 }
 
+#if defined(USE_JEMALLOC)
+void inputCatSds(void *result, const char *str) {
+    /* result is actually a (sds *), so re-cast it here */
+    sds *info = (sds *)result;
+    *info = sdscat(*info, str);
+}
+#endif
+
 void debugCommand(client *c) {
     if (c->argc == 1) {
         addReplyError(c,"You must specify a subcommand for DEBUG. Try DEBUG HELP for info.");
@@ -275,8 +276,6 @@ void debugCommand(client *c) {
         blen++; addReplyStatus(c,
         "segfault -- Crash the server with sigsegv.");
         blen++; addReplyStatus(c,
-        "panic -- Crash the server simulating a panic.");
-        blen++; addReplyStatus(c,
         "restart  -- Graceful restart: save config, db, restart.");
         blen++; addReplyStatus(c,
         "crash-and-recovery <milliseconds> -- Hard crash and restart after <milliseconds> delay.");
@@ -291,9 +290,7 @@ void debugCommand(client *c) {
         blen++; addReplyStatus(c,
         "sdslen <key> -- Show low level SDS string info representing key and value.");
         blen++; addReplyStatus(c,
-        "ziplist <key> -- Show low level info about the ziplist encoding.");
-        blen++; addReplyStatus(c,
-        "populate <count> [prefix] [size] -- Create <count> string keys named key:<num>. If a prefix is specified is used instead of the 'key' prefix.");
+        "populate <count> [prefix] -- Create <count> string keys named key:<num>. If a prefix is specified is used instead of the 'key' prefix.");
         blen++; addReplyStatus(c,
         "digest   -- Outputs an hex signature representing the current DB content.");
         blen++; addReplyStatus(c,
@@ -308,11 +305,13 @@ void debugCommand(client *c) {
         "structsize -- Return the size of different Redis core C structures.");
         blen++; addReplyStatus(c,
         "htstats <dbid> -- Return hash table statistics of the specified Redis database.");
+        blen++; addReplyStatus(c,
+        "jemalloc info  -- Show internal jemalloc statistics.");
+        blen++; addReplyStatus(c,
+        "jemalloc purge -- Force jemalloc to release unused memory.");
         setDeferredMultiBulkLength(c,blenp,blen);
     } else if (!strcasecmp(c->argv[1]->ptr,"segfault")) {
         *((char*)-1) = 'x';
-    } else if (!strcasecmp(c->argv[1]->ptr,"panic")) {
-        serverPanic("DEBUG PANIC called at Unix time %ld", time(NULL));
     } else if (!strcasecmp(c->argv[1]->ptr,"restart") ||
                !strcasecmp(c->argv[1]->ptr,"crash-and-recover"))
     {
@@ -335,14 +334,12 @@ void debugCommand(client *c) {
         if (c->argc >= 3) c->argv[2] = tryObjectEncoding(c->argv[2]);
         serverAssertWithInfo(c,c->argv[0],1 == 2);
     } else if (!strcasecmp(c->argv[1]->ptr,"reload")) {
-        rdbSaveInfo rsi, *rsiptr;
-        rsiptr = rdbPopulateSaveInfo(&rsi);
-        if (rdbSave(server.rdb_filename,rsiptr) != C_OK) {
+        if (rdbSave(server.rdb_filename) != C_OK) {
             addReply(c,shared.err);
             return;
         }
-        emptyDb(-1,EMPTYDB_NO_FLAGS,NULL);
-        if (rdbLoad(server.rdb_filename,NULL) != C_OK) {
+        emptyDb(NULL);
+        if (rdbLoad(server.rdb_filename) != C_OK) {
             addReplyError(c,"Error trying to load the RDB dump");
             return;
         }
@@ -350,7 +347,7 @@ void debugCommand(client *c) {
         addReply(c,shared.ok);
     } else if (!strcasecmp(c->argv[1]->ptr,"loadaof")) {
         if (server.aof_state == AOF_ON) flushAppendOnlyFile(1);
-        emptyDb(-1,EMPTYDB_NO_FLAGS,NULL);
+        emptyDb(NULL);
         if (loadAppendOnlyFile(server.aof_filename) != C_OK) {
             addReply(c,shared.err);
             return;
@@ -426,29 +423,15 @@ void debugCommand(client *c) {
             addReplyError(c,"Not an sds encoded string.");
         } else {
             addReplyStatusFormat(c,
-                "key_sds_len:%lld, key_sds_avail:%lld, key_zmalloc: %lld, "
-                "val_sds_len:%lld, val_sds_avail:%lld, val_zmalloc: %lld",
+                "key_sds_len:%lld, key_sds_avail:%lld, "
+                "val_sds_len:%lld, val_sds_avail:%lld",
                 (long long) sdslen(key),
                 (long long) sdsavail(key),
-                (long long) sdsZmallocSize(key),
                 (long long) sdslen(val->ptr),
-                (long long) sdsavail(val->ptr),
-                (long long) getStringObjectSdsUsedMemory(val));
-        }
-    } else if (!strcasecmp(c->argv[1]->ptr,"ziplist") && c->argc == 3) {
-        robj *o;
-
-        if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nokeyerr))
-                == NULL) return;
-
-        if (o->encoding != OBJ_ENCODING_ZIPLIST) {
-            addReplyError(c,"Not an sds encoded string.");
-        } else {
-            ziplistRepr(o->ptr);
-            addReplyStatus(c,"Ziplist structure printed on stdout");
+                (long long) sdsavail(val->ptr));
         }
     } else if (!strcasecmp(c->argv[1]->ptr,"populate") &&
-               c->argc >= 3 && c->argc <= 5) {
+               (c->argc == 3 || c->argc == 4)) {
         long keys, j;
         robj *key, *val;
         char buf[128];
@@ -457,25 +440,15 @@ void debugCommand(client *c) {
             return;
         dictExpand(c->db->dict,keys);
         for (j = 0; j < keys; j++) {
-            long valsize = 0;
             snprintf(buf,sizeof(buf),"%s:%lu",
                 (c->argc == 3) ? "key" : (char*)c->argv[3]->ptr, j);
             key = createStringObject(buf,strlen(buf));
-            if (c->argc == 5)
-                if (getLongFromObjectOrReply(c, c->argv[4], &valsize, NULL) != C_OK)
-                    return;
             if (lookupKeyWrite(c->db,key) != NULL) {
                 decrRefCount(key);
                 continue;
             }
             snprintf(buf,sizeof(buf),"value:%lu",j);
-            if (valsize==0)
-                val = createStringObject(buf,strlen(buf));
-            else {
-                int buflen = strlen(buf);
-                val = createStringObject(NULL,valsize);
-                memcpy(val->ptr, buf, valsize<=buflen? valsize: buflen);
-            }
+            val = createStringObject(buf,strlen(buf));
             dbAdd(c->db,key,val);
             signalModifiedKey(c->db,key);
             decrRefCount(key);
@@ -549,6 +522,30 @@ void debugCommand(client *c) {
         stats = sdscat(stats,buf);
 
         addReplyBulkSds(c,stats);
+    } else if (!strcasecmp(c->argv[1]->ptr,"jemalloc") && c->argc == 3) {
+#if defined(USE_JEMALLOC)
+        if (!strcasecmp(c->argv[2]->ptr, "info")) {
+            sds info = sdsempty();
+            je_malloc_stats_print(inputCatSds, &info, NULL);
+            addReplyBulkSds(c, info);
+        } else if (!strcasecmp(c->argv[2]->ptr, "purge")) {
+            char tmp[32];
+            unsigned narenas = 0;
+            size_t sz = sizeof(unsigned);
+            if (!je_mallctl("arenas.narenas", &narenas, &sz, NULL, 0)) {
+                sprintf(tmp, "arena.%d.purge", narenas);
+                if (!je_mallctl(tmp, NULL, 0, NULL, 0)) {
+                    addReply(c, shared.ok);
+                    return;
+                }
+            }
+            addReplyError(c, "Error purging dirty pages");
+        } else {
+            addReplyErrorFormat(c, "Valid jemalloc debug fields: info, purge");
+        }
+#else
+        addReplyErrorFormat(c, "jemalloc support not available");
+#endif
     } else {
         addReplyErrorFormat(c, "Unknown DEBUG subcommand or wrong number of arguments for '%s'",
             (char*)c->argv[1]->ptr);
@@ -557,7 +554,7 @@ void debugCommand(client *c) {
 
 /* =========================== Crash handling  ============================== */
 
-void _serverAssert(const char *estr, const char *file, int line) {
+void _serverAssert(char *estr, char *file, int line) {
     bugReportStart();
     serverLog(LL_WARNING,"=== ASSERTION FAILED ===");
     serverLog(LL_WARNING,"==> %s:%d '%s' is not true",file,line,estr);
@@ -570,7 +567,7 @@ void _serverAssert(const char *estr, const char *file, int line) {
     *((char*)-1) = 'x';
 }
 
-void _serverAssertPrintClientInfo(const client *c) {
+void _serverAssertPrintClientInfo(client *c) {
     int j;
 
     bugReportStart();
@@ -594,7 +591,7 @@ void _serverAssertPrintClientInfo(const client *c) {
     }
 }
 
-void serverLogObjectDebugInfo(const robj *o) {
+void serverLogObjectDebugInfo(robj *o) {
     serverLog(LL_WARNING,"Object type: %d", o->type);
     serverLog(LL_WARNING,"Object encoding: %d", o->encoding);
     serverLog(LL_WARNING,"Object refcount: %d", o->refcount);
@@ -614,33 +611,27 @@ void serverLogObjectDebugInfo(const robj *o) {
     } else if (o->type == OBJ_ZSET) {
         serverLog(LL_WARNING,"Sorted set size: %d", (int) zsetLength(o));
         if (o->encoding == OBJ_ENCODING_SKIPLIST)
-            serverLog(LL_WARNING,"Skiplist level: %d", (int) ((const zset*)o->ptr)->zsl->level);
+            serverLog(LL_WARNING,"Skiplist level: %d", (int) ((zset*)o->ptr)->zsl->level);
     }
 }
 
-void _serverAssertPrintObject(const robj *o) {
+void _serverAssertPrintObject(robj *o) {
     bugReportStart();
     serverLog(LL_WARNING,"=== ASSERTION FAILED OBJECT CONTEXT ===");
     serverLogObjectDebugInfo(o);
 }
 
-void _serverAssertWithInfo(const client *c, const robj *o, const char *estr, const char *file, int line) {
+void _serverAssertWithInfo(client *c, robj *o, char *estr, char *file, int line) {
     if (c) _serverAssertPrintClientInfo(c);
     if (o) _serverAssertPrintObject(o);
     _serverAssert(estr,file,line);
 }
 
-void _serverPanic(const char *file, int line, const char *msg, ...) {
-    va_list ap;
-    va_start(ap,msg);
-    char fmtmsg[256];
-    vsnprintf(fmtmsg,sizeof(fmtmsg),msg,ap);
-    va_end(ap);
-
+void _serverPanic(char *msg, char *file, int line) {
     bugReportStart();
     serverLog(LL_WARNING,"------------------------------------------------");
     serverLog(LL_WARNING,"!!! Software Failure. Press left mouse button to continue");
-    serverLog(LL_WARNING,"Guru Meditation: %s #%s:%d",fmtmsg,file,line);
+    serverLog(LL_WARNING,"Guru Meditation: %s #%s:%d",msg,file,line);
 #ifdef HAVE_BACKTRACE
     serverLog(LL_WARNING,"(forcing SIGSEGV in order to print the stack trace)");
 #endif
@@ -1040,6 +1031,8 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
     /* Log INFO and CLIENT LIST */
     serverLogRaw(LL_WARNING|LL_RAW, "\n------ INFO OUTPUT ------\n");
     infostring = genRedisInfoString("all");
+    infostring = sdscatprintf(infostring, "hash_init_value: %u\n",
+        dictGetHashFunctionSeed());
     serverLogRaw(LL_WARNING|LL_RAW, infostring);
     serverLogRaw(LL_WARNING|LL_RAW, "\n------ CLIENT LIST OUTPUT ------\n");
     clients = getAllClientsInfoString();

src/defrag.c

@@ -1,579 +0,0 @@
-/* 
- * Active memory defragmentation
- * Try to find key / value allocations that need to be re-allocated in order 
- * to reduce external fragmentation.
- * We do that by scanning the keyspace and for each pointer we have, we can try to
- * ask the allocator if moving it to a new address will help reduce fragmentation.
- *
- * Copyright (c) 2017, Oran Agra
- * Copyright (c) 2017, Redis Labs, Inc
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "server.h"
-#include <time.h>
-#include <assert.h>
-#include <stddef.h>
-
-#ifdef HAVE_DEFRAG
-
-/* this method was added to jemalloc in order to help us understand which
- * pointers are worthwhile moving and which aren't */
-int je_get_defrag_hint(void* ptr, int *bin_util, int *run_util);
-
-/* Defrag helper for generic allocations.
- *
- * returns NULL in case the allocatoin wasn't moved.
- * when it returns a non-null value, the old pointer was already released
- * and should NOT be accessed. */
-void* activeDefragAlloc(void *ptr) {
-    int bin_util, run_util;
-    size_t size;
-    void *newptr;
-    if(!je_get_defrag_hint(ptr, &bin_util, &run_util)) {
-        server.stat_active_defrag_misses++;
-        return NULL;
-    }
-    /* if this run is more utilized than the average utilization in this bin
-     * (or it is full), skip it. This will eventually move all the allocations
-     * from relatively empty runs into relatively full runs. */
-    if (run_util > bin_util || run_util == 1<<16) {
-        server.stat_active_defrag_misses++;
-        return NULL;
-    }
-    /* move this allocation to a new allocation.
-     * make sure not to use the thread cache. so that we don't get back the same
-     * pointers we try to free */
-    size = zmalloc_size(ptr);
-    newptr = zmalloc_no_tcache(size);
-    memcpy(newptr, ptr, size);
-    zfree_no_tcache(ptr);
-    return newptr;
-}
-
-/*Defrag helper for sds strings
- *
- * returns NULL in case the allocatoin wasn't moved.
- * when it returns a non-null value, the old pointer was already released
- * and should NOT be accessed. */
-sds activeDefragSds(sds sdsptr) {
-    void* ptr = sdsAllocPtr(sdsptr);
-    void* newptr = activeDefragAlloc(ptr);
-    if (newptr) {
-        size_t offset = sdsptr - (char*)ptr;
-        sdsptr = (char*)newptr + offset;
-        return sdsptr;
-    }
-    return NULL;
-}
-
-/* Defrag helper for robj and/or string objects
- *
- * returns NULL in case the allocatoin wasn't moved.
- * when it returns a non-null value, the old pointer was already released
- * and should NOT be accessed. */
-robj *activeDefragStringOb(robj* ob, int *defragged) {
-    robj *ret = NULL;
-    if (ob->refcount!=1)
-        return NULL;
-
-    /* try to defrag robj (only if not an EMBSTR type (handled below). */
-    if (ob->type!=OBJ_STRING || ob->encoding!=OBJ_ENCODING_EMBSTR) {
-        if ((ret = activeDefragAlloc(ob))) {
-            ob = ret;
-            (*defragged)++;
-        }
-    }
-
-    /* try to defrag string object */
-    if (ob->type == OBJ_STRING) {
-        if(ob->encoding==OBJ_ENCODING_RAW) {
-            sds newsds = activeDefragSds((sds)ob->ptr);
-            if (newsds) {
-                ob->ptr = newsds;
-                (*defragged)++;
-            }
-        } else if (ob->encoding==OBJ_ENCODING_EMBSTR) {
-            /* The sds is embedded in the object allocation, calculate the
-             * offset and update the pointer in the new allocation. */
-            long ofs = (intptr_t)ob->ptr - (intptr_t)ob;
-            if ((ret = activeDefragAlloc(ob))) {
-                ret->ptr = (void*)((intptr_t)ret + ofs);
-                (*defragged)++;
-            }
-        } else if (ob->encoding!=OBJ_ENCODING_INT) {
-            serverPanic("Unknown string encoding");
-        }
-    }
-    return ret;
-}
-
-/* Defrag helper for dictEntries to be used during dict iteration (called on
- * each step). Teturns a stat of how many pointers were moved. */
-int dictIterDefragEntry(dictIterator *iter) {
-    /* This function is a little bit dirty since it messes with the internals
-     * of the dict and it's iterator, but the benefit is that it is very easy
-     * to use, and require no other chagnes in the dict. */
-    int defragged = 0;
-    dictht *ht;
-    /* Handle the next entry (if there is one), and update the pointer in the
-     * current entry. */
-    if (iter->nextEntry) {
-        dictEntry *newde = activeDefragAlloc(iter->nextEntry);
-        if (newde) {
-            defragged++;
-            iter->nextEntry = newde;
-            iter->entry->next = newde;
-        }
-    }
-    /* handle the case of the first entry in the hash bucket. */
-    ht = &iter->d->ht[iter->table];
-    if (ht->table[iter->index] == iter->entry) {
-        dictEntry *newde = activeDefragAlloc(iter->entry);
-        if (newde) {
-            iter->entry = newde;
-            ht->table[iter->index] = newde;
-            defragged++;
-        }
-    }
-    return defragged;
-}
-
-/* Defrag helper for dict main allocations (dict struct, and hash tables).
- * receives a pointer to the dict* and implicitly updates it when the dict
- * struct itself was moved. Returns a stat of how many pointers were moved. */
-int dictDefragTables(dict** dictRef) {
-    dict *d = *dictRef;
-    dictEntry **newtable;
-    int defragged = 0;
-    /* handle the dict struct */
-    dict *newd = activeDefragAlloc(d);
-    if (newd)
-        defragged++, *dictRef = d = newd;
-    /* handle the first hash table */
-    newtable = activeDefragAlloc(d->ht[0].table);
-    if (newtable)
-        defragged++, d->ht[0].table = newtable;
-    /* handle the second hash table */
-    if (d->ht[1].table) {
-        newtable = activeDefragAlloc(d->ht[1].table);
-        if (newtable)
-            defragged++, d->ht[1].table = newtable;
-    }
-    return defragged;
-}
-
-/* Internal function used by zslDefrag */
-void zslUpdateNode(zskiplist *zsl, zskiplistNode *oldnode, zskiplistNode *newnode, zskiplistNode **update) {
-    int i;
-    for (i = 0; i < zsl->level; i++) {
-        if (update[i]->level[i].forward == oldnode)
-            update[i]->level[i].forward = newnode;
-    }
-    serverAssert(zsl->header!=oldnode);
-    if (newnode->level[0].forward) {
-        serverAssert(newnode->level[0].forward->backward==oldnode);
-        newnode->level[0].forward->backward = newnode;
-    } else {
-        serverAssert(zsl->tail==oldnode);
-        zsl->tail = newnode;
-    }
-}
-
-/* Defrag helper for sorted set.
- * Update the robj pointer, defrag the skiplist struct and return the new score
- * reference. We may not access oldele pointer (not even the pointer stored in
- * the skiplist), as it was already freed. Newele may be null, in which case we
- * only need to defrag the skiplist, but not update the obj pointer.
- * When return value is non-NULL, it is the score reference that must be updated
- * in the dict record. */
-double *zslDefrag(zskiplist *zsl, double score, sds oldele, sds newele) {
-    zskiplistNode *update[ZSKIPLIST_MAXLEVEL], *x, *newx;
-    int i;
-    sds ele = newele? newele: oldele;
-
-    /* find the skiplist node referring to the object that was moved,
-     * and all pointers that need to be updated if we'll end up moving the skiplist node. */
-    x = zsl->header;
-    for (i = zsl->level-1; i >= 0; i--) {
-        while (x->level[i].forward &&
-            x->level[i].forward->ele != oldele && /* make sure not to access the
-                                                     ->obj pointer if it matches
-                                                     oldele */
-            (x->level[i].forward->score < score ||
-                (x->level[i].forward->score == score &&
-                sdscmp(x->level[i].forward->ele,ele) < 0)))
-            x = x->level[i].forward;
-        update[i] = x;
-    }
-
-    /* update the robj pointer inside the skip list record. */
-    x = x->level[0].forward;
-    serverAssert(x && score == x->score && x->ele==oldele);
-    if (newele)
-        x->ele = newele;
-
-    /* try to defrag the skiplist record itself */
-    newx = activeDefragAlloc(x);
-    if (newx) {
-        zslUpdateNode(zsl, x, newx, update);
-        return &newx->score;
-    }
-    return NULL;
-}
-
-/* Utility function that replaces an old key pointer in the dictionary with a
- * new pointer. Additionally, we try to defrag the dictEntry in that dict.
- * Oldkey mey be a dead pointer and should not be accessed (we get a
- * pre-calculated hash value). Newkey may be null if the key pointer wasn't
- * moved. Return value is the the dictEntry if found, or NULL if not found.
- * NOTE: this is very ugly code, but it let's us avoid the complication of
- * doing a scan on another dict. */
-dictEntry* replaceSateliteDictKeyPtrAndOrDefragDictEntry(dict *d, sds oldkey, sds newkey, unsigned int hash, int *defragged) {
-    dictEntry **deref = dictFindEntryRefByPtrAndHash(d, oldkey, hash);
-    if (deref) {
-        dictEntry *de = *deref;
-        dictEntry *newde = activeDefragAlloc(de);
-        if (newde) {
-            de = *deref = newde;
-            (*defragged)++;
-        }
-        if (newkey)
-            de->key = newkey;
-        return de;
-    }
-    return NULL;
-}
-
-/* for each key we scan in the main dict, this function will attempt to defrag
- * all the various pointers it has. Returns a stat of how many pointers were
- * moved. */
-int defragKey(redisDb *db, dictEntry *de) {
-    sds keysds = dictGetKey(de);
-    robj *newob, *ob;
-    unsigned char *newzl;
-    dict *d;
-    dictIterator *di;
-    int defragged = 0;
-    sds newsds;
-
-    /* Try to defrag the key name. */
-    newsds = activeDefragSds(keysds);
-    if (newsds)
-        defragged++, de->key = newsds;
-    if (dictSize(db->expires)) {
-         /* Dirty code:
-          * I can't search in db->expires for that key after i already released
-          * the pointer it holds it won't be able to do the string compare */
-        unsigned int hash = dictGetHash(db->dict, de->key);
-        replaceSateliteDictKeyPtrAndOrDefragDictEntry(db->expires, keysds, newsds, hash, &defragged);
-    }
-
-    /* Try to defrag robj and / or string value. */
-    ob = dictGetVal(de);
-    if ((newob = activeDefragStringOb(ob, &defragged))) {
-        de->v.val = newob;
-        ob = newob;
-    }
-
-    if (ob->type == OBJ_STRING) {
-        /* Already handled in activeDefragStringOb. */
-    } else if (ob->type == OBJ_LIST) {
-        if (ob->encoding == OBJ_ENCODING_QUICKLIST) {
-            quicklist *ql = ob->ptr, *newql;
-            quicklistNode *node = ql->head, *newnode;
-            if ((newql = activeDefragAlloc(ql)))
-                defragged++, ob->ptr = ql = newql;
-            while (node) {
-                if ((newnode = activeDefragAlloc(node))) {
-                    if (newnode->prev)
-                        newnode->prev->next = newnode;
-                    else
-                        ql->head = newnode;
-                    if (newnode->next)
-                        newnode->next->prev = newnode;
-                    else
-                        ql->tail = newnode;
-                    node = newnode;
-                    defragged++;
-                }
-                if ((newzl = activeDefragAlloc(node->zl)))
-                    defragged++, node->zl = newzl;
-                node = node->next;
-            }
-        } else if (ob->encoding == OBJ_ENCODING_ZIPLIST) {
-            if ((newzl = activeDefragAlloc(ob->ptr)))
-                defragged++, ob->ptr = newzl;
-        } else {
-            serverPanic("Unknown list encoding");
-        }
-    } else if (ob->type == OBJ_SET) {
-        if (ob->encoding == OBJ_ENCODING_HT) {
-            d = ob->ptr;
-            di = dictGetIterator(d);
-            while((de = dictNext(di)) != NULL) {
-                sds sdsele = dictGetKey(de);
-                if ((newsds = activeDefragSds(sdsele)))
-                    defragged++, de->key = newsds;
-                defragged += dictIterDefragEntry(di);
-            }
-            dictReleaseIterator(di);
-            dictDefragTables((dict**)&ob->ptr);
-        } else if (ob->encoding == OBJ_ENCODING_INTSET) {
-            intset *is = ob->ptr;
-            intset *newis = activeDefragAlloc(is);
-            if (newis)
-                defragged++, ob->ptr = newis;
-        } else {
-            serverPanic("Unknown set encoding");
-        }
-    } else if (ob->type == OBJ_ZSET) {
-        if (ob->encoding == OBJ_ENCODING_ZIPLIST) {
-            if ((newzl = activeDefragAlloc(ob->ptr)))
-                defragged++, ob->ptr = newzl;
-        } else if (ob->encoding == OBJ_ENCODING_SKIPLIST) {
-            zset *zs = (zset*)ob->ptr;
-            zset *newzs;
-            zskiplist *newzsl;
-            struct zskiplistNode *newheader;
-            if ((newzs = activeDefragAlloc(zs)))
-                defragged++, ob->ptr = zs = newzs;
-            if ((newzsl = activeDefragAlloc(zs->zsl)))
-                defragged++, zs->zsl = newzsl;
-            if ((newheader = activeDefragAlloc(zs->zsl->header)))
-                defragged++, zs->zsl->header = newheader;
-            d = zs->dict;
-            di = dictGetIterator(d);
-            while((de = dictNext(di)) != NULL) {
-                double* newscore;
-                sds sdsele = dictGetKey(de);
-                if ((newsds = activeDefragSds(sdsele)))
-                    defragged++, de->key = newsds;
-                newscore = zslDefrag(zs->zsl, *(double*)dictGetVal(de), sdsele, newsds);
-                if (newscore) {
-                    dictSetVal(d, de, newscore);
-                    defragged++;
-                }
-                defragged += dictIterDefragEntry(di);
-            }
-            dictReleaseIterator(di);
-            dictDefragTables(&zs->dict);
-        } else {
-            serverPanic("Unknown sorted set encoding");
-        }
-    } else if (ob->type == OBJ_HASH) {
-        if (ob->encoding == OBJ_ENCODING_ZIPLIST) {
-            if ((newzl = activeDefragAlloc(ob->ptr)))
-                defragged++, ob->ptr = newzl;
-        } else if (ob->encoding == OBJ_ENCODING_HT) {
-            d = ob->ptr;
-            di = dictGetIterator(d);
-            while((de = dictNext(di)) != NULL) {
-                sds sdsele = dictGetKey(de);
-                if ((newsds = activeDefragSds(sdsele)))
-                    defragged++, de->key = newsds;
-                sdsele = dictGetVal(de);
-                if ((newsds = activeDefragSds(sdsele)))
-                    defragged++, de->v.val = newsds;
-                defragged += dictIterDefragEntry(di);
-            }
-            dictReleaseIterator(di);
-            dictDefragTables((dict**)&ob->ptr);
-        } else {
-            serverPanic("Unknown hash encoding");
-        }
-    } else if (ob->type == OBJ_MODULE) {
-        /* Currently defragmenting modules private data types
-         * is not supported. */
-    } else {
-        serverPanic("Unknown object type");
-    }
-    return defragged;
-}
-
-/* Defrag scan callback for the main db dictionary. */
-void defragScanCallback(void *privdata, const dictEntry *de) {
-    int defragged = defragKey((redisDb*)privdata, (dictEntry*)de);
-    server.stat_active_defrag_hits += defragged;
-    if(defragged)
-        server.stat_active_defrag_key_hits++;
-    else
-        server.stat_active_defrag_key_misses++;
-}
-
-/* Defrag scan callback for for each hash table bicket,
- * used in order to defrag the dictEntry allocations. */
-void defragDictBucketCallback(void *privdata, dictEntry **bucketref) {
-    UNUSED(privdata);
-    while(*bucketref) {
-        dictEntry *de = *bucketref, *newde;
-        if ((newde = activeDefragAlloc(de))) {
-            *bucketref = newde;
-        }
-        bucketref = &(*bucketref)->next;
-    }
-}
-
-/* Utility function to get the fragmentation ratio from jemalloc.
- * It is critical to do that by comparing only heap maps that belown to
- * jemalloc, and skip ones the jemalloc keeps as spare. Since we use this
- * fragmentation ratio in order to decide if a defrag action should be taken
- * or not, a false detection can cause the defragmenter to waste a lot of CPU
- * without the possibility of getting any results. */
-float getAllocatorFragmentation(size_t *out_frag_bytes) {
-    size_t epoch = 1, allocated = 0, resident = 0, active = 0, sz = sizeof(size_t);
-    /* Update the statistics cached by mallctl. */
-    je_mallctl("epoch", &epoch, &sz, &epoch, sz);
-    /* Unlike RSS, this does not include RSS from shared libraries and other non
-     * heap mappings. */
-    je_mallctl("stats.resident", &resident, &sz, NULL, 0);
-    /* Unlike resident, this doesn't not include the pages jemalloc reserves
-     * for re-use (purge will clean that). */
-    je_mallctl("stats.active", &active, &sz, NULL, 0);
-    /* Unlike zmalloc_used_memory, this matches the stats.resident by taking
-     * into account all allocations done by this process (not only zmalloc). */
-    je_mallctl("stats.allocated", &allocated, &sz, NULL, 0);
-    float frag_pct = ((float)active / allocated)*100 - 100;
-    size_t frag_bytes = active - allocated;
-    float rss_pct = ((float)resident / allocated)*100 - 100;
-    size_t rss_bytes = resident - allocated;
-    if(out_frag_bytes)
-        *out_frag_bytes = frag_bytes;
-    serverLog(LL_DEBUG,
-        "allocated=%zu, active=%zu, resident=%zu, frag=%.0f%% (%.0f%% rss), frag_bytes=%zu (%zu%% rss)",
-        allocated, active, resident, frag_pct, rss_pct, frag_bytes, rss_bytes);
-    return frag_pct;
-}
-
-#define INTERPOLATE(x, x1, x2, y1, y2) ( (y1) + ((x)-(x1)) * ((y2)-(y1)) / ((x2)-(x1)) )
-#define LIMIT(y, min, max) ((y)<(min)? min: ((y)>(max)? max: (y)))
-
-/* Perform incremental defragmentation work from the serverCron.
- * This works in a similar way to activeExpireCycle, in the sense that
- * we do incremental work across calls. */
-void activeDefragCycle(void) {
-    static int current_db = -1;
-    static unsigned long cursor = 0;
-    static redisDb *db = NULL;
-    static long long start_scan, start_stat;
-    unsigned int iterations = 0;
-    unsigned long long defragged = server.stat_active_defrag_hits;
-    long long start, timelimit;
-
-    if (server.aof_child_pid!=-1 || server.rdb_child_pid!=-1)
-        return; /* Defragging memory while there's a fork will just do damage. */
-
-    /* Once a second, check if we the fragmentation justfies starting a scan
-     * or making it more aggressive. */
-    run_with_period(1000) {
-        size_t frag_bytes;
-        float frag_pct = getAllocatorFragmentation(&frag_bytes);
-        /* If we're not already running, and below the threshold, exit. */
-        if (!server.active_defrag_running) {
-            if(frag_pct < server.active_defrag_threshold_lower || frag_bytes < server.active_defrag_ignore_bytes)
-                return;
-        }
-
-        /* Calculate the adaptive aggressiveness of the defrag */
-        int cpu_pct = INTERPOLATE(frag_pct,
-                server.active_defrag_threshold_lower,
-                server.active_defrag_threshold_upper,
-                server.active_defrag_cycle_min,
-                server.active_defrag_cycle_max);
-        cpu_pct = LIMIT(cpu_pct,
-                server.active_defrag_cycle_min,
-                server.active_defrag_cycle_max);
-         /* We allow increasing the aggressiveness during a scan, but don't
-          * reduce it. */
-        if (!server.active_defrag_running ||
-            cpu_pct > server.active_defrag_running)
-        {
-            server.active_defrag_running = cpu_pct;
-            serverLog(LL_VERBOSE,
-                "Starting active defrag, frag=%.0f%%, frag_bytes=%zu, cpu=%d%%",
-                frag_pct, frag_bytes, cpu_pct);
-        }
-    }
-    if (!server.active_defrag_running)
-        return;
-
-    /* See activeExpireCycle for how timelimit is handled. */
-    start = ustime();
-    timelimit = 1000000*server.active_defrag_running/server.hz/100;
-    if (timelimit <= 0) timelimit = 1;
-
-    do {
-        if (!cursor) {
-            /* Move on to next database, and stop if we reached the last one. */
-            if (++current_db >= server.dbnum) {
-                long long now = ustime();
-                size_t frag_bytes;
-                float frag_pct = getAllocatorFragmentation(&frag_bytes);
-                serverLog(LL_VERBOSE,
-                    "Active defrag done in %dms, reallocated=%d, frag=%.0f%%, frag_bytes=%zu",
-                    (int)((now - start_scan)/1000), (int)(server.stat_active_defrag_hits - start_stat), frag_pct, frag_bytes);
-
-                start_scan = now;
-                current_db = -1;
-                cursor = 0;
-                db = NULL;
-                server.active_defrag_running = 0;
-                return;
-            }
-            else if (current_db==0) {
-                /* Start a scan from the first database. */
-                start_scan = ustime();
-                start_stat = server.stat_active_defrag_hits;
-            }
-
-            db = &server.db[current_db];
-            cursor = 0;
-        }
-
-        do {
-            cursor = dictScan(db->dict, cursor, defragScanCallback, defragDictBucketCallback, db);
-            /* Once in 16 scan iterations, or 1000 pointer reallocations
-             * (if we have a lot of pointers in one hash bucket), check if we
-             * reached the tiem limit. */
-            if (cursor && (++iterations > 16 || server.stat_active_defrag_hits - defragged > 1000)) {
-                if ((ustime() - start) > timelimit) {
-                    return;
-                }
-                iterations = 0;
-                defragged = server.stat_active_defrag_hits;
-            }
-        } while(cursor);
-    } while(1);
-}
-
-#else /* HAVE_DEFRAG */
-
-void activeDefragCycle(void) {
-    /* Not implemented yet. */
-}
-
-#endif

src/dict.c

@@ -37,19 +37,15 @@
 
 #include <stdio.h>
 #include <stdlib.h>
-#include <stdint.h>
 #include <string.h>
 #include <stdarg.h>
 #include <limits.h>
 #include <sys/time.h>
+#include <ctype.h>
 
 #include "dict.h"
 #include "zmalloc.h"
-#ifndef DICT_BENCHMARK_MAIN
 #include "redisassert.h"
-#else
-#include <assert.h>
-#endif
 
 /* Using dictEnableResize() / dictDisableResize() we make possible to
  * enable/disable resizing of the hash table as needed. This is very important
@@ -66,33 +62,94 @@ static unsigned int dict_force_resize_ratio = 5;
 
 static int _dictExpandIfNeeded(dict *ht);
 static unsigned long _dictNextPower(unsigned long size);
-static int _dictKeyIndex(dict *ht, const void *key, unsigned int hash, dictEntry **existing);
+static int _dictKeyIndex(dict *ht, const void *key);
 static int _dictInit(dict *ht, dictType *type, void *privDataPtr);
 
 /* -------------------------- hash functions -------------------------------- */
 
-static uint8_t dict_hash_function_seed[16];
-
-void dictSetHashFunctionSeed(uint8_t *seed) {
-    memcpy(dict_hash_function_seed,seed,sizeof(dict_hash_function_seed));
+/* Thomas Wang's 32 bit Mix Function */
+unsigned int dictIntHashFunction(unsigned int key)
+{
+    key += ~(key << 15);
+    key ^=  (key >> 10);
+    key +=  (key << 3);
+    key ^=  (key >> 6);
+    key += ~(key << 11);
+    key ^=  (key >> 16);
+    return key;
 }
 
-uint8_t *dictGetHashFunctionSeed(void) {
+static uint32_t dict_hash_function_seed = 5381;
+
+void dictSetHashFunctionSeed(uint32_t seed) {
+    dict_hash_function_seed = seed;
+}
+
+uint32_t dictGetHashFunctionSeed(void) {
     return dict_hash_function_seed;
 }
 
-/* The default hashing function uses SipHash implementation
- * in siphash.c. */
+/* MurmurHash2, by Austin Appleby
+ * Note - This code makes a few assumptions about how your machine behaves -
+ * 1. We can read a 4-byte value from any address without crashing
+ * 2. sizeof(int) == 4
+ *
+ * And it has a few limitations -
+ *
+ * 1. It will not work incrementally.
+ * 2. It will not produce the same results on little-endian and big-endian
+ *    machines.
+ */
+unsigned int dictGenHashFunction(const void *key, int len) {
+    /* 'm' and 'r' are mixing constants generated offline.
+     They're not really 'magic', they just happen to work well.  */
+    uint32_t seed = dict_hash_function_seed;
+    const uint32_t m = 0x5bd1e995;
+    const int r = 24;
 
-uint64_t siphash(const uint8_t *in, const size_t inlen, const uint8_t *k);
-uint64_t siphash_nocase(const uint8_t *in, const size_t inlen, const uint8_t *k);
+    /* Initialize the hash to a 'random' value */
+    uint32_t h = seed ^ len;
 
-uint64_t dictGenHashFunction(const void *key, int len) {
-    return siphash(key,len,dict_hash_function_seed);
+    /* Mix 4 bytes at a time into the hash */
+    const unsigned char *data = (const unsigned char *)key;
+
+    while(len >= 4) {
+        uint32_t k = *(uint32_t*)data;
+
+        k *= m;
+        k ^= k >> r;
+        k *= m;
+
+        h *= m;
+        h ^= k;
+
+        data += 4;
+        len -= 4;
+    }
+
+    /* Handle the last few bytes of the input array  */
+    switch(len) {
+    case 3: h ^= data[2] << 16;
+    case 2: h ^= data[1] << 8;
+    case 1: h ^= data[0]; h *= m;
+    };
+
+    /* Do a few final mixes of the hash to ensure the last few
+     * bytes are well-incorporated. */
+    h ^= h >> 13;
+    h *= m;
+    h ^= h >> 15;
+
+    return (unsigned int)h;
 }
 
-uint64_t dictGenCaseHashFunction(const unsigned char *buf, int len) {
-    return siphash_nocase(buf,len,dict_hash_function_seed);
+/* And a case insensitive hash function (based on djb hash) */
+unsigned int dictGenCaseHashFunction(const unsigned char *buf, int len) {
+    unsigned int hash = (unsigned int)dict_hash_function_seed;
+
+    while (len--)
+        hash = ((hash << 5) + hash) + (tolower(*buf++)); /* hash * 33 + c */
+    return hash;
 }
 
 /* ----------------------------- API implementation ------------------------- */
@@ -264,32 +321,29 @@ static void _dictRehashStep(dict *d) {
 /* Add an element to the target hash table */
 int dictAdd(dict *d, void *key, void *val)
 {
-    dictEntry *entry = dictAddRaw(d,key,NULL);
+    dictEntry *entry = dictAddRaw(d,key);
 
     if (!entry) return DICT_ERR;
     dictSetVal(d, entry, val);
     return DICT_OK;
 }
 
-/* Low level add or find:
- * This function adds the entry but instead of setting a value returns the
- * dictEntry structure to the user, that will make sure to fill the value
- * field as he wishes.
+/* Low level add. This function adds the entry but instead of setting
+ * a value returns the dictEntry structure to the user, that will make
+ * sure to fill the value field as he wishes.
  *
  * This function is also directly exposed to the user API to be called
  * mainly in order to store non-pointers inside the hash value, example:
  *
- * entry = dictAddRaw(dict,mykey,NULL);
+ * entry = dictAddRaw(dict,mykey);
  * if (entry != NULL) dictSetSignedIntegerVal(entry,1000);
  *
  * Return values:
  *
- * If key already exists NULL is returned, and "*existing" is populated
- * with the existing entry if existing is not NULL.
- *
+ * If key already exists NULL is returned.
  * If key was added, the hash entry is returned to be manipulated by the caller.
  */
-dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing)
+dictEntry *dictAddRaw(dict *d, void *key)
 {
     int index;
     dictEntry *entry;
@@ -299,7 +353,7 @@ dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing)
 
     /* Get the index of the new element, or -1 if
      * the element already exists. */
-    if ((index = _dictKeyIndex(d, key, dictHashKey(d,key), existing)) == -1)
+    if ((index = _dictKeyIndex(d, key)) == -1)
         return NULL;
 
     /* Allocate the memory and store the new entry.
@@ -317,57 +371,51 @@ dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing)
     return entry;
 }
 
-/* Add or Overwrite:
- * Add an element, discarding the old value if the key already exists.
+/* Add an element, discarding the old if the key already exists.
  * Return 1 if the key was added from scratch, 0 if there was already an
  * element with such key and dictReplace() just performed a value update
  * operation. */
 int dictReplace(dict *d, void *key, void *val)
 {
-    dictEntry *entry, *existing, auxentry;
+    dictEntry *entry, auxentry;
 
     /* Try to add the element. If the key
      * does not exists dictAdd will suceed. */
-    entry = dictAddRaw(d,key,&existing);
-    if (entry) {
-        dictSetVal(d, entry, val);
+    if (dictAdd(d, key, val) == DICT_OK)
         return 1;
-    }
-
+    /* It already exists, get the entry */
+    entry = dictFind(d, key);
     /* Set the new value and free the old one. Note that it is important
      * to do that in this order, as the value may just be exactly the same
      * as the previous one. In this context, think to reference counting,
      * you want to increment (set), and then decrement (free), and not the
      * reverse. */
-    auxentry = *existing;
-    dictSetVal(d, existing, val);
+    auxentry = *entry;
+    dictSetVal(d, entry, val);
     dictFreeVal(d, &auxentry);
     return 0;
 }
 
-/* Add or Find:
- * dictAddOrFind() is simply a version of dictAddRaw() that always
+/* dictReplaceRaw() is simply a version of dictAddRaw() that always
  * returns the hash entry of the specified key, even if the key already
  * exists and can't be added (in that case the entry of the already
  * existing key is returned.)
  *
  * See dictAddRaw() for more information. */
-dictEntry *dictAddOrFind(dict *d, void *key) {
-    dictEntry *entry, *existing;
-    entry = dictAddRaw(d,key,&existing);
-    return entry ? entry : existing;
+dictEntry *dictReplaceRaw(dict *d, void *key) {
+    dictEntry *entry = dictFind(d,key);
+
+    return entry ? entry : dictAddRaw(d,key);
 }
 
-/* Search and remove an element. This is an helper function for
- * dictDelete() and dictUnlink(), please check the top comment
- * of those functions. */
-static dictEntry *dictGenericDelete(dict *d, const void *key, int nofree) {
+/* Search and remove an element */
+static int dictGenericDelete(dict *d, const void *key, int nofree)
+{
     unsigned int h, idx;
     dictEntry *he, *prevHe;
     int table;
 
-    if (d->ht[0].used == 0 && d->ht[1].used == 0) return NULL;
-
+    if (d->ht[0].size == 0) return DICT_ERR; /* d->ht[0].table is NULL */
     if (dictIsRehashing(d)) _dictRehashStep(d);
     h = dictHashKey(d, key);
 
@@ -385,59 +433,27 @@ static dictEntry *dictGenericDelete(dict *d, const void *key, int nofree) {
                 if (!nofree) {
                     dictFreeKey(d, he);
                     dictFreeVal(d, he);
-                    zfree(he);
                 }
+                zfree(he);
                 d->ht[table].used--;
-                return he;
+                return DICT_OK;
             }
             prevHe = he;
             he = he->next;
         }
         if (!dictIsRehashing(d)) break;
     }
-    return NULL; /* not found */
+    return DICT_ERR; /* not found */
 }
 
-/* Remove an element, returning DICT_OK on success or DICT_ERR if the
- * element was not found. */
 int dictDelete(dict *ht, const void *key) {
-    return dictGenericDelete(ht,key,0) ? DICT_OK : DICT_ERR;
+    return dictGenericDelete(ht,key,0);
 }
 
-/* Remove an element from the table, but without actually releasing
- * the key, value and dictionary entry. The dictionary entry is returned
- * if the element was found (and unlinked from the table), and the user
- * should later call `dictFreeUnlinkedEntry()` with it in order to release it.
- * Otherwise if the key is not found, NULL is returned.
- *
- * This function is useful when we want to remove something from the hash
- * table but want to use its value before actually deleting the entry.
- * Without this function the pattern would require two lookups:
- *
- *  entry = dictFind(...);
- *  // Do something with entry
- *  dictDelete(dictionary,entry);
- *
- * Thanks to this function it is possible to avoid this, and use
- * instead:
- *
- * entry = dictUnlink(dictionary,entry);
- * // Do something with entry
- * dictFreeUnlinkedEntry(entry); // <- This does not need to lookup again.
- */
-dictEntry *dictUnlink(dict *ht, const void *key) {
+int dictDeleteNoFree(dict *ht, const void *key) {
     return dictGenericDelete(ht,key,1);
 }
 
-/* You need to call this function to really free the entry after a call
- * to dictUnlink(). It's safe to call this function with 'he' = NULL. */
-void dictFreeUnlinkedEntry(dict *d, dictEntry *he) {
-    if (he == NULL) return;
-    dictFreeKey(d, he);
-    dictFreeVal(d, he);
-    zfree(he);
-}
-
 /* Destroy an entire dictionary */
 int _dictClear(dict *d, dictht *ht, void(callback)(void *)) {
     unsigned long i;
@@ -836,11 +852,10 @@ static unsigned long rev(unsigned long v) {
 unsigned long dictScan(dict *d,
                        unsigned long v,
                        dictScanFunction *fn,
-                       dictScanBucketFunction* bucketfn,
                        void *privdata)
 {
     dictht *t0, *t1;
-    const dictEntry *de, *next;
+    const dictEntry *de;
     unsigned long m0, m1;
 
     if (dictSize(d) == 0) return 0;
@@ -850,12 +865,10 @@ unsigned long dictScan(dict *d,
         m0 = t0->sizemask;
 
         /* Emit entries at cursor */
-        if (bucketfn) bucketfn(privdata, &t0->table[v & m0]);
         de = t0->table[v & m0];
         while (de) {
-            next = de->next;
             fn(privdata, de);
-            de = next;
+            de = de->next;
         }
 
     } else {
@@ -872,24 +885,20 @@ unsigned long dictScan(dict *d,
         m1 = t1->sizemask;
 
         /* Emit entries at cursor */
-        if (bucketfn) bucketfn(privdata, &t0->table[v & m0]);
         de = t0->table[v & m0];
         while (de) {
-            next = de->next;
             fn(privdata, de);
-            de = next;
+            de = de->next;
         }
 
         /* Iterate over indices in larger table that are the expansion
          * of the index pointed to by the cursor in the smaller table */
         do {
             /* Emit entries at cursor */
-            if (bucketfn) bucketfn(privdata, &t1->table[v & m1]);
             de = t1->table[v & m1];
             while (de) {
-                next = de->next;
                 fn(privdata, de);
-                de = next;
+                de = de->next;
             }
 
             /* Increment bits not covered by the smaller mask */
@@ -940,7 +949,7 @@ static unsigned long _dictNextPower(unsigned long size)
 {
     unsigned long i = DICT_HT_INITIAL_SIZE;
 
-    if (size >= LONG_MAX) return LONG_MAX + 1LU;
+    if (size >= LONG_MAX) return LONG_MAX;
     while(1) {
         if (i >= size)
             return i;
@@ -950,29 +959,27 @@ static unsigned long _dictNextPower(unsigned long size)
 
 /* Returns the index of a free slot that can be populated with
  * a hash entry for the given 'key'.
- * If the key already exists, -1 is returned
- * and the optional output parameter may be filled.
+ * If the key already exists, -1 is returned.
  *
  * Note that if we are in the process of rehashing the hash table, the
  * index is always returned in the context of the second (new) hash table. */
-static int _dictKeyIndex(dict *d, const void *key, unsigned int hash, dictEntry **existing)
+static int _dictKeyIndex(dict *d, const void *key)
 {
-    unsigned int idx, table;
+    unsigned int h, idx, table;
     dictEntry *he;
-    if (existing) *existing = NULL;
 
     /* Expand the hash table if needed */
     if (_dictExpandIfNeeded(d) == DICT_ERR)
         return -1;
+    /* Compute the key hash value */
+    h = dictHashKey(d, key);
     for (table = 0; table <= 1; table++) {
-        idx = hash & d->ht[table].sizemask;
+        idx = h & d->ht[table].sizemask;
         /* Search if this slot does not already contain the given key */
         he = d->ht[table].table[idx];
         while(he) {
-            if (key==he->key || dictCompareKeys(d, key, he->key)) {
-                if (existing) *existing = he;
+            if (key==he->key || dictCompareKeys(d, key, he->key))
                 return -1;
-            }
             he = he->next;
         }
         if (!dictIsRehashing(d)) break;
@@ -995,35 +1002,6 @@ void dictDisableResize(void) {
     dict_can_resize = 0;
 }
 
-unsigned int dictGetHash(dict *d, const void *key) {
-    return dictHashKey(d, key);
-}
-
-/* Finds the dictEntry reference by using pointer and pre-calculated hash.
- * oldkey is a dead pointer and should not be accessed.
- * the hash value should be provided using dictGetHash.
- * no string / key comparison is performed.
- * return value is the reference to the dictEntry if found, or NULL if not found. */
-dictEntry **dictFindEntryRefByPtrAndHash(dict *d, const void *oldptr, unsigned int hash) {
-    dictEntry *he, **heref;
-    unsigned int idx, table;
-
-    if (d->ht[0].used + d->ht[1].used == 0) return NULL; /* dict is empty */
-    for (table = 0; table <= 1; table++) {
-        idx = hash & d->ht[table].sizemask;
-        heref = &d->ht[table].table[idx];
-        he = *heref;
-        while(he) {
-            if (oldptr==he->key)
-                return heref;
-            heref = &he->next;
-            he = *heref;
-        }
-        if (!dictIsRehashing(d)) return NULL;
-    }
-    return NULL;
-}
-
 /* ------------------------------- Debugging ---------------------------------*/
 
 #define DICT_STATS_VECTLEN 50
@@ -1102,120 +1080,3 @@ void dictGetStats(char *buf, size_t bufsize, dict *d) {
     /* Make sure there is a NULL term at the end. */
     if (orig_bufsize) orig_buf[orig_bufsize-1] = '\0';
 }
-
-/* ------------------------------- Benchmark ---------------------------------*/
-
-#ifdef DICT_BENCHMARK_MAIN
-
-#include "sds.h"
-
-uint64_t hashCallback(const void *key) {
-    return dictGenHashFunction((unsigned char*)key, sdslen((char*)key));
-}
-
-int compareCallback(void *privdata, const void *key1, const void *key2) {
-    int l1,l2;
-    DICT_NOTUSED(privdata);
-
-    l1 = sdslen((sds)key1);
-    l2 = sdslen((sds)key2);
-    if (l1 != l2) return 0;
-    return memcmp(key1, key2, l1) == 0;
-}
-
-void freeCallback(void *privdata, void *val) {
-    DICT_NOTUSED(privdata);
-
-    sdsfree(val);
-}
-
-dictType BenchmarkDictType = {
-    hashCallback,
-    NULL,
-    NULL,
-    compareCallback,
-    freeCallback,
-    NULL
-};
-
-#define start_benchmark() start = timeInMilliseconds()
-#define end_benchmark(msg) do { \
-    elapsed = timeInMilliseconds()-start; \
-    printf(msg ": %ld items in %lld ms\n", count, elapsed); \
-} while(0);
-
-/* dict-benchmark [count] */
-int main(int argc, char **argv) {
-    long j;
-    long long start, elapsed;
-    dict *dict = dictCreate(&BenchmarkDictType,NULL);
-    long count = 0;
-
-    if (argc == 2) {
-        count = strtol(argv[1],NULL,10);
-    } else {
-        count = 5000000;
-    }
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        int retval = dictAdd(dict,sdsfromlonglong(j),(void*)j);
-        assert(retval == DICT_OK);
-    }
-    end_benchmark("Inserting");
-    assert((long)dictSize(dict) == count);
-
-    /* Wait for rehashing. */
-    while (dictIsRehashing(dict)) {
-        dictRehashMilliseconds(dict,100);
-    }
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        sds key = sdsfromlonglong(j);
-        dictEntry *de = dictFind(dict,key);
-        assert(de != NULL);
-        sdsfree(key);
-    }
-    end_benchmark("Linear access of existing elements");
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        sds key = sdsfromlonglong(j);
-        dictEntry *de = dictFind(dict,key);
-        assert(de != NULL);
-        sdsfree(key);
-    }
-    end_benchmark("Linear access of existing elements (2nd round)");
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        sds key = sdsfromlonglong(rand() % count);
-        dictEntry *de = dictFind(dict,key);
-        assert(de != NULL);
-        sdsfree(key);
-    }
-    end_benchmark("Random access of existing elements");
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        sds key = sdsfromlonglong(rand() % count);
-        key[0] = 'X';
-        dictEntry *de = dictFind(dict,key);
-        assert(de == NULL);
-        sdsfree(key);
-    }
-    end_benchmark("Accessing missing");
-
-    start_benchmark();
-    for (j = 0; j < count; j++) {
-        sds key = sdsfromlonglong(j);
-        int retval = dictDelete(dict,key);
-        assert(retval == DICT_OK);
-        key[0] += 17; /* Change first number to letter. */
-        retval = dictAdd(dict,key,(void*)j);
-        assert(retval == DICT_OK);
-    }
-    end_benchmark("Removing and adding");
-}
-#endif

src/dict.h

@@ -56,7 +56,7 @@ typedef struct dictEntry {
 } dictEntry;
 
 typedef struct dictType {
-    uint64_t (*hashFunction)(const void *key);
+    unsigned int (*hashFunction)(const void *key);
     void *(*keyDup)(void *privdata, const void *key);
     void *(*valDup)(void *privdata, const void *obj);
     int (*keyCompare)(void *privdata, const void *key1, const void *key2);
@@ -78,7 +78,7 @@ typedef struct dict {
     void *privdata;
     dictht ht[2];
     long rehashidx; /* rehashing not in progress if rehashidx == -1 */
-    unsigned long iterators; /* number of iterators currently running */
+    int iterators; /* number of iterators currently running */
 } dict;
 
 /* If safe is set to 1 this is a safe iterator, that means, you can call
@@ -95,7 +95,6 @@ typedef struct dictIterator {
 } dictIterator;
 
 typedef void (dictScanFunction)(void *privdata, const dictEntry *de);
-typedef void (dictScanBucketFunction)(void *privdata, dictEntry **bucketref);
 
 /* This is the initial size of every hash table */
 #define DICT_HT_INITIAL_SIZE     4
@@ -107,19 +106,19 @@ typedef void (dictScanBucketFunction)(void *privdata, dictEntry **bucketref);
 
 #define dictSetVal(d, entry, _val_) do { \
     if ((d)->type->valDup) \
-        (entry)->v.val = (d)->type->valDup((d)->privdata, _val_); \
+        entry->v.val = (d)->type->valDup((d)->privdata, _val_); \
     else \
-        (entry)->v.val = (_val_); \
+        entry->v.val = (_val_); \
 } while(0)
 
 #define dictSetSignedIntegerVal(entry, _val_) \
-    do { (entry)->v.s64 = _val_; } while(0)
+    do { entry->v.s64 = _val_; } while(0)
 
 #define dictSetUnsignedIntegerVal(entry, _val_) \
-    do { (entry)->v.u64 = _val_; } while(0)
+    do { entry->v.u64 = _val_; } while(0)
 
 #define dictSetDoubleVal(entry, _val_) \
-    do { (entry)->v.d = _val_; } while(0)
+    do { entry->v.d = _val_; } while(0)
 
 #define dictFreeKey(d, entry) \
     if ((d)->type->keyDestructor) \
@@ -127,9 +126,9 @@ typedef void (dictScanBucketFunction)(void *privdata, dictEntry **bucketref);
 
 #define dictSetKey(d, entry, _key_) do { \
     if ((d)->type->keyDup) \
-        (entry)->key = (d)->type->keyDup((d)->privdata, _key_); \
+        entry->key = (d)->type->keyDup((d)->privdata, _key_); \
     else \
-        (entry)->key = (_key_); \
+        entry->key = (_key_); \
 } while(0)
 
 #define dictCompareKeys(d, key1, key2) \
@@ -151,12 +150,11 @@ typedef void (dictScanBucketFunction)(void *privdata, dictEntry **bucketref);
 dict *dictCreate(dictType *type, void *privDataPtr);
 int dictExpand(dict *d, unsigned long size);
 int dictAdd(dict *d, void *key, void *val);
-dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing);
-dictEntry *dictAddOrFind(dict *d, void *key);
+dictEntry *dictAddRaw(dict *d, void *key);
 int dictReplace(dict *d, void *key, void *val);
+dictEntry *dictReplaceRaw(dict *d, void *key);
 int dictDelete(dict *d, const void *key);
-dictEntry *dictUnlink(dict *ht, const void *key);
-void dictFreeUnlinkedEntry(dict *d, dictEntry *he);
+int dictDeleteNoFree(dict *d, const void *key);
 void dictRelease(dict *d);
 dictEntry * dictFind(dict *d, const void *key);
 void *dictFetchValue(dict *d, const void *key);
@@ -168,18 +166,16 @@ void dictReleaseIterator(dictIterator *iter);
 dictEntry *dictGetRandomKey(dict *d);
 unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count);
 void dictGetStats(char *buf, size_t bufsize, dict *d);
-uint64_t dictGenHashFunction(const void *key, int len);
-uint64_t dictGenCaseHashFunction(const unsigned char *buf, int len);
+unsigned int dictGenHashFunction(const void *key, int len);
+unsigned int dictGenCaseHashFunction(const unsigned char *buf, int len);
 void dictEmpty(dict *d, void(callback)(void*));
 void dictEnableResize(void);
 void dictDisableResize(void);
 int dictRehash(dict *d, int n);
 int dictRehashMilliseconds(dict *d, int ms);
-void dictSetHashFunctionSeed(uint8_t *seed);
-uint8_t *dictGetHashFunctionSeed(void);
-unsigned long dictScan(dict *d, unsigned long v, dictScanFunction *fn, dictScanBucketFunction *bucketfn, void *privdata);
-unsigned int dictGetHash(dict *d, const void *key);
-dictEntry **dictFindEntryRefByPtrAndHash(dict *d, const void *oldptr, unsigned int hash);
+void dictSetHashFunctionSeed(unsigned int initval);
+unsigned int dictGetHashFunctionSeed(void);
+unsigned long dictScan(dict *d, unsigned long v, dictScanFunction *fn, void *privdata);
 
 /* Hash table types */
 extern dictType dictTypeHeapStringCopyKey;

src/evict.c

@@ -1,567 +0,0 @@
-/* Maxmemory directive handling (LRU eviction and other policies).
- *
- * ----------------------------------------------------------------------------
- *
- * Copyright (c) 2009-2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "server.h"
-#include "bio.h"
-#include "atomicvar.h"
-
-/* ----------------------------------------------------------------------------
- * Data structures
- * --------------------------------------------------------------------------*/
-
-/* To improve the quality of the LRU approximation we take a set of keys
- * that are good candidate for eviction across freeMemoryIfNeeded() calls.
- *
- * Entries inside the eviciton pool are taken ordered by idle time, putting
- * greater idle times to the right (ascending order).
- *
- * When an LFU policy is used instead, a reverse frequency indication is used
- * instead of the idle time, so that we still evict by larger value (larger
- * inverse frequency means to evict keys with the least frequent accesses).
- *
- * Empty entries have the key pointer set to NULL. */
-#define EVPOOL_SIZE 16
-#define EVPOOL_CACHED_SDS_SIZE 255
-struct evictionPoolEntry {
-    unsigned long long idle;    /* Object idle time (inverse frequency for LFU) */
-    sds key;                    /* Key name. */
-    sds cached;                 /* Cached SDS object for key name. */
-    int dbid;                   /* Key DB number. */
-};
-
-static struct evictionPoolEntry *EvictionPoolLRU;
-
-unsigned long LFUDecrAndReturn(robj *o);
-
-/* ----------------------------------------------------------------------------
- * Implementation of eviction, aging and LRU
- * --------------------------------------------------------------------------*/
-
-/* Return the LRU clock, based on the clock resolution. This is a time
- * in a reduced-bits format that can be used to set and check the
- * object->lru field of redisObject structures. */
-unsigned int getLRUClock(void) {
-    return (mstime()/LRU_CLOCK_RESOLUTION) & LRU_CLOCK_MAX;
-}
-
-/* This function is used to obtain the current LRU clock.
- * If the current resolution is lower than the frequency we refresh the
- * LRU clock (as it should be in production servers) we return the
- * precomputed value, otherwise we need to resort to a system call. */
-unsigned int LRU_CLOCK(void) {
-    unsigned int lruclock;
-    if (1000/server.hz <= LRU_CLOCK_RESOLUTION) {
-        atomicGet(server.lruclock,lruclock);
-    } else {
-        lruclock = getLRUClock();
-    }
-    return lruclock;
-}
-
-/* Given an object returns the min number of milliseconds the object was never
- * requested, using an approximated LRU algorithm. */
-unsigned long long estimateObjectIdleTime(robj *o) {
-    unsigned long long lruclock = LRU_CLOCK();
-    if (lruclock >= o->lru) {
-        return (lruclock - o->lru) * LRU_CLOCK_RESOLUTION;
-    } else {
-        return (lruclock + (LRU_CLOCK_MAX - o->lru)) *
-                    LRU_CLOCK_RESOLUTION;
-    }
-}
-
-/* freeMemoryIfNeeded() gets called when 'maxmemory' is set on the config
- * file to limit the max memory used by the server, before processing a
- * command.
- *
- * The goal of the function is to free enough memory to keep Redis under the
- * configured memory limit.
- *
- * The function starts calculating how many bytes should be freed to keep
- * Redis under the limit, and enters a loop selecting the best keys to
- * evict accordingly to the configured policy.
- *
- * If all the bytes needed to return back under the limit were freed the
- * function returns C_OK, otherwise C_ERR is returned, and the caller
- * should block the execution of commands that will result in more memory
- * used by the server.
- *
- * ------------------------------------------------------------------------
- *
- * LRU approximation algorithm
- *
- * Redis uses an approximation of the LRU algorithm that runs in constant
- * memory. Every time there is a key to expire, we sample N keys (with
- * N very small, usually in around 5) to populate a pool of best keys to
- * evict of M keys (the pool size is defined by EVPOOL_SIZE).
- *
- * The N keys sampled are added in the pool of good keys to expire (the one
- * with an old access time) if they are better than one of the current keys
- * in the pool.
- *
- * After the pool is populated, the best key we have in the pool is expired.
- * However note that we don't remove keys from the pool when they are deleted
- * so the pool may contain keys that no longer exist.
- *
- * When we try to evict a key, and all the entries in the pool don't exist
- * we populate it again. This time we'll be sure that the pool has at least
- * one key that can be evicted, if there is at least one key that can be
- * evicted in the whole database. */
-
-/* Create a new eviction pool. */
-void evictionPoolAlloc(void) {
-    struct evictionPoolEntry *ep;
-    int j;
-
-    ep = zmalloc(sizeof(*ep)*EVPOOL_SIZE);
-    for (j = 0; j < EVPOOL_SIZE; j++) {
-        ep[j].idle = 0;
-        ep[j].key = NULL;
-        ep[j].cached = sdsnewlen(NULL,EVPOOL_CACHED_SDS_SIZE);
-        ep[j].dbid = 0;
-    }
-    EvictionPoolLRU = ep;
-}
-
-/* This is an helper function for freeMemoryIfNeeded(), it is used in order
- * to populate the evictionPool with a few entries every time we want to
- * expire a key. Keys with idle time smaller than one of the current
- * keys are added. Keys are always added if there are free entries.
- *
- * We insert keys on place in ascending order, so keys with the smaller
- * idle time are on the left, and keys with the higher idle time on the
- * right. */
-
-void evictionPoolPopulate(int dbid, dict *sampledict, dict *keydict, struct evictionPoolEntry *pool) {
-    int j, k, count;
-    dictEntry *samples[server.maxmemory_samples];
-
-    count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);
-    for (j = 0; j < count; j++) {
-        unsigned long long idle;
-        sds key;
-        robj *o;
-        dictEntry *de;
-
-        de = samples[j];
-        key = dictGetKey(de);
-
-        /* If the dictionary we are sampling from is not the main
-         * dictionary (but the expires one) we need to lookup the key
-         * again in the key dictionary to obtain the value object. */
-        if (server.maxmemory_policy != MAXMEMORY_VOLATILE_TTL) {
-            if (sampledict != keydict) de = dictFind(keydict, key);
-            o = dictGetVal(de);
-        }
-
-        /* Calculate the idle time according to the policy. This is called
-         * idle just because the code initially handled LRU, but is in fact
-         * just a score where an higher score means better candidate. */
-        if (server.maxmemory_policy & MAXMEMORY_FLAG_LRU) {
-            idle = estimateObjectIdleTime(o);
-        } else if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-            /* When we use an LRU policy, we sort the keys by idle time
-             * so that we expire keys starting from greater idle time.
-             * However when the policy is an LFU one, we have a frequency
-             * estimation, and we want to evict keys with lower frequency
-             * first. So inside the pool we put objects using the inverted
-             * frequency subtracting the actual frequency to the maximum
-             * frequency of 255. */
-            idle = 255-LFUDecrAndReturn(o);
-        } else if (server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL) {
-            /* In this case the sooner the expire the better. */
-            idle = ULLONG_MAX - (long)dictGetVal(de);
-        } else {
-            serverPanic("Unknown eviction policy in evictionPoolPopulate()");
-        }
-
-        /* Insert the element inside the pool.
-         * First, find the first empty bucket or the first populated
-         * bucket that has an idle time smaller than our idle time. */
-        k = 0;
-        while (k < EVPOOL_SIZE &&
-               pool[k].key &&
-               pool[k].idle < idle) k++;
-        if (k == 0 && pool[EVPOOL_SIZE-1].key != NULL) {
-            /* Can't insert if the element is < the worst element we have
-             * and there are no empty buckets. */
-            continue;
-        } else if (k < EVPOOL_SIZE && pool[k].key == NULL) {
-            /* Inserting into empty position. No setup needed before insert. */
-        } else {
-            /* Inserting in the middle. Now k points to the first element
-             * greater than the element to insert.  */
-            if (pool[EVPOOL_SIZE-1].key == NULL) {
-                /* Free space on the right? Insert at k shifting
-                 * all the elements from k to end to the right. */
-
-                /* Save SDS before overwriting. */
-                sds cached = pool[EVPOOL_SIZE-1].cached;
-                memmove(pool+k+1,pool+k,
-                    sizeof(pool[0])*(EVPOOL_SIZE-k-1));
-                pool[k].cached = cached;
-            } else {
-                /* No free space on right? Insert at k-1 */
-                k--;
-                /* Shift all elements on the left of k (included) to the
-                 * left, so we discard the element with smaller idle time. */
-                sds cached = pool[0].cached; /* Save SDS before overwriting. */
-                if (pool[0].key != pool[0].cached) sdsfree(pool[0].key);
-                memmove(pool,pool+1,sizeof(pool[0])*k);
-                pool[k].cached = cached;
-            }
-        }
-
-        /* Try to reuse the cached SDS string allocated in the pool entry,
-         * because allocating and deallocating this object is costly
-         * (according to the profiler, not my fantasy. Remember:
-         * premature optimizbla bla bla bla. */
-        int klen = sdslen(key);
-        if (klen > EVPOOL_CACHED_SDS_SIZE) {
-            pool[k].key = sdsdup(key);
-        } else {
-            memcpy(pool[k].cached,key,klen+1);
-            sdssetlen(pool[k].cached,klen);
-            pool[k].key = pool[k].cached;
-        }
-        pool[k].idle = idle;
-        pool[k].dbid = dbid;
-    }
-}
-
-/* ----------------------------------------------------------------------------
- * LFU (Least Frequently Used) implementation.
-
- * We have 24 total bits of space in each object in order to implement
- * an LFU (Least Frequently Used) eviction policy, since we re-use the
- * LRU field for this purpose.
- *
- * We split the 24 bits into two fields:
- *
- *          16 bits      8 bits
- *     +----------------+--------+
- *     + Last decr time | LOG_C  |
- *     +----------------+--------+
- *
- * LOG_C is a logarithmic counter that provides an indication of the access
- * frequency. However this field must also be decremented otherwise what used
- * to be a frequently accessed key in the past, will remain ranked like that
- * forever, while we want the algorithm to adapt to access pattern changes.
- *
- * So the remaining 16 bits are used in order to store the "decrement time",
- * a reduced-precision Unix time (we take 16 bits of the time converted
- * in minutes since we don't care about wrapping around) where the LOG_C
- * counter is halved if it has an high value, or just decremented if it
- * has a low value.
- *
- * New keys don't start at zero, in order to have the ability to collect
- * some accesses before being trashed away, so they start at COUNTER_INIT_VAL.
- * The logarithmic increment performed on LOG_C takes care of COUNTER_INIT_VAL
- * when incrementing the key, so that keys starting at COUNTER_INIT_VAL
- * (or having a smaller value) have a very high chance of being incremented
- * on access.
- *
- * During decrement, the value of the logarithmic counter is halved if
- * its current value is greater than two times the COUNTER_INIT_VAL, otherwise
- * it is just decremented by one.
- * --------------------------------------------------------------------------*/
-
-/* Return the current time in minutes, just taking the least significant
- * 16 bits. The returned time is suitable to be stored as LDT (last decrement
- * time) for the LFU implementation. */
-unsigned long LFUGetTimeInMinutes(void) {
-    return (server.unixtime/60) & 65535;
-}
-
-/* Given an object last decrement time, compute the minimum number of minutes
- * that elapsed since the last decrement. Handle overflow (ldt greater than
- * the current 16 bits minutes time) considering the time as wrapping
- * exactly once. */
-unsigned long LFUTimeElapsed(unsigned long ldt) {
-    unsigned long now = LFUGetTimeInMinutes();
-    if (now >= ldt) return now-ldt;
-    return 65535-ldt+now;
-}
-
-/* Logarithmically increment a counter. The greater is the current counter value
- * the less likely is that it gets really implemented. Saturate it at 255. */
-uint8_t LFULogIncr(uint8_t counter) {
-    if (counter == 255) return 255;
-    double r = (double)rand()/RAND_MAX;
-    double baseval = counter - LFU_INIT_VAL;
-    if (baseval < 0) baseval = 0;
-    double p = 1.0/(baseval*server.lfu_log_factor+1);
-    if (r < p) counter++;
-    return counter;
-}
-
-/* If the object decrement time is reached, decrement the LFU counter and
- * update the decrement time field. Return the object frequency counter.
- *
- * This function is used in order to scan the dataset for the best object
- * to fit: as we check for the candidate, we incrementally decrement the
- * counter of the scanned objects if needed. */
-#define LFU_DECR_INTERVAL 1
-unsigned long LFUDecrAndReturn(robj *o) {
-    unsigned long ldt = o->lru >> 8;
-    unsigned long counter = o->lru & 255;
-    if (LFUTimeElapsed(ldt) >= server.lfu_decay_time && counter) {
-        if (counter > LFU_INIT_VAL*2) {
-            counter /= 2;
-            if (counter < LFU_INIT_VAL*2) counter = LFU_INIT_VAL*2;
-        } else {
-            counter--;
-        }
-        o->lru = (LFUGetTimeInMinutes()<<8) | counter;
-    }
-    return counter;
-}
-
-/* ----------------------------------------------------------------------------
- * The external API for eviction: freeMemroyIfNeeded() is called by the
- * server when there is data to add in order to make space if needed.
- * --------------------------------------------------------------------------*/
-
-/* We don't want to count AOF buffers and slaves output buffers as
- * used memory: the eviction should use mostly data size. This function
- * returns the sum of AOF and slaves buffer. */
-size_t freeMemoryGetNotCountedMemory(void) {
-    size_t overhead = 0;
-    int slaves = listLength(server.slaves);
-
-    if (slaves) {
-        listIter li;
-        listNode *ln;
-
-        listRewind(server.slaves,&li);
-        while((ln = listNext(&li))) {
-            client *slave = listNodeValue(ln);
-            overhead += getClientOutputBufferMemoryUsage(slave);
-        }
-    }
-    if (server.aof_state != AOF_OFF) {
-        overhead += sdslen(server.aof_buf)+aofRewriteBufferSize();
-    }
-    return overhead;
-}
-
-int freeMemoryIfNeeded(void) {
-    size_t mem_reported, mem_used, mem_tofree, mem_freed;
-    mstime_t latency, eviction_latency;
-    long long delta;
-    int slaves = listLength(server.slaves);
-
-    /* When clients are paused the dataset should be static not just from the
-     * POV of clients not being able to write, but also from the POV of
-     * expires and evictions of keys not being performed. */
-    if (clientsArePaused()) return C_OK;
-
-    /* Check if we are over the memory usage limit. If we are not, no need
-     * to subtract the slaves output buffers. We can just return ASAP. */
-    mem_reported = zmalloc_used_memory();
-    if (mem_reported <= server.maxmemory) return C_OK;
-
-    /* Remove the size of slaves output buffers and AOF buffer from the
-     * count of used memory. */
-    mem_used = mem_reported;
-    size_t overhead = freeMemoryGetNotCountedMemory();
-    mem_used = (mem_used > overhead) ? mem_used-overhead : 0;
-
-    /* Check if we are still over the memory limit. */
-    if (mem_used <= server.maxmemory) return C_OK;
-
-    /* Compute how much memory we need to free. */
-    mem_tofree = mem_used - server.maxmemory;
-    mem_freed = 0;
-
-    if (server.maxmemory_policy == MAXMEMORY_NO_EVICTION)
-        goto cant_free; /* We need to free memory, but policy forbids. */
-
-    latencyStartMonitor(latency);
-    while (mem_freed < mem_tofree) {
-        int j, k, i, keys_freed = 0;
-        static int next_db = 0;
-        sds bestkey = NULL;
-        int bestdbid;
-        redisDb *db;
-        dict *dict;
-        dictEntry *de;
-
-        if (server.maxmemory_policy & (MAXMEMORY_FLAG_LRU|MAXMEMORY_FLAG_LFU) ||
-            server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL)
-        {
-            struct evictionPoolEntry *pool = EvictionPoolLRU;
-
-            while(bestkey == NULL) {
-                unsigned long total_keys = 0, keys;
-
-                /* We don't want to make local-db choices when expiring keys,
-                 * so to start populate the eviction pool sampling keys from
-                 * every DB. */
-                for (i = 0; i < server.dbnum; i++) {
-                    db = server.db+i;
-                    dict = (server.maxmemory_policy & MAXMEMORY_FLAG_ALLKEYS) ?
-                            db->dict : db->expires;
-                    if ((keys = dictSize(dict)) != 0) {
-                        evictionPoolPopulate(i, dict, db->dict, pool);
-                        total_keys += keys;
-                    }
-                }
-                if (!total_keys) break; /* No keys to evict. */
-
-                /* Go backward from best to worst element to evict. */
-                for (k = EVPOOL_SIZE-1; k >= 0; k--) {
-                    if (pool[k].key == NULL) continue;
-                    bestdbid = pool[k].dbid;
-
-                    if (server.maxmemory_policy & MAXMEMORY_FLAG_ALLKEYS) {
-                        de = dictFind(server.db[pool[k].dbid].dict,
-                            pool[k].key);
-                    } else {
-                        de = dictFind(server.db[pool[k].dbid].expires,
-                            pool[k].key);
-                    }
-
-                    /* Remove the entry from the pool. */
-                    if (pool[k].key != pool[k].cached)
-                        sdsfree(pool[k].key);
-                    pool[k].key = NULL;
-                    pool[k].idle = 0;
-
-                    /* If the key exists, is our pick. Otherwise it is
-                     * a ghost and we need to try the next element. */
-                    if (de) {
-                        bestkey = dictGetKey(de);
-                        break;
-                    } else {
-                        /* Ghost... Iterate again. */
-                    }
-                }
-            }
-        }
-
-        /* volatile-random and allkeys-random policy */
-        else if (server.maxmemory_policy == MAXMEMORY_ALLKEYS_RANDOM ||
-                 server.maxmemory_policy == MAXMEMORY_VOLATILE_RANDOM)
-        {
-            /* When evicting a random key, we try to evict a key for
-             * each DB, so we use the static 'next_db' variable to
-             * incrementally visit all DBs. */
-            for (i = 0; i < server.dbnum; i++) {
-                j = (++next_db) % server.dbnum;
-                db = server.db+j;
-                dict = (server.maxmemory_policy == MAXMEMORY_ALLKEYS_RANDOM) ?
-                        db->dict : db->expires;
-                if (dictSize(dict) != 0) {
-                    de = dictGetRandomKey(dict);
-                    bestkey = dictGetKey(de);
-                    bestdbid = j;
-                    break;
-                }
-            }
-        }
-
-        /* Finally remove the selected key. */
-        if (bestkey) {
-            db = server.db+bestdbid;
-            robj *keyobj = createStringObject(bestkey,sdslen(bestkey));
-            propagateExpire(db,keyobj,server.lazyfree_lazy_eviction);
-            /* We compute the amount of memory freed by db*Delete() alone.
-             * It is possible that actually the memory needed to propagate
-             * the DEL in AOF and replication link is greater than the one
-             * we are freeing removing the key, but we can't account for
-             * that otherwise we would never exit the loop.
-             *
-             * AOF and Output buffer memory will be freed eventually so
-             * we only care about memory used by the key space. */
-            delta = (long long) zmalloc_used_memory();
-            latencyStartMonitor(eviction_latency);
-            if (server.lazyfree_lazy_eviction)
-                dbAsyncDelete(db,keyobj);
-            else
-                dbSyncDelete(db,keyobj);
-            latencyEndMonitor(eviction_latency);
-            latencyAddSampleIfNeeded("eviction-del",eviction_latency);
-            latencyRemoveNestedEvent(latency,eviction_latency);
-            delta -= (long long) zmalloc_used_memory();
-            mem_freed += delta;
-            server.stat_evictedkeys++;
-            notifyKeyspaceEvent(NOTIFY_EVICTED, "evicted",
-                keyobj, db->id);
-            decrRefCount(keyobj);
-            keys_freed++;
-
-            /* When the memory to free starts to be big enough, we may
-             * start spending so much time here that is impossible to
-             * deliver data to the slaves fast enough, so we force the
-             * transmission here inside the loop. */
-            if (slaves) flushSlavesOutputBuffers();
-
-            /* Normally our stop condition is the ability to release
-             * a fixed, pre-computed amount of memory. However when we
-             * are deleting objects in another thread, it's better to
-             * check, from time to time, if we already reached our target
-             * memory, since the "mem_freed" amount is computed only
-             * across the dbAsyncDelete() call, while the thread can
-             * release the memory all the time. */
-            if (server.lazyfree_lazy_eviction && !(keys_freed % 16)) {
-                overhead = freeMemoryGetNotCountedMemory();
-                mem_used = zmalloc_used_memory();
-                mem_used = (mem_used > overhead) ? mem_used-overhead : 0;
-                if (mem_used <= server.maxmemory) {
-                    mem_freed = mem_tofree;
-                }
-            }
-        }
-
-        if (!keys_freed) {
-            latencyEndMonitor(latency);
-            latencyAddSampleIfNeeded("eviction-cycle",latency);
-            goto cant_free; /* nothing to free... */
-        }
-    }
-    latencyEndMonitor(latency);
-    latencyAddSampleIfNeeded("eviction-cycle",latency);
-    return C_OK;
-
-cant_free:
-    /* We are here if we are not able to reclaim memory. There is only one
-     * last thing we can try: check if the lazyfree thread has jobs in queue
-     * and wait... */
-    while(bioPendingJobsOfType(BIO_LAZY_FREE)) {
-        if (((mem_reported - zmalloc_used_memory()) + mem_freed) >= mem_tofree)
-            break;
-        usleep(1000);
-    }
-    return C_ERR;
-}
-

src/expire.c

@@ -1,504 +0,0 @@
-/* Implementation of EXPIRE (keys with fixed time to live).
- *
- * ----------------------------------------------------------------------------
- *
- * Copyright (c) 2009-2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "server.h"
-
-/*-----------------------------------------------------------------------------
- * Incremental collection of expired keys.
- *
- * When keys are accessed they are expired on-access. However we need a
- * mechanism in order to ensure keys are eventually removed when expired even
- * if no access is performed on them.
- *----------------------------------------------------------------------------*/
-
-/* Helper function for the activeExpireCycle() function.
- * This function will try to expire the key that is stored in the hash table
- * entry 'de' of the 'expires' hash table of a Redis database.
- *
- * If the key is found to be expired, it is removed from the database and
- * 1 is returned. Otherwise no operation is performed and 0 is returned.
- *
- * When a key is expired, server.stat_expiredkeys is incremented.
- *
- * The parameter 'now' is the current time in milliseconds as is passed
- * to the function to avoid too many gettimeofday() syscalls. */
-int activeExpireCycleTryExpire(redisDb *db, dictEntry *de, long long now) {
-    long long t = dictGetSignedIntegerVal(de);
-    if (now > t) {
-        sds key = dictGetKey(de);
-        robj *keyobj = createStringObject(key,sdslen(key));
-
-        propagateExpire(db,keyobj,server.lazyfree_lazy_expire);
-        if (server.lazyfree_lazy_expire)
-            dbAsyncDelete(db,keyobj);
-        else
-            dbSyncDelete(db,keyobj);
-        notifyKeyspaceEvent(NOTIFY_EXPIRED,
-            "expired",keyobj,db->id);
-        decrRefCount(keyobj);
-        server.stat_expiredkeys++;
-        return 1;
-    } else {
-        return 0;
-    }
-}
-
-/* Try to expire a few timed out keys. The algorithm used is adaptive and
- * will use few CPU cycles if there are few expiring keys, otherwise
- * it will get more aggressive to avoid that too much memory is used by
- * keys that can be removed from the keyspace.
- *
- * No more than CRON_DBS_PER_CALL databases are tested at every
- * iteration.
- *
- * This kind of call is used when Redis detects that timelimit_exit is
- * true, so there is more work to do, and we do it more incrementally from
- * the beforeSleep() function of the event loop.
- *
- * Expire cycle type:
- *
- * If type is ACTIVE_EXPIRE_CYCLE_FAST the function will try to run a
- * "fast" expire cycle that takes no longer than EXPIRE_FAST_CYCLE_DURATION
- * microseconds, and is not repeated again before the same amount of time.
- *
- * If type is ACTIVE_EXPIRE_CYCLE_SLOW, that normal expire cycle is
- * executed, where the time limit is a percentage of the REDIS_HZ period
- * as specified by the ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC define. */
-
-void activeExpireCycle(int type) {
-    /* This function has some global state in order to continue the work
-     * incrementally across calls. */
-    static unsigned int current_db = 0; /* Last DB tested. */
-    static int timelimit_exit = 0;      /* Time limit hit in previous call? */
-    static long long last_fast_cycle = 0; /* When last fast cycle ran. */
-
-    int j, iteration = 0;
-    int dbs_per_call = CRON_DBS_PER_CALL;
-    long long start = ustime(), timelimit;
-
-    /* When clients are paused the dataset should be static not just from the
-     * POV of clients not being able to write, but also from the POV of
-     * expires and evictions of keys not being performed. */
-    if (clientsArePaused()) return;
-
-    if (type == ACTIVE_EXPIRE_CYCLE_FAST) {
-        /* Don't start a fast cycle if the previous cycle did not exited
-         * for time limt. Also don't repeat a fast cycle for the same period
-         * as the fast cycle total duration itself. */
-        if (!timelimit_exit) return;
-        if (start < last_fast_cycle + ACTIVE_EXPIRE_CYCLE_FAST_DURATION*2) return;
-        last_fast_cycle = start;
-    }
-
-    /* We usually should test CRON_DBS_PER_CALL per iteration, with
-     * two exceptions:
-     *
-     * 1) Don't test more DBs than we have.
-     * 2) If last time we hit the time limit, we want to scan all DBs
-     * in this iteration, as there is work to do in some DB and we don't want
-     * expired keys to use memory for too much time. */
-    if (dbs_per_call > server.dbnum || timelimit_exit)
-        dbs_per_call = server.dbnum;
-
-    /* We can use at max ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC percentage of CPU time
-     * per iteration. Since this function gets called with a frequency of
-     * server.hz times per second, the following is the max amount of
-     * microseconds we can spend in this function. */
-    timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;
-    timelimit_exit = 0;
-    if (timelimit <= 0) timelimit = 1;
-
-    if (type == ACTIVE_EXPIRE_CYCLE_FAST)
-        timelimit = ACTIVE_EXPIRE_CYCLE_FAST_DURATION; /* in microseconds. */
-
-    for (j = 0; j < dbs_per_call; j++) {
-        int expired;
-        redisDb *db = server.db+(current_db % server.dbnum);
-
-        /* Increment the DB now so we are sure if we run out of time
-         * in the current DB we'll restart from the next. This allows to
-         * distribute the time evenly across DBs. */
-        current_db++;
-
-        /* Continue to expire if at the end of the cycle more than 25%
-         * of the keys were expired. */
-        do {
-            unsigned long num, slots;
-            long long now, ttl_sum;
-            int ttl_samples;
-
-            /* If there is nothing to expire try next DB ASAP. */
-            if ((num = dictSize(db->expires)) == 0) {
-                db->avg_ttl = 0;
-                break;
-            }
-            slots = dictSlots(db->expires);
-            now = mstime();
-
-            /* When there are less than 1% filled slots getting random
-             * keys is expensive, so stop here waiting for better times...
-             * The dictionary will be resized asap. */
-            if (num && slots > DICT_HT_INITIAL_SIZE &&
-                (num*100/slots < 1)) break;
-
-            /* The main collection cycle. Sample random keys among keys
-             * with an expire set, checking for expired ones. */
-            expired = 0;
-            ttl_sum = 0;
-            ttl_samples = 0;
-
-            if (num > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP)
-                num = ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP;
-
-            while (num--) {
-                dictEntry *de;
-                long long ttl;
-
-                if ((de = dictGetRandomKey(db->expires)) == NULL) break;
-                ttl = dictGetSignedIntegerVal(de)-now;
-                if (activeExpireCycleTryExpire(db,de,now)) expired++;
-                if (ttl > 0) {
-                    /* We want the average TTL of keys yet not expired. */
-                    ttl_sum += ttl;
-                    ttl_samples++;
-                }
-            }
-
-            /* Update the average TTL stats for this database. */
-            if (ttl_samples) {
-                long long avg_ttl = ttl_sum/ttl_samples;
-
-                /* Do a simple running average with a few samples.
-                 * We just use the current estimate with a weight of 2%
-                 * and the previous estimate with a weight of 98%. */
-                if (db->avg_ttl == 0) db->avg_ttl = avg_ttl;
-                db->avg_ttl = (db->avg_ttl/50)*49 + (avg_ttl/50);
-            }
-
-            /* We can't block forever here even if there are many keys to
-             * expire. So after a given amount of milliseconds return to the
-             * caller waiting for the other active expire cycle. */
-            iteration++;
-            if ((iteration & 0xf) == 0) { /* check once every 16 iterations. */
-                long long elapsed = ustime()-start;
-
-                latencyAddSampleIfNeeded("expire-cycle",elapsed/1000);
-                if (elapsed > timelimit) timelimit_exit = 1;
-            }
-            if (timelimit_exit) return;
-            /* We don't repeat the cycle if there are less than 25% of keys
-             * found expired in the current DB. */
-        } while (expired > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP/4);
-    }
-}
-
-/*-----------------------------------------------------------------------------
- * Expires of keys created in writable slaves
- *
- * Normally slaves do not process expires: they wait the masters to synthesize
- * DEL operations in order to retain consistency. However writable slaves are
- * an exception: if a key is created in the slave and an expire is assigned
- * to it, we need a way to expire such a key, since the master does not know
- * anything about such a key.
- *
- * In order to do so, we track keys created in the slave side with an expire
- * set, and call the expireSlaveKeys() function from time to time in order to
- * reclaim the keys if they already expired.
- *
- * Note that the use case we are trying to cover here, is a popular one where
- * slaves are put in writable mode in order to compute slow operations in
- * the slave side that are mostly useful to actually read data in a more
- * processed way. Think at sets intersections in a tmp key, with an expire so
- * that it is also used as a cache to avoid intersecting every time.
- *
- * This implementation is currently not perfect but a lot better than leaking
- * the keys as implemented in 3.2.
- *----------------------------------------------------------------------------*/
-
-/* The dictionary where we remember key names and database ID of keys we may
- * want to expire from the slave. Since this function is not often used we
- * don't even care to initialize the database at startup. We'll do it once
- * the feature is used the first time, that is, when rememberSlaveKeyWithExpire()
- * is called.
- *
- * The dictionary has an SDS string representing the key as the hash table
- * key, while the value is a 64 bit unsigned integer with the bits corresponding
- * to the DB where the keys may exist set to 1. Currently the keys created
- * with a DB id > 63 are not expired, but a trivial fix is to set the bitmap
- * to the max 64 bit unsigned value when we know there is a key with a DB
- * ID greater than 63, and check all the configured DBs in such a case. */
-dict *slaveKeysWithExpire = NULL;
-
-/* Check the set of keys created by the master with an expire set in order to
- * check if they should be evicted. */
-void expireSlaveKeys(void) {
-    if (slaveKeysWithExpire == NULL ||
-        dictSize(slaveKeysWithExpire) == 0) return;
-
-    int cycles = 0, noexpire = 0;
-    mstime_t start = mstime();
-    while(1) {
-        dictEntry *de = dictGetRandomKey(slaveKeysWithExpire);
-        sds keyname = dictGetKey(de);
-        uint64_t dbids = dictGetUnsignedIntegerVal(de);
-        uint64_t new_dbids = 0;
-
-        /* Check the key against every database corresponding to the
-         * bits set in the value bitmap. */
-        int dbid = 0;
-        while(dbids && dbid < server.dbnum) {
-            if ((dbids & 1) != 0) {
-                redisDb *db = server.db+dbid;
-                dictEntry *expire = dictFind(db->expires,keyname);
-                int expired = 0;
-
-                if (expire &&
-                    activeExpireCycleTryExpire(server.db+dbid,expire,start))
-                {
-                    expired = 1;
-                }
-
-                /* If the key was not expired in this DB, we need to set the
-                 * corresponding bit in the new bitmap we set as value.
-                 * At the end of the loop if the bitmap is zero, it means we
-                 * no longer need to keep track of this key. */
-                if (expire && !expired) {
-                    noexpire++;
-                    new_dbids |= (uint64_t)1 << dbid;
-                }
-            }
-            dbid++;
-            dbids >>= 1;
-        }
-
-        /* Set the new bitmap as value of the key, in the dictionary
-         * of keys with an expire set directly in the writable slave. Otherwise
-         * if the bitmap is zero, we no longer need to keep track of it. */
-        if (new_dbids)
-            dictSetUnsignedIntegerVal(de,new_dbids);
-        else
-            dictDelete(slaveKeysWithExpire,keyname);
-
-        /* Stop conditions: found 3 keys we cna't expire in a row or
-         * time limit was reached. */
-        cycles++;
-        if (noexpire > 3) break;
-        if ((cycles % 64) == 0 && mstime()-start > 1) break;
-        if (dictSize(slaveKeysWithExpire) == 0) break;
-    }
-}
-
-/* Track keys that received an EXPIRE or similar command in the context
- * of a writable slave. */
-void rememberSlaveKeyWithExpire(redisDb *db, robj *key) {
-    if (slaveKeysWithExpire == NULL) {
-        static dictType dt = {
-            dictSdsHash,                /* hash function */
-            NULL,                       /* key dup */
-            NULL,                       /* val dup */
-            dictSdsKeyCompare,          /* key compare */
-            dictSdsDestructor,          /* key destructor */
-            NULL                        /* val destructor */
-        };
-        slaveKeysWithExpire = dictCreate(&dt,NULL);
-    }
-    if (db->id > 63) return;
-
-    dictEntry *de = dictAddOrFind(slaveKeysWithExpire,key->ptr);
-    /* If the entry was just created, set it to a copy of the SDS string
-     * representing the key: we don't want to need to take those keys
-     * in sync with the main DB. The keys will be removed by expireSlaveKeys()
-     * as it scans to find keys to remove. */
-    if (de->key == key->ptr) {
-        de->key = sdsdup(key->ptr);
-        dictSetUnsignedIntegerVal(de,0);
-    }
-
-    uint64_t dbids = dictGetUnsignedIntegerVal(de);
-    dbids |= (uint64_t)1 << db->id;
-    dictSetUnsignedIntegerVal(de,dbids);
-}
-
-/* Return the number of keys we are tracking. */
-size_t getSlaveKeyWithExpireCount(void) {
-    if (slaveKeysWithExpire == NULL) return 0;
-    return dictSize(slaveKeysWithExpire);
-}
-
-/* Remove the keys in the hash table. We need to do that when data is
- * flushed from the server. We may receive new keys from the master with
- * the same name/db and it is no longer a good idea to expire them.
- *
- * Note: technically we should handle the case of a single DB being flushed
- * but it is not worth it since anyway race conditions using the same set
- * of key names in a wriatable slave and in its master will lead to
- * inconsistencies. This is just a best-effort thing we do. */
-void flushSlaveKeysWithExpireList(void) {
-    if (slaveKeysWithExpire) {
-        dictRelease(slaveKeysWithExpire);
-        slaveKeysWithExpire = NULL;
-    }
-}
-
-/*-----------------------------------------------------------------------------
- * Expires Commands
- *----------------------------------------------------------------------------*/
-
-/* This is the generic command implementation for EXPIRE, PEXPIRE, EXPIREAT
- * and PEXPIREAT. Because the commad second argument may be relative or absolute
- * the "basetime" argument is used to signal what the base time is (either 0
- * for *AT variants of the command, or the current time for relative expires).
- *
- * unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for
- * the argv[2] parameter. The basetime is always specified in milliseconds. */
-void expireGenericCommand(client *c, long long basetime, int unit) {
-    robj *key = c->argv[1], *param = c->argv[2];
-    long long when; /* unix time in milliseconds when the key will expire. */
-
-    if (getLongLongFromObjectOrReply(c, param, &when, NULL) != C_OK)
-        return;
-
-    if (unit == UNIT_SECONDS) when *= 1000;
-    when += basetime;
-
-    /* No key, return zero. */
-    if (lookupKeyWrite(c->db,key) == NULL) {
-        addReply(c,shared.czero);
-        return;
-    }
-
-    /* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past
-     * should never be executed as a DEL when load the AOF or in the context
-     * of a slave instance.
-     *
-     * Instead we take the other branch of the IF statement setting an expire
-     * (possibly in the past) and wait for an explicit DEL from the master. */
-    if (when <= mstime() && !server.loading && !server.masterhost) {
-        robj *aux;
-
-        int deleted = server.lazyfree_lazy_expire ? dbAsyncDelete(c->db,key) :
-                                                    dbSyncDelete(c->db,key);
-        serverAssertWithInfo(c,key,deleted);
-        server.dirty++;
-
-        /* Replicate/AOF this as an explicit DEL or UNLINK. */
-        aux = server.lazyfree_lazy_expire ? shared.unlink : shared.del;
-        rewriteClientCommandVector(c,2,aux,key);
-        signalModifiedKey(c->db,key);
-        notifyKeyspaceEvent(NOTIFY_GENERIC,"del",key,c->db->id);
-        addReply(c, shared.cone);
-        return;
-    } else {
-        setExpire(c,c->db,key,when);
-        addReply(c,shared.cone);
-        signalModifiedKey(c->db,key);
-        notifyKeyspaceEvent(NOTIFY_GENERIC,"expire",key,c->db->id);
-        server.dirty++;
-        return;
-    }
-}
-
-/* EXPIRE key seconds */
-void expireCommand(client *c) {
-    expireGenericCommand(c,mstime(),UNIT_SECONDS);
-}
-
-/* EXPIREAT key time */
-void expireatCommand(client *c) {
-    expireGenericCommand(c,0,UNIT_SECONDS);
-}
-
-/* PEXPIRE key milliseconds */
-void pexpireCommand(client *c) {
-    expireGenericCommand(c,mstime(),UNIT_MILLISECONDS);
-}
-
-/* PEXPIREAT key ms_time */
-void pexpireatCommand(client *c) {
-    expireGenericCommand(c,0,UNIT_MILLISECONDS);
-}
-
-/* Implements TTL and PTTL */
-void ttlGenericCommand(client *c, int output_ms) {
-    long long expire, ttl = -1;
-
-    /* If the key does not exist at all, return -2 */
-    if (lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH) == NULL) {
-        addReplyLongLong(c,-2);
-        return;
-    }
-    /* The key exists. Return -1 if it has no expire, or the actual
-     * TTL value otherwise. */
-    expire = getExpire(c->db,c->argv[1]);
-    if (expire != -1) {
-        ttl = expire-mstime();
-        if (ttl < 0) ttl = 0;
-    }
-    if (ttl == -1) {
-        addReplyLongLong(c,-1);
-    } else {
-        addReplyLongLong(c,output_ms ? ttl : ((ttl+500)/1000));
-    }
-}
-
-/* TTL key */
-void ttlCommand(client *c) {
-    ttlGenericCommand(c, 0);
-}
-
-/* PTTL key */
-void pttlCommand(client *c) {
-    ttlGenericCommand(c, 1);
-}
-
-/* PERSIST key */
-void persistCommand(client *c) {
-    if (lookupKeyWrite(c->db,c->argv[1])) {
-        if (removeExpire(c->db,c->argv[1])) {
-            addReply(c,shared.cone);
-            server.dirty++;
-        } else {
-            addReply(c,shared.czero);
-        }
-    } else {
-        addReply(c,shared.czero);
-    }
-}
-
-/* TOUCH key1 [key2 key3 ... keyN] */
-void touchCommand(client *c) {
-    int touched = 0;
-    for (int j = 1; j < c->argc; j++)
-        if (lookupKeyRead(c->db,c->argv[j]) != NULL) touched++;
-    addReplyLongLong(c,touched);
-}
-

src/geo.c

@@ -113,7 +113,7 @@ int extractLongLatOrReply(client *c, robj **argv, double *xy) {
 int longLatFromMember(robj *zobj, robj *member, double *xy) {
     double score = 0;
 
-    if (zsetScore(zobj, member->ptr, &score) == C_ERR) return C_ERR;
+    if (zsetScore(zobj, member, &score) == C_ERR) return C_ERR;
     if (!decodeGeohash(score, xy)) return C_ERR;
     return C_OK;
 }
@@ -161,7 +161,7 @@ double extractDistanceOrReply(client *c, robj **argv,
         addReplyError(c,"radius cannot be negative");
         return -1;
     }
-
+    
     double to_meters = extractUnitOrReply(c,argv[1]);
     if (to_meters < 0) {
         return -1;
@@ -269,14 +269,16 @@ int geoGetPointsInRange(robj *zobj, double min, double max, double lon, double l
         }
 
         while (ln) {
-            sds ele = ln->ele;
+            robj *o = ln->obj;
             /* Abort when the node is no longer in range. */
             if (!zslValueLteMax(ln->score, &range))
                 break;
 
-            ele = sdsdup(ele);
-            if (geoAppendIfWithinRadius(ga,lon,lat,radius,ln->score,ele)
-                == C_ERR) sdsfree(ele);
+            member = (o->encoding == OBJ_ENCODING_INT) ?
+                        sdsfromlonglong((long)o->ptr) :
+                        sdsdup(o->ptr);
+            if (geoAppendIfWithinRadius(ga,lon,lat,radius,ln->score,member)
+                == C_ERR) sdsfree(member);
             ln = ln->level[0].forward;
         }
     }
@@ -326,7 +328,6 @@ int membersOfGeoHashBox(robj *zobj, GeoHashBits hash, geoArray *ga, double lon,
 int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, double radius, geoArray *ga) {
     GeoHashBits neighbors[9];
     unsigned int i, count = 0, last_processed = 0;
-    int debugmsg = 0;
 
     neighbors[0] = n.hash;
     neighbors[1] = n.neighbors.north;
@@ -341,26 +342,8 @@ int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, d
     /* For each neighbor (*and* our own hashbox), get all the matching
      * members and add them to the potential result list. */
     for (i = 0; i < sizeof(neighbors) / sizeof(*neighbors); i++) {
-        if (HASHISZERO(neighbors[i])) {
-            if (debugmsg) D("neighbors[%d] is zero",i);
+        if (HASHISZERO(neighbors[i]))
             continue;
-        }
-
-        /* Debugging info. */
-        if (debugmsg) {
-            GeoHashRange long_range, lat_range;
-            geohashGetCoordRange(&long_range,&lat_range);
-            GeoHashArea myarea = {{0}};
-            geohashDecode(long_range, lat_range, neighbors[i], &myarea);
-
-            /* Dump center square. */
-            D("neighbors[%d]:\n",i);
-            D("area.longitude.min: %f\n", myarea.longitude.min);
-            D("area.longitude.max: %f\n", myarea.longitude.max);
-            D("area.latitude.min: %f\n", myarea.latitude.min);
-            D("area.latitude.max: %f\n", myarea.latitude.max);
-            D("\n");
-        }
 
         /* When a huge Radius (in the 5000 km range or more) is used,
          * adjacent neighbors can be the same, leading to duplicated
@@ -369,11 +352,7 @@ int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, d
         if (last_processed &&
             neighbors[i].bits == neighbors[last_processed].bits &&
             neighbors[i].step == neighbors[last_processed].step)
-        {
-            if (debugmsg)
-                D("Skipping processing of %d, same as previous\n",i);
             continue;
-        }
         count += membersOfGeoHashBox(zobj, neighbors[i], ga, lon, lat, radius);
         last_processed = i;
     }
@@ -452,14 +431,13 @@ void geoaddCommand(client *c) {
 #define SORT_ASC 1
 #define SORT_DESC 2
 
-#define RADIUS_COORDS (1<<0)    /* Search around coordinates. */
-#define RADIUS_MEMBER (1<<1)    /* Search around member. */
-#define RADIUS_NOSTORE (1<<2)   /* Do not acceot STORE/STOREDIST option. */
+#define RADIUS_COORDS 1
+#define RADIUS_MEMBER 2
 
 /* GEORADIUS key x y radius unit [WITHDIST] [WITHHASH] [WITHCOORD] [ASC|DESC]
  *                               [COUNT count] [STORE key] [STOREDIST key]
  * GEORADIUSBYMEMBER key member radius unit ... options ... */
-void georadiusGeneric(client *c, int flags) {
+void georadiusGeneric(client *c, int type) {
     robj *key = c->argv[1];
     robj *storekey = NULL;
     int storedist = 0; /* 0 for STORE, 1 for STOREDIST. */
@@ -474,11 +452,11 @@ void georadiusGeneric(client *c, int flags) {
     /* Find long/lat to use for radius search based on inquiry type */
     int base_args;
     double xy[2] = { 0 };
-    if (flags & RADIUS_COORDS) {
+    if (type == RADIUS_COORDS) {
         base_args = 6;
         if (extractLongLatOrReply(c, c->argv + 2, xy) == C_ERR)
             return;
-    } else if (flags & RADIUS_MEMBER) {
+    } else if (type == RADIUS_MEMBER) {
         base_args = 5;
         robj *member = c->argv[2];
         if (longLatFromMember(zobj, member, xy) == C_ERR) {
@@ -486,7 +464,7 @@ void georadiusGeneric(client *c, int flags) {
             return;
         }
     } else {
-        addReplyError(c, "Unknown georadius search type");
+        addReplyError(c, "unknown georadius search type");
         return;
     }
 
@@ -523,17 +501,11 @@ void georadiusGeneric(client *c, int flags) {
                     return;
                 }
                 i++;
-            } else if (!strcasecmp(arg, "store") &&
-                       (i+1) < remaining &&
-                       !(flags & RADIUS_NOSTORE))
-            {
+            } else if (!strcasecmp(arg, "store") && (i+1) < remaining) {
                 storekey = c->argv[base_args+i+1];
                 storedist = 0;
                 i++;
-            } else if (!strcasecmp(arg, "storedist") &&
-                       (i+1) < remaining &&
-                       !(flags & RADIUS_NOSTORE))
-            {
+            } else if (!strcasecmp(arg, "storedist") && (i+1) < remaining) {
                 storekey = c->argv[base_args+i+1];
                 storedist = 1;
                 i++;
@@ -648,10 +620,13 @@ void georadiusGeneric(client *c, int flags) {
             gp->dist /= conversion; /* Fix according to unit. */
             double score = storedist ? gp->dist : gp->score;
             size_t elelen = sdslen(gp->member);
+            robj *ele = createObject(OBJ_STRING,gp->member);
 
             if (maxelelen < elelen) maxelelen = elelen;
-            znode = zslInsert(zs->zsl,score,gp->member);
-            serverAssert(dictAdd(zs->dict,gp->member,&znode->score) == DICT_OK);
+            incrRefCount(ele); /* Set refcount to 2 since we reference the
+                                  object both in the skiplist and dict. */
+            znode = zslInsert(zs->zsl,score,ele);
+            serverAssert(dictAdd(zs->dict,ele,&znode->score) == DICT_OK);
             gp->member = NULL;
         }
 
@@ -678,20 +653,10 @@ void georadiusCommand(client *c) {
 }
 
 /* GEORADIUSBYMEMBER wrapper function. */
-void georadiusbymemberCommand(client *c) {
+void georadiusByMemberCommand(client *c) {
     georadiusGeneric(c, RADIUS_MEMBER);
 }
 
-/* GEORADIUS_RO wrapper function. */
-void georadiusroCommand(client *c) {
-    georadiusGeneric(c, RADIUS_COORDS|RADIUS_NOSTORE);
-}
-
-/* GEORADIUSBYMEMBER_RO wrapper function. */
-void georadiusbymemberroCommand(client *c) {
-    georadiusGeneric(c, RADIUS_MEMBER|RADIUS_NOSTORE);
-}
-
 /* GEOHASH key ele1 ele2 ... eleN
  *
  * Returns an array with an 11 characters geohash representation of the
@@ -701,15 +666,16 @@ void geohashCommand(client *c) {
     int j;
 
     /* Look up the requested zset */
-    robj *zobj = lookupKeyRead(c->db, c->argv[1]);
-    if (zobj && checkType(c, zobj, OBJ_ZSET)) return;
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptymultibulk))
+        == NULL || checkType(c, zobj, OBJ_ZSET)) return;
 
     /* Geohash elements one after the other, using a null bulk reply for
      * missing elements. */
     addReplyMultiBulkLen(c,c->argc-2);
     for (j = 2; j < c->argc; j++) {
         double score;
-        if (!zobj || zsetScore(zobj, c->argv[j]->ptr, &score) == C_ERR) {
+        if (zsetScore(zobj, c->argv[j], &score) == C_ERR) {
             addReply(c,shared.nullbulk);
         } else {
             /* The internal format we use for geocoding is a bit different
@@ -754,15 +720,16 @@ void geoposCommand(client *c) {
     int j;
 
     /* Look up the requested zset */
-    robj *zobj = lookupKeyRead(c->db, c->argv[1]);
-    if (zobj && checkType(c, zobj, OBJ_ZSET)) return;
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptymultibulk))
+        == NULL || checkType(c, zobj, OBJ_ZSET)) return;
 
     /* Report elements one after the other, using a null bulk reply for
      * missing elements. */
     addReplyMultiBulkLen(c,c->argc-2);
     for (j = 2; j < c->argc; j++) {
         double score;
-        if (!zobj || zsetScore(zobj, c->argv[j]->ptr, &score) == C_ERR) {
+        if (zsetScore(zobj, c->argv[j], &score) == C_ERR) {
             addReply(c,shared.nullmultibulk);
         } else {
             /* Decode... */
@@ -797,13 +764,13 @@ void geodistCommand(client *c) {
 
     /* Look up the requested zset */
     robj *zobj = NULL;
-    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.nullbulk))
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptybulk))
         == NULL || checkType(c, zobj, OBJ_ZSET)) return;
 
     /* Get the scores. We need both otherwise NULL is returned. */
     double score1, score2, xyxy[4];
-    if (zsetScore(zobj, c->argv[2]->ptr, &score1) == C_ERR ||
-        zsetScore(zobj, c->argv[3]->ptr, &score2) == C_ERR)
+    if (zsetScore(zobj, c->argv[2], &score1) == C_ERR ||
+        zsetScore(zobj, c->argv[3], &score2) == C_ERR)
     {
         addReply(c,shared.nullbulk);
         return;

src/hyperloglog.c

@@ -401,11 +401,7 @@ uint64_t MurmurHash64A (const void * key, int len, unsigned int seed) {
         uint64_t k;
 
 #if (BYTE_ORDER == LITTLE_ENDIAN)
-	#ifdef USE_ALIGNED_ACCESS
-	memcpy(&k,data,sizeof(uint64_t));
-	#else
         k = *((uint64_t*)data);
-	#endif
 #else
         k = (uint64_t) data[0];
         k |= (uint64_t) data[1] << 8;
@@ -998,21 +994,32 @@ uint64_t hllCount(struct hllhdr *hdr, int *invalid) {
         serverPanic("Unknown HyperLogLog encoding in hllCount()");
     }
 
-    /* Apply loglog-beta to the raw estimate. See:
-     * "LogLog-Beta and More: A New Algorithm for Cardinality Estimation
-     * Based on LogLog Counting" Jason Qin, Denys Kim, Yumei Tung
-     * arXiv:1612.02284 */
-    double zl = log(ez + 1);
-    double beta = -0.370393911*ez +
-                   0.070471823*zl +
-                   0.17393686*pow(zl,2) +
-                   0.16339839*pow(zl,3) +
-                  -0.09237745*pow(zl,4) +
-                   0.03738027*pow(zl,5) +
-                  -0.005384159*pow(zl,6) +
-                   0.00042419*pow(zl,7);
+    /* Muliply the inverse of E for alpha_m * m^2 to have the raw estimate. */
+    E = (1/E)*alpha*m*m;
 
-    E  = llroundl(alpha*m*(m-ez)*(1/(E+beta)));
+    /* Use the LINEARCOUNTING algorithm for small cardinalities.
+     * For larger values but up to 72000 HyperLogLog raw approximation is
+     * used since linear counting error starts to increase. However HyperLogLog
+     * shows a strong bias in the range 2.5*16384 - 72000, so we try to
+     * compensate for it. */
+    if (E < m*2.5 && ez != 0) {
+        E = m*log(m/ez); /* LINEARCOUNTING() */
+    } else if (m == 16384 && E < 72000) {
+        /* We did polynomial regression of the bias for this range, this
+         * way we can compute the bias for a given cardinality and correct
+         * according to it. Only apply the correction for P=14 that's what
+         * we use and the value the correction was verified with. */
+        double bias = 5.9119*1.0e-18*(E*E*E*E)
+                      -1.4253*1.0e-12*(E*E*E)+
+                      1.2940*1.0e-7*(E*E)
+                      -5.2921*1.0e-3*E+
+                      83.3216;
+        E -= E*(bias/100);
+    }
+    /* We don't apply the correction for E > 1/30 of 2^32 since we use
+     * a 64 bit function and 6 bit counters. To apply the correction for
+     * 1/30 of 2^64 is not needed since it would require a huge set
+     * to approach such a value. */
     return (uint64_t) E;
 }
 
@@ -1121,7 +1128,6 @@ int isHLLObjectOrReply(client *c, robj *o) {
     if (checkType(c,o,OBJ_STRING))
         return C_ERR; /* Error already sent. */
 
-    if (!sdsEncodedObject(o)) goto invalid;
     if (stringObjectLen(o) < sizeof(*hdr)) goto invalid;
     hdr = o->ptr;
 

src/intset.c

@@ -261,7 +261,7 @@ int64_t intsetRandom(intset *is) {
     return _intsetGet(is,rand()%intrev32ifbe(is->length));
 }
 
-/* Get the value at the given position. When this position is
+/* Sets the value to the value at the given position. When this position is
  * out of range the function returns 0, when in range it returns 1. */
 uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value) {
     if (pos < intrev32ifbe(is->length)) {
@@ -272,7 +272,7 @@ uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value) {
 }
 
 /* Return intset length */
-uint32_t intsetLen(const intset *is) {
+uint32_t intsetLen(intset *is) {
     return intrev32ifbe(is->length);
 }
 

src/intset.h

@@ -44,7 +44,7 @@ intset *intsetRemove(intset *is, int64_t value, int *success);
 uint8_t intsetFind(intset *is, int64_t value);
 int64_t intsetRandom(intset *is);
 uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value);
-uint32_t intsetLen(const intset *is);
+uint32_t intsetLen(intset *is);
 size_t intsetBlobLen(intset *is);
 
 #ifdef REDIS_TEST

src/latency.c

@@ -41,7 +41,7 @@ int dictStringKeyCompare(void *privdata, const void *key1, const void *key2) {
     return strcmp(key1,key2) == 0;
 }
 
-uint64_t dictStringHash(const void *key) {
+unsigned int dictStringHash(const void *key) {
     return dictGenHashFunction(key, strlen(key));
 }
 
@@ -79,7 +79,7 @@ int THPIsEnabled(void) {
  * value of the function is non-zero, the process is being targeted by
  * THP support, and is likely to have memory usage / latency issues. */
 int THPGetAnonHugePagesSize(void) {
-    return zmalloc_get_smap_bytes_by_field("AnonHugePages:",-1);
+    return zmalloc_get_smap_bytes_by_field("AnonHugePages:");
 }
 
 /* ---------------------------- Latency API --------------------------------- */

src/lazyfree.c

@@ -1,135 +0,0 @@
-#include "server.h"
-#include "bio.h"
-#include "atomicvar.h"
-#include "cluster.h"
-
-static size_t lazyfree_objects = 0;
-pthread_mutex_t lazyfree_objects_mutex = PTHREAD_MUTEX_INITIALIZER;
-
-/* Return the number of currently pending objects to free. */
-size_t lazyfreeGetPendingObjectsCount(void) {
-    size_t aux;
-    atomicGet(lazyfree_objects,aux);
-    return aux;
-}
-
-/* Return the amount of work needed in order to free an object.
- * The return value is not always the actual number of allocations the
- * object is compoesd of, but a number proportional to it.
- *
- * For strings the function always returns 1.
- *
- * For aggregated objects represented by hash tables or other data structures
- * the function just returns the number of elements the object is composed of.
- *
- * Objects composed of single allocations are always reported as having a
- * single item even if they are actaully logical composed of multiple
- * elements.
- *
- * For lists the funciton returns the number of elements in the quicklist
- * representing the list. */
-size_t lazyfreeGetFreeEffort(robj *obj) {
-    if (obj->type == OBJ_LIST) {
-        quicklist *ql = obj->ptr;
-        return ql->len;
-    } else if (obj->type == OBJ_SET && obj->encoding == OBJ_ENCODING_HT) {
-        dict *ht = obj->ptr;
-        return dictSize(ht);
-    } else if (obj->type == OBJ_ZSET && obj->encoding == OBJ_ENCODING_SKIPLIST){
-        zset *zs = obj->ptr;
-        return zs->zsl->length;
-    } else if (obj->type == OBJ_HASH && obj->encoding == OBJ_ENCODING_HT) {
-        dict *ht = obj->ptr;
-        return dictSize(ht);
-    } else {
-        return 1; /* Everything else is a single allocation. */
-    }
-}
-
-/* Delete a key, value, and associated expiration entry if any, from the DB.
- * If there are enough allocations to free the value object may be put into
- * a lazy free list instead of being freed synchronously. The lazy free list
- * will be reclaimed in a different bio.c thread. */
-#define LAZYFREE_THRESHOLD 64
-int dbAsyncDelete(redisDb *db, robj *key) {
-    /* Deleting an entry from the expires dict will not free the sds of
-     * the key, because it is shared with the main dictionary. */
-    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
-
-    /* If the value is composed of a few allocations, to free in a lazy way
-     * is actually just slower... So under a certain limit we just free
-     * the object synchronously. */
-    dictEntry *de = dictUnlink(db->dict,key->ptr);
-    if (de) {
-        robj *val = dictGetVal(de);
-        size_t free_effort = lazyfreeGetFreeEffort(val);
-
-        /* If releasing the object is too much work, let's put it into the
-         * lazy free list. */
-        if (free_effort > LAZYFREE_THRESHOLD) {
-            atomicIncr(lazyfree_objects,1);
-            bioCreateBackgroundJob(BIO_LAZY_FREE,val,NULL,NULL);
-            dictSetVal(db->dict,de,NULL);
-        }
-    }
-
-    /* Release the key-val pair, or just the key if we set the val
-     * field to NULL in order to lazy free it later. */
-    if (de) {
-        dictFreeUnlinkedEntry(db->dict,de);
-        if (server.cluster_enabled) slotToKeyDel(key);
-        return 1;
-    } else {
-        return 0;
-    }
-}
-
-/* Empty a Redis DB asynchronously. What the function does actually is to
- * create a new empty set of hash tables and scheduling the old ones for
- * lazy freeing. */
-void emptyDbAsync(redisDb *db) {
-    dict *oldht1 = db->dict, *oldht2 = db->expires;
-    db->dict = dictCreate(&dbDictType,NULL);
-    db->expires = dictCreate(&keyptrDictType,NULL);
-    atomicIncr(lazyfree_objects,dictSize(oldht1));
-    bioCreateBackgroundJob(BIO_LAZY_FREE,NULL,oldht1,oldht2);
-}
-
-/* Empty the slots-keys map of Redis CLuster by creating a new empty one
- * and scheduiling the old for lazy freeing. */
-void slotToKeyFlushAsync(void) {
-    rax *old = server.cluster->slots_to_keys;
-
-    server.cluster->slots_to_keys = raxNew();
-    memset(server.cluster->slots_keys_count,0,
-           sizeof(server.cluster->slots_keys_count));
-    atomicIncr(lazyfree_objects,old->numele);
-    bioCreateBackgroundJob(BIO_LAZY_FREE,NULL,NULL,old);
-}
-
-/* Release objects from the lazyfree thread. It's just decrRefCount()
- * updating the count of objects to release. */
-void lazyfreeFreeObjectFromBioThread(robj *o) {
-    decrRefCount(o);
-    atomicDecr(lazyfree_objects,1);
-}
-
-/* Release a database from the lazyfree thread. The 'db' pointer is the
- * database which was substitutied with a fresh one in the main thread
- * when the database was logically deleted. 'sl' is a skiplist used by
- * Redis Cluster in order to take the hash slots -> keys mapping. This
- * may be NULL if Redis Cluster is disabled. */
-void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2) {
-    size_t numkeys = dictSize(ht1);
-    dictRelease(ht1);
-    dictRelease(ht2);
-    atomicDecr(lazyfree_objects,numkeys);
-}
-
-/* Release the skiplist mapping Redis Cluster keys to slots in the
- * lazyfree thread. */
-void lazyfreeFreeSlotsMapFromBioThread(rax *rt) {
-    size_t len = rt->numele;
-    raxFree(rt);
-    atomicDecr(lazyfree_objects,len);
-}

src/modules/.gitignore

@@ -1,2 +0,0 @@
-*.so
-*.xo

src/modules/Makefile

@@ -1,42 +0,0 @@
-
-# find the OS
-uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
-
-# Compile flags for linux / osx
-ifeq ($(uname_S),Linux)
-	SHOBJ_CFLAGS ?= -W -Wall -fno-common -g -ggdb -std=c99 -O2
-	SHOBJ_LDFLAGS ?= -shared
-else
-	SHOBJ_CFLAGS ?= -W -Wall -dynamic -fno-common -g -ggdb -std=c99 -O2
-	SHOBJ_LDFLAGS ?= -bundle -undefined dynamic_lookup
-endif
-
-.SUFFIXES: .c .so .xo .o
-
-all: helloworld.so hellotype.so helloblock.so testmodule.so
-
-.c.xo:
-	$(CC) -I. $(CFLAGS) $(SHOBJ_CFLAGS) -fPIC -c $< -o $@
-
-helloworld.xo: ../redismodule.h
-
-helloworld.so: helloworld.xo
-	$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
-
-hellotype.xo: ../redismodule.h
-
-hellotype.so: hellotype.xo
-	$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
-
-helloblock.xo: ../redismodule.h
-
-helloblock.so: helloblock.xo
-	$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lpthread -lc
-
-testmodule.xo: ../redismodule.h
-
-testmodule.so: testmodule.xo
-	$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
-
-clean:
-	rm -rf *.xo *.so

src/modules/gendoc.rb

@@ -1,51 +0,0 @@
-# gendoc.rb -- Converts the top-comments inside module.c to modules API
-#              reference documentaiton in markdown format.
-
-# Convert the C comment to markdown
-def markdown(s)
-    s = s.gsub(/\*\/$/,"")
-    s = s.gsub(/^ \* {0,1}/,"")
-    s = s.gsub(/^\/\* /,"")
-    s.chop! while s[-1] == "\n" || s[-1] == " "
-    lines = s.split("\n")
-    newlines = []
-    lines.each{|l|
-        if l[0] != ' '
-            l = l.gsub(/RM_[A-z()]+/){|x| "`#{x}`"}
-            l = l.gsub(/RedisModule_[A-z()]+/){|x| "`#{x}`"}
-            l = l.gsub(/REDISMODULE_[A-z]+/){|x| "`#{x}`"}
-        end
-        newlines << l
-    }
-    return newlines.join("\n")
-end
-
-# Given the source code array and the index at which an exported symbol was
-# detected, extracts and outputs the documentation.
-def docufy(src,i)
-    m = /RM_[A-z0-9]+/.match(src[i])
-    name = m[0]
-    name = name.sub("RM_","RedisModule_")
-    proto = src[i].sub("{","").strip+";\n"
-    proto = proto.sub("RM_","RedisModule_")
-    puts "## `#{name}`\n\n"
-    puts "    #{proto}\n"
-    comment = ""
-    while true
-        i = i-1
-        comment = src[i]+comment
-        break if src[i] =~ /\/\*/
-    end
-    comment = markdown(comment)
-    puts comment+"\n\n"
-end
-
-puts "# Modules API reference\n\n"
-src = File.open("../module.c").to_a
-src.each_with_index{|line,i|
-    if line =~ /RM_/ && line[0] != ' ' && line[0] != '#' && line[0] != '/'
-        if src[i-1] =~ /\*\//
-            docufy(src,i)
-        end
-    end
-}

src/modules/helloblock.c

@@ -1,196 +0,0 @@
-/* Helloblock module -- An example of blocking command implementation
- * with threads.
- *
- * -----------------------------------------------------------------------------
- *
- * Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#define REDISMODULE_EXPERIMENTAL_API
-#include "../redismodule.h"
-#include <stdio.h>
-#include <stdlib.h>
-#include <pthread.h>
-#include <unistd.h>
-
-/* Reply callback for blocking command HELLO.BLOCK */
-int HelloBlock_Reply(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-    int *myint = RedisModule_GetBlockedClientPrivateData(ctx);
-    return RedisModule_ReplyWithLongLong(ctx,*myint);
-}
-
-/* Timeout callback for blocking command HELLO.BLOCK */
-int HelloBlock_Timeout(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-    return RedisModule_ReplyWithSimpleString(ctx,"Request timedout");
-}
-
-/* Private data freeing callback for HELLO.BLOCK command. */
-void HelloBlock_FreeData(void *privdata) {
-    RedisModule_Free(privdata);
-}
-
-/* The thread entry point that actually executes the blocking part
- * of the command HELLO.BLOCK. */
-void *HelloBlock_ThreadMain(void *arg) {
-    void **targ = arg;
-    RedisModuleBlockedClient *bc = targ[0];
-    long long delay = (unsigned long)targ[1];
-    RedisModule_Free(targ);
-
-    sleep(delay);
-    int *r = RedisModule_Alloc(sizeof(int));
-    *r = rand();
-    RedisModule_UnblockClient(bc,r);
-    return NULL;
-}
-
-/* HELLO.BLOCK <delay> <timeout> -- Block for <count> seconds, then reply with
- * a random number. Timeout is the command timeout, so that you can test
- * what happens when the delay is greater than the timeout. */
-int HelloBlock_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-    long long delay;
-    long long timeout;
-
-    if (RedisModule_StringToLongLong(argv[1],&delay) != REDISMODULE_OK) {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid count");
-    }
-
-    if (RedisModule_StringToLongLong(argv[2],&timeout) != REDISMODULE_OK) {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid count");
-    }
-
-    pthread_t tid;
-    RedisModuleBlockedClient *bc = RedisModule_BlockClient(ctx,HelloBlock_Reply,HelloBlock_Timeout,HelloBlock_FreeData,timeout);
-
-    /* Now that we setup a blocking client, we need to pass the control
-     * to the thread. However we need to pass arguments to the thread:
-     * the delay and a reference to the blocked client handle. */
-    void **targ = RedisModule_Alloc(sizeof(void*)*2);
-    targ[0] = bc;
-    targ[1] = (void*)(unsigned long) delay;
-
-    if (pthread_create(&tid,NULL,HelloBlock_ThreadMain,targ) != 0) {
-        RedisModule_AbortBlock(bc);
-        return RedisModule_ReplyWithError(ctx,"-ERR Can't start thread");
-    }
-    return REDISMODULE_OK;
-}
-
-/* The thread entry point that actually executes the blocking part
- * of the command HELLO.KEYS.
- *
- * Note: this implementation is very simple on purpose, so no duplicated
- * keys (returned by SCAN) are filtered. However adding such a functionality
- * would be trivial just using any data structure implementing a dictionary
- * in order to filter the duplicated items. */
-void *HelloKeys_ThreadMain(void *arg) {
-    RedisModuleBlockedClient *bc = arg;
-    RedisModuleCtx *ctx = RedisModule_GetThreadSafeContext(bc);
-    long long cursor = 0;
-    size_t replylen = 0;
-
-    RedisModule_ReplyWithArray(ctx,REDISMODULE_POSTPONED_ARRAY_LEN);
-    do {
-        RedisModule_ThreadSafeContextLock(ctx);
-        RedisModuleCallReply *reply = RedisModule_Call(ctx,
-            "SCAN","l",(long long)cursor);
-        RedisModule_ThreadSafeContextUnlock(ctx);
-
-        RedisModuleCallReply *cr_cursor =
-            RedisModule_CallReplyArrayElement(reply,0);
-        RedisModuleCallReply *cr_keys =
-            RedisModule_CallReplyArrayElement(reply,1);
-
-        RedisModuleString *s = RedisModule_CreateStringFromCallReply(cr_cursor);
-        RedisModule_StringToLongLong(s,&cursor);
-        RedisModule_FreeString(ctx,s);
-
-        size_t items = RedisModule_CallReplyLength(cr_keys);
-        for (size_t j = 0; j < items; j++) {
-            RedisModuleCallReply *ele =
-                RedisModule_CallReplyArrayElement(cr_keys,j);
-            RedisModule_ReplyWithCallReply(ctx,ele);
-            replylen++;
-        }
-        RedisModule_FreeCallReply(reply);
-    } while (cursor != 0);
-    RedisModule_ReplySetArrayLength(ctx,replylen);
-
-    RedisModule_FreeThreadSafeContext(ctx);
-    RedisModule_UnblockClient(bc,NULL);
-    return NULL;
-}
-
-/* HELLO.KEYS -- Return all the keys in the current database without blocking
- * the server. The keys do not represent a point-in-time state so only the keys
- * that were in the database from the start to the end are guaranteed to be
- * there. */
-int HelloKeys_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    if (argc != 1) return RedisModule_WrongArity(ctx);
-
-    pthread_t tid;
-
-    /* Note that when blocking the client we do not set any callback: no
-     * timeout is possible since we passed '0', nor we need a reply callback
-     * because we'll use the thread safe context to accumulate a reply. */
-    RedisModuleBlockedClient *bc = RedisModule_BlockClient(ctx,NULL,NULL,NULL,0);
-
-    /* Now that we setup a blocking client, we need to pass the control
-     * to the thread. However we need to pass arguments to the thread:
-     * the reference to the blocked client handle. */
-    if (pthread_create(&tid,NULL,HelloKeys_ThreadMain,bc) != 0) {
-        RedisModule_AbortBlock(bc);
-        return RedisModule_ReplyWithError(ctx,"-ERR Can't start thread");
-    }
-    return REDISMODULE_OK;
-}
-
-/* This function must be present on each Redis module. It is used in order to
- * register the commands into the Redis server. */
-int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    if (RedisModule_Init(ctx,"helloblock",1,REDISMODULE_APIVER_1)
-        == REDISMODULE_ERR) return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.block",
-        HelloBlock_RedisCommand,"",0,0,0) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-    if (RedisModule_CreateCommand(ctx,"hello.keys",
-        HelloKeys_RedisCommand,"",0,0,0) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    return REDISMODULE_OK;
-}

src/modules/hellotype.c

@@ -1,286 +0,0 @@
-/* This file implements a new module native data type called "HELLOTYPE".
- * The data structure implemented is a very simple ordered linked list of
- * 64 bit integers, in order to have something that is real world enough, but
- * at the same time, extremely simple to understand, to show how the API
- * works, how a new data type is created, and how to write basic methods
- * for RDB loading, saving and AOF rewriting.
- *
- * -----------------------------------------------------------------------------
- *
- * Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "../redismodule.h"
-#include <stdio.h>
-#include <stdlib.h>
-#include <ctype.h>
-#include <string.h>
-#include <stdint.h>
-
-static RedisModuleType *HelloType;
-
-/* ========================== Internal data structure  =======================
- * This is just a linked list of 64 bit integers where elements are inserted
- * in-place, so it's ordered. There is no pop/push operation but just insert
- * because it is enough to show the implementation of new data types without
- * making things complex. */
-
-struct HelloTypeNode {
-    int64_t value;
-    struct HelloTypeNode *next;
-};
-
-struct HelloTypeObject {
-    struct HelloTypeNode *head;
-    size_t len; /* Number of elements added. */
-};
-
-struct HelloTypeObject *createHelloTypeObject(void) {
-    struct HelloTypeObject *o;
-    o = RedisModule_Alloc(sizeof(*o));
-    o->head = NULL;
-    o->len = 0;
-    return o;
-}
-
-void HelloTypeInsert(struct HelloTypeObject *o, int64_t ele) {
-    struct HelloTypeNode *next = o->head, *newnode, *prev = NULL;
-
-    while(next && next->value < ele) {
-        prev = next;
-        next = next->next;
-    }
-    newnode = RedisModule_Alloc(sizeof(*newnode));
-    newnode->value = ele;
-    newnode->next = next;
-    if (prev) {
-        prev->next = newnode;
-    } else {
-        o->head = newnode;
-    }
-    o->len++;
-}
-
-void HelloTypeReleaseObject(struct HelloTypeObject *o) {
-    struct HelloTypeNode *cur, *next;
-    cur = o->head;
-    while(cur) {
-        next = cur->next;
-        RedisModule_Free(cur);
-        cur = next;
-    }
-    RedisModule_Free(o);
-}
-
-/* ========================= "hellotype" type commands ======================= */
-
-/* HELLOTYPE.INSERT key value */
-int HelloTypeInsert_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    int type = RedisModule_KeyType(key);
-    if (type != REDISMODULE_KEYTYPE_EMPTY &&
-        RedisModule_ModuleTypeGetType(key) != HelloType)
-    {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    long long value;
-    if ((RedisModule_StringToLongLong(argv[2],&value) != REDISMODULE_OK)) {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid value: must be a signed 64 bit integer");
-    }
-
-    /* Create an empty value object if the key is currently empty. */
-    struct HelloTypeObject *hto;
-    if (type == REDISMODULE_KEYTYPE_EMPTY) {
-        hto = createHelloTypeObject();
-        RedisModule_ModuleTypeSetValue(key,HelloType,hto);
-    } else {
-        hto = RedisModule_ModuleTypeGetValue(key);
-    }
-
-    /* Insert the new element. */
-    HelloTypeInsert(hto,value);
-
-    RedisModule_ReplyWithLongLong(ctx,hto->len);
-    RedisModule_ReplicateVerbatim(ctx);
-    return REDISMODULE_OK;
-}
-
-/* HELLOTYPE.RANGE key first count */
-int HelloTypeRange_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    int type = RedisModule_KeyType(key);
-    if (type != REDISMODULE_KEYTYPE_EMPTY &&
-        RedisModule_ModuleTypeGetType(key) != HelloType)
-    {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    long long first, count;
-    if (RedisModule_StringToLongLong(argv[2],&first) != REDISMODULE_OK ||
-        RedisModule_StringToLongLong(argv[3],&count) != REDISMODULE_OK ||
-        first < 0 || count < 0)
-    {
-        return RedisModule_ReplyWithError(ctx,
-            "ERR invalid first or count parameters");
-    }
-
-    struct HelloTypeObject *hto = RedisModule_ModuleTypeGetValue(key);
-    struct HelloTypeNode *node = hto ? hto->head : NULL;
-    RedisModule_ReplyWithArray(ctx,REDISMODULE_POSTPONED_ARRAY_LEN);
-    long long arraylen = 0;
-    while(node && count--) {
-        RedisModule_ReplyWithLongLong(ctx,node->value);
-        arraylen++;
-        node = node->next;
-    }
-    RedisModule_ReplySetArrayLength(ctx,arraylen);
-    return REDISMODULE_OK;
-}
-
-/* HELLOTYPE.LEN key */
-int HelloTypeLen_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-
-    if (argc != 2) return RedisModule_WrongArity(ctx);
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    int type = RedisModule_KeyType(key);
-    if (type != REDISMODULE_KEYTYPE_EMPTY &&
-        RedisModule_ModuleTypeGetType(key) != HelloType)
-    {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    struct HelloTypeObject *hto = RedisModule_ModuleTypeGetValue(key);
-    RedisModule_ReplyWithLongLong(ctx,hto ? hto->len : 0);
-    return REDISMODULE_OK;
-}
-
-
-/* ========================== "hellotype" type methods ======================= */
-
-void *HelloTypeRdbLoad(RedisModuleIO *rdb, int encver) {
-    if (encver != 0) {
-        /* RedisModule_Log("warning","Can't load data with version %d", encver);*/
-        return NULL;
-    }
-    uint64_t elements = RedisModule_LoadUnsigned(rdb);
-    struct HelloTypeObject *hto = createHelloTypeObject();
-    while(elements--) {
-        int64_t ele = RedisModule_LoadSigned(rdb);
-        HelloTypeInsert(hto,ele);
-    }
-    return hto;
-}
-
-void HelloTypeRdbSave(RedisModuleIO *rdb, void *value) {
-    struct HelloTypeObject *hto = value;
-    struct HelloTypeNode *node = hto->head;
-    RedisModule_SaveUnsigned(rdb,hto->len);
-    while(node) {
-        RedisModule_SaveSigned(rdb,node->value);
-        node = node->next;
-    }
-}
-
-void HelloTypeAofRewrite(RedisModuleIO *aof, RedisModuleString *key, void *value) {
-    struct HelloTypeObject *hto = value;
-    struct HelloTypeNode *node = hto->head;
-    while(node) {
-        RedisModule_EmitAOF(aof,"HELLOTYPE.INSERT","sl",key,node->value);
-        node = node->next;
-    }
-}
-
-/* The goal of this function is to return the amount of memory used by
- * the HelloType value. */
-size_t HelloTypeMemUsage(const void *value) {
-    const struct HelloTypeObject *hto = value;
-    struct HelloTypeNode *node = hto->head;
-    return sizeof(*hto) + sizeof(*node)*hto->len;
-}
-
-void HelloTypeFree(void *value) {
-    HelloTypeReleaseObject(value);
-}
-
-void HelloTypeDigest(RedisModuleDigest *md, void *value) {
-    struct HelloTypeObject *hto = value;
-    struct HelloTypeNode *node = hto->head;
-    while(node) {
-        RedisModule_DigestAddLongLong(md,node->value);
-        node = node->next;
-    }
-    RedisModule_DigestEndSequence(md);
-}
-
-/* This function must be present on each Redis module. It is used in order to
- * register the commands into the Redis server. */
-int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    if (RedisModule_Init(ctx,"hellotype",1,REDISMODULE_APIVER_1)
-        == REDISMODULE_ERR) return REDISMODULE_ERR;
-
-    RedisModuleTypeMethods tm = {
-        .version = REDISMODULE_TYPE_METHOD_VERSION,
-        .rdb_load = HelloTypeRdbLoad,
-        .rdb_save = HelloTypeRdbSave,
-        .aof_rewrite = HelloTypeAofRewrite,
-        .mem_usage = HelloTypeMemUsage,
-        .free = HelloTypeFree,
-        .digest = HelloTypeDigest
-    };
-
-    HelloType = RedisModule_CreateDataType(ctx,"hellotype",0,&tm);
-    if (HelloType == NULL) return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hellotype.insert",
-        HelloTypeInsert_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hellotype.range",
-        HelloTypeRange_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hellotype.len",
-        HelloTypeLen_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    return REDISMODULE_OK;
-}

src/modules/helloworld.c

@@ -1,621 +0,0 @@
-/* Helloworld module -- A few examples of the Redis Modules API in the form
- * of commands showing how to accomplish common tasks.
- *
- * This module does not do anything useful, if not for a few commands. The
- * examples are designed in order to show the API.
- *
- * -----------------------------------------------------------------------------
- *
- * Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "../redismodule.h"
-#include <stdio.h>
-#include <stdlib.h>
-#include <ctype.h>
-#include <string.h>
-
-/* HELLO.SIMPLE is among the simplest commands you can implement.
- * It just returns the currently selected DB id, a functionality which is
- * missing in Redis. The command uses two important API calls: one to
- * fetch the currently selected DB, the other in order to send the client
- * an integer reply as response. */
-int HelloSimple_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-    RedisModule_ReplyWithLongLong(ctx,RedisModule_GetSelectedDb(ctx));
-    return REDISMODULE_OK;
-}
-
-/* HELLO.PUSH.NATIVE re-implements RPUSH, and shows the low level modules API
- * where you can "open" keys, make low level operations, create new keys by
- * pushing elements into non-existing keys, and so forth.
- *
- * You'll find this command to be roughly as fast as the actual RPUSH
- * command. */
-int HelloPushNative_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
-{
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-
-    RedisModule_ListPush(key,REDISMODULE_LIST_TAIL,argv[2]);
-    size_t newlen = RedisModule_ValueLength(key);
-    RedisModule_CloseKey(key);
-    RedisModule_ReplyWithLongLong(ctx,newlen);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.PUSH.CALL implements RPUSH using an higher level approach, calling
- * a Redis command instead of working with the key in a low level way. This
- * approach is useful when you need to call Redis commands that are not
- * available as low level APIs, or when you don't need the maximum speed
- * possible but instead prefer implementation simplicity. */
-int HelloPushCall_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
-{
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-
-    RedisModuleCallReply *reply;
-
-    reply = RedisModule_Call(ctx,"RPUSH","ss",argv[1],argv[2]);
-    long long len = RedisModule_CallReplyInteger(reply);
-    RedisModule_FreeCallReply(reply);
-    RedisModule_ReplyWithLongLong(ctx,len);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.PUSH.CALL2
- * This is exaxctly as HELLO.PUSH.CALL, but shows how we can reply to the
- * client using directly a reply object that Call() returned. */
-int HelloPushCall2_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
-{
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-
-    RedisModuleCallReply *reply;
-
-    reply = RedisModule_Call(ctx,"RPUSH","ss",argv[1],argv[2]);
-    RedisModule_ReplyWithCallReply(ctx,reply);
-    RedisModule_FreeCallReply(reply);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.LIST.SUM.LEN returns the total length of all the items inside
- * a Redis list, by using the high level Call() API.
- * This command is an example of the array reply access. */
-int HelloListSumLen_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
-{
-    if (argc != 2) return RedisModule_WrongArity(ctx);
-
-    RedisModuleCallReply *reply;
-
-    reply = RedisModule_Call(ctx,"LRANGE","sll",argv[1],(long long)0,(long long)-1);
-    size_t strlen = 0;
-    size_t items = RedisModule_CallReplyLength(reply);
-    size_t j;
-    for (j = 0; j < items; j++) {
-        RedisModuleCallReply *ele = RedisModule_CallReplyArrayElement(reply,j);
-        strlen += RedisModule_CallReplyLength(ele);
-    }
-    RedisModule_FreeCallReply(reply);
-    RedisModule_ReplyWithLongLong(ctx,strlen);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.LIST.SPLICE srclist dstlist count
- * Moves 'count' elements from the tail of 'srclist' to the head of
- * 'dstlist'. If less than count elements are available, it moves as much
- * elements as possible. */
-int HelloListSplice_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-
-    RedisModuleKey *srckey = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    RedisModuleKey *dstkey = RedisModule_OpenKey(ctx,argv[2],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-
-    /* Src and dst key must be empty or lists. */
-    if ((RedisModule_KeyType(srckey) != REDISMODULE_KEYTYPE_LIST &&
-         RedisModule_KeyType(srckey) != REDISMODULE_KEYTYPE_EMPTY) ||
-        (RedisModule_KeyType(dstkey) != REDISMODULE_KEYTYPE_LIST &&
-         RedisModule_KeyType(dstkey) != REDISMODULE_KEYTYPE_EMPTY))
-    {
-        RedisModule_CloseKey(srckey);
-        RedisModule_CloseKey(dstkey);
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    long long count;
-    if ((RedisModule_StringToLongLong(argv[3],&count) != REDISMODULE_OK) ||
-        (count < 0)) {
-        RedisModule_CloseKey(srckey);
-        RedisModule_CloseKey(dstkey);
-        return RedisModule_ReplyWithError(ctx,"ERR invalid count");
-    }
-
-    while(count-- > 0) {
-        RedisModuleString *ele;
-
-        ele = RedisModule_ListPop(srckey,REDISMODULE_LIST_TAIL);
-        if (ele == NULL) break;
-        RedisModule_ListPush(dstkey,REDISMODULE_LIST_HEAD,ele);
-        RedisModule_FreeString(ctx,ele);
-    }
-
-    size_t len = RedisModule_ValueLength(srckey);
-    RedisModule_CloseKey(srckey);
-    RedisModule_CloseKey(dstkey);
-    RedisModule_ReplyWithLongLong(ctx,len);
-    return REDISMODULE_OK;
-}
-
-/* Like the HELLO.LIST.SPLICE above, but uses automatic memory management
- * in order to avoid freeing stuff. */
-int HelloListSpliceAuto_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-
-    RedisModule_AutoMemory(ctx);
-
-    RedisModuleKey *srckey = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    RedisModuleKey *dstkey = RedisModule_OpenKey(ctx,argv[2],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-
-    /* Src and dst key must be empty or lists. */
-    if ((RedisModule_KeyType(srckey) != REDISMODULE_KEYTYPE_LIST &&
-         RedisModule_KeyType(srckey) != REDISMODULE_KEYTYPE_EMPTY) ||
-        (RedisModule_KeyType(dstkey) != REDISMODULE_KEYTYPE_LIST &&
-         RedisModule_KeyType(dstkey) != REDISMODULE_KEYTYPE_EMPTY))
-    {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    long long count;
-    if ((RedisModule_StringToLongLong(argv[3],&count) != REDISMODULE_OK) ||
-        (count < 0))
-    {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid count");
-    }
-
-    while(count-- > 0) {
-        RedisModuleString *ele;
-
-        ele = RedisModule_ListPop(srckey,REDISMODULE_LIST_TAIL);
-        if (ele == NULL) break;
-        RedisModule_ListPush(dstkey,REDISMODULE_LIST_HEAD,ele);
-    }
-
-    size_t len = RedisModule_ValueLength(srckey);
-    RedisModule_ReplyWithLongLong(ctx,len);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.RAND.ARRAY <count>
- * Shows how to generate arrays as commands replies.
- * It just outputs <count> random numbers. */
-int HelloRandArray_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 2) return RedisModule_WrongArity(ctx);
-    long long count;
-    if (RedisModule_StringToLongLong(argv[1],&count) != REDISMODULE_OK ||
-        count < 0)
-        return RedisModule_ReplyWithError(ctx,"ERR invalid count");
-
-    /* To reply with an array, we call RedisModule_ReplyWithArray() followed
-     * by other "count" calls to other reply functions in order to generate
-     * the elements of the array. */
-    RedisModule_ReplyWithArray(ctx,count);
-    while(count--) RedisModule_ReplyWithLongLong(ctx,rand());
-    return REDISMODULE_OK;
-}
-
-/* This is a simple command to test replication. Because of the "!" modified
- * in the RedisModule_Call() call, the two INCRs get replicated.
- * Also note how the ECHO is replicated in an unexpected position (check
- * comments the function implementation). */
-int HelloRepl1_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
-{
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-    RedisModule_AutoMemory(ctx);
-
-    /* This will be replicated *after* the two INCR statements, since
-     * the Call() replication has precedence, so the actual replication
-     * stream will be:
-     *
-     * MULTI
-     * INCR foo
-     * INCR bar
-     * ECHO c foo
-     * EXEC
-     */
-    RedisModule_Replicate(ctx,"ECHO","c","foo");
-
-    /* Using the "!" modifier we replicate the command if it
-     * modified the dataset in some way. */
-    RedisModule_Call(ctx,"INCR","c!","foo");
-    RedisModule_Call(ctx,"INCR","c!","bar");
-
-    RedisModule_ReplyWithLongLong(ctx,0);
-
-    return REDISMODULE_OK;
-}
-
-/* Another command to show replication. In this case, we call
- * RedisModule_ReplicateVerbatim() to mean we want just the command to be
- * propagated to slaves / AOF exactly as it was called by the user.
- *
- * This command also shows how to work with string objects.
- * It takes a list, and increments all the elements (that must have
- * a numerical value) by 1, returning the sum of all the elements
- * as reply.
- *
- * Usage: HELLO.REPL2 <list-key> */
-int HelloRepl2_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 2) return RedisModule_WrongArity(ctx);
-
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-
-    if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_LIST)
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-
-    size_t listlen = RedisModule_ValueLength(key);
-    long long sum = 0;
-
-    /* Rotate and increment. */
-    while(listlen--) {
-        RedisModuleString *ele = RedisModule_ListPop(key,REDISMODULE_LIST_TAIL);
-        long long val;
-        if (RedisModule_StringToLongLong(ele,&val) != REDISMODULE_OK) val = 0;
-        val++;
-        sum += val;
-        RedisModuleString *newele = RedisModule_CreateStringFromLongLong(ctx,val);
-        RedisModule_ListPush(key,REDISMODULE_LIST_HEAD,newele);
-    }
-    RedisModule_ReplyWithLongLong(ctx,sum);
-    RedisModule_ReplicateVerbatim(ctx);
-    return REDISMODULE_OK;
-}
-
-/* This is an example of strings DMA access. Given a key containing a string
- * it toggles the case of each character from lower to upper case or the
- * other way around.
- *
- * No automatic memory management is used in this example (for the sake
- * of variety).
- *
- * HELLO.TOGGLE.CASE key */
-int HelloToggleCase_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (argc != 2) return RedisModule_WrongArity(ctx);
-
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-
-    int keytype = RedisModule_KeyType(key);
-    if (keytype != REDISMODULE_KEYTYPE_STRING &&
-        keytype != REDISMODULE_KEYTYPE_EMPTY)
-    {
-        RedisModule_CloseKey(key);
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    if (keytype == REDISMODULE_KEYTYPE_STRING) {
-        size_t len, j;
-        char *s = RedisModule_StringDMA(key,&len,REDISMODULE_WRITE);
-        for (j = 0; j < len; j++) {
-            if (isupper(s[j])) {
-                s[j] = tolower(s[j]);
-            } else {
-                s[j] = toupper(s[j]);
-            }
-        }
-    }
-
-    RedisModule_CloseKey(key);
-    RedisModule_ReplyWithSimpleString(ctx,"OK");
-    RedisModule_ReplicateVerbatim(ctx);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.MORE.EXPIRE key milliseconds.
- *
- * If they key has already an associated TTL, extends it by "milliseconds"
- * milliseconds. Otherwise no operation is performed. */
-int HelloMoreExpire_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-    if (argc != 3) return RedisModule_WrongArity(ctx);
-
-    mstime_t addms, expire;
-
-    if (RedisModule_StringToLongLong(argv[2],&addms) != REDISMODULE_OK)
-        return RedisModule_ReplyWithError(ctx,"ERR invalid expire time");
-
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    expire = RedisModule_GetExpire(key);
-    if (expire != REDISMODULE_NO_EXPIRE) {
-        expire += addms;
-        RedisModule_SetExpire(key,expire);
-    }
-    return RedisModule_ReplyWithSimpleString(ctx,"OK");
-}
-
-/* HELLO.ZSUMRANGE key startscore endscore
- * Return the sum of all the scores elements between startscore and endscore.
- *
- * The computation is performed two times, one time from start to end and
- * another time backward. The two scores, returned as a two element array,
- * should match.*/
-int HelloZsumRange_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    double score_start, score_end;
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-
-    if (RedisModule_StringToDouble(argv[2],&score_start) != REDISMODULE_OK ||
-        RedisModule_StringToDouble(argv[3],&score_end) != REDISMODULE_OK)
-    {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid range");
-    }
-
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_ZSET) {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    double scoresum_a = 0;
-    double scoresum_b = 0;
-
-    RedisModule_ZsetFirstInScoreRange(key,score_start,score_end,0,0);
-    while(!RedisModule_ZsetRangeEndReached(key)) {
-        double score;
-        RedisModuleString *ele = RedisModule_ZsetRangeCurrentElement(key,&score);
-        RedisModule_FreeString(ctx,ele);
-        scoresum_a += score;
-        RedisModule_ZsetRangeNext(key);
-    }
-    RedisModule_ZsetRangeStop(key);
-
-    RedisModule_ZsetLastInScoreRange(key,score_start,score_end,0,0);
-    while(!RedisModule_ZsetRangeEndReached(key)) {
-        double score;
-        RedisModuleString *ele = RedisModule_ZsetRangeCurrentElement(key,&score);
-        RedisModule_FreeString(ctx,ele);
-        scoresum_b += score;
-        RedisModule_ZsetRangePrev(key);
-    }
-
-    RedisModule_ZsetRangeStop(key);
-
-    RedisModule_CloseKey(key);
-
-    RedisModule_ReplyWithArray(ctx,2);
-    RedisModule_ReplyWithDouble(ctx,scoresum_a);
-    RedisModule_ReplyWithDouble(ctx,scoresum_b);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.LEXRANGE key min_lex max_lex min_age max_age
- * This command expects a sorted set stored at key in the following form:
- * - All the elements have score 0.
- * - Elements are pairs of "<name>:<age>", for example "Anna:52".
- * The command will return all the sorted set items that are lexicographically
- * between the specified range (using the same format as ZRANGEBYLEX)
- * and having an age between min_age and max_age. */
-int HelloLexRange_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-
-    if (argc != 6) return RedisModule_WrongArity(ctx);
-
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_ZSET) {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    if (RedisModule_ZsetFirstInLexRange(key,argv[2],argv[3]) != REDISMODULE_OK) {
-        return RedisModule_ReplyWithError(ctx,"invalid range");
-    }
-
-    int arraylen = 0;
-    RedisModule_ReplyWithArray(ctx,REDISMODULE_POSTPONED_ARRAY_LEN);
-    while(!RedisModule_ZsetRangeEndReached(key)) {
-        double score;
-        RedisModuleString *ele = RedisModule_ZsetRangeCurrentElement(key,&score);
-        RedisModule_ReplyWithString(ctx,ele);
-        RedisModule_FreeString(ctx,ele);
-        RedisModule_ZsetRangeNext(key);
-        arraylen++;
-    }
-    RedisModule_ZsetRangeStop(key);
-    RedisModule_ReplySetArrayLength(ctx,arraylen);
-    RedisModule_CloseKey(key);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.HCOPY key srcfield dstfield
- * This is just an example command that sets the hash field dstfield to the
- * same value of srcfield. If srcfield does not exist no operation is
- * performed.
- *
- * The command returns 1 if the copy is performed (srcfield exists) otherwise
- * 0 is returned. */
-int HelloHCopy_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
-        REDISMODULE_READ|REDISMODULE_WRITE);
-    int type = RedisModule_KeyType(key);
-    if (type != REDISMODULE_KEYTYPE_HASH &&
-        type != REDISMODULE_KEYTYPE_EMPTY)
-    {
-        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
-    }
-
-    /* Get the old field value. */
-    RedisModuleString *oldval;
-    RedisModule_HashGet(key,REDISMODULE_HASH_NONE,argv[2],&oldval,NULL);
-    if (oldval) {
-        RedisModule_HashSet(key,REDISMODULE_HASH_NONE,argv[3],oldval,NULL);
-    }
-    RedisModule_ReplyWithLongLong(ctx,oldval != NULL);
-    return REDISMODULE_OK;
-}
-
-/* HELLO.LEFTPAD str len ch
- * This is an implementation of the infamous LEFTPAD function, that
- * was at the center of an issue with the npm modules system in March 2016.
- *
- * LEFTPAD is a good example of using a Redis Modules API called
- * "pool allocator", that was a famous way to allocate memory in yet another
- * open source project, the Apache web server.
- *
- * The concept is very simple: there is memory that is useful to allocate
- * only in the context of serving a request, and must be freed anyway when
- * the callback implementing the command returns. So in that case the module
- * does not need to retain a reference to these allocations, it is just
- * required to free the memory before returning. When this is the case the
- * module can call RedisModule_PoolAlloc() instead, that works like malloc()
- * but will automatically free the memory when the module callback returns.
- *
- * Note that PoolAlloc() does not necessarily require AutoMemory to be
- * active. */
-int HelloLeftPad_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
-    long long padlen;
-
-    if (argc != 4) return RedisModule_WrongArity(ctx);
-
-    if ((RedisModule_StringToLongLong(argv[2],&padlen) != REDISMODULE_OK) ||
-        (padlen< 0)) {
-        return RedisModule_ReplyWithError(ctx,"ERR invalid padding length");
-    }
-    size_t strlen, chlen;
-    const char *str = RedisModule_StringPtrLen(argv[1], &strlen);
-    const char *ch = RedisModule_StringPtrLen(argv[3], &chlen);
-
-    /* If the string is already larger than the target len, just return
-     * the string itself. */
-    if (strlen >= (size_t)padlen)
-        return RedisModule_ReplyWithString(ctx,argv[1]);
-
-    /* Padding must be a single character in this simple implementation. */
-    if (chlen != 1)
-        return RedisModule_ReplyWithError(ctx,
-            "ERR padding must be a single char");
-
-    /* Here we use our pool allocator, for our throw-away allocation. */
-    padlen -= strlen;
-    char *buf = RedisModule_PoolAlloc(ctx,padlen+strlen);
-    for (long long j = 0; j < padlen; j++) buf[j] = *ch;
-    memcpy(buf+padlen,str,strlen);
-
-    RedisModule_ReplyWithStringBuffer(ctx,buf,padlen+strlen);
-    return REDISMODULE_OK;
-}
-
-/* This function must be present on each Redis module. It is used in order to
- * register the commands into the Redis server. */
-int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    if (RedisModule_Init(ctx,"helloworld",1,REDISMODULE_APIVER_1)
-        == REDISMODULE_ERR) return REDISMODULE_ERR;
-
-    /* Log the list of parameters passing loading the module. */
-    for (int j = 0; j < argc; j++) {
-        const char *s = RedisModule_StringPtrLen(argv[j],NULL);
-        printf("Module loaded with ARGV[%d] = %s\n", j, s);
-    }
-
-    if (RedisModule_CreateCommand(ctx,"hello.simple",
-        HelloSimple_RedisCommand,"readonly",0,0,0) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.push.native",
-        HelloPushNative_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.push.call",
-        HelloPushCall_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.push.call2",
-        HelloPushCall2_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.list.sum.len",
-        HelloListSumLen_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.list.splice",
-        HelloListSplice_RedisCommand,"write deny-oom",1,2,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.list.splice.auto",
-        HelloListSpliceAuto_RedisCommand,
-        "write deny-oom",1,2,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.rand.array",
-        HelloRandArray_RedisCommand,"readonly",0,0,0) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.repl1",
-        HelloRepl1_RedisCommand,"write",0,0,0) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.repl2",
-        HelloRepl2_RedisCommand,"write",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.toggle.case",
-        HelloToggleCase_RedisCommand,"write",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.more.expire",
-        HelloMoreExpire_RedisCommand,"write",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.zsumrange",
-        HelloZsumRange_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.lexrange",
-        HelloLexRange_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.hcopy",
-        HelloHCopy_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"hello.leftpad",
-        HelloLeftPad_RedisCommand,"",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    return REDISMODULE_OK;
-}

src/modules/testmodule.c

@@ -1,319 +0,0 @@
-/* Module designed to test the Redis modules subsystem.
- *
- * -----------------------------------------------------------------------------
- *
- * Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "../redismodule.h"
-#include <string.h>
-
-/* --------------------------------- Helpers -------------------------------- */
-
-/* Return true if the reply and the C null term string matches. */
-int TestMatchReply(RedisModuleCallReply *reply, char *str) {
-    RedisModuleString *mystr;
-    mystr = RedisModule_CreateStringFromCallReply(reply);
-    if (!mystr) return 0;
-    const char *ptr = RedisModule_StringPtrLen(mystr,NULL);
-    return strcmp(ptr,str) == 0;
-}
-
-/* ------------------------------- Test units ------------------------------- */
-
-/* TEST.CALL -- Test Call() API. */
-int TestCall(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    RedisModule_AutoMemory(ctx);
-    RedisModuleCallReply *reply;
-
-    RedisModule_Call(ctx,"DEL","c","mylist");
-    RedisModuleString *mystr = RedisModule_CreateString(ctx,"foo",3);
-    RedisModule_Call(ctx,"RPUSH","csl","mylist",mystr,(long long)1234);
-    reply = RedisModule_Call(ctx,"LRANGE","ccc","mylist","0","-1");
-    long long items = RedisModule_CallReplyLength(reply);
-    if (items != 2) goto fail;
-
-    RedisModuleCallReply *item0, *item1;
-
-    item0 = RedisModule_CallReplyArrayElement(reply,0);
-    item1 = RedisModule_CallReplyArrayElement(reply,1);
-    if (!TestMatchReply(item0,"foo")) goto fail;
-    if (!TestMatchReply(item1,"1234")) goto fail;
-
-    RedisModule_ReplyWithSimpleString(ctx,"OK");
-    return REDISMODULE_OK;
-
-fail:
-    RedisModule_ReplyWithSimpleString(ctx,"ERR");
-    return REDISMODULE_OK;
-}
-
-/* TEST.STRING.APPEND -- Test appending to an existing string object. */
-int TestStringAppend(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    RedisModuleString *s = RedisModule_CreateString(ctx,"foo",3);
-    RedisModule_StringAppendBuffer(ctx,s,"bar",3);
-    RedisModule_ReplyWithString(ctx,s);
-    RedisModule_FreeString(ctx,s);
-    return REDISMODULE_OK;
-}
-
-/* TEST.STRING.APPEND.AM -- Test append with retain when auto memory is on. */
-int TestStringAppendAM(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    RedisModule_AutoMemory(ctx);
-    RedisModuleString *s = RedisModule_CreateString(ctx,"foo",3);
-    RedisModule_RetainString(ctx,s);
-    RedisModule_StringAppendBuffer(ctx,s,"bar",3);
-    RedisModule_ReplyWithString(ctx,s);
-    RedisModule_FreeString(ctx,s);
-    return REDISMODULE_OK;
-}
-
-/* TEST.STRING.PRINTF -- Test string formatting. */
-int TestStringPrintf(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    RedisModule_AutoMemory(ctx);
-    if (argc < 3) {
-        return RedisModule_WrongArity(ctx);
-    }
-    RedisModuleString *s = RedisModule_CreateStringPrintf(ctx, 
-        "Got %d args. argv[1]: %s, argv[2]: %s", 
-        argc, 
-        RedisModule_StringPtrLen(argv[1], NULL),
-        RedisModule_StringPtrLen(argv[2], NULL)
-    );
-
-    RedisModule_ReplyWithString(ctx,s);
-
-    return REDISMODULE_OK;
-}
-
-
-/* TEST.CTXFLAGS -- Test GetContextFlags. */
-int TestCtxFlags(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argc);
-    REDISMODULE_NOT_USED(argv);
-  
-    RedisModule_AutoMemory(ctx);
-  
-    int ok = 1;
-    const char *errString = NULL;
-  
-  #define FAIL(msg)    \
-    {                  \
-      ok = 0;          \
-      errString = msg; \
-      goto end;        \
-    }
-  
-    int flags = RedisModule_GetContextFlags(ctx);
-    if (flags == 0) {
-      FAIL("Got no flags");
-    }
-  
-    if (flags & REDISMODULE_CTX_FLAGS_LUA) FAIL("Lua flag was set");
-    if (flags & REDISMODULE_CTX_FLAGS_MULTI) FAIL("Multi flag was set");
-  
-    if (flags & REDISMODULE_CTX_FLAGS_AOF) FAIL("AOF Flag was set")
-    /* Enable AOF to test AOF flags */
-    RedisModule_Call(ctx, "config", "ccc", "set", "appendonly", "yes");
-    flags = RedisModule_GetContextFlags(ctx);
-    if (!(flags & REDISMODULE_CTX_FLAGS_AOF))
-      FAIL("AOF Flag not set after config set");
-  
-    if (flags & REDISMODULE_CTX_FLAGS_RDB) FAIL("RDB Flag was set");
-    /* Enable RDB to test RDB flags */
-    RedisModule_Call(ctx, "config", "ccc", "set", "save", "900 1");
-    flags = RedisModule_GetContextFlags(ctx);
-    if (!(flags & REDISMODULE_CTX_FLAGS_RDB))
-      FAIL("RDB Flag was not set after config set");
-  
-    if (!(flags & REDISMODULE_CTX_FLAGS_MASTER)) FAIL("Master flag was not set");
-    if (flags & REDISMODULE_CTX_FLAGS_SLAVE) FAIL("Slave flag was set");
-    if (flags & REDISMODULE_CTX_FLAGS_READONLY) FAIL("Read-only flag was set");
-    if (flags & REDISMODULE_CTX_FLAGS_CLUSTER) FAIL("Cluster flag was set");
-  
-    if (flags & REDISMODULE_CTX_FLAGS_MAXMEMORY) FAIL("Maxmemory flag was set");
-    ;
-    RedisModule_Call(ctx, "config", "ccc", "set", "maxmemory", "100000000");
-    flags = RedisModule_GetContextFlags(ctx);
-    if (!(flags & REDISMODULE_CTX_FLAGS_MAXMEMORY))
-      FAIL("Maxmemory flag was not set after config set");
-  
-    if (flags & REDISMODULE_CTX_FLAGS_EVICT) FAIL("Eviction flag was set");
-    RedisModule_Call(ctx, "config", "ccc", "set", "maxmemory-policy",
-                     "allkeys-lru");
-    flags = RedisModule_GetContextFlags(ctx);
-    if (!(flags & REDISMODULE_CTX_FLAGS_EVICT))
-      FAIL("Eviction flag was not set after config set");
-  
-  end:
-    /* Revert config changes */
-    RedisModule_Call(ctx, "config", "ccc", "set", "appendonly", "no");
-    RedisModule_Call(ctx, "config", "ccc", "set", "save", "");
-    RedisModule_Call(ctx, "config", "ccc", "set", "maxmemory", "0");
-    RedisModule_Call(ctx, "config", "ccc", "set", "maxmemory-policy", "noeviction");
-  
-    if (!ok) {
-      RedisModule_Log(ctx, "warning", "Failed CTXFLAGS Test. Reason: %s",
-                      errString);
-      return RedisModule_ReplyWithSimpleString(ctx, "ERR");
-    }
-  
-    return RedisModule_ReplyWithSimpleString(ctx, "OK");
-  }
-
-
-/* ----------------------------- Test framework ----------------------------- */
-
-/* Return 1 if the reply matches the specified string, otherwise log errors
- * in the server log and return 0. */
-int TestAssertStringReply(RedisModuleCtx *ctx, RedisModuleCallReply *reply, char *str, size_t len) {
-    RedisModuleString *mystr, *expected;
-
-    if (RedisModule_CallReplyType(reply) != REDISMODULE_REPLY_STRING) {
-        RedisModule_Log(ctx,"warning","Unexpected reply type %d",
-            RedisModule_CallReplyType(reply));
-        return 0;
-    }
-    mystr = RedisModule_CreateStringFromCallReply(reply);
-    expected = RedisModule_CreateString(ctx,str,len);
-    if (RedisModule_StringCompare(mystr,expected) != 0) {
-        const char *mystr_ptr = RedisModule_StringPtrLen(mystr,NULL);
-        const char *expected_ptr = RedisModule_StringPtrLen(expected,NULL);
-        RedisModule_Log(ctx,"warning",
-            "Unexpected string reply '%s' (instead of '%s')",
-            mystr_ptr, expected_ptr);
-        return 0;
-    }
-    return 1;
-}
-
-/* Return 1 if the reply matches the specified integer, otherwise log errors
- * in the server log and return 0. */
-int TestAssertIntegerReply(RedisModuleCtx *ctx, RedisModuleCallReply *reply, long long expected) {
-    if (RedisModule_CallReplyType(reply) != REDISMODULE_REPLY_INTEGER) {
-        RedisModule_Log(ctx,"warning","Unexpected reply type %d",
-            RedisModule_CallReplyType(reply));
-        return 0;
-    }
-    long long val = RedisModule_CallReplyInteger(reply);
-    if (val != expected) {
-        RedisModule_Log(ctx,"warning",
-            "Unexpected integer reply '%lld' (instead of '%lld')",
-            val, expected);
-        return 0;
-    }
-    return 1;
-}
-
-#define T(name,...) \
-    do { \
-        RedisModule_Log(ctx,"warning","Testing %s", name); \
-        reply = RedisModule_Call(ctx,name,__VA_ARGS__); \
-    } while (0);
-
-/* TEST.IT -- Run all the tests. */
-int TestIt(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    RedisModule_AutoMemory(ctx);
-    RedisModuleCallReply *reply;
-
-    /* Make sure the DB is empty before to proceed. */
-    T("dbsize","");
-    if (!TestAssertIntegerReply(ctx,reply,0)) goto fail;
-
-    T("ping","");
-    if (!TestAssertStringReply(ctx,reply,"PONG",4)) goto fail;
-
-    T("test.call","");
-    if (!TestAssertStringReply(ctx,reply,"OK",2)) goto fail;
-
-    T("test.ctxflags","");
-    if (!TestAssertStringReply(ctx,reply,"OK",2)) goto fail;
-
-    T("test.string.append","");
-    if (!TestAssertStringReply(ctx,reply,"foobar",6)) goto fail;
-
-    T("test.string.append.am","");
-    if (!TestAssertStringReply(ctx,reply,"foobar",6)) goto fail;
-
-    T("test.string.printf", "cc", "foo", "bar");
-    if (!TestAssertStringReply(ctx,reply,"Got 3 args. argv[1]: foo, argv[2]: bar",38)) goto fail;
-
-    RedisModule_ReplyWithSimpleString(ctx,"ALL TESTS PASSED");
-    return REDISMODULE_OK;
-
-fail:
-    RedisModule_ReplyWithSimpleString(ctx,
-        "SOME TEST NOT PASSED! Check server logs");
-    return REDISMODULE_OK;
-}
-
-int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
-    REDISMODULE_NOT_USED(argv);
-    REDISMODULE_NOT_USED(argc);
-
-    if (RedisModule_Init(ctx,"test",1,REDISMODULE_APIVER_1)
-        == REDISMODULE_ERR) return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.call",
-        TestCall,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.string.append",
-        TestStringAppend,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.string.append.am",
-        TestStringAppendAM,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.string.printf",
-        TestStringPrintf,"write deny-oom",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.ctxflags",
-        TestCtxFlags,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    if (RedisModule_CreateCommand(ctx,"test.it",
-        TestIt,"readonly",1,1,1) == REDISMODULE_ERR)
-        return REDISMODULE_ERR;
-
-    return REDISMODULE_OK;
-}

src/multi.c

@@ -117,7 +117,6 @@ void execCommand(client *c) {
     int orig_argc;
     struct redisCommand *orig_cmd;
     int must_propagate = 0; /* Need to propagate MULTI/EXEC to AOF / slaves? */
-    int was_master = server.masterhost == NULL;
 
     if (!(c->flags & CLIENT_MULTI)) {
         addReplyError(c,"EXEC without MULTI");
@@ -148,12 +147,11 @@ void execCommand(client *c) {
         c->argv = c->mstate.commands[j].argv;
         c->cmd = c->mstate.commands[j].cmd;
 
-        /* Propagate a MULTI request once we encounter the first command which
-         * is not readonly nor an administrative one.
+        /* Propagate a MULTI request once we encounter the first write op.
          * This way we'll deliver the MULTI/..../EXEC block as a whole and
          * both the AOF and the replication link will have the same consistency
          * and atomicity guarantees. */
-        if (!must_propagate && !(c->cmd->flags & (CMD_READONLY|CMD_ADMIN))) {
+        if (!must_propagate && !(c->cmd->flags & CMD_READONLY)) {
             execCommandPropagateMulti(c);
             must_propagate = 1;
         }
@@ -169,22 +167,9 @@ void execCommand(client *c) {
     c->argc = orig_argc;
     c->cmd = orig_cmd;
     discardTransaction(c);
-
     /* Make sure the EXEC command will be propagated as well if MULTI
      * was already propagated. */
-    if (must_propagate) {
-        int is_master = server.masterhost == NULL;
-        server.dirty++;
-        /* If inside the MULTI/EXEC block this instance was suddenly
-         * switched from master to slave (using the SLAVEOF command), the
-         * initial MULTI was propagated into the replication backlog, but the
-         * rest was not. We need to make sure to at least terminate the
-         * backlog with the final EXEC. */
-        if (server.repl_backlog && was_master && !is_master) {
-            char *execcmd = "*1\r\n$4\r\nEXEC\r\n";
-            feedReplicationBacklog(execcmd,strlen(execcmd));
-        }
-    }
+    if (must_propagate) server.dirty++;
 
 handle_monitor:
     /* Send EXEC to clients waiting data from MONITOR. We do it here

src/networking.c

@@ -28,12 +28,10 @@
  */
 
 #include "server.h"
-#include "atomicvar.h"
 #include <sys/uio.h>
 #include <math.h>
-#include <ctype.h>
 
-static void setProtocolError(const char *errstr, client *c, int pos);
+static void setProtocolError(client *c, int pos);
 
 /* Return the size consumed from the allocator, for the specified SDS string,
  * including internal fragmentation. This function is used in order to compute
@@ -54,13 +52,9 @@ size_t getStringObjectSdsUsedMemory(robj *o) {
     }
 }
 
-/* Client.reply list dup and free methods. */
 void *dupClientReplyValue(void *o) {
-    return sdsdup(o);
-}
-
-void freeClientReplyValue(void *o) {
-    sdsfree(o);
+    incrRefCount((robj*)o);
+    return o;
 }
 
 int listMatchObjects(void *a, void *b) {
@@ -89,14 +83,11 @@ client *createClient(int fd) {
     }
 
     selectDb(c,0);
-    uint64_t client_id;
-    atomicGetIncr(server.next_client_id,client_id,1);
-    c->id = client_id;
+    c->id = server.next_client_id++;
     c->fd = fd;
     c->name = NULL;
     c->bufpos = 0;
     c->querybuf = sdsempty();
-    c->pending_querybuf = sdsempty();
     c->querybuf_peak = 0;
     c->reqtype = 0;
     c->argc = 0;
@@ -111,7 +102,6 @@ client *createClient(int fd) {
     c->replstate = REPL_STATE_NONE;
     c->repl_put_online_on_ack = 0;
     c->reploff = 0;
-    c->read_reploff = 0;
     c->repl_ack_off = 0;
     c->repl_ack_time = 0;
     c->slave_listening_port = 0;
@@ -120,17 +110,17 @@ client *createClient(int fd) {
     c->reply = listCreate();
     c->reply_bytes = 0;
     c->obuf_soft_limit_reached_time = 0;
-    listSetFreeMethod(c->reply,freeClientReplyValue);
+    listSetFreeMethod(c->reply,decrRefCountVoid);
     listSetDupMethod(c->reply,dupClientReplyValue);
     c->btype = BLOCKED_NONE;
     c->bpop.timeout = 0;
-    c->bpop.keys = dictCreate(&objectKeyPointerValueDictType,NULL);
+    c->bpop.keys = dictCreate(&setDictType,NULL);
     c->bpop.target = NULL;
     c->bpop.numreplicas = 0;
     c->bpop.reploffset = 0;
     c->woff = 0;
     c->watched_keys = listCreate();
-    c->pubsub_channels = dictCreate(&objectKeyPointerValueDictType,NULL);
+    c->pubsub_channels = dictCreate(&setDictType,NULL);
     c->pubsub_patterns = listCreate();
     c->peerid = NULL;
     listSetFreeMethod(c->pubsub_patterns,decrRefCountVoid);
@@ -155,7 +145,7 @@ client *createClient(int fd) {
  * event handler in the following cases:
  *
  * 1) The event handler should already be installed since the output buffer
- *    already contains something.
+ *    already contained something.
  * 2) The client is a slave but not yet online, so we want to just accumulate
  *    writes in the buffer but not actually sending them yet.
  *
@@ -165,7 +155,7 @@ client *createClient(int fd) {
 int prepareClientToWrite(client *c) {
     /* If it's the Lua client we always return ok without installing any
      * handler since there is no socket at all. */
-    if (c->flags & (CLIENT_LUA|CLIENT_MODULE)) return C_OK;
+    if (c->flags & CLIENT_LUA) return C_OK;
 
     /* CLIENT REPLY OFF / SKIP handling: don't send replies. */
     if (c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) return C_ERR;
@@ -200,6 +190,22 @@ int prepareClientToWrite(client *c) {
     return C_OK;
 }
 
+/* Create a duplicate of the last object in the reply list when
+ * it is not exclusively owned by the reply list. */
+robj *dupLastObjectIfNeeded(list *reply) {
+    robj *new, *cur;
+    listNode *ln;
+    serverAssert(listLength(reply) > 0);
+    ln = listLast(reply);
+    cur = listNodeValue(ln);
+    if (cur->refcount > 1) {
+        new = dupStringObject(cur);
+        decrRefCount(cur);
+        listNodeValue(ln) = new;
+    }
+    return listNodeValue(ln);
+}
+
 /* -----------------------------------------------------------------------------
  * Low level functions to add more data to output buffers.
  * -------------------------------------------------------------------------- */
@@ -222,26 +228,30 @@ int _addReplyToBuffer(client *c, const char *s, size_t len) {
 }
 
 void _addReplyObjectToList(client *c, robj *o) {
+    robj *tail;
+
     if (c->flags & CLIENT_CLOSE_AFTER_REPLY) return;
 
     if (listLength(c->reply) == 0) {
-        sds s = sdsdup(o->ptr);
-        listAddNodeTail(c->reply,s);
-        c->reply_bytes += sdslen(s);
+        incrRefCount(o);
+        listAddNodeTail(c->reply,o);
+        c->reply_bytes += getStringObjectSdsUsedMemory(o);
     } else {
-        listNode *ln = listLast(c->reply);
-        sds tail = listNodeValue(ln);
+        tail = listNodeValue(listLast(c->reply));
 
-        /* Append to this object when possible. If tail == NULL it was
-         * set via addDeferredMultiBulkLength(). */
-        if (tail && sdslen(tail)+sdslen(o->ptr) <= PROTO_REPLY_CHUNK_BYTES) {
-            tail = sdscatsds(tail,o->ptr);
-            listNodeValue(ln) = tail;
-            c->reply_bytes += sdslen(o->ptr);
+        /* Append to this object when possible. */
+        if (tail->ptr != NULL &&
+            tail->encoding == OBJ_ENCODING_RAW &&
+            sdslen(tail->ptr)+sdslen(o->ptr) <= PROTO_REPLY_CHUNK_BYTES)
+        {
+            c->reply_bytes -= sdsZmallocSize(tail->ptr);
+            tail = dupLastObjectIfNeeded(c->reply);
+            tail->ptr = sdscatlen(tail->ptr,o->ptr,sdslen(o->ptr));
+            c->reply_bytes += sdsZmallocSize(tail->ptr);
         } else {
-            sds s = sdsdup(o->ptr);
-            listAddNodeTail(c->reply,s);
-            c->reply_bytes += sdslen(s);
+            incrRefCount(o);
+            listAddNodeTail(c->reply,o);
+            c->reply_bytes += getStringObjectSdsUsedMemory(o);
         }
     }
     asyncCloseClientOnOutputBufferLimitReached(c);
@@ -250,54 +260,62 @@ void _addReplyObjectToList(client *c, robj *o) {
 /* This method takes responsibility over the sds. When it is no longer
  * needed it will be free'd, otherwise it ends up in a robj. */
 void _addReplySdsToList(client *c, sds s) {
+    robj *tail;
+
     if (c->flags & CLIENT_CLOSE_AFTER_REPLY) {
         sdsfree(s);
         return;
     }
 
     if (listLength(c->reply) == 0) {
-        listAddNodeTail(c->reply,s);
-        c->reply_bytes += sdslen(s);
+        listAddNodeTail(c->reply,createObject(OBJ_STRING,s));
+        c->reply_bytes += sdsZmallocSize(s);
     } else {
-        listNode *ln = listLast(c->reply);
-        sds tail = listNodeValue(ln);
+        tail = listNodeValue(listLast(c->reply));
 
-        /* Append to this object when possible. If tail == NULL it was
-         * set via addDeferredMultiBulkLength(). */
-        if (tail && sdslen(tail)+sdslen(s) <= PROTO_REPLY_CHUNK_BYTES) {
-            tail = sdscatsds(tail,s);
-            listNodeValue(ln) = tail;
-            c->reply_bytes += sdslen(s);
+        /* Append to this object when possible. */
+        if (tail->ptr != NULL && tail->encoding == OBJ_ENCODING_RAW &&
+            sdslen(tail->ptr)+sdslen(s) <= PROTO_REPLY_CHUNK_BYTES)
+        {
+            c->reply_bytes -= sdsZmallocSize(tail->ptr);
+            tail = dupLastObjectIfNeeded(c->reply);
+            tail->ptr = sdscatlen(tail->ptr,s,sdslen(s));
+            c->reply_bytes += sdsZmallocSize(tail->ptr);
             sdsfree(s);
         } else {
-            listAddNodeTail(c->reply,s);
-            c->reply_bytes += sdslen(s);
+            listAddNodeTail(c->reply,createObject(OBJ_STRING,s));
+            c->reply_bytes += sdsZmallocSize(s);
         }
     }
     asyncCloseClientOnOutputBufferLimitReached(c);
 }
 
 void _addReplyStringToList(client *c, const char *s, size_t len) {
+    robj *tail;
+
     if (c->flags & CLIENT_CLOSE_AFTER_REPLY) return;
 
     if (listLength(c->reply) == 0) {
-        sds node = sdsnewlen(s,len);
-        listAddNodeTail(c->reply,node);
-        c->reply_bytes += len;
-    } else {
-        listNode *ln = listLast(c->reply);
-        sds tail = listNodeValue(ln);
+        robj *o = createStringObject(s,len);
 
-        /* Append to this object when possible. If tail == NULL it was
-         * set via addDeferredMultiBulkLength(). */
-        if (tail && sdslen(tail)+len <= PROTO_REPLY_CHUNK_BYTES) {
-            tail = sdscatlen(tail,s,len);
-            listNodeValue(ln) = tail;
-            c->reply_bytes += len;
+        listAddNodeTail(c->reply,o);
+        c->reply_bytes += getStringObjectSdsUsedMemory(o);
+    } else {
+        tail = listNodeValue(listLast(c->reply));
+
+        /* Append to this object when possible. */
+        if (tail->ptr != NULL && tail->encoding == OBJ_ENCODING_RAW &&
+            sdslen(tail->ptr)+len <= PROTO_REPLY_CHUNK_BYTES)
+        {
+            c->reply_bytes -= sdsZmallocSize(tail->ptr);
+            tail = dupLastObjectIfNeeded(c->reply);
+            tail->ptr = sdscatlen(tail->ptr,s,len);
+            c->reply_bytes += sdsZmallocSize(tail->ptr);
         } else {
-            sds node = sdsnewlen(s,len);
-            listAddNodeTail(c->reply,node);
-            c->reply_bytes += len;
+            robj *o = createStringObject(s,len);
+
+            listAddNodeTail(c->reply,o);
+            c->reply_bytes += getStringObjectSdsUsedMemory(o);
         }
     }
     asyncCloseClientOnOutputBufferLimitReached(c);
@@ -358,14 +376,6 @@ void addReplySds(client *c, sds s) {
     }
 }
 
-/* This low level function just adds whatever protocol you send it to the
- * client buffer, trying the static buffer initially, and using the string
- * of objects if not possible.
- *
- * It is efficient because does not create an SDS object nor an Redis object
- * if not needed. The object will only be created by calling
- * _addReplyStringToList() if we fail to extend the existing tail object
- * in the list of objects. */
 void addReplyString(client *c, const char *s, size_t len) {
     if (prepareClientToWrite(c) != C_OK) return;
     if (_addReplyToBuffer(c,s,len) != C_OK)
@@ -424,32 +434,32 @@ void *addDeferredMultiBulkLength(client *c) {
      * ready to be sent, since we are sure that before returning to the
      * event loop setDeferredMultiBulkLength() will be called. */
     if (prepareClientToWrite(c) != C_OK) return NULL;
-    listAddNodeTail(c->reply,NULL); /* NULL is our placeholder. */
+    listAddNodeTail(c->reply,createObject(OBJ_STRING,NULL));
     return listLast(c->reply);
 }
 
 /* Populate the length object and try gluing it to the next chunk. */
 void setDeferredMultiBulkLength(client *c, void *node, long length) {
     listNode *ln = (listNode*)node;
-    sds len, next;
+    robj *len, *next;
 
-    /* Abort when *node is NULL: when the client should not accept writes
-     * we return NULL in addDeferredMultiBulkLength() */
+    /* Abort when *node is NULL (see addDeferredMultiBulkLength). */
     if (node == NULL) return;
 
-    len = sdscatprintf(sdsnewlen("*",1),"%ld\r\n",length);
-    listNodeValue(ln) = len;
-    c->reply_bytes += sdslen(len);
+    len = listNodeValue(ln);
+    len->ptr = sdscatprintf(sdsempty(),"*%ld\r\n",length);
+    len->encoding = OBJ_ENCODING_RAW; /* in case it was an EMBSTR. */
+    c->reply_bytes += sdsZmallocSize(len->ptr);
     if (ln->next != NULL) {
         next = listNodeValue(ln->next);
 
         /* Only glue when the next node is non-NULL (an sds in this case) */
-        if (next != NULL) {
-            len = sdscatsds(len,next);
+        if (next->ptr != NULL) {
+            c->reply_bytes -= sdsZmallocSize(len->ptr);
+            c->reply_bytes -= getStringObjectSdsUsedMemory(next);
+            len->ptr = sdscatlen(len->ptr,next->ptr,sdslen(next->ptr));
+            c->reply_bytes += sdsZmallocSize(len->ptr);
             listDelNode(c->reply,ln->next);
-            listNodeValue(ln) = len;
-            /* No need to update c->reply_bytes: we are just moving the same
-             * amount of bytes from one node to another. */
         }
     }
     asyncCloseClientOnOutputBufferLimitReached(c);
@@ -561,7 +571,8 @@ void addReplyBulkCBuffer(client *c, const void *p, size_t len) {
 
 /* Add sds to reply (takes ownership of sds and frees it) */
 void addReplyBulkSds(client *c, sds s)  {
-    addReplyLongLongWithPrefix(c,sdslen(s),'$');
+    addReplySds(c,sdscatfmt(sdsempty(),"$%u\r\n",
+        (unsigned long)sdslen(s)));
     addReplySds(c,s);
     addReply(c,shared.crlf);
 }
@@ -800,7 +811,6 @@ void freeClient(client *c) {
 
     /* Free the query buffer */
     sdsfree(c->querybuf);
-    sdsfree(c->pending_querybuf);
     c->querybuf = NULL;
 
     /* Deallocate structures used to block on blocking ops. */
@@ -892,7 +902,8 @@ void freeClientsInAsyncFreeQueue(void) {
 int writeToClient(int fd, client *c, int handler_installed) {
     ssize_t nwritten = 0, totwritten = 0;
     size_t objlen;
-    sds o;
+    size_t objmem;
+    robj *o;
 
     while(clientHasPendingReplies(c)) {
         if (c->bufpos > 0) {
@@ -909,14 +920,16 @@ int writeToClient(int fd, client *c, int handler_installed) {
             }
         } else {
             o = listNodeValue(listFirst(c->reply));
-            objlen = sdslen(o);
+            objlen = sdslen(o->ptr);
+            objmem = getStringObjectSdsUsedMemory(o);
 
             if (objlen == 0) {
                 listDelNode(c->reply,listFirst(c->reply));
+                c->reply_bytes -= objmem;
                 continue;
             }
 
-            nwritten = write(fd, o + c->sentlen, objlen - c->sentlen);
+            nwritten = write(fd, ((char*)o->ptr)+c->sentlen,objlen-c->sentlen);
             if (nwritten <= 0) break;
             c->sentlen += nwritten;
             totwritten += nwritten;
@@ -925,11 +938,7 @@ int writeToClient(int fd, client *c, int handler_installed) {
             if (c->sentlen == objlen) {
                 listDelNode(c->reply,listFirst(c->reply));
                 c->sentlen = 0;
-                c->reply_bytes -= objlen;
-                /* If there are no longer objects in the list, we expect
-                 * the count of reply bytes to be exactly zero. */
-                if (listLength(c->reply) == 0)
-                    serverAssert(c->reply_bytes == 0);
+                c->reply_bytes -= objmem;
             }
         }
         /* Note that we avoid to send more than NET_MAX_WRITES_PER_EVENT
@@ -940,11 +949,11 @@ int writeToClient(int fd, client *c, int handler_installed) {
          *
          * However if we are over the maxmemory limit we ignore that and
          * just deliver as much data as it is possible to deliver. */
+        server.stat_net_output_bytes += totwritten;
         if (totwritten > NET_MAX_WRITES_PER_EVENT &&
             (server.maxmemory == 0 ||
              zmalloc_used_memory() < server.maxmemory)) break;
     }
-    server.stat_net_output_bytes += totwritten;
     if (nwritten == -1) {
         if (errno == EAGAIN) {
             nwritten = 0;
@@ -1036,13 +1045,6 @@ void resetClient(client *c) {
     }
 }
 
-/* Like processMultibulkBuffer(), but for the inline protocol instead of RESP,
- * this function consumes the client query buffer and creates a command ready
- * to be executed inside the client structure. Returns C_OK if the command
- * is ready to be executed, or C_ERR if there is still protocol to read to
- * have a well formed command. The function also returns C_ERR when there is
- * a protocol error: in such a case the client structure is setup to reply
- * with the error and close the connection. */
 int processInlineBuffer(client *c) {
     char *newline;
     int argc, j;
@@ -1056,7 +1058,7 @@ int processInlineBuffer(client *c) {
     if (newline == NULL) {
         if (sdslen(c->querybuf) > PROTO_INLINE_MAX_SIZE) {
             addReplyError(c,"Protocol error: too big inline request");
-            setProtocolError("too big inline request",c,0);
+            setProtocolError(c,0);
         }
         return C_ERR;
     }
@@ -1072,7 +1074,7 @@ int processInlineBuffer(client *c) {
     sdsfree(aux);
     if (argv == NULL) {
         addReplyError(c,"Protocol error: unbalanced quotes in request");
-        setProtocolError("unbalanced quotes in inline request",c,0);
+        setProtocolError(c,0);
         return C_ERR;
     }
 
@@ -1106,46 +1108,17 @@ int processInlineBuffer(client *c) {
 
 /* Helper function. Trims query buffer to make the function that processes
  * multi bulk requests idempotent. */
-#define PROTO_DUMP_LEN 128
-static void setProtocolError(const char *errstr, client *c, int pos) {
+static void setProtocolError(client *c, int pos) {
     if (server.verbosity <= LL_VERBOSE) {
         sds client = catClientInfoString(sdsempty(),c);
-
-        /* Sample some protocol to given an idea about what was inside. */
-        char buf[256];
-        if (sdslen(c->querybuf) < PROTO_DUMP_LEN) {
-            snprintf(buf,sizeof(buf),"Query buffer during protocol error: '%s'", c->querybuf);
-        } else {
-            snprintf(buf,sizeof(buf),"Query buffer during protocol error: '%.*s' (... more %zu bytes ...) '%.*s'", PROTO_DUMP_LEN/2, c->querybuf, sdslen(c->querybuf)-PROTO_DUMP_LEN, PROTO_DUMP_LEN/2, c->querybuf+sdslen(c->querybuf)-PROTO_DUMP_LEN/2);
-        }
-
-        /* Remove non printable chars. */
-        char *p = buf;
-        while (*p != '\0') {
-            if (!isprint(*p)) *p = '.';
-            p++;
-        }
-
-        /* Log all the client and protocol info. */
         serverLog(LL_VERBOSE,
-            "Protocol error (%s) from client: %s. %s", errstr, client, buf);
+            "Protocol error from client: %s", client);
         sdsfree(client);
     }
     c->flags |= CLIENT_CLOSE_AFTER_REPLY;
     sdsrange(c->querybuf,pos,-1);
 }
 
-/* Process the query buffer for client 'c', setting up the client argument
- * vector for command execution. Returns C_OK if after running the function
- * the client has a well-formed ready to be processed command, otherwise
- * C_ERR if there is still to read more buffer to get the full command.
- * The function also returns C_ERR when there is a protocol error: in such a
- * case the client structure is setup to reply with the error and close
- * the connection.
- *
- * This function is called if processInputBuffer() detects that the next
- * command is in RESP format, so the first byte in the command is found
- * to be '*'. Otherwise for inline commands processInlineBuffer() is called. */
 int processMultibulkBuffer(client *c) {
     char *newline = NULL;
     int pos = 0, ok;
@@ -1160,7 +1133,7 @@ int processMultibulkBuffer(client *c) {
         if (newline == NULL) {
             if (sdslen(c->querybuf) > PROTO_INLINE_MAX_SIZE) {
                 addReplyError(c,"Protocol error: too big mbulk count string");
-                setProtocolError("too big mbulk count string",c,0);
+                setProtocolError(c,0);
             }
             return C_ERR;
         }
@@ -1175,7 +1148,7 @@ int processMultibulkBuffer(client *c) {
         ok = string2ll(c->querybuf+1,newline-(c->querybuf+1),&ll);
         if (!ok || ll > 1024*1024) {
             addReplyError(c,"Protocol error: invalid multibulk length");
-            setProtocolError("invalid mbulk count",c,pos);
+            setProtocolError(c,pos);
             return C_ERR;
         }
 
@@ -1201,7 +1174,7 @@ int processMultibulkBuffer(client *c) {
                 if (sdslen(c->querybuf) > PROTO_INLINE_MAX_SIZE) {
                     addReplyError(c,
                         "Protocol error: too big bulk count string");
-                    setProtocolError("too big bulk count string",c,0);
+                    setProtocolError(c,0);
                     return C_ERR;
                 }
                 break;
@@ -1215,14 +1188,14 @@ int processMultibulkBuffer(client *c) {
                 addReplyErrorFormat(c,
                     "Protocol error: expected '$', got '%c'",
                     c->querybuf[pos]);
-                setProtocolError("expected $ but got something else",c,pos);
+                setProtocolError(c,pos);
                 return C_ERR;
             }
 
             ok = string2ll(c->querybuf+pos+1,newline-(c->querybuf+pos+1),&ll);
             if (!ok || ll < 0 || ll > 512*1024*1024) {
                 addReplyError(c,"Protocol error: invalid bulk length");
-                setProtocolError("invalid bulk length",c,pos);
+                setProtocolError(c,pos);
                 return C_ERR;
             }
 
@@ -1280,14 +1253,10 @@ int processMultibulkBuffer(client *c) {
     /* We're done when c->multibulk == 0 */
     if (c->multibulklen == 0) return C_OK;
 
-    /* Still not ready to process the command */
+    /* Still not read to process the command */
     return C_ERR;
 }
 
-/* This function is called every time, in the client structure 'c', there is
- * more query buffer to process, because we read more data from the socket
- * or because a client was blocked and later reactivated, so there could be
- * pending query buffer, already representing a full command, to process. */
 void processInputBuffer(client *c) {
     server.current_client = c;
     /* Keep processing while there is something in the input buffer */
@@ -1300,10 +1269,8 @@ void processInputBuffer(client *c) {
 
         /* CLIENT_CLOSE_AFTER_REPLY closes the connection once the reply is
          * written to the client. Make sure to not let the reply grow after
-         * this flag has been set (i.e. don't process more commands).
-         *
-         * The same applies for clients we want to terminate ASAP. */
-        if (c->flags & (CLIENT_CLOSE_AFTER_REPLY|CLIENT_CLOSE_ASAP)) break;
+         * this flag has been set (i.e. don't process more commands). */
+        if (c->flags & CLIENT_CLOSE_AFTER_REPLY) break;
 
         /* Determine request type when unknown. */
         if (!c->reqtype) {
@@ -1327,22 +1294,10 @@ void processInputBuffer(client *c) {
             resetClient(c);
         } else {
             /* Only reset the client when the command was executed. */
-            if (processCommand(c) == C_OK) {
-                if (c->flags & CLIENT_MASTER && !(c->flags & CLIENT_MULTI)) {
-                    /* Update the applied replication offset of our master. */
-                    c->reploff = c->read_reploff - sdslen(c->querybuf);
-                }
-
-                /* Don't reset the client structure for clients blocked in a
-                 * module blocking command, so that the reply callback will
-                 * still be able to access the client argv and argc field.
-                 * The client will be reset in unblockClientFromModule(). */
-                if (!(c->flags & CLIENT_BLOCKED) || c->btype != BLOCKED_MODULE)
-                    resetClient(c);
-            }
-            /* freeMemoryIfNeeded may flush slave output buffers. This may
-             * result into a slave, that may be the active client, to be
-             * freed. */
+            if (processCommand(c) == C_OK)
+                resetClient(c);
+            /* freeMemoryIfNeeded may flush slave output buffers. This may result
+             * into a slave, that may be the active client, to be freed. */
             if (server.current_client == NULL) break;
         }
     }
@@ -1387,17 +1342,11 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
         serverLog(LL_VERBOSE, "Client closed connection");
         freeClient(c);
         return;
-    } else if (c->flags & CLIENT_MASTER) {
-        /* Append the query buffer to the pending (not applied) buffer
-         * of the master. We'll use this buffer later in order to have a
-         * copy of the string applied by the last command executed. */
-        c->pending_querybuf = sdscatlen(c->pending_querybuf,
-                                        c->querybuf+qblen,nread);
     }
 
     sdsIncrLen(c->querybuf,nread);
     c->lastinteraction = server.unixtime;
-    if (c->flags & CLIENT_MASTER) c->read_reploff += nread;
+    if (c->flags & CLIENT_MASTER) c->reploff += nread;
     server.stat_net_input_bytes += nread;
     if (sdslen(c->querybuf) > server.client_max_querybuf_len) {
         sds ci = catClientInfoString(sdsempty(),c), bytes = sdsempty();
@@ -1409,25 +1358,7 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
         freeClient(c);
         return;
     }
-
-    /* Time to process the buffer. If the client is a master we need to
-     * compute the difference between the applied offset before and after
-     * processing the buffer, to understand how much of the replication stream
-     * was actually applied to the master state: this quantity, and its
-     * corresponding part of the replication stream, will be propagated to
-     * the sub-slaves and to the replication backlog. */
-    if (!(c->flags & CLIENT_MASTER)) {
-        processInputBuffer(c);
-    } else {
-        size_t prev_offset = c->reploff;
-        processInputBuffer(c);
-        size_t applied = c->reploff - prev_offset;
-        if (applied) {
-            replicationFeedSlavesFromMasterStream(server.slaves,
-                    c->pending_querybuf, applied);
-            sdsrange(c->pending_querybuf,applied,-1);
-        }
-    }
+    processInputBuffer(c);
 }
 
 void getClientsMaxBuffers(unsigned long *longest_output_list,
@@ -1706,26 +1637,6 @@ void clientCommand(client *c) {
     }
 }
 
-/* This callback is bound to POST and "Host:" command names. Those are not
- * really commands, but are used in security attacks in order to talk to
- * Redis instances via HTTP, with a technique called "cross protocol scripting"
- * which exploits the fact that services like Redis will discard invalid
- * HTTP headers and will process what follows.
- *
- * As a protection against this attack, Redis will terminate the connection
- * when a POST or "Host:" header is seen, and will log the event from
- * time to time (to avoid creating a DOS as a result of too many logs). */
-void securityWarningCommand(client *c) {
-    static time_t logged_time;
-    time_t now = time(NULL);
-
-    if (labs(now-logged_time) > 60) {
-        serverLog(LL_WARNING,"Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted.");
-        logged_time = now;
-    }
-    freeClientAsync(c);
-}
-
 /* Rewrite the command vector of the client. All the new objects ref count
  * is incremented. The old command vector is freed, and the old objects
  * ref count is decremented. */
@@ -1811,9 +1722,7 @@ void rewriteClientCommandArgument(client *c, int i, robj *newval) {
  * the caller wishes. The main usage of this function currently is
  * enforcing the client output length limits. */
 unsigned long getClientOutputBufferMemoryUsage(client *c) {
-    unsigned long list_item_size = sizeof(listNode)+5;
-    /* The +5 above means we assume an sds16 hdr, may not be true
-     * but is not going to be a problem. */
+    unsigned long list_item_size = sizeof(listNode)+sizeof(robj);
 
     return c->reply_bytes + (list_item_size*listLength(c->reply));
 }

src/object.c

@@ -36,8 +36,6 @@
 #define strtold(a,b) ((long double)strtod((a),(b)))
 #endif
 
-/* ===================== Creation and parsing of objects ==================== */
-
 robj *createObject(int type, void *ptr) {
     robj *o = zmalloc(sizeof(*o));
     o->type = type;
@@ -45,37 +43,15 @@ robj *createObject(int type, void *ptr) {
     o->ptr = ptr;
     o->refcount = 1;
 
-    /* Set the LRU to the current lruclock (minutes resolution), or
-     * alternatively the LFU counter. */
-    if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-        o->lru = (LFUGetTimeInMinutes()<<8) | LFU_INIT_VAL;
-    } else {
-        o->lru = LRU_CLOCK();
-    }
-    return o;
-}
-
-/* Set a special refcount in the object to make it "shared":
- * incrRefCount and decrRefCount() will test for this special refcount
- * and will not touch the object. This way it is free to access shared
- * objects such as small integers from different threads without any
- * mutex.
- *
- * A common patter to create shared objects:
- *
- * robj *myobject = makeObjectShared(createObject(...));
- *
- */
-robj *makeObjectShared(robj *o) {
-    serverAssert(o->refcount == 1);
-    o->refcount = OBJ_SHARED_REFCOUNT;
+    /* Set the LRU to the current lruclock (minutes resolution). */
+    o->lru = LRU_CLOCK();
     return o;
 }
 
 /* Create a string object with encoding OBJ_ENCODING_RAW, that is a plain
  * string object where o->ptr points to a proper sds string. */
 robj *createRawStringObject(const char *ptr, size_t len) {
-    return createObject(OBJ_STRING, sdsnewlen(ptr,len));
+    return createObject(OBJ_STRING,sdsnewlen(ptr,len));
 }
 
 /* Create a string object with encoding OBJ_ENCODING_EMBSTR, that is
@@ -89,11 +65,7 @@ robj *createEmbeddedStringObject(const char *ptr, size_t len) {
     o->encoding = OBJ_ENCODING_EMBSTR;
     o->ptr = sh+1;
     o->refcount = 1;
-    if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-        o->lru = (LFUGetTimeInMinutes()<<8) | LFU_INIT_VAL;
-    } else {
-        o->lru = LRU_CLOCK();
-    }
+    o->lru = LRU_CLOCK();
 
     sh->len = len;
     sh->alloc = len;
@@ -108,7 +80,7 @@ robj *createEmbeddedStringObject(const char *ptr, size_t len) {
 }
 
 /* Create a string object with EMBSTR encoding if it is smaller than
- * OBJ_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
+ * REIDS_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
  * used.
  *
  * The current limit of 39 is chosen so that the biggest string object
@@ -146,7 +118,37 @@ robj *createStringObjectFromLongLong(long long value) {
  * The 'humanfriendly' option is used for INCRBYFLOAT and HINCRBYFLOAT. */
 robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
     char buf[256];
-    int len = ld2string(buf,sizeof(buf),value,humanfriendly);
+    int len;
+
+    if (isinf(value)) {
+        /* Libc in odd systems (Hi Solaris!) will format infinite in a
+         * different way, so better to handle it in an explicit way. */
+        if (value > 0) {
+            memcpy(buf,"inf",3);
+            len = 3;
+        } else {
+            memcpy(buf,"-inf",4);
+            len = 4;
+        }
+    } else if (humanfriendly) {
+        /* We use 17 digits precision since with 128 bit floats that precision
+         * after rounding is able to represent most small decimal numbers in a
+         * way that is "non surprising" for the user (that is, most small
+         * decimal numbers will be represented in a way that when converted
+         * back into a string are exactly the same as what the user typed.) */
+        len = snprintf(buf,sizeof(buf),"%.17Lf", value);
+        /* Now remove trailing zeroes after the '.' */
+        if (strchr(buf,'.') != NULL) {
+            char *p = buf+len-1;
+            while(*p == '0') {
+                p--;
+                len--;
+            }
+            if (*p == '.') len--;
+        }
+    } else {
+        len = snprintf(buf,sizeof(buf),"%.17Lg", value);
+    }
     return createStringObject(buf,len);
 }
 
@@ -158,7 +160,7 @@ robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
  * will always result in a fresh object that is unshared (refcount == 1).
  *
  * The resulting object always has refcount set to 1. */
-robj *dupStringObject(const robj *o) {
+robj *dupStringObject(robj *o) {
     robj *d;
 
     serverAssert(o->type == OBJ_STRING);
@@ -232,13 +234,6 @@ robj *createZsetZiplistObject(void) {
     return o;
 }
 
-robj *createModuleObject(moduleType *mt, void *value) {
-    moduleValue *mv = zmalloc(sizeof(*mv));
-    mv->type = mt;
-    mv->value = value;
-    return createObject(OBJ_MODULE,mv);
-}
-
 void freeStringObject(robj *o) {
     if (o->encoding == OBJ_ENCODING_RAW) {
         sdsfree(o->ptr);
@@ -246,9 +241,11 @@ void freeStringObject(robj *o) {
 }
 
 void freeListObject(robj *o) {
-    if (o->encoding == OBJ_ENCODING_QUICKLIST) {
+    switch (o->encoding) {
+    case OBJ_ENCODING_QUICKLIST:
         quicklistRelease(o->ptr);
-    } else {
+        break;
+    default:
         serverPanic("Unknown list encoding type");
     }
 }
@@ -297,17 +294,12 @@ void freeHashObject(robj *o) {
     }
 }
 
-void freeModuleObject(robj *o) {
-    moduleValue *mv = o->ptr;
-    mv->type->free(mv->value);
-    zfree(mv);
-}
-
 void incrRefCount(robj *o) {
-    if (o->refcount != OBJ_SHARED_REFCOUNT) o->refcount++;
+    o->refcount++;
 }
 
 void decrRefCount(robj *o) {
+    if (o->refcount <= 0) serverPanic("decrRefCount against refcount <= 0");
     if (o->refcount == 1) {
         switch(o->type) {
         case OBJ_STRING: freeStringObject(o); break;
@@ -315,13 +307,11 @@ void decrRefCount(robj *o) {
         case OBJ_SET: freeSetObject(o); break;
         case OBJ_ZSET: freeZsetObject(o); break;
         case OBJ_HASH: freeHashObject(o); break;
-        case OBJ_MODULE: freeModuleObject(o); break;
         default: serverPanic("Unknown object type"); break;
         }
         zfree(o);
     } else {
-        if (o->refcount <= 0) serverPanic("decrRefCount against refcount <= 0");
-        if (o->refcount != OBJ_SHARED_REFCOUNT) o->refcount--;
+        o->refcount--;
     }
 }
 
@@ -357,17 +347,13 @@ int checkType(client *c, robj *o, int type) {
     return 0;
 }
 
-int isSdsRepresentableAsLongLong(sds s, long long *llval) {
-    return string2ll(s,sdslen(s),llval) ? C_OK : C_ERR;
-}
-
 int isObjectRepresentableAsLongLong(robj *o, long long *llval) {
     serverAssertWithInfo(NULL,o,o->type == OBJ_STRING);
     if (o->encoding == OBJ_ENCODING_INT) {
         if (llval) *llval = (long) o->ptr;
         return C_OK;
     } else {
-        return isSdsRepresentableAsLongLong(o->ptr,llval);
+        return string2ll(o->ptr,sdslen(o->ptr),llval) ? C_OK : C_ERR;
     }
 }
 
@@ -403,7 +389,8 @@ robj *tryObjectEncoding(robj *o) {
          * because every object needs to have a private LRU field for the LRU
          * algorithm to work well. */
         if ((server.maxmemory == 0 ||
-            !(server.maxmemory_policy & MAXMEMORY_FLAG_NO_SHARED_INTEGERS)) &&
+             (server.maxmemory_policy != MAXMEMORY_VOLATILE_LRU &&
+              server.maxmemory_policy != MAXMEMORY_ALLKEYS_LRU)) &&
             value >= 0 &&
             value < OBJ_SHARED_INTEGERS)
         {
@@ -547,7 +534,7 @@ size_t stringObjectLen(robj *o) {
     }
 }
 
-int getDoubleFromObject(const robj *o, double *target) {
+int getDoubleFromObject(robj *o, double *target) {
     double value;
     char *eptr;
 
@@ -558,11 +545,11 @@ int getDoubleFromObject(const robj *o, double *target) {
         if (sdsEncodedObject(o)) {
             errno = 0;
             value = strtod(o->ptr, &eptr);
-            if (sdslen(o->ptr) == 0 ||
-                isspace(((const char*)o->ptr)[0]) ||
+            if (isspace(((char*)o->ptr)[0]) ||
                 eptr[0] != '\0' ||
                 (errno == ERANGE &&
                     (value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
+                errno == EINVAL ||
                 isnan(value))
                 return C_ERR;
         } else if (o->encoding == OBJ_ENCODING_INT) {
@@ -600,12 +587,8 @@ int getLongDoubleFromObject(robj *o, long double *target) {
         if (sdsEncodedObject(o)) {
             errno = 0;
             value = strtold(o->ptr, &eptr);
-            if (sdslen(o->ptr) == 0 ||
-                isspace(((const char*)o->ptr)[0]) ||
-                eptr[0] != '\0' ||
-                (errno == ERANGE &&
-                    (value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
-                isnan(value))
+            if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' ||
+                errno == ERANGE || isnan(value))
                 return C_ERR;
         } else if (o->encoding == OBJ_ENCODING_INT) {
             value = (long)o->ptr;
@@ -694,307 +677,18 @@ char *strEncoding(int encoding) {
     }
 }
 
-/* =========================== Memory introspection ========================== */
-
-/* Returns the size in bytes consumed by the key's value in RAM.
- * Note that the returned value is just an approximation, especially in the
- * case of aggregated data types where only "sample_size" elements
- * are checked and averaged to estimate the total size. */
-#define OBJ_COMPUTE_SIZE_DEF_SAMPLES 5 /* Default sample size. */
-size_t objectComputeSize(robj *o, size_t sample_size) {
-    sds ele, ele2;
-    dict *d;
-    dictIterator *di;
-    struct dictEntry *de;
-    size_t asize = 0, elesize = 0, samples = 0;
-
-    if (o->type == OBJ_STRING) {
-        if(o->encoding == OBJ_ENCODING_INT) {
-            asize = sizeof(*o);
-        } else if(o->encoding == OBJ_ENCODING_RAW) {
-            asize = sdsAllocSize(o->ptr)+sizeof(*o);
-        } else if(o->encoding == OBJ_ENCODING_EMBSTR) {
-            asize = sdslen(o->ptr)+2+sizeof(*o);
-        } else {
-            serverPanic("Unknown string encoding");
-        }
-    } else if (o->type == OBJ_LIST) {
-        if (o->encoding == OBJ_ENCODING_QUICKLIST) {
-            quicklist *ql = o->ptr;
-            quicklistNode *node = ql->head;
-            asize = sizeof(*o)+sizeof(quicklist);
-            do {
-                elesize += sizeof(quicklistNode)+ziplistBlobLen(node->zl);
-                samples++;
-            } while ((node = node->next) && samples < sample_size);
-            asize += (double)elesize/samples*listTypeLength(o);
-        } else if (o->encoding == OBJ_ENCODING_ZIPLIST) {
-            asize = sizeof(*o)+ziplistBlobLen(o->ptr);
-        } else {
-            serverPanic("Unknown list encoding");
-        }
-    } else if (o->type == OBJ_SET) {
-        if (o->encoding == OBJ_ENCODING_HT) {
-            d = o->ptr;
-            di = dictGetIterator(d);
-            asize = sizeof(*o)+sizeof(dict)+(sizeof(struct dictEntry*)*dictSlots(d));
-            while((de = dictNext(di)) != NULL && samples < sample_size) {
-                ele = dictGetKey(de);
-                elesize += sizeof(struct dictEntry) + sdsAllocSize(ele);
-                samples++;
-            }
-            dictReleaseIterator(di);
-            if (samples) asize += (double)elesize/samples*dictSize(d);
-        } else if (o->encoding == OBJ_ENCODING_INTSET) {
-            intset *is = o->ptr;
-            asize = sizeof(*o)+sizeof(*is)+is->encoding*is->length;
-        } else {
-            serverPanic("Unknown set encoding");
-        }
-    } else if (o->type == OBJ_ZSET) {
-        if (o->encoding == OBJ_ENCODING_ZIPLIST) {
-            asize = sizeof(*o)+(ziplistBlobLen(o->ptr));
-        } else if (o->encoding == OBJ_ENCODING_SKIPLIST) {
-            d = ((zset*)o->ptr)->dict;
-            zskiplist *zsl = ((zset*)o->ptr)->zsl;
-            zskiplistNode *znode = zsl->header->level[0].forward;
-            asize = sizeof(*o)+sizeof(zset)+(sizeof(struct dictEntry*)*dictSlots(d));
-            while(znode != NULL && samples < sample_size) {
-                elesize += sdsAllocSize(znode->ele);
-                elesize += sizeof(struct dictEntry) + zmalloc_size(znode);
-                samples++;
-                znode = znode->level[0].forward;
-            }
-            if (samples) asize += (double)elesize/samples*dictSize(d);
-        } else {
-            serverPanic("Unknown sorted set encoding");
-        }
-    } else if (o->type == OBJ_HASH) {
-        if (o->encoding == OBJ_ENCODING_ZIPLIST) {
-            asize = sizeof(*o)+(ziplistBlobLen(o->ptr));
-        } else if (o->encoding == OBJ_ENCODING_HT) {
-            d = o->ptr;
-            di = dictGetIterator(d);
-            asize = sizeof(*o)+sizeof(dict)+(sizeof(struct dictEntry*)*dictSlots(d));
-            while((de = dictNext(di)) != NULL && samples < sample_size) {
-                ele = dictGetKey(de);
-                ele2 = dictGetVal(de);
-                elesize += sdsAllocSize(ele) + sdsAllocSize(ele2);
-                elesize += sizeof(struct dictEntry);
-                samples++;
-            }
-            dictReleaseIterator(di);
-            if (samples) asize += (double)elesize/samples*dictSize(d);
-        } else {
-            serverPanic("Unknown hash encoding");
-        }
-    } else if (o->type == OBJ_MODULE) {
-        moduleValue *mv = o->ptr;
-        moduleType *mt = mv->type;
-        if (mt->mem_usage != NULL) {
-            asize = mt->mem_usage(mv->value);
-        } else {
-            asize = 0;
-        }
+/* Given an object returns the min number of milliseconds the object was never
+ * requested, using an approximated LRU algorithm. */
+unsigned long long estimateObjectIdleTime(robj *o) {
+    unsigned long long lruclock = LRU_CLOCK();
+    if (lruclock >= o->lru) {
+        return (lruclock - o->lru) * LRU_CLOCK_RESOLUTION;
     } else {
-        serverPanic("Unknown object type");
+        return (lruclock + (LRU_CLOCK_MAX - o->lru)) *
+                    LRU_CLOCK_RESOLUTION;
     }
-    return asize;
 }
 
-/* Release data obtained with getMemoryOverheadData(). */
-void freeMemoryOverheadData(struct redisMemOverhead *mh) {
-    zfree(mh->db);
-    zfree(mh);
-}
-
-/* Return a struct redisMemOverhead filled with memory overhead
- * information used for the MEMORY OVERHEAD and INFO command. The returned
- * structure pointer should be freed calling freeMemoryOverheadData(). */
-struct redisMemOverhead *getMemoryOverheadData(void) {
-    int j;
-    size_t mem_total = 0;
-    size_t mem = 0;
-    size_t zmalloc_used = zmalloc_used_memory();
-    struct redisMemOverhead *mh = zcalloc(sizeof(*mh));
-
-    mh->total_allocated = zmalloc_used;
-    mh->startup_allocated = server.initial_memory_usage;
-    mh->peak_allocated = server.stat_peak_memory;
-    mh->fragmentation =
-        zmalloc_get_fragmentation_ratio(server.resident_set_size);
-    mem_total += server.initial_memory_usage;
-
-    mem = 0;
-    if (server.repl_backlog)
-        mem += zmalloc_size(server.repl_backlog);
-    mh->repl_backlog = mem;
-    mem_total += mem;
-
-    mem = 0;
-    if (listLength(server.slaves)) {
-        listIter li;
-        listNode *ln;
-
-        listRewind(server.slaves,&li);
-        while((ln = listNext(&li))) {
-            client *c = listNodeValue(ln);
-            mem += getClientOutputBufferMemoryUsage(c);
-            mem += sdsAllocSize(c->querybuf);
-            mem += sizeof(client);
-        }
-    }
-    mh->clients_slaves = mem;
-    mem_total+=mem;
-
-    mem = 0;
-    if (listLength(server.clients)) {
-        listIter li;
-        listNode *ln;
-
-        listRewind(server.clients,&li);
-        while((ln = listNext(&li))) {
-            client *c = listNodeValue(ln);
-            if (c->flags & CLIENT_SLAVE)
-                continue;
-            mem += getClientOutputBufferMemoryUsage(c);
-            mem += sdsAllocSize(c->querybuf);
-            mem += sizeof(client);
-        }
-    }
-    mh->clients_normal = mem;
-    mem_total+=mem;
-
-    mem = 0;
-    if (server.aof_state != AOF_OFF) {
-        mem += sdslen(server.aof_buf);
-        mem += aofRewriteBufferSize();
-    }
-    mh->aof_buffer = mem;
-    mem_total+=mem;
-
-    for (j = 0; j < server.dbnum; j++) {
-        redisDb *db = server.db+j;
-        long long keyscount = dictSize(db->dict);
-        if (keyscount==0) continue;
-
-        mh->total_keys += keyscount;
-        mh->db = zrealloc(mh->db,sizeof(mh->db[0])*(mh->num_dbs+1));
-        mh->db[mh->num_dbs].dbid = j;
-
-        mem = dictSize(db->dict) * sizeof(dictEntry) +
-              dictSlots(db->dict) * sizeof(dictEntry*) +
-              dictSize(db->dict) * sizeof(robj);
-        mh->db[mh->num_dbs].overhead_ht_main = mem;
-        mem_total+=mem;
-
-        mem = dictSize(db->expires) * sizeof(dictEntry) +
-              dictSlots(db->expires) * sizeof(dictEntry*);
-        mh->db[mh->num_dbs].overhead_ht_expires = mem;
-        mem_total+=mem;
-
-        mh->num_dbs++;
-    }
-
-    mh->overhead_total = mem_total;
-    mh->dataset = zmalloc_used - mem_total;
-    mh->peak_perc = (float)zmalloc_used*100/mh->peak_allocated;
-
-    /* Metrics computed after subtracting the startup memory from
-     * the total memory. */
-    size_t net_usage = 1;
-    if (zmalloc_used > mh->startup_allocated)
-        net_usage = zmalloc_used - mh->startup_allocated;
-    mh->dataset_perc = (float)mh->dataset*100/net_usage;
-    mh->bytes_per_key = mh->total_keys ? (net_usage / mh->total_keys) : 0;
-
-    return mh;
-}
-
-/* Helper for "MEMORY allocator-stats", used as a callback for the jemalloc
- * stats output. */
-void inputCatSds(void *result, const char *str) {
-    /* result is actually a (sds *), so re-cast it here */
-    sds *info = (sds *)result;
-    *info = sdscat(*info, str);
-}
-
-/* This implements MEMORY DOCTOR. An human readable analysis of the Redis
- * memory condition. */
-sds getMemoryDoctorReport(void) {
-    int empty = 0;          /* Instance is empty or almost empty. */
-    int big_peak = 0;       /* Memory peak is much larger than used mem. */
-    int high_frag = 0;      /* High fragmentation. */
-    int big_slave_buf = 0;  /* Slave buffers are too big. */
-    int big_client_buf = 0; /* Client buffers are too big. */
-    int num_reports = 0;
-    struct redisMemOverhead *mh = getMemoryOverheadData();
-
-    if (mh->total_allocated < (1024*1024*5)) {
-        empty = 1;
-        num_reports++;
-    } else {
-        /* Peak is > 150% of current used memory? */
-        if (((float)mh->peak_allocated / mh->total_allocated) > 1.5) {
-            big_peak = 1;
-            num_reports++;
-        }
-
-        /* Fragmentation is higher than 1.4? */
-        if (mh->fragmentation > 1.4) {
-            high_frag = 1;
-            num_reports++;
-        }
-
-        /* Clients using more than 200k each average? */
-        long numslaves = listLength(server.slaves);
-        long numclients = listLength(server.clients)-numslaves;
-        if (mh->clients_normal / numclients > (1024*200)) {
-            big_client_buf = 1;
-            num_reports++;
-        }
-
-        /* Slaves using more than 10 MB each? */
-        if (numslaves > 0 && mh->clients_slaves / numslaves > (1024*1024*10)) {
-            big_slave_buf = 1;
-            num_reports++;
-        }
-    }
-
-    sds s;
-    if (num_reports == 0) {
-        s = sdsnew(
-        "Hi Sam, I can't find any memory issue in your instance. "
-        "I can only account for what occurs on this base.\n");
-    } else if (empty == 1) {
-        s = sdsnew(
-        "Hi Sam, this instance is empty or is using very little memory, "
-        "my issues detector can't be used in these conditions. "
-        "Please, leave for your mission on Earth and fill it with some data. "
-        "The new Sam and I will be back to our programming as soon as I "
-        "finished rebooting.\n");
-    } else {
-        s = sdsnew("Sam, I detected a few issues in this Redis instance memory implants:\n\n");
-        if (big_peak) {
-            s = sdscat(s," * Peak memory: In the past this instance used more than 150% the memory that is currently using. The allocator is normally not able to release memory after a peak, so you can expect to see a big fragmentation ratio, however this is actually harmless and is only due to the memory peak, and if the Redis instance Resident Set Size (RSS) is currently bigger than expected, the memory will be used as soon as you fill the Redis instance with more data. If the memory peak was only occasional and you want to try to reclaim memory, please try the MEMORY PURGE command, otherwise the only other option is to shutdown and restart the instance.\n\n");
-        }
-        if (high_frag) {
-            s = sdscatprintf(s," * High fragmentation: This instance has a memory fragmentation greater than 1.4 (this means that the Resident Set Size of the Redis process is much larger than the sum of the logical allocations Redis performed). This problem is usually due either to a large peak memory (check if there is a peak memory entry above in the report) or may result from a workload that causes the allocator to fragment memory a lot. If the problem is a large peak memory, then there is no issue. Otherwise, make sure you are using the Jemalloc allocator and not the default libc malloc. Note: The currently used allocator is \"%s\".\n\n", ZMALLOC_LIB);
-        }
-        if (big_slave_buf) {
-            s = sdscat(s," * Big slave buffers: The slave output buffers in this instance are greater than 10MB for each slave (on average). This likely means that there is some slave instance that is struggling receiving data, either because it is too slow or because of networking issues. As a result, data piles on the master output buffers. Please try to identify what slave is not receiving data correctly and why. You can use the INFO output in order to check the slaves delays and the CLIENT LIST command to check the output buffers of each slave.\n\n");
-        }
-        if (big_client_buf) {
-            s = sdscat(s," * Big client buffers: The clients output buffers in this instance are greater than 200K per client (on average). This may result from different causes, like Pub/Sub clients subscribed to channels bot not receiving data fast enough, so that data piles on the Redis instance output buffer, or clients sending commands with large replies or very large sequences of commands in the same pipeline. Please use the CLIENT LIST command in order to investigate the issue if it causes problems in your instance, or to understand better why certain clients are using a big amount of memory.\n\n");
-        }
-        s = sdscat(s,"I'm here to keep you safe, Sam. I want to help you.\n");
-    }
-    freeMemoryOverheadData(mh);
-    return s;
-}
-
-/* ======================= The OBJECT and MEMORY commands =================== */
-
 /* This is a helper function for the OBJECT command. We need to lookup keys
  * without any modification of LRU or other parameters. */
 robj *objectCommandLookup(client *c, robj *key) {
@@ -1027,158 +721,9 @@ void objectCommand(client *c) {
     } else if (!strcasecmp(c->argv[1]->ptr,"idletime") && c->argc == 3) {
         if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nullbulk))
                 == NULL) return;
-        if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
-            addReplyError(c,"An LFU maxmemory policy is selected, idle time not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.");
-            return;
-        }
         addReplyLongLong(c,estimateObjectIdleTime(o)/1000);
-    } else if (!strcasecmp(c->argv[1]->ptr,"freq") && c->argc == 3) {
-        if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nullbulk))
-                == NULL) return;
-        if (server.maxmemory_policy & MAXMEMORY_FLAG_LRU) {
-            addReplyError(c,"An LRU maxmemory policy is selected, access frequency not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.");
-            return;
-        }
-        addReplyLongLong(c,o->lru&255);
     } else {
-        addReplyError(c,"Syntax error. Try OBJECT (refcount|encoding|idletime|freq)");
+        addReplyError(c,"Syntax error. Try OBJECT (refcount|encoding|idletime)");
     }
 }
 
-/* The memory command will eventually be a complete interface for the
- * memory introspection capabilities of Redis.
- *
- * Usage: MEMORY usage <key> */
-void memoryCommand(client *c) {
-    robj *o;
-
-    if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
-        long long samples = OBJ_COMPUTE_SIZE_DEF_SAMPLES;
-        for (int j = 3; j < c->argc; j++) {
-            if (!strcasecmp(c->argv[j]->ptr,"samples") &&
-                j+1 < c->argc)
-            {
-                if (getLongLongFromObjectOrReply(c,c->argv[j+1],&samples,NULL)
-                     == C_ERR) return;
-                if (samples < 0) {
-                    addReply(c,shared.syntaxerr);
-                    return;
-                }
-                if (samples == 0) samples = LLONG_MAX;;
-                j++; /* skip option argument. */
-            } else {
-                addReply(c,shared.syntaxerr);
-                return;
-            }
-        }
-        if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nullbulk))
-                == NULL) return;
-        size_t usage = objectComputeSize(o,samples);
-        usage += sdsAllocSize(c->argv[2]->ptr);
-        usage += sizeof(dictEntry);
-        addReplyLongLong(c,usage);
-    } else if (!strcasecmp(c->argv[1]->ptr,"stats") && c->argc == 2) {
-        struct redisMemOverhead *mh = getMemoryOverheadData();
-
-        addReplyMultiBulkLen(c,(14+mh->num_dbs)*2);
-
-        addReplyBulkCString(c,"peak.allocated");
-        addReplyLongLong(c,mh->peak_allocated);
-
-        addReplyBulkCString(c,"total.allocated");
-        addReplyLongLong(c,mh->total_allocated);
-
-        addReplyBulkCString(c,"startup.allocated");
-        addReplyLongLong(c,mh->startup_allocated);
-
-        addReplyBulkCString(c,"replication.backlog");
-        addReplyLongLong(c,mh->repl_backlog);
-
-        addReplyBulkCString(c,"clients.slaves");
-        addReplyLongLong(c,mh->clients_slaves);
-
-        addReplyBulkCString(c,"clients.normal");
-        addReplyLongLong(c,mh->clients_normal);
-
-        addReplyBulkCString(c,"aof.buffer");
-        addReplyLongLong(c,mh->aof_buffer);
-
-        for (size_t j = 0; j < mh->num_dbs; j++) {
-            char dbname[32];
-            snprintf(dbname,sizeof(dbname),"db.%zd",mh->db[j].dbid);
-            addReplyBulkCString(c,dbname);
-            addReplyMultiBulkLen(c,4);
-
-            addReplyBulkCString(c,"overhead.hashtable.main");
-            addReplyLongLong(c,mh->db[j].overhead_ht_main);
-
-            addReplyBulkCString(c,"overhead.hashtable.expires");
-            addReplyLongLong(c,mh->db[j].overhead_ht_expires);
-        }
-
-        addReplyBulkCString(c,"overhead.total");
-        addReplyLongLong(c,mh->overhead_total);
-
-        addReplyBulkCString(c,"keys.count");
-        addReplyLongLong(c,mh->total_keys);
-
-        addReplyBulkCString(c,"keys.bytes-per-key");
-        addReplyLongLong(c,mh->bytes_per_key);
-
-        addReplyBulkCString(c,"dataset.bytes");
-        addReplyLongLong(c,mh->dataset);
-
-        addReplyBulkCString(c,"dataset.percentage");
-        addReplyDouble(c,mh->dataset_perc);
-
-        addReplyBulkCString(c,"peak.percentage");
-        addReplyDouble(c,mh->peak_perc);
-
-        addReplyBulkCString(c,"fragmentation");
-        addReplyDouble(c,mh->fragmentation);
-
-        freeMemoryOverheadData(mh);
-    } else if (!strcasecmp(c->argv[1]->ptr,"malloc-stats") && c->argc == 2) {
-#if defined(USE_JEMALLOC)
-        sds info = sdsempty();
-        je_malloc_stats_print(inputCatSds, &info, NULL);
-        addReplyBulkSds(c, info);
-#else
-        addReplyBulkCString(c,"Stats not supported for the current allocator");
-#endif
-    } else if (!strcasecmp(c->argv[1]->ptr,"doctor") && c->argc == 2) {
-        sds report = getMemoryDoctorReport();
-        addReplyBulkSds(c,report);
-    } else if (!strcasecmp(c->argv[1]->ptr,"purge") && c->argc == 2) {
-#if defined(USE_JEMALLOC)
-        char tmp[32];
-        unsigned narenas = 0;
-        size_t sz = sizeof(unsigned);
-        if (!je_mallctl("arenas.narenas", &narenas, &sz, NULL, 0)) {
-            sprintf(tmp, "arena.%d.purge", narenas);
-            if (!je_mallctl(tmp, NULL, 0, NULL, 0)) {
-                addReply(c, shared.ok);
-                return;
-            }
-        }
-        addReplyError(c, "Error purging dirty pages");
-#else
-        addReply(c, shared.ok);
-        /* Nothing to do for other allocators. */
-#endif
-    } else if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc == 2) {
-        addReplyMultiBulkLen(c,5);
-        addReplyBulkCString(c,
-"MEMORY DOCTOR                        - Outputs memory problems report");
-        addReplyBulkCString(c,
-"MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key");
-        addReplyBulkCString(c,
-"MEMORY STATS                         - Show memory usage details");
-        addReplyBulkCString(c,
-"MEMORY PURGE                         - Ask the allocator to release memory");
-        addReplyBulkCString(c,
-"MEMORY MALLOC-STATS                  - Show allocator internal stats");
-    } else {
-        addReplyError(c,"Syntax error. Try MEMORY HELP");
-    }
-}

src/quicklist.c

@@ -149,7 +149,7 @@ REDIS_STATIC quicklistNode *quicklistCreateNode(void) {
 }
 
 /* Return cached quicklist count */
-unsigned int quicklistCount(const quicklist *ql) { return ql->count; }
+unsigned int quicklistCount(quicklist *ql) { return ql->count; }
 
 /* Free entire quicklist. */
 void quicklistRelease(quicklist *quicklist) {
@@ -1192,12 +1192,12 @@ quicklist *quicklistDup(quicklist *orig) {
          current = current->next) {
         quicklistNode *node = quicklistCreateNode();
 
-        if (current->encoding == QUICKLIST_NODE_ENCODING_LZF) {
-            quicklistLZF *lzf = (quicklistLZF *)current->zl;
+        if (node->encoding == QUICKLIST_NODE_ENCODING_LZF) {
+            quicklistLZF *lzf = (quicklistLZF *)node->zl;
             size_t lzf_sz = sizeof(*lzf) + lzf->sz;
             node->zl = zmalloc(lzf_sz);
             memcpy(node->zl, current->zl, lzf_sz);
-        } else if (current->encoding == QUICKLIST_NODE_ENCODING_RAW) {
+        } else if (node->encoding == QUICKLIST_NODE_ENCODING_RAW) {
             node->zl = zmalloc(current->sz);
             memcpy(node->zl, current->zl, current->sz);
         }

src/quicklist.h

@@ -154,7 +154,7 @@ int quicklistPopCustom(quicklist *quicklist, int where, unsigned char **data,
                        void *(*saver)(unsigned char *data, unsigned int sz));
 int quicklistPop(quicklist *quicklist, int where, unsigned char **data,
                  unsigned int *sz, long long *slong);
-unsigned int quicklistCount(const quicklist *ql);
+unsigned int quicklistCount(quicklist *ql);
 int quicklistCompare(unsigned char *p1, unsigned char *p2, int p2_len);
 size_t quicklistGetLzf(const quicklistNode *node, void **data);
 

src/rax.h

@@ -1,160 +0,0 @@
-#ifndef RAX_H
-#define RAX_H
-
-#include <stdint.h>
-
-/* Representation of a radix tree as implemented in this file, that contains
- * the strings "foo", "foobar" and "footer" after the insertion of each
- * word. When the node represents a key inside the radix tree, we write it
- * between [], otherwise it is written between ().
- *
- * This is the vanilla representation:
- *
- *              (f) ""
- *                \
- *                (o) "f"
- *                  \
- *                  (o) "fo"
- *                    \
- *                  [t   b] "foo"
- *                  /     \
- *         "foot" (e)     (a) "foob"
- *                /         \
- *      "foote" (r)         (r) "fooba"
- *              /             \
- *    "footer" []             [] "foobar"
- *
- * However, this implementation implements a very common optimization where
- * successive nodes having a single child are "compressed" into the node
- * itself as a string of characters, each representing a next-level child,
- * and only the link to the node representing the last character node is
- * provided inside the representation. So the above representation is turend
- * into:
- *
- *                  ["foo"] ""
- *                     |
- *                  [t   b] "foo"
- *                  /     \
- *        "foot" ("er")    ("ar") "foob"
- *                 /          \
- *       "footer" []          [] "foobar"
- *
- * However this optimization makes the implementation a bit more complex.
- * For instance if a key "first" is added in the above radix tree, a
- * "node splitting" operation is needed, since the "foo" prefix is no longer
- * composed of nodes having a single child one after the other. This is the
- * above tree and the resulting node splitting after this event happens:
- *
- *
- *                    (f) ""
- *                    /
- *                 (i o) "f"
- *                 /   \
- *    "firs"  ("rst")  (o) "fo"
- *              /        \
- *    "first" []       [t   b] "foo"
- *                     /     \
- *           "foot" ("er")    ("ar") "foob"
- *                    /          \
- *          "footer" []          [] "foobar"
- *
- * Similarly after deletion, if a new chain of nodes having a single child
- * is created (the chain must also not include nodes that represent keys),
- * it must be compressed back into a single node.
- *
- */
-
-#define RAX_NODE_MAX_SIZE ((1<<29)-1)
-typedef struct raxNode {
-    uint32_t iskey:1;     /* Does this node contain a key? */
-    uint32_t isnull:1;    /* Associated value is NULL (don't store it). */
-    uint32_t iscompr:1;   /* Node is compressed. */
-    uint32_t size:29;     /* Number of children, or compressed string len. */
-    /* Data layout is as follows:
-     *
-     * If node is not compressed we have 'size' bytes, one for each children
-     * character, and 'size' raxNode pointers, point to each child node.
-     * Note how the character is not stored in the children but in the
-     * edge of the parents:
-     *
-     * [header strlen=0][abc][a-ptr][b-ptr][c-ptr](value-ptr?)
-     *
-     * if node is compressed (strlen != 0) the node has 1 children.
-     * In that case the 'size' bytes of the string stored immediately at
-     * the start of the data section, represent a sequence of successive
-     * nodes linked one after the other, for which only the last one in
-     * the sequence is actually represented as a node, and pointed to by
-     * the current compressed node.
-     *
-     * [header strlen=3][xyz][z-ptr](value-ptr?)
-     *
-     * Both compressed and not compressed nodes can represent a key
-     * with associated data in the radix tree at any level (not just terminal
-     * nodes).
-     *
-     * If the node has an associated key (iskey=1) and is not NULL
-     * (isnull=0), then after the raxNode pointers poiting to the
-     * childen, an additional value pointer is present (as you can see
-     * in the representation above as "value-ptr" field).
-     */
-    unsigned char data[];
-} raxNode;
-
-typedef struct rax {
-    raxNode *head;
-    uint64_t numele;
-    uint64_t numnodes;
-} rax;
-
-/* Stack data structure used by raxLowWalk() in order to, optionally, return
- * a list of parent nodes to the caller. The nodes do not have a "parent"
- * field for space concerns, so we use the auxiliary stack when needed. */
-#define RAX_STACK_STATIC_ITEMS 32
-typedef struct raxStack {
-    void **stack; /* Points to static_items or an heap allocated array. */
-    size_t items, maxitems; /* Number of items contained and total space. */
-    /* Up to RAXSTACK_STACK_ITEMS items we avoid to allocate on the heap
-     * and use this static array of pointers instead. */
-    void *static_items[RAX_STACK_STATIC_ITEMS];
-    int oom; /* True if pushing into this stack failed for OOM at some point. */
-} raxStack;
-
-/* Radix tree iterator state is encapsulated into this data structure. */
-#define RAX_ITER_STATIC_LEN 128
-#define RAX_ITER_JUST_SEEKED (1<<0) /* Iterator was just seeked. Return current
-                                       element for the first iteration and
-                                       clear the flag. */
-#define RAX_ITER_EOF (1<<1)    /* End of iteration reached. */
-#define RAX_ITER_SAFE (1<<2)   /* Safe iterator, allows operations while
-                                  iterating. But it is slower. */
-typedef struct raxIterator {
-    int flags;
-    rax *rt;                /* Radix tree we are iterating. */
-    unsigned char *key;     /* The current string. */
-    void *data;             /* Data associated to this key. */
-    size_t key_len;         /* Current key length. */
-    size_t key_max;         /* Max key len the current key buffer can hold. */
-    unsigned char key_static_string[RAX_ITER_STATIC_LEN];
-    raxNode *node;          /* Current node. Only for unsafe iteration. */
-    raxStack stack;         /* Stack used for unsafe iteration. */
-} raxIterator;
-
-/* A special pointer returned for not found items. */
-extern void *raxNotFound;
-
-/* Exported API. */
-rax *raxNew(void);
-int raxInsert(rax *rax, unsigned char *s, size_t len, void *data, void **old);
-int raxRemove(rax *rax, unsigned char *s, size_t len, void **old);
-void *raxFind(rax *rax, unsigned char *s, size_t len);
-void raxFree(rax *rax);
-void raxStart(raxIterator *it, rax *rt);
-int raxSeek(raxIterator *it, const char *op, unsigned char *ele, size_t len);
-int raxNext(raxIterator *it);
-int raxPrev(raxIterator *it);
-int raxRandomWalk(raxIterator *it, size_t steps);
-int raxCompare(raxIterator *iter, const char *op, unsigned char *key, size_t key_len);
-void raxStop(raxIterator *it);
-void raxShow(rax *rax);
-
-#endif

src/rax_malloc.h

@@ -1,44 +0,0 @@
-/* Rax -- A radix tree implementation.
- *
- * Copyright (c) 2017, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-/* Allocator selection.
- *
- * This file is used in order to change the Rax allocator at compile time.
- * Just define the following defines to what you want to use. Also add
- * the include of your alternate allocator if needed (not needed in order
- * to use the default libc allocator). */
-
-#ifndef RAX_ALLOC_H
-#define RAX_ALLOC_H
-#include "zmalloc.h"
-#define rax_malloc zmalloc
-#define rax_realloc zrealloc
-#define rax_free zfree
-#endif

src/rdb.h

@@ -38,17 +38,16 @@
 
 /* The current RDB version. When the format changes in a way that is no longer
  * backward compatible this number gets incremented. */
-#define RDB_VERSION 8
+#define RDB_VERSION 7
 
 /* Defines related to the dump file format. To store 32 bits lengths for short
  * keys requires a lot of space, so we check the most significant 2 bits of
  * the first byte to interpreter the length:
  *
- * 00|XXXXXX => if the two MSB are 00 the len is the 6 bits of this byte
- * 01|XXXXXX XXXXXXXX =>  01, the len is 14 byes, 6 bits + 8 bits of next byte
- * 10|000000 [32 bit integer] => A full 32 bit len in net byte order will follow
- * 10|000001 [64 bit integer] => A full 64 bit len in net byte order will follow
- * 11|OBKIND this means: specially encoded object will follow. The six bits
+ * 00|000000 => if the two MSB are 00 the len is the 6 bits of this byte
+ * 01|000000 00000000 =>  01, the len is 14 byes, 6 bits + 8 bits of next byte
+ * 10|000000 [32 bit integer] => if it's 01, a full 32 bit len will follow
+ * 11|000000 this means: specially encoded object will follow. The six bits
  *           number specify the kind of object that follows.
  *           See the RDB_ENC_* defines.
  *
@@ -56,13 +55,12 @@
  * values, will fit inside. */
 #define RDB_6BITLEN 0
 #define RDB_14BITLEN 1
-#define RDB_32BITLEN 0x80
-#define RDB_64BITLEN 0x81
+#define RDB_32BITLEN 2
 #define RDB_ENCVAL 3
-#define RDB_LENERR UINT64_MAX
+#define RDB_LENERR UINT_MAX
 
 /* When a length of a string object stored on disk has the first two bits
- * set, the remaining six bits specify a special encoding for the object
+ * set, the remaining two bits specify a special encoding for the object
  * accordingly to the following defines: */
 #define RDB_ENC_INT8 0        /* 8 bit signed integer */
 #define RDB_ENC_INT16 1       /* 16 bit signed integer */
@@ -76,10 +74,6 @@
 #define RDB_TYPE_SET    2
 #define RDB_TYPE_ZSET   3
 #define RDB_TYPE_HASH   4
-#define RDB_TYPE_ZSET_2 5 /* ZSET version 2 with doubles stored in binary. */
-#define RDB_TYPE_MODULE 6
-#define RDB_TYPE_MODULE_2 7 /* Module value with annotations for parsing without
-                               the generating module being loaded. */
 /* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdbIsObjectType() BELOW */
 
 /* Object types for encoded objects. */
@@ -92,7 +86,7 @@
 /* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdbIsObjectType() BELOW */
 
 /* Test if a type is an object type. */
-#define rdbIsObjectType(t) ((t >= 0 && t <= 7) || (t >= 9 && t <= 14))
+#define rdbIsObjectType(t) ((t >= 0 && t <= 4) || (t >= 9 && t <= 14))
 
 /* Special RDB opcodes (saved/loaded with rdbSaveType/rdbLoadType). */
 #define RDB_OPCODE_AUX        250
@@ -102,51 +96,24 @@
 #define RDB_OPCODE_SELECTDB   254
 #define RDB_OPCODE_EOF        255
 
-/* Module serialized values sub opcodes */
-#define RDB_MODULE_OPCODE_EOF   0   /* End of module value. */
-#define RDB_MODULE_OPCODE_SINT  1   /* Signed integer. */
-#define RDB_MODULE_OPCODE_UINT  2   /* Unsigned integer. */
-#define RDB_MODULE_OPCODE_FLOAT 3   /* Float. */
-#define RDB_MODULE_OPCODE_DOUBLE 4  /* Double. */
-#define RDB_MODULE_OPCODE_STRING 5  /* String. */
-
-/* rdbLoad...() functions flags. */
-#define RDB_LOAD_NONE   0
-#define RDB_LOAD_ENC    (1<<0)
-#define RDB_LOAD_PLAIN  (1<<1)
-#define RDB_LOAD_SDS    (1<<2)
-
-#define RDB_SAVE_NONE 0
-#define RDB_SAVE_AOF_PREAMBLE (1<<0)
-
 int rdbSaveType(rio *rdb, unsigned char type);
 int rdbLoadType(rio *rdb);
 int rdbSaveTime(rio *rdb, time_t t);
 time_t rdbLoadTime(rio *rdb);
-int rdbSaveLen(rio *rdb, uint64_t len);
-uint64_t rdbLoadLen(rio *rdb, int *isencoded);
-int rdbLoadLenByRef(rio *rdb, int *isencoded, uint64_t *lenptr);
+int rdbSaveLen(rio *rdb, uint32_t len);
+uint32_t rdbLoadLen(rio *rdb, int *isencoded);
 int rdbSaveObjectType(rio *rdb, robj *o);
 int rdbLoadObjectType(rio *rdb);
-int rdbLoad(char *filename, rdbSaveInfo *rsi);
-int rdbSaveBackground(char *filename, rdbSaveInfo *rsi);
-int rdbSaveToSlavesSockets(rdbSaveInfo *rsi);
+int rdbLoad(char *filename);
+int rdbSaveBackground(char *filename);
+int rdbSaveToSlavesSockets(void);
 void rdbRemoveTempFile(pid_t childpid);
-int rdbSave(char *filename, rdbSaveInfo *rsi);
+int rdbSave(char *filename);
 ssize_t rdbSaveObject(rio *rdb, robj *o);
 size_t rdbSavedObjectLen(robj *o);
 robj *rdbLoadObject(int type, rio *rdb);
 void backgroundSaveDoneHandler(int exitcode, int bysignal);
 int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val, long long expiretime, long long now);
 robj *rdbLoadStringObject(rio *rdb);
-int rdbSaveStringObject(rio *rdb, robj *obj);
-ssize_t rdbSaveRawString(rio *rdb, unsigned char *s, size_t len);
-void *rdbGenericLoadStringObject(rio *rdb, int flags, size_t *lenptr);
-int rdbSaveBinaryDoubleValue(rio *rdb, double val);
-int rdbLoadBinaryDoubleValue(rio *rdb, double *val);
-int rdbSaveBinaryFloatValue(rio *rdb, float val);
-int rdbLoadBinaryFloatValue(rio *rdb, float *val);
-int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi);
-rdbSaveInfo *rdbPopulateSaveInfo(rdbSaveInfo *rsi);
 
 #endif

src/redis-benchmark.c

@@ -565,15 +565,15 @@ invalid:
 
 usage:
     printf(
-"Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests>] [-k <boolean>]\n\n"
+"Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>]\n\n"
 " -h <hostname>      Server hostname (default 127.0.0.1)\n"
 " -p <port>          Server port (default 6379)\n"
 " -s <socket>        Server socket (overrides host and port)\n"
 " -a <password>      Password for Redis Auth\n"
 " -c <clients>       Number of parallel connections (default 50)\n"
 " -n <requests>      Total number of requests (default 100000)\n"
-" -d <size>          Data size of SET/GET value in bytes (default 3)\n"
-" --dbnum <db>       SELECT the specified db number (default 0)\n"
+" -d <size>          Data size of SET/GET value in bytes (default 2)\n"
+" -dbnum <db>        SELECT the specified db number (default 0)\n"
 " -k <boolean>       1=keep alive 0=reconnect (default 1)\n"
 " -r <keyspacelen>   Use random keys for SET/GET/INCR, random values for SADD\n"
 "  Using this option the benchmark will expand the string __rand_int__\n"
@@ -779,13 +779,6 @@ int main(int argc, const char **argv) {
             free(cmd);
         }
 
-        if (test_is_selected("hset")) {
-            len = redisFormatCommand(&cmd,
-                "HSET myset:__rand_int__ element:__rand_int__ %s",data);
-            benchmark("HSET",cmd,len);
-            free(cmd);
-        }
-
         if (test_is_selected("spop")) {
             len = redisFormatCommand(&cmd,"SPOP myset");
             benchmark("SPOP",cmd,len);

src/redis-check-aof.c

@@ -28,8 +28,13 @@
  * POSSIBILITY OF SUCH DAMAGE.
  */
 
-#include "server.h"
+#include "fmacros.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
 #include <sys/stat.h>
+#include "config.h"
 
 #define ERROR(...) { \
     char __buf[1024]; \
@@ -55,7 +60,7 @@ int readLong(FILE *fp, char prefix, long *target) {
         return 0;
     }
     if (buf[0] != prefix) {
-        ERROR("Expected prefix '%c', got: '%c'",prefix,buf[0]);
+        ERROR("Expected prefix '%c', got: '%c'",buf[0],prefix);
         return 0;
     }
     *target = strtol(buf+1,&eptr,10);
@@ -82,7 +87,7 @@ int readString(FILE *fp, char** target) {
 
     /* Increase length to also consume \r\n */
     len += 2;
-    *target = (char*)zmalloc(len);
+    *target = (char*)malloc(len);
     if (!readBytes(fp,*target,len)) {
         return 0;
     }
@@ -122,12 +127,12 @@ off_t process(FILE *fp) {
                     }
                 }
             }
-            zfree(str);
+            free(str);
         }
 
         /* Stop if the loop did not finish */
         if (i < argc) {
-            if (str) zfree(str);
+            if (str) free(str);
             break;
         }
     }
@@ -141,7 +146,7 @@ off_t process(FILE *fp) {
     return pos;
 }
 
-int redis_check_aof_main(int argc, char **argv) {
+int main(int argc, char **argv) {
     char *filename;
     int fix = 0;
 
@@ -180,25 +185,6 @@ int redis_check_aof_main(int argc, char **argv) {
         exit(1);
     }
 
-    /* This AOF file may have an RDB preamble. Check this to start, and if this
-     * is the case, start processing the RDB part. */
-    if (size >= 8) {    /* There must be at least room for the RDB header. */
-        char sig[5];
-        int has_preamble = fread(sig,sizeof(sig),1,fp) == 1 &&
-                            memcmp(sig,"REDIS",sizeof(sig)) == 0;
-        rewind(fp);
-        if (has_preamble) {
-            printf("The AOF appears to start with an RDB preamble.\n"
-                   "Checking the RDB preamble to start:\n");
-            if (redis_check_rdb_main(argc,argv,fp) == C_ERR) {
-                printf("RDB preamble of AOF file is not sane, aborting.\n");
-                exit(1);
-            } else {
-                printf("RDB preamble is OK, proceeding with AOF tail...\n");
-            }
-        }
-    }
-
     off_t pos = process(fp);
     off_t diff = size-pos;
     printf("AOF analyzed: size=%lld, ok_up_to=%lld, diff=%lld\n",
@@ -220,8 +206,7 @@ int redis_check_aof_main(int argc, char **argv) {
                 printf("Successfully truncated AOF\n");
             }
         } else {
-            printf("AOF is not valid. "
-                   "Use the --fix option to try fixing it.\n");
+            printf("AOF is not valid\n");
             exit(1);
         }
     } else {
@@ -229,5 +214,5 @@ int redis_check_aof_main(int argc, char **argv) {
     }
 
     fclose(fp);
-    exit(0);
+    return 0;
 }

src/redis-check-rdb.c

@@ -172,19 +172,16 @@ void rdbCheckSetupSignals(void) {
     sigaction(SIGILL, &act, NULL);
 }
 
-/* Check the specified RDB file. Return 0 if the RDB looks sane, otherwise
- * 1 is returned.
- * The file is specified as a filename in 'rdbfilename' if 'fp' is not NULL,
- * otherwise the already open file 'fp' is checked. */
-int redis_check_rdb(char *rdbfilename, FILE *fp) {
+/* Check the specified RDB file. */
+int redis_check_rdb(char *rdbfilename) {
     uint64_t dbid;
     int type, rdbver;
     char buf[1024];
     long long expiretime, now = mstime();
-    static rio rdb; /* Pointed by global struct riostate. */
+    FILE *fp;
+    rio rdb;
 
-    int closefile = (fp == NULL);
-    if (fp == NULL && (fp = fopen(rdbfilename,"r")) == NULL) return 1;
+    if ((fp = fopen(rdbfilename,"r")) == NULL) return C_ERR;
 
     rioInitWithFile(&rdb,fp);
     rdbstate.rio = &rdb;
@@ -312,7 +309,7 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
         }
     }
 
-    if (closefile) fclose(fp);
+    fclose(fp);
     return 0;
 
 eoferr: /* unexpected end of file is handled here with a fatal exit */
@@ -325,36 +322,24 @@ eoferr: /* unexpected end of file is handled here with a fatal exit */
 }
 
 /* RDB check main: called form redis.c when Redis is executed with the
- * redis-check-rdb alias, on during RDB loading errors.
+ * redis-check-rdb alias.
  *
- * The function works in two ways: can be called with argc/argv as a
- * standalone executable, or called with a non NULL 'fp' argument if we
- * already have an open file to check. This happens when the function
- * is used to check an RDB preamble inside an AOF file.
- *
- * When called with fp = NULL, the function never returns, but exits with the
- * status code according to success (RDB is sane) or error (RDB is corrupted).
- * Otherwise if called with a non NULL fp, the function returns C_OK or
- * C_ERR depending on the success or failure. */
-int redis_check_rdb_main(int argc, char **argv, FILE *fp) {
-    if (argc != 2 && fp == NULL) {
+ * The function never returns, but exits with the status code according
+ * to success (RDB is sane) or error (RDB is corrupted). */
+int redis_check_rdb_main(int argc, char **argv) {
+    if (argc != 2) {
         fprintf(stderr, "Usage: %s <rdb-file-name>\n", argv[0]);
         exit(1);
     }
-    /* In order to call the loading functions we need to create the shared
-     * integer objects, however since this function may be called from
-     * an already initialized Redis instance, check if we really need to. */
-    if (shared.integers[0] == NULL)
-        createSharedObjects();
+    createSharedObjects(); /* Needed for loading. */
     server.loading_process_events_interval_bytes = 0;
     rdbCheckMode = 1;
     rdbCheckInfo("Checking RDB file %s", argv[1]);
     rdbCheckSetupSignals();
-    int retval = redis_check_rdb(argv[1],fp);
+    int retval = redis_check_rdb(argv[1]);
     if (retval == 0) {
         rdbCheckInfo("\\o/ RDB looks OK! \\o/");
         rdbShowGenericInfo();
     }
-    if (fp) return (retval == 0) ? C_OK : C_ERR;
     exit(retval);
 }

src/redis-cli.c

@@ -275,10 +275,6 @@ static void cliIntegrateHelp(void) {
      * don't already match what we have. */
     for (size_t j = 0; j < reply->elements; j++) {
         redisReply *entry = reply->element[j];
-        if (entry->type != REDIS_REPLY_ARRAY || entry->elements < 4 ||
-            entry->element[0]->type != REDIS_REPLY_STRING ||
-            entry->element[1]->type != REDIS_REPLY_INTEGER ||
-            entry->element[3]->type != REDIS_REPLY_INTEGER) return;
         char *cmdname = entry->element[0]->str;
         int i;
 
@@ -340,7 +336,7 @@ static void cliOutputGenericHelp(void) {
         "      \"help <tab>\" to get a list of possible help topics\n"
         "      \"quit\" to exit\n"
         "\n"
-        "To set redis-cli preferences:\n"
+        "To set redis-cli perferences:\n"
         "      \":set hints\" enable online hints\n"
         "      \":set nohints\" disable online hints\n"
         "Set your preferences in ~/.redisclirc\n",
@@ -632,6 +628,7 @@ sds sdscatcolor(sds o, char *s, size_t len, char *color) {
     int bold = strstr(color,"bold") != NULL;
     int ccode = 37; /* Defaults to white. */
     if (strstr(color,"red")) ccode = 31;
+    else if (strstr(color,"red")) ccode = 31;
     else if (strstr(color,"green")) ccode = 32;
     else if (strstr(color,"yellow")) ccode = 33;
     else if (strstr(color,"blue")) ccode = 34;
@@ -846,10 +843,8 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
     output_raw = 0;
     if (!strcasecmp(command,"info") ||
         (argc >= 2 && !strcasecmp(command,"debug") &&
-                       !strcasecmp(argv[1],"htstats")) ||
-        (argc >= 2 && !strcasecmp(command,"memory") &&
-                      (!strcasecmp(argv[1],"malloc-stats") ||
-                       !strcasecmp(argv[1],"doctor"))) ||
+                      ((!strcasecmp(argv[1],"jemalloc") && !strcasecmp(argv[2],"info")) ||
+                       !strcasecmp(argv[1],"htstats"))) ||
         (argc == 2 && !strcasecmp(command,"cluster") &&
                       (!strcasecmp(argv[1],"nodes") ||
                        !strcasecmp(argv[1],"info"))) ||
@@ -1122,12 +1117,6 @@ static void usage(void) {
 "  --csv              Output in CSV format.\n"
 "  --stat             Print rolling stats about server: mem, clients, ...\n"
 "  --latency          Enter a special mode continuously sampling latency.\n"
-"                     If you use this mode in an interactive session it runs\n"
-"                     forever displaying real-time stats. Otherwise if --raw or\n"
-"                     --csv is specified, or if you redirect the output to a non\n"
-"                     TTY, it samples the latency for 1 second (you can use\n"
-"                     -i to change the interval), then produces a single output\n"
-"                     and exits.\n"
 "  --latency-history  Like --latency but tracking latency changes over time.\n"
 "                     Default time interval is 15 sec. Change it using -i.\n"
 "  --latency-dist     Shows latency as a spectrum, requires xterm 256 colors.\n"
@@ -1231,7 +1220,7 @@ static sds *cliSplitArgs(char *line, int *argc) {
     }
 }
 
-/* Set the CLI preferences. This function is invoked when an interactive
+/* Set the CLI perferences. This function is invoked when an interactive
  * ":command" is called, or when reading ~/.redisclirc file, in order to
  * set user preferences. */
 void cliSetPreferences(char **argv, int argc, int interactive) {
@@ -1266,7 +1255,6 @@ void cliLoadPreferences(void) {
             if (argc > 0) cliSetPreferences(argv,argc,0);
             sdsfreesplitres(argv,argc);
         }
-        fclose(fp);
     }
     sdsfree(rcfile);
 }
@@ -1278,11 +1266,6 @@ static void repl(void) {
     int argc;
     sds *argv;
 
-    /* Initialize the help and, if possible, use the COMMAND command in order
-     * to retrieve missing entries. */
-    cliInitHelp();
-    cliIntegrateHelp();
-
     config.interactive = 1;
     linenoiseSetMultiLine(1);
     linenoiseSetCompletionCallback(completionCallback);
@@ -1337,10 +1320,9 @@ static void repl(void) {
                 } else {
                     long long start_time = mstime(), elapsed;
                     int repeat, skipargs = 0;
-                    char *endptr;
 
-                    repeat = strtol(argv[0], &endptr, 10);
-                    if (argc > 1 && *endptr == '\0' && repeat) {
+                    repeat = atoi(argv[0]);
+                    if (argc > 1 && repeat) {
                         skipargs = 1;
                     } else {
                         repeat = 1;
@@ -1359,9 +1341,7 @@ static void repl(void) {
                     }
 
                     elapsed = mstime()-start_time;
-                    if (elapsed >= 500 &&
-                        config.output == OUTPUT_STANDARD)
-                    {
+                    if (elapsed >= 500) {
                         printf("(%.2fs)\n",(double)elapsed/1000);
                     }
                 }
@@ -1478,18 +1458,6 @@ static int evalMode(int argc, char **argv) {
  * Latency and latency history modes
  *--------------------------------------------------------------------------- */
 
-static void latencyModePrint(long long min, long long max, double avg, long long count) {
-    if (config.output == OUTPUT_STANDARD) {
-        printf("min: %lld, max: %lld, avg: %.2f (%lld samples)",
-                min, max, avg, count);
-        fflush(stdout);
-    } else if (config.output == OUTPUT_CSV) {
-        printf("%lld,%lld,%.2f,%lld\n", min, max, avg, count);
-    } else if (config.output == OUTPUT_RAW) {
-        printf("%lld %lld %.2f %lld\n", min, max, avg, count);
-    }
-}
-
 #define LATENCY_SAMPLE_RATE 10 /* milliseconds. */
 #define LATENCY_HISTORY_DEFAULT_INTERVAL 15000 /* milliseconds. */
 static void latencyMode(void) {
@@ -1501,14 +1469,6 @@ static void latencyMode(void) {
     double avg;
     long long history_start = mstime();
 
-    /* Set a default for the interval in case of --latency option
-     * with --raw, --csv or when it is redirected to non tty. */
-    if (config.interval == 0) {
-        config.interval = 1000;
-    } else {
-        config.interval /= 1000; /* We need to convert to milliseconds. */
-    }
-
     if (!context) exit(1);
     while(1) {
         start = mstime();
@@ -1529,19 +1489,9 @@ static void latencyMode(void) {
             tot += latency;
             avg = (double) tot/count;
         }
-
-        if (config.output == OUTPUT_STANDARD) {
-            printf("\x1b[0G\x1b[2K"); /* Clear the line. */
-            latencyModePrint(min,max,avg,count);
-        } else {
-            if (config.latency_history) {
-                latencyModePrint(min,max,avg,count);
-            } else if (mstime()-history_start > config.interval) {
-                latencyModePrint(min,max,avg,count);
-                exit(0);
-            }
-        }
-
+        printf("\x1b[0G\x1b[2Kmin: %lld, max: %lld, avg: %.2f (%lld samples)",
+            min, max, avg, count);
+        fflush(stdout);
         if (config.latency_history && mstime()-history_start > history_interval)
         {
             printf(" -- %.2f seconds range\n", (float)(mstime()-history_start)/1000);
@@ -1806,7 +1756,6 @@ static void getRDB(void) {
     }
     close(s); /* Close the file descriptor ASAP as fsync() may take time. */
     fsync(fd);
-    close(fd);
     fprintf(stderr,"Transfer finished with success.\n");
     exit(0);
 }
@@ -2063,13 +2012,8 @@ static void getKeyTypes(redisReply *keys, int *types) {
                 keys->element[i]->str, context->err, context->errstr);
             exit(1);
         } else if(reply->type != REDIS_REPLY_STATUS) {
-            if(reply->type == REDIS_REPLY_ERROR) {
-                fprintf(stderr, "TYPE returned an error: %s\n", reply->str);
-            } else {
-                fprintf(stderr,
-                    "Invalid reply type (%d) for TYPE on key '%s'!\n",
-                    reply->type, keys->element[i]->str);
-            }
+            fprintf(stderr, "Invalid reply type (%d) for TYPE on key '%s'!\n",
+                reply->type, keys->element[i]->str);
             exit(1);
         }
 
@@ -2458,7 +2402,7 @@ long long powerLawRand(long long min, long long max, double alpha) {
 /* Generates a key name among a set of lru_test_sample_size keys, using
  * an 80-20 distribution. */
 void LRUTestGenKey(char *buf, size_t buflen) {
-    snprintf(buf, buflen, "lru:%lld",
+    snprintf(buf, buflen, "lru:%lld\n",
         powerLawRand(1, config.lru_test_sample_size, 6.2));
 }
 
@@ -2480,11 +2424,8 @@ static void LRUTestMode(void) {
         while(mstime() - start_cycle < 1000) {
             /* Write cycle. */
             for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
-                char val[6];
-                val[5] = '\0';
-                for (int i = 0; i < 5; i++) val[i] = 'A'+rand()%('z'-'A');
                 LRUTestGenKey(key,sizeof(key));
-                redisAppendCommand(context, "SET %s %s",key,val);
+                redisAppendCommand(context, "SET %s val",key);
             }
             for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++)
                 redisGetReply(context, (void**)&reply);
@@ -2655,6 +2596,11 @@ int main(int argc, char **argv) {
     argc -= firstarg;
     argv += firstarg;
 
+    /* Initialize the help and, if possible, use the COMMAND command in order
+     * to retrieve missing entries. */
+    cliInitHelp();
+    cliIntegrateHelp();
+
     /* Latency mode */
     if (config.latency_mode) {
         if (cliConnect(0) == REDIS_ERR) exit(1);