Compare commits

..
4 Commits
78 changed files with 2120 additions and 4690 deletions
-4
View File
@@ -1,8 +1,5 @@
.*.swp
*.o
*.xo
*.so
*.d
*.log
dump.rdb
redis-benchmark
@@ -31,4 +28,3 @@ deps/lua/src/liblua.a
.prerequisites
*.dSYM
Makefile.dep
.vscode/*
-18
View File
@@ -35,11 +35,6 @@ It is as simple as:
% make
To build with TLS support, you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu) and run:
% make BUILD_TLS=yes
You can run a 32 bit Redis binary using:
% make 32bit
@@ -48,13 +43,6 @@ After building Redis, it is a good idea to test it using:
% make test
If TLS is built, running the tests with TLS enabled (you will need `tcl-tls`
installed):
% ./utils/gen-test-certs.sh
% ./runtest --tls
Fixing build problems with dependencies or cached build options
---------
@@ -137,12 +125,6 @@ as options using the command line. Examples:
All the options in redis.conf are also supported as options using the command
line, with exactly the same name.
Running Redis with TLS:
------------------
Please consult the [TLS.md](TLS.md) file for more information on
how to use Redis with TLS.
Playing with Redis
------------------
+31 -14
View File
@@ -1,5 +1,8 @@
TLS Support
===========
TLS Support -- Work In Progress
===============================
This is a brief note to capture current thoughts/ideas and track pending action
items.
Getting Started
---------------
@@ -66,23 +69,37 @@ probably not be so hard. For cluster keys migration it might be more difficult,
but there are probably other good reasons to improve that part anyway.
To-Do List
----------
==========
- [ ] Add session caching support. Check if/how it's handled by clients to
assess how useful/important it is.
- [ ] redis-benchmark support. The current implementation is a mix of using
hiredis for parsing and basic networking (establishing connections), but
directly manipulating sockets for most actions. This will need to be cleaned
up for proper TLS support. The best approach is probably to migrate to hiredis
async mode.
- [ ] redis-cli `--slave` and `--rdb` support.
Additional TLS Features
-----------------------
Multi-port
----------
1. Add metrics to INFO?
2. Add session caching support. Check if/how it's handled by clients to assess
how useful/important it is.
redis-benchmark
---------------
The current implementation is a mix of using hiredis for parsing and basic
networking (establishing connections), but directly manipulating sockets for
most actions.
This will need to be cleaned up for proper TLS support. The best approach is
probably to migrate to hiredis async mode.
redis-cli
---------
1. Add support for TLS in --slave and --rdb modes.
Others
------
Consider the implications of allowing TLS to be configured on a separate port,
making Redis listening on multiple ports:
making Redis listening on multiple ports.
This impacts many things, like
1. Startup banner port notification
2. Proctitle
3. How slaves announce themselves
+26 -106
View File
@@ -155,22 +155,23 @@ tcp-keepalive 300
# tls-ca-cert-file ca.crt
# tls-ca-cert-dir /etc/ssl/certs
# By default, clients (including replica servers) on a TLS port are required
# to authenticate using valid client side certificates.
# If TLS/SSL clients are required to authenticate using a client side
# certificate, use this directive.
#
# It is possible to disable authentication using this directive.
# Note: this applies to all incoming clients, including replicas.
#
# tls-auth-clients no
# tls-auth-clients yes
# By default, a Redis replica does not attempt to establish a TLS connection
# with its master.
#
# Use the following directive to enable TLS on replication links.
# If TLS/SSL should be used when connecting as a replica to a master, enable
# this configuration directive:
#
# tls-replication yes
# By default, the Redis Cluster bus uses a plain TCP connection. To enable
# TLS for the bus protocol, use the following directive:
# If TLS/SSL should be used for the Redis Cluster bus, enable this configuration
# directive.
#
# NOTE: If TLS/SSL is enabled for Cluster Bus, mutual authentication is always
# enforced.
#
# tls-cluster yes
@@ -196,7 +197,7 @@ tcp-keepalive 300
# When choosing a cipher, use the server's preference instead of the client
# preference. By default, the server follows the client's preference.
#
# tls-prefer-server-ciphers yes
# tls-prefer-server-cipher yes
################################# GENERAL #####################################
@@ -810,13 +811,13 @@ replica-priority 100
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select one from the following behaviors:
# is reached. You can select among five behaviors:
#
# volatile-lru -> Evict using approximated LRU, only keys with an expire set.
# volatile-lru -> Evict using approximated LRU among the keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU, only keys with an expire set.
# volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key having an expire set.
# volatile-random -> Remove a random key among the ones with an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
@@ -871,23 +872,6 @@ replica-priority 100
#
# replica-ignore-maxmemory yes
# Redis reclaims expired keys in two ways: upon access when those keys are
# found to be expired, and also in background, in what is called the
# "active expire key". The key space is slowly and interactively scanned
# looking for expired keys to reclaim, so that it is possible to free memory
# of keys that are expired and will never be accessed again in a short time.
#
# The default effort of the expire cycle will try to avoid having more than
# ten percent of expired keys still in memory, and will try to avoid consuming
# more than 25% of total memory and to add latency to the system. However
# it is possible to increase the expire "effort" that is normally set to
# "1", to a greater value, up to the value "10". At its maximum value the
# system will use more CPU, longer cycles (and technically may introduce
# more latency), and will tollerate less already expired keys still present
# in the system. It's a tradeoff betweeen memory, CPU and latecy.
#
# active-expire-effort 1
############################# LAZY FREEING ####################################
# Redis has two primitives to delete keys. One is called DEL and is a blocking
@@ -937,52 +921,6 @@ lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
################################ THREADED I/O #################################
# Redis is mostly single threaded, however there are certain threaded
# operations such as UNLINK, slow I/O accesses and other things that are
# performed on side threads.
#
# Now it is also possible to handle Redis clients socket reads and writes
# in different I/O threads. Since especially writing is so slow, normally
# Redis users use pipelining in order to speedup the Redis performances per
# core, and spawn multiple instances in order to scale more. Using I/O
# threads it is possible to easily speedup two times Redis without resorting
# to pipelining nor sharding of the instance.
#
# By default threading is disabled, we suggest enabling it only in machines
# that have at least 4 or more cores, leaving at least one spare core.
# Using more than 8 threads is unlikely to help much. We also recommend using
# threaded I/O only if you actually have performance problems, with Redis
# instances being able to use a quite big percentage of CPU time, otherwise
# there is no point in using this feature.
#
# So for instance if you have a four cores boxes, try to use 2 or 3 I/O
# threads, if you have a 8 cores, try to use 6 threads. In order to
# enable I/O threads use the following configuration directive:
#
# io-threads 4
#
# Setting io-threads to 1 will just use the main thread as usually.
# When I/O threads are enabled, we only use threads for writes, that is
# to thread the write(2) syscall and transfer the client buffers to the
# socket. However it is also possible to enable threading of reads and
# protocol parsing using the following configuration directive, by setting
# it to yes:
#
# io-threads-do-reads no
#
# Usually threading reads doesn't help much.
#
# NOTE 1: This configuration directive cannot be changed at runtime via
# CONFIG SET. Aso this feature currently does not work when SSL is
# enabled.
#
# NOTE 2: If you want to test the Redis speedup using redis-benchmark, make
# sure you also run the benchmark itself in threaded mode, using the
# --threads option to match the number of Redis theads, otherwise you'll not
# be able to notice the improvements.
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
@@ -1239,22 +1177,6 @@ lua-time-limit 5000
#
# cluster-replica-no-failover no
# This option, when set to yes, allows nodes to serve read traffic while the
# the cluster is in a down state, as long as it believes it owns the slots.
#
# This is useful for two cases. The first case is for when an application
# doesn't require consistency of data during node failures or network partitions.
# One example of this is a cache, where as long as the node has the data it
# should be able to serve it.
#
# The second use case is for configurations that don't meet the recommended
# three shards but want to enable cluster mode and scale later. A
# master outage in a 1 or 2 shard configuration causes a read/write outage to the
# entire cluster without this option set, with it set there is only a write outage.
# Without a quorum of masters, slot ownership will not change automatically.
#
# cluster-allow-reads-when-down no
# In order to setup your cluster make sure to read the documentation
# available at http://redis.io web site.
@@ -1361,11 +1283,7 @@ latency-monitor-threshold 0
# z Sorted set commands
# x Expired events (events generated every time a key expires)
# e Evicted events (events generated when a key is evicted for maxmemory)
# t Stream commands
# m Key-miss events (Note: It is not included in the 'A' class)
# A Alias for g$lshzxet, so that the "AKE" string means all the events
# (Except key-miss events which are excluded from 'A' due to their
# unique nature).
# A Alias for g$lshzxe, so that the "AKE" string means all the events.
#
# The "notify-keyspace-events" takes as argument a string that is composed
# of zero or multiple characters. The empty string means that notifications
@@ -1688,6 +1606,10 @@ rdb-save-incremental-fsync yes
########################### ACTIVE DEFRAGMENTATION #######################
#
# WARNING THIS FEATURE IS EXPERIMENTAL. However it was stress tested
# even in production and manually tested by multiple engineers for some
# time.
#
# What is active defragmentation?
# -------------------------------
#
@@ -1727,7 +1649,7 @@ rdb-save-incremental-fsync yes
# a good idea to leave the defaults untouched.
# Enabled active defragmentation
# activedefrag no
# activedefrag yes
# Minimum amount of fragmentation waste to start active defrag
# active-defrag-ignore-bytes 100mb
@@ -1738,13 +1660,11 @@ rdb-save-incremental-fsync yes
# Maximum percentage of fragmentation at which we use maximum effort
# active-defrag-threshold-upper 100
# Minimal effort for defrag in CPU percentage, to be used when the lower
# threshold is reached
# active-defrag-cycle-min 1
# Minimal effort for defrag in CPU percentage
# active-defrag-cycle-min 5
# Maximal effort for defrag in CPU percentage, to be used when the upper
# threshold is reached
# active-defrag-cycle-max 25
# Maximal effort for defrag in CPU percentage
# active-defrag-cycle-max 75
# Maximum number of set/hash/zset/list fields that will be processed from
# the main dictionary scan
-3
View File
@@ -22,7 +22,4 @@ $TCLSH tests/test_helper.tcl \
--single unit/moduleapi/hooks \
--single unit/moduleapi/misc \
--single unit/moduleapi/blockonkeys \
--single unit/moduleapi/scan \
--single unit/moduleapi/datatype \
--single unit/moduleapi/auth \
"${@}"
+1 -39
View File
@@ -32,7 +32,6 @@ OPT=$(OPTIMIZATION)
PREFIX?=/usr/local
INSTALL_BIN=$(PREFIX)/bin
INSTALL=install
PKG_CONFIG?=pkg-config
# Default allocator defaults to Jemalloc if it's not an ARM
MALLOC=libc
@@ -78,15 +77,6 @@ FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
FINAL_LIBS=-lm
DEBUG=-g -ggdb
# Linux ARM needs -latomic at linking time
ifneq (,$(filter aarch64 armv,$(uname_M)))
FINAL_LIBS+=-latomic
else
ifneq (,$(findstring armv,$(uname_M)))
FINAL_LIBS+=-latomic
endif
endif
ifeq ($(uname_S),SunOS)
# SunOS
ifneq ($(@@),32bit)
@@ -141,30 +131,6 @@ endif
# Include paths to dependencies
FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src
# Determine systemd support and/or build preference (defaulting to auto-detection)
BUILD_WITH_SYSTEMD=no
# If 'USE_SYSTEMD' in the environment is neither "no" nor "yes", try to
# auto-detect libsystemd's presence and link accordingly.
ifneq ($(USE_SYSTEMD),no)
LIBSYSTEMD_PKGCONFIG := $(shell $(PKG_CONFIG) --exists libsystemd && echo $$?)
# If libsystemd cannot be detected, continue building without support for it
# (unless a later check tells us otherwise)
ifeq ($(LIBSYSTEMD_PKGCONFIG),0)
BUILD_WITH_SYSTEMD=yes
endif
endif
ifeq ($(USE_SYSTEMD),yes)
ifneq ($(LIBSYSTEMD_PKGCONFIG),0)
$(error USE_SYSTEMD is set to "$(USE_SYSTEMD)", but $(PKG_CONFIG) cannot find libsystemd)
endif
# Force building with libsystemd
BUILD_WITH_SYSTEMD=yes
endif
ifeq ($(BUILD_WITH_SYSTEMD),yes)
FINAL_LIBS+=$(shell $(PKG_CONFIG) --libs libsystemd)
FINAL_CFLAGS+= -DHAVE_LIBSYSTEMD
endif
ifeq ($(MALLOC),tcmalloc)
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc
@@ -283,18 +249,14 @@ $(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
dict-benchmark: dict.c zmalloc.c sds.c siphash.c
$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS)
DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d)
-include $(DEP)
# Because the jemalloc.h header is generated as a part of the jemalloc build,
# building it should complete before building any other object. Instead of
# depending on a single artifact, build all dependencies first.
%.o: %.c .make-prerequisites
$(REDIS_CC) -MMD -o $@ -c $<
$(REDIS_CC) -c $<
clean:
rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep dict-benchmark
rm -f $(DEP)
.PHONY: clean
+3 -203
View File
@@ -49,8 +49,6 @@ list *UsersToLoad; /* This is a list of users found in the configuration file
array of SDS pointers: the first is the user name,
all the remaining pointers are ACL rules in the same
format as ACLSetUser(). */
list *ACLLog; /* Our security log, the user is able to inspect that
using the ACL LOG command .*/
struct ACLCategoryItem {
const char *name;
@@ -95,7 +93,6 @@ struct ACLUserFlag {
void ACLResetSubcommandsForCommand(user *u, unsigned long id);
void ACLResetSubcommands(user *u);
void ACLAddAllowedSubcommand(user *u, unsigned long id, const char *sub);
void ACLFreeLogEntry(void *le);
/* The length of the string representation of a hashed password. */
#define HASH_PASSWORD_LEN SHA256_BLOCK_SIZE*2
@@ -186,12 +183,12 @@ int ACLListMatchSds(void *a, void *b) {
return sdscmp(a,b) == 0;
}
/* Method to free list elements from ACL users password/patterns lists. */
/* Method to free list elements from ACL users password/ptterns lists. */
void ACLListFreeSds(void *item) {
sdsfree(item);
}
/* Method to duplicate list elements from ACL users password/patterns lists. */
/* Method to duplicate list elements from ACL users password/ptterns lists. */
void *ACLListDupSds(void *item) {
return sdsdup(item);
}
@@ -923,7 +920,6 @@ void ACLInitDefaultUser(void) {
void ACLInit(void) {
Users = raxNew();
UsersToLoad = listCreate();
ACLLog = listCreate();
ACLInitDefaultUser();
}
@@ -982,7 +978,6 @@ int ACLAuthenticateUser(client *c, robj *username, robj *password) {
moduleNotifyUserChanged(c);
return C_OK;
} else {
addACLLogEntry(c,ACL_DENIED_AUTH,0,username->ptr);
return C_ERR;
}
}
@@ -1039,7 +1034,7 @@ user *ACLGetUserByName(const char *name, size_t namelen) {
* command cannot be executed because the user is not allowed to run such
* command, the second if the command is denied because the user is trying
* to access keys that are not among the specified patterns. */
int ACLCheckCommandPerm(client *c, int *keyidxptr) {
int ACLCheckCommandPerm(client *c) {
user *u = c->user;
uint64_t id = c->cmd->id;
@@ -1099,7 +1094,6 @@ int ACLCheckCommandPerm(client *c, int *keyidxptr) {
}
}
if (!match) {
if (keyidxptr) *keyidxptr = keyidx[j];
getKeysFreeResult(keyidx);
return ACL_DENIED_KEY;
}
@@ -1460,131 +1454,6 @@ void ACLLoadUsersAtStartup(void) {
}
}
/* =============================================================================
* ACL log
* ==========================================================================*/
#define ACL_LOG_CTX_TOPLEVEL 0
#define ACL_LOG_CTX_LUA 1
#define ACL_LOG_CTX_MULTI 2
#define ACL_LOG_GROUPING_MAX_TIME_DELTA 60000
/* This structure defines an entry inside the ACL log. */
typedef struct ACLLogEntry {
uint64_t count; /* Number of times this happened recently. */
int reason; /* Reason for denying the command. ACL_DENIED_*. */
int context; /* Toplevel, Lua or MULTI/EXEC? ACL_LOG_CTX_*. */
sds object; /* The key name or command name. */
sds username; /* User the client is authenticated with. */
mstime_t ctime; /* Milliseconds time of last update to this entry. */
sds cinfo; /* Client info (last client if updated). */
} ACLLogEntry;
/* This function will check if ACL entries 'a' and 'b' are similar enough
* that we should actually update the existing entry in our ACL log instead
* of creating a new one. */
int ACLLogMatchEntry(ACLLogEntry *a, ACLLogEntry *b) {
if (a->reason != b->reason) return 0;
if (a->context != b->context) return 0;
mstime_t delta = a->ctime - b->ctime;
if (delta < 0) delta = -delta;
if (delta > ACL_LOG_GROUPING_MAX_TIME_DELTA) return 0;
if (sdscmp(a->object,b->object) != 0) return 0;
if (sdscmp(a->username,b->username) != 0) return 0;
return 1;
}
/* Release an ACL log entry. */
void ACLFreeLogEntry(void *leptr) {
ACLLogEntry *le = leptr;
sdsfree(le->object);
sdsfree(le->username);
sdsfree(le->cinfo);
zfree(le);
}
/* Adds a new entry in the ACL log, making sure to delete the old entry
* if we reach the maximum length allowed for the log. This function attempts
* to find similar entries in the current log in order to bump the counter of
* the log entry instead of creating many entries for very similar ACL
* rules issues.
*
* The keypos argument is only used when the reason is ACL_DENIED_KEY, since
* it allows the function to log the key name that caused the problem.
* Similarly the username is only passed when we failed to authenticate the
* user with AUTH or HELLO, for the ACL_DENIED_AUTH reason. Otherwise
* it will just be NULL.
*/
void addACLLogEntry(client *c, int reason, int keypos, sds username) {
/* Create a new entry. */
struct ACLLogEntry *le = zmalloc(sizeof(*le));
le->count = 1;
le->reason = reason;
le->username = sdsdup(reason == ACL_DENIED_AUTH ? username : c->user->name);
le->ctime = mstime();
switch(reason) {
case ACL_DENIED_CMD: le->object = sdsnew(c->cmd->name); break;
case ACL_DENIED_KEY: le->object = sdsnew(c->argv[keypos]->ptr); break;
case ACL_DENIED_AUTH: le->object = sdsnew(c->argv[0]->ptr); break;
default: le->object = sdsempty();
}
client *realclient = c;
if (realclient->flags & CLIENT_LUA) realclient = server.lua_caller;
le->cinfo = catClientInfoString(sdsempty(),realclient);
if (c->flags & CLIENT_MULTI) {
le->context = ACL_LOG_CTX_MULTI;
} else if (c->flags & CLIENT_LUA) {
le->context = ACL_LOG_CTX_LUA;
} else {
le->context = ACL_LOG_CTX_TOPLEVEL;
}
/* Try to match this entry with past ones, to see if we can just
* update an existing entry instead of creating a new one. */
long toscan = 10; /* Do a limited work trying to find duplicated. */
listIter li;
listNode *ln;
listRewind(ACLLog,&li);
ACLLogEntry *match = NULL;
while (toscan-- && (ln = listNext(&li)) != NULL) {
ACLLogEntry *current = listNodeValue(ln);
if (ACLLogMatchEntry(current,le)) {
match = current;
listDelNode(ACLLog,ln);
listAddNodeHead(ACLLog,current);
break;
}
}
/* If there is a match update the entry, otherwise add it as a
* new one. */
if (match) {
/* We update a few fields of the existing entry and bump the
* counter of events for this entry. */
sdsfree(match->cinfo);
match->cinfo = le->cinfo;
match->ctime = le->ctime;
match->count++;
/* Release the old entry. */
le->cinfo = NULL;
ACLFreeLogEntry(le);
} else {
/* Add it to our list of entires. We'll have to trim the list
* to its maximum size. */
listAddNodeHead(ACLLog, le);
while(listLength(ACLLog) > server.acllog_max_len) {
listNode *ln = listLast(ACLLog);
ACLLogEntry *le = listNodeValue(ln);
ACLFreeLogEntry(le);
listDelNode(ACLLog,ln);
}
}
}
/* =============================================================================
* ACL related commands
* ==========================================================================*/
@@ -1592,7 +1461,6 @@ void addACLLogEntry(client *c, int reason, int keypos, sds username) {
/* ACL -- show and modify the configuration of ACL users.
* ACL HELP
* ACL LOAD
* ACL SAVE
* ACL LIST
* ACL USERS
* ACL CAT [<category>]
@@ -1601,7 +1469,6 @@ void addACLLogEntry(client *c, int reason, int keypos, sds username) {
* ACL GETUSER <username>
* ACL GENPASS
* ACL WHOAMI
* ACL LOG [<count> | RESET]
*/
void aclCommand(client *c) {
char *sub = c->argv[1]->ptr;
@@ -1788,75 +1655,9 @@ void aclCommand(client *c) {
char pass[32]; /* 128 bits of actual pseudo random data. */
getRandomHexChars(pass,sizeof(pass));
addReplyBulkCBuffer(c,pass,sizeof(pass));
} else if (!strcasecmp(sub,"log") && (c->argc == 2 || c->argc ==3)) {
long count = 10; /* Number of entries to emit by default. */
/* Parse the only argument that LOG may have: it could be either
* the number of entires the user wants to display, or alternatively
* the "RESET" command in order to flush the old entires. */
if (c->argc == 3) {
if (!strcasecmp(c->argv[2]->ptr,"reset")) {
listSetFreeMethod(ACLLog,ACLFreeLogEntry);
listEmpty(ACLLog);
listSetFreeMethod(ACLLog,NULL);
addReply(c,shared.ok);
return;
} else if (getLongFromObjectOrReply(c,c->argv[2],&count,NULL)
!= C_OK)
{
return;
}
if (count < 0) count = 0;
}
/* Fix the count according to the number of entries we got. */
if ((size_t)count > listLength(ACLLog))
count = listLength(ACLLog);
addReplyArrayLen(c,count);
listIter li;
listNode *ln;
listRewind(ACLLog,&li);
mstime_t now = mstime();
while (count-- && (ln = listNext(&li)) != NULL) {
ACLLogEntry *le = listNodeValue(ln);
addReplyMapLen(c,7);
addReplyBulkCString(c,"count");
addReplyLongLong(c,le->count);
addReplyBulkCString(c,"reason");
char *reasonstr;
switch(le->reason) {
case ACL_DENIED_CMD: reasonstr="command"; break;
case ACL_DENIED_KEY: reasonstr="key"; break;
case ACL_DENIED_AUTH: reasonstr="auth"; break;
}
addReplyBulkCString(c,reasonstr);
addReplyBulkCString(c,"context");
char *ctxstr;
switch(le->context) {
case ACL_LOG_CTX_TOPLEVEL: ctxstr="toplevel"; break;
case ACL_LOG_CTX_MULTI: ctxstr="multi"; break;
case ACL_LOG_CTX_LUA: ctxstr="lua"; break;
default: ctxstr="unknown";
}
addReplyBulkCString(c,ctxstr);
addReplyBulkCString(c,"object");
addReplyBulkCBuffer(c,le->object,sdslen(le->object));
addReplyBulkCString(c,"username");
addReplyBulkCBuffer(c,le->username,sdslen(le->username));
addReplyBulkCString(c,"age-seconds");
double age = (double)(now - le->ctime)/1000;
addReplyDouble(c,age);
addReplyBulkCString(c,"client-info");
addReplyBulkCBuffer(c,le->cinfo,sdslen(le->cinfo));
}
} else if (!strcasecmp(sub,"help")) {
const char *help[] = {
"LOAD -- Reload users from the ACL file.",
"SAVE -- Save the current config to the ACL file."
"LIST -- Show user details in config file format.",
"USERS -- List all the registered usernames.",
"SETUSER <username> [attribs ...] -- Create or modify a user.",
@@ -1866,7 +1667,6 @@ void aclCommand(client *c) {
"CAT <category> -- List commands inside category.",
"GENPASS -- Generate a secure user password.",
"WHOAMI -- Return the current connection username.",
"LOG [<count> | RESET] -- Show the ACL log entries.",
NULL
};
addReplyHelp(c,help);
+1 -1
View File
@@ -66,7 +66,7 @@ typedef struct list {
#define listSetMatchMethod(l,m) ((l)->match = (m))
#define listGetDupMethod(l) ((l)->dup)
#define listGetFreeMethod(l) ((l)->free)
#define listGetFree(l) ((l)->free)
#define listGetMatchMethod(l) ((l)->match)
/* Prototypes */
+6 -17
View File
@@ -242,7 +242,6 @@ void stopAppendOnly(void) {
server.aof_fd = -1;
server.aof_selected_db = -1;
server.aof_state = AOF_OFF;
server.aof_rewrite_scheduled = 0;
killAppendOnlyChild();
}
@@ -782,26 +781,18 @@ int loadAppendOnlyFile(char *filename) {
argc = atoi(buf+1);
if (argc < 1) goto fmterr;
/* Load the next command in the AOF as our fake client
* argv. */
argv = zmalloc(sizeof(robj*)*argc);
fakeClient->argc = argc;
fakeClient->argv = argv;
for (j = 0; j < argc; j++) {
/* Parse the argument len. */
char *readres = fgets(buf,sizeof(buf),fp);
if (readres == NULL || buf[0] != '$') {
if (fgets(buf,sizeof(buf),fp) == NULL) {
fakeClient->argc = j; /* Free up to j-1. */
freeFakeClientArgv(fakeClient);
if (readres == NULL)
goto readerr;
else
goto fmterr;
goto readerr;
}
if (buf[0] != '$') goto fmterr;
len = strtol(buf+1,NULL,10);
/* Read it into a string object. */
argsds = sdsnewlen(SDS_NOINIT,len);
if (len && fread(argsds,len,1,fp) == 0) {
sdsfree(argsds);
@@ -810,12 +801,10 @@ int loadAppendOnlyFile(char *filename) {
goto readerr;
}
argv[j] = createObject(OBJ_STRING,argsds);
/* Discard CRLF. */
if (fread(buf,2,1,fp) == 0) {
fakeClient->argc = j+1; /* Free up to j. */
freeFakeClientArgv(fakeClient);
goto readerr;
goto readerr; /* discard CRLF */
}
}
@@ -1150,7 +1139,7 @@ int rioWriteBulkStreamID(rio *r,streamID *id) {
int retval;
sds replyid = sdscatfmt(sdsempty(),"%U-%U",id->ms,id->seq);
retval = rioWriteBulkString(r,replyid,sdslen(replyid));
if ((retval = rioWriteBulkString(r,replyid,sdslen(replyid))) == 0) return 0;
sdsfree(replyid);
return retval;
}
@@ -1777,7 +1766,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
server.aof_selected_db = -1; /* Make sure SELECT is re-issued */
aofUpdateCurrentSize();
server.aof_rewrite_base_size = server.aof_current_size;
server.aof_fsync_offset = server.aof_current_size;
server.aof_current_size = server.aof_current_size;
/* Clear regular AOF buffer since its contents was just written to
* the new AOF from the background rewrite buffer. */
-5
View File
@@ -27,9 +27,6 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __BIO_H
#define __BIO_H
/* Exported API */
void bioInit(void);
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
@@ -43,5 +40,3 @@ void bioKillThreads(void);
#define BIO_AOF_FSYNC 1 /* Deferred AOF fsync. */
#define BIO_LAZY_FREE 2 /* Deferred objects freeing. */
#define BIO_NUM_OPS 3
#endif
+1 -12
View File
@@ -388,7 +388,7 @@ void serveClientsBlockedOnStreamKey(robj *o, readyList *rl) {
if (streamCompareID(&s->last_id, gt) > 0) {
streamID start = *gt;
streamIncrID(&start);
start.seq++; /* Can't overflow, it's an uint64_t */
/* Lookup the consumer for the group, if any. */
streamConsumer *consumer = NULL;
@@ -514,16 +514,6 @@ void handleClientsBlockedOnKeys(void) {
* we can safely call signalKeyAsReady() against this key. */
dictDelete(rl->db->ready_keys,rl->key);
/* Even if we are not inside call(), increment the call depth
* in order to make sure that keys are expired against a fixed
* reference time, and not against the wallclock time. This
* way we can lookup an object multiple times (BRPOPLPUSH does
* that) without the risk of it being freed in the second
* lookup, invalidating the first one.
* See https://github.com/antirez/redis/pull/6554. */
server.fixed_time_expire++;
updateCachedTime(0);
/* Serve clients blocked on list key. */
robj *o = lookupKeyWrite(rl->db,rl->key);
@@ -539,7 +529,6 @@ void handleClientsBlockedOnKeys(void) {
* module is trying to accomplish right now. */
serveClientsBlockedOnKeyByModule(rl);
}
server.fixed_time_expire--;
/* Free this item. */
decrRefCount(rl->key);
+12 -47
View File
@@ -157,10 +157,7 @@ int clusterLoadConfig(char *filename) {
}
/* Regular config lines have at least eight fields */
if (argc < 8) {
sdsfreesplitres(argv,argc);
goto fmterr;
}
if (argc < 8) goto fmterr;
/* Create this node if it does not exist */
n = clusterLookupNode(argv[0]);
@@ -169,10 +166,7 @@ int clusterLoadConfig(char *filename) {
clusterAddNode(n);
}
/* Address and port */
if ((p = strrchr(argv[1],':')) == NULL) {
sdsfreesplitres(argv,argc);
goto fmterr;
}
if ((p = strrchr(argv[1],':')) == NULL) goto fmterr;
*p = '\0';
memcpy(n->ip,argv[1],strlen(argv[1])+1);
char *port = p+1;
@@ -253,10 +247,7 @@ int clusterLoadConfig(char *filename) {
*p = '\0';
direction = p[1]; /* Either '>' or '<' */
slot = atoi(argv[j]+1);
if (slot < 0 || slot >= CLUSTER_SLOTS) {
sdsfreesplitres(argv,argc);
goto fmterr;
}
if (slot < 0 || slot >= CLUSTER_SLOTS) goto fmterr;
p += 3;
cn = clusterLookupNode(p);
if (!cn) {
@@ -276,12 +267,8 @@ int clusterLoadConfig(char *filename) {
} else {
start = stop = atoi(argv[j]);
}
if (start < 0 || start >= CLUSTER_SLOTS ||
stop < 0 || stop >= CLUSTER_SLOTS)
{
sdsfreesplitres(argv,argc);
goto fmterr;
}
if (start < 0 || start >= CLUSTER_SLOTS) goto fmterr;
if (stop < 0 || stop >= CLUSTER_SLOTS) goto fmterr;
while(start <= stop) clusterAddSlot(n, start++);
}
@@ -4979,7 +4966,7 @@ void restoreCommand(client *c) {
if (!absttl) ttl+=mstime();
setExpire(c,c->db,c->argv[1],ttl);
}
objectSetLRUOrLFU(obj,lfu_freq,lru_idle,lru_clock,1000);
objectSetLRUOrLFU(obj,lfu_freq,lru_idle,lru_clock);
signalModifiedKey(c->db,c->argv[1]);
addReply(c,shared.ok);
server.dirty++;
@@ -5489,8 +5476,8 @@ void readwriteCommand(client *c) {
* already "down" but it is fragile to rely on the update of the global state,
* so we also handle it here.
*
* CLUSTER_REDIR_DOWN_STATE and CLUSTER_REDIR_DOWN_RO_STATE if the cluster is
* down but the user attempts to execute a command that addresses one or more keys. */
* CLUSTER_REDIR_DOWN_STATE if the cluster is down but the user attempts to
* execute a command that addresses one or more keys. */
clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *error_code) {
clusterNode *n = NULL;
robj *firstkey = NULL;
@@ -5608,27 +5595,10 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in
* without redirections or errors in all the cases. */
if (n == NULL) return myself;
/* Cluster is globally down but we got keys? We only serve the request
* if it is a read command and when allow_reads_when_down is enabled. */
/* Cluster is globally down but we got keys? We can't serve the request. */
if (server.cluster->state != CLUSTER_OK) {
if (!server.cluster_allow_reads_when_down) {
/* The cluster is configured to block commands when the
* cluster is down. */
if (error_code) *error_code = CLUSTER_REDIR_DOWN_STATE;
return NULL;
} else if (!(cmd->flags & CMD_READONLY) && !(cmd->proc == evalCommand)
&& !(cmd->proc == evalShaCommand))
{
/* The cluster is configured to allow read only commands
* but this command is neither readonly, nor EVAL or
* EVALSHA. */
if (error_code) *error_code = CLUSTER_REDIR_DOWN_RO_STATE;
return NULL;
} else {
/* Fall through and allow the command to be executed:
* this happens when server.cluster_allow_reads_when_down is
* true and the command is a readonly command or EVAL / EVALSHA. */
}
if (error_code) *error_code = CLUSTER_REDIR_DOWN_STATE;
return NULL;
}
/* Return the hashslot by reference. */
@@ -5697,8 +5667,6 @@ void clusterRedirectClient(client *c, clusterNode *n, int hashslot, int error_co
addReplySds(c,sdsnew("-TRYAGAIN Multiple keys request during rehashing of slot\r\n"));
} else if (error_code == CLUSTER_REDIR_DOWN_STATE) {
addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down\r\n"));
} else if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) {
addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down and only accepts read commands\r\n"));
} else if (error_code == CLUSTER_REDIR_DOWN_UNBOUND) {
addReplySds(c,sdsnew("-CLUSTERDOWN Hash slot not served\r\n"));
} else if (error_code == CLUSTER_REDIR_MOVED ||
@@ -5733,10 +5701,7 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
dictEntry *de;
dictIterator *di;
/* If the cluster is down, unblock the client with the right error.
* If the cluster is configured to allow reads on cluster down, we
* still want to emit this error since a write will be required
* to unblock them which may never come. */
/* If the cluster is down, unblock the client with the right error. */
if (server.cluster->state == CLUSTER_FAIL) {
clusterRedirectClient(c,NULL,0,CLUSTER_REDIR_DOWN_STATE);
return 1;
+5 -1
View File
@@ -13,10 +13,15 @@
/* The following defines are amount of time, sometimes expressed as
* multiplicators of the node timeout value (when ending with MULT). */
#define CLUSTER_DEFAULT_NODE_TIMEOUT 15000
#define CLUSTER_DEFAULT_SLAVE_VALIDITY 10 /* Slave max data age factor. */
#define CLUSTER_DEFAULT_REQUIRE_FULL_COVERAGE 1
#define CLUSTER_DEFAULT_SLAVE_NO_FAILOVER 0 /* Failover by default. */
#define CLUSTER_FAIL_REPORT_VALIDITY_MULT 2 /* Fail report validity. */
#define CLUSTER_FAIL_UNDO_TIME_MULT 2 /* Undo fail if master is back. */
#define CLUSTER_FAIL_UNDO_TIME_ADD 10 /* Some additional time. */
#define CLUSTER_FAILOVER_DELAY 5 /* Seconds */
#define CLUSTER_DEFAULT_MIGRATION_BARRIER 1
#define CLUSTER_MF_TIMEOUT 5000 /* Milliseconds to do a manual failover. */
#define CLUSTER_MF_PAUSE_MULT 2 /* Master pause manual failover mult. */
#define CLUSTER_SLAVE_MIGRATION_DELAY 5000 /* Delay for slave migration. */
@@ -29,7 +34,6 @@
#define CLUSTER_REDIR_MOVED 4 /* -MOVED redirection required. */
#define CLUSTER_REDIR_DOWN_STATE 5 /* -CLUSTERDOWN, global state. */
#define CLUSTER_REDIR_DOWN_UNBOUND 6 /* -CLUSTERDOWN, unbound slot. */
#define CLUSTER_REDIR_DOWN_RO_STATE 7 /* -CLUSTERDOWN, allow reads. */
struct clusterNode;
+1020 -848
View File
File diff suppressed because it is too large Load Diff
+27 -90
View File
@@ -216,28 +216,20 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
*
* 1) The ref count of the value object is incremented.
* 2) clients WATCHing for the destination key notified.
* 3) The expire time of the key is reset (the key is made persistent),
* unless 'keepttl' is true.
* 3) The expire time of the key is reset (the key is made persistent).
*
* All the new keys in the database should be created via this interface. */
void genericSetKey(redisDb *db, robj *key, robj *val, int keepttl) {
void setKey(redisDb *db, robj *key, robj *val) {
if (lookupKeyWrite(db,key) == NULL) {
dbAdd(db,key,val);
} else {
dbOverwrite(db,key,val);
}
incrRefCount(val);
if (!keepttl) removeExpire(db,key);
removeExpire(db,key);
signalModifiedKey(db,key);
}
/* Common case for genericSetKey() where the TTL is not retained. */
void setKey(redisDb *db, robj *key, robj *val) {
genericSetKey(db,key,val,0);
}
/* Return true if the specified key exists in the specified database.
* LRU/LFU info is not updated in any way. */
int dbExists(redisDb *db, robj *key) {
return dictFind(db->dict,key->ptr) != NULL;
}
@@ -347,10 +339,7 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o) {
* DB number if we want to flush only a single Redis database number.
*
* Flags are be EMPTYDB_NO_FLAGS if no special flags are specified or
* 1. EMPTYDB_ASYNC if we want the memory to be freed in a different thread.
* 2. EMPTYDB_BACKUP if we want to empty the backup dictionaries created by
* disklessLoadMakeBackups. In that case we only free memory and avoid
* firing module events.
* EMPTYDB_ASYNC if we want the memory to be freed in a different thread
* and the function to return ASAP.
*
* On success the fuction returns the number of keys removed from the
@@ -358,8 +347,6 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o) {
* DB number is out of range, and errno is set to EINVAL. */
long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(void*)) {
int async = (flags & EMPTYDB_ASYNC);
int backup = (flags & EMPTYDB_BACKUP); /* Just free the memory, nothing else */
RedisModuleFlushInfoV1 fi = {REDISMODULE_FLUSHINFO_VERSION,!async,dbnum};
long long removed = 0;
if (dbnum < -1 || dbnum >= server.dbnum) {
@@ -367,18 +354,16 @@ long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(
return -1;
}
/* Pre-flush actions */
if (!backup) {
/* Fire the flushdb modules event. */
moduleFireServerEvent(REDISMODULE_EVENT_FLUSHDB,
REDISMODULE_SUBEVENT_FLUSHDB_START,
&fi);
/* Fire the flushdb modules event. */
RedisModuleFlushInfoV1 fi = {REDISMODULE_FLUSHINFO_VERSION,!async,dbnum};
moduleFireServerEvent(REDISMODULE_EVENT_FLUSHDB,
REDISMODULE_SUBEVENT_FLUSHDB_START,
&fi);
/* Make sure the WATCHed keys are affected by the FLUSH* commands.
* Note that we need to call the function while the keys are still
* there. */
signalFlushedDb(dbnum);
}
/* Make sure the WATCHed keys are affected by the FLUSH* commands.
* Note that we need to call the function while the keys are still
* there. */
signalFlushedDb(dbnum);
int startdb, enddb;
if (dbnum == -1) {
@@ -397,24 +382,20 @@ long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(
dictEmpty(dbarray[j].expires,callback);
}
}
/* Post-flush actions */
if (!backup) {
if (server.cluster_enabled) {
if (async) {
slotToKeyFlushAsync();
} else {
slotToKeyFlush();
}
if (server.cluster_enabled) {
if (async) {
slotToKeyFlushAsync();
} else {
slotToKeyFlush();
}
if (dbnum == -1) flushSlaveKeysWithExpireList();
/* Also fire the end event. Note that this event will fire almost
* immediately after the start event if the flush is asynchronous. */
moduleFireServerEvent(REDISMODULE_EVENT_FLUSHDB,
REDISMODULE_SUBEVENT_FLUSHDB_END,
&fi);
}
if (dbnum == -1) flushSlaveKeysWithExpireList();
/* Also fire the end event. Note that this event will fire almost
* immediately after the start event if the flush is asynchronous. */
moduleFireServerEvent(REDISMODULE_EVENT_FLUSHDB,
REDISMODULE_SUBEVENT_FLUSHDB_END,
&fi);
return removed;
}
@@ -613,7 +594,7 @@ void keysCommand(client *c) {
void *replylen = addReplyDeferredLen(c);
di = dictGetSafeIterator(c->db->dict);
allkeys = (pattern[0] == '*' && plen == 1);
allkeys = (pattern[0] == '*' && pattern[1] == '\0');
while((de = dictNext(di)) != NULL) {
sds key = dictGetKey(de);
robj *keyobj;
@@ -1053,13 +1034,6 @@ void moveCommand(client *c) {
/* OK! key moved, free the entry in the source DB */
dbDelete(src,c->argv[1]);
signalModifiedKey(src,c->argv[1]);
signalModifiedKey(dst,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_GENERIC,
"move_from",c->argv[1],src->id);
notifyKeyspaceEvent(NOTIFY_GENERIC,
"move_to",c->argv[1],dst->id);
server.dirty++;
addReply(c,shared.cone);
}
@@ -1103,12 +1077,10 @@ int dbSwapDatabases(long id1, long id2) {
db1->dict = db2->dict;
db1->expires = db2->expires;
db1->avg_ttl = db2->avg_ttl;
db1->expires_cursor = db2->expires_cursor;
db2->dict = aux.dict;
db2->expires = aux.expires;
db2->avg_ttl = aux.avg_ttl;
db2->expires_cursor = aux.expires_cursor;
/* Now we need to handle clients blocked on lists: as an effect
* of swapping the two DBs, a client that was waiting for list
@@ -1224,7 +1196,6 @@ void propagateExpire(redisDb *db, robj *key, int lazy) {
/* Check if the key is expired. */
int keyIsExpired(redisDb *db, robj *key) {
mstime_t when = getExpire(db,key);
mstime_t now;
if (when < 0) return 0; /* No expire for this key */
@@ -1236,26 +1207,8 @@ int keyIsExpired(redisDb *db, robj *key) {
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
if (server.lua_caller) {
now = server.lua_time_start;
}
/* If we are in the middle of a command execution, we still want to use
* a reference time that does not change: in that case we just use the
* cached time, that we update before each call in the call() function.
* This way we avoid that commands such as RPOPLPUSH or similar, that
* may re-open the same key multiple times, can invalidate an already
* open object in a next call, if the next call will see the key expired,
* while the first did not. */
else if (server.fixed_time_expire > 0) {
now = server.mstime;
}
/* For the other cases, we want to use the most fresh time we have. */
else {
now = mstime();
}
mstime_t now = server.lua_caller ? server.lua_time_start : mstime();
/* The key expired if the current (virtual or real) time is greater
* than the expire time of the key. */
return now > when;
}
@@ -1533,22 +1486,6 @@ int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numk
return keys;
}
/* Helper function to extract keys from memory command.
* MEMORY USAGE <key> */
int *memoryGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int *keys;
UNUSED(cmd);
if (argc >= 3 && !strcasecmp(argv[1]->ptr,"usage")) {
keys = zmalloc(sizeof(int) * 1);
keys[0] = 2;
*numkeys = 1;
return keys;
}
*numkeys = 0;
return NULL;
}
/* XREAD [BLOCK <milliseconds>] [COUNT <count>] [GROUP <groupname> <ttl>]
* STREAMS key_1 key_2 ... key_N ID_1 ID_2 ... ID_N */
int *xreadGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
+2 -3
View File
@@ -355,7 +355,6 @@ void debugCommand(client *c) {
"CRASH-AND-RECOVER <milliseconds> -- Hard crash and restart after <milliseconds> delay.",
"DIGEST -- Output a hex signature representing the current DB content.",
"DIGEST-VALUE <key-1> ... <key-N>-- Output a hex signature of the values of all the specified keys.",
"DEBUG PROTOCOL [string|integer|double|bignum|null|array|set|map|attrib|push|verbatim|true|false]",
"ERROR <string> -- Return a Redis protocol error with <string> as message. Useful for clients unit tests to simulate Redis errors.",
"LOG <message> -- write message to the server log.",
"HTSTATS <dbid> -- Return hash table statistics of the specified Redis database.",
@@ -587,7 +586,7 @@ NULL
}
} else if (!strcasecmp(c->argv[1]->ptr,"protocol") && c->argc == 3) {
/* DEBUG PROTOCOL [string|integer|double|bignum|null|array|set|map|
* attrib|push|verbatim|true|false] */
* attrib|push|verbatim|true|false|state|err|bloberr] */
char *name = c->argv[2]->ptr;
if (!strcasecmp(name,"string")) {
addReplyBulkCString(c,"Hello World");
@@ -635,7 +634,7 @@ NULL
} else if (!strcasecmp(name,"verbatim")) {
addReplyVerbatim(c,"This is a verbatim\nstring",25,"txt");
} else {
addReplyError(c,"Wrong protocol type name. Please use one of the following: string|integer|double|bignum|null|array|set|map|attrib|push|verbatim|true|false");
addReplyError(c,"Wrong protocol type name. Please use one of the following: string|integer|double|bignum|null|array|set|map|attrib|push|verbatim|true|false|state|err|bloberr");
}
} else if (!strcasecmp(c->argv[1]->ptr,"sleep") && c->argc == 3) {
double dtime = strtod(c->argv[2]->ptr,NULL);
+14 -30
View File
@@ -919,12 +919,10 @@ int defragLaterItem(dictEntry *de, unsigned long *cursor, long long endtime) {
return 0;
}
/* static variables serving defragLaterStep to continue scanning a key from were we stopped last time. */
static sds defrag_later_current_key = NULL;
static unsigned long defrag_later_cursor = 0;
/* returns 0 if no more work needs to be been done, and 1 if time is up and more work is needed. */
int defragLaterStep(redisDb *db, long long endtime) {
static sds current_key = NULL;
static unsigned long cursor = 0;
unsigned int iterations = 0;
unsigned long long prev_defragged = server.stat_active_defrag_hits;
unsigned long long prev_scanned = server.stat_active_defrag_scanned;
@@ -932,15 +930,16 @@ int defragLaterStep(redisDb *db, long long endtime) {
do {
/* if we're not continuing a scan from the last call or loop, start a new one */
if (!defrag_later_cursor) {
if (!cursor) {
listNode *head = listFirst(db->defrag_later);
/* Move on to next key */
if (defrag_later_current_key) {
serverAssert(defrag_later_current_key == head->value);
if (current_key) {
serverAssert(current_key == head->value);
sdsfree(head->value);
listDelNode(db->defrag_later, head);
defrag_later_cursor = 0;
defrag_later_current_key = NULL;
cursor = 0;
current_key = NULL;
}
/* stop if we reached the last one. */
@@ -949,21 +948,21 @@ int defragLaterStep(redisDb *db, long long endtime) {
return 0;
/* start a new key */
defrag_later_current_key = head->value;
defrag_later_cursor = 0;
current_key = head->value;
cursor = 0;
}
/* each time we enter this function we need to fetch the key from the dict again (if it still exists) */
dictEntry *de = dictFind(db->dict, defrag_later_current_key);
dictEntry *de = dictFind(db->dict, current_key);
key_defragged = server.stat_active_defrag_hits;
do {
int quit = 0;
if (defragLaterItem(de, &defrag_later_cursor, endtime))
if (defragLaterItem(de, &cursor, endtime))
quit = 1; /* time is up, we didn't finish all the work */
/* Don't start a new BIG key in this loop, this is because the
* next key can be a list, and scanLaterList must be done in once cycle */
if (!defrag_later_cursor)
if (!cursor)
quit = 1;
/* Once in 16 scan iterations, 512 pointer reallocations, or 64 fields
@@ -983,7 +982,7 @@ int defragLaterStep(redisDb *db, long long endtime) {
prev_defragged = server.stat_active_defrag_hits;
prev_scanned = server.stat_active_defrag_scanned;
}
} while(defrag_later_cursor);
} while(cursor);
if(key_defragged != server.stat_active_defrag_hits)
server.stat_active_defrag_key_hits++;
else
@@ -1040,21 +1039,6 @@ void activeDefragCycle(void) {
mstime_t latency;
int quit = 0;
if (!server.active_defrag_enabled) {
if (server.active_defrag_running) {
/* if active defrag was disabled mid-run, start from fresh next time. */
server.active_defrag_running = 0;
if (db)
listEmpty(db->defrag_later);
defrag_later_current_key = NULL;
defrag_later_cursor = 0;
current_db = -1;
cursor = 0;
db = NULL;
}
return;
}
if (hasActiveChildProcess())
return; /* Defragging memory while there's a fork will just do damage. */
+35 -118
View File
@@ -78,63 +78,24 @@ int activeExpireCycleTryExpire(redisDb *db, dictEntry *de, long long now) {
* it will get more aggressive to avoid that too much memory is used by
* keys that can be removed from the keyspace.
*
* Every expire cycle tests multiple databases: the next call will start
* again from the next db, with the exception of exists for time limit: in that
* case we restart again from the last database we were processing. Anyway
* no more than CRON_DBS_PER_CALL databases are tested at every iteration.
* No more than CRON_DBS_PER_CALL databases are tested at every
* iteration.
*
* The function can perform more or less work, depending on the "type"
* argument. It can execute a "fast cycle" or a "slow cycle". The slow
* cycle is the main way we collect expired cycles: this happens with
* the "server.hz" frequency (usually 10 hertz).
* This kind of call is used when Redis detects that timelimit_exit is
* true, so there is more work to do, and we do it more incrementally from
* the beforeSleep() function of the event loop.
*
* However the slow cycle can exit for timeout, since it used too much time.
* For this reason the function is also invoked to perform a fast cycle
* at every event loop cycle, in the beforeSleep() function. The fast cycle
* will try to perform less work, but will do it much more often.
*
* The following are the details of the two expire cycles and their stop
* conditions:
* Expire cycle type:
*
* If type is ACTIVE_EXPIRE_CYCLE_FAST the function will try to run a
* "fast" expire cycle that takes no longer than EXPIRE_FAST_CYCLE_DURATION
* microseconds, and is not repeated again before the same amount of time.
* The cycle will also refuse to run at all if the latest slow cycle did not
* terminate because of a time limit condition.
*
* If type is ACTIVE_EXPIRE_CYCLE_SLOW, that normal expire cycle is
* executed, where the time limit is a percentage of the REDIS_HZ period
* as specified by the ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC define. In the
* fast cycle, the check of every database is interrupted once the number
* of already expired keys in the database is estimated to be lower than
* a given percentage, in order to avoid doing too much work to gain too
* little memory.
*
* The configured expire "effort" will modify the baseline parameters in
* order to do more work in both the fast and slow expire cycles.
*/
#define ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP 20 /* Keys for each DB loop. */
#define ACTIVE_EXPIRE_CYCLE_FAST_DURATION 1000 /* Microseconds. */
#define ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC 25 /* Max % of CPU to use. */
#define ACTIVE_EXPIRE_CYCLE_ACCEPTABLE_STALE 10 /* % of stale keys after which
we do extra efforts. */
* as specified by the ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC define. */
void activeExpireCycle(int type) {
/* Adjust the running parameters according to the configured expire
* effort. The default effort is 1, and the maximum configurable effort
* is 10. */
unsigned long
effort = server.active_expire_effort-1, /* Rescale from 0 to 9. */
config_keys_per_loop = ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP +
ACTIVE_EXPIRE_CYCLE_KEYS_PER_LOOP/4*effort,
config_cycle_fast_duration = ACTIVE_EXPIRE_CYCLE_FAST_DURATION +
ACTIVE_EXPIRE_CYCLE_FAST_DURATION/4*effort,
config_cycle_slow_time_perc = ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC +
2*effort,
config_cycle_acceptable_stale = ACTIVE_EXPIRE_CYCLE_ACCEPTABLE_STALE-
effort;
/* This function has some global state in order to continue the work
* incrementally across calls. */
static unsigned int current_db = 0; /* Last DB tested. */
@@ -152,16 +113,10 @@ void activeExpireCycle(int type) {
if (type == ACTIVE_EXPIRE_CYCLE_FAST) {
/* Don't start a fast cycle if the previous cycle did not exit
* for time limit, unless the percentage of estimated stale keys is
* too high. Also never repeat a fast cycle for the same period
* for time limit. Also don't repeat a fast cycle for the same period
* as the fast cycle total duration itself. */
if (!timelimit_exit &&
server.stat_expired_stale_perc < config_cycle_acceptable_stale)
return;
if (start < last_fast_cycle + (long long)config_cycle_fast_duration*2)
return;
if (!timelimit_exit) return;
if (start < last_fast_cycle + ACTIVE_EXPIRE_CYCLE_FAST_DURATION*2) return;
last_fast_cycle = start;
}
@@ -175,16 +130,16 @@ void activeExpireCycle(int type) {
if (dbs_per_call > server.dbnum || timelimit_exit)
dbs_per_call = server.dbnum;
/* We can use at max 'config_cycle_slow_time_perc' percentage of CPU
* time per iteration. Since this function gets called with a frequency of
/* We can use at max ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC percentage of CPU time
* per iteration. Since this function gets called with a frequency of
* server.hz times per second, the following is the max amount of
* microseconds we can spend in this function. */
timelimit = config_cycle_slow_time_perc*1000000/server.hz/100;
timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;
timelimit_exit = 0;
if (timelimit <= 0) timelimit = 1;
if (type == ACTIVE_EXPIRE_CYCLE_FAST)
timelimit = config_cycle_fast_duration; /* in microseconds. */
timelimit = ACTIVE_EXPIRE_CYCLE_FAST_DURATION; /* in microseconds. */
/* Accumulate some global stats as we expire keys, to have some idea
* about the number of keys that are already logically expired, but still
@@ -193,9 +148,7 @@ void activeExpireCycle(int type) {
long total_expired = 0;
for (j = 0; j < dbs_per_call && timelimit_exit == 0; j++) {
/* Expired and checked in a single loop. */
unsigned long expired, sampled;
int expired;
redisDb *db = server.db+(current_db % server.dbnum);
/* Increment the DB now so we are sure if we run out of time
@@ -203,10 +156,8 @@ void activeExpireCycle(int type) {
* distribute the time evenly across DBs. */
current_db++;
/* Continue to expire if at the end of the cycle there are still
* a big percentage of keys to expire, compared to the number of keys
* we scanned. The percentage, stored in config_cycle_acceptable_stale
* is not fixed, but depends on the Redis configured "expire effort". */
/* Continue to expire if at the end of the cycle more than 25%
* of the keys were expired. */
do {
unsigned long num, slots;
long long now, ttl_sum;
@@ -221,8 +172,8 @@ void activeExpireCycle(int type) {
slots = dictSlots(db->expires);
now = mstime();
/* When there are less than 1% filled slots, sampling the key
* space is expensive, so stop here waiting for better times...
/* When there are less than 1% filled slots getting random
* keys is expensive, so stop here waiting for better times...
* The dictionary will be resized asap. */
if (num && slots > DICT_HT_INITIAL_SIZE &&
(num*100/slots < 1)) break;
@@ -230,58 +181,27 @@ void activeExpireCycle(int type) {
/* The main collection cycle. Sample random keys among keys
* with an expire set, checking for expired ones. */
expired = 0;
sampled = 0;
ttl_sum = 0;
ttl_samples = 0;
if (num > config_keys_per_loop)
num = config_keys_per_loop;
if (num > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP)
num = ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP;
/* Here we access the low level representation of the hash table
* for speed concerns: this makes this code coupled with dict.c,
* but it hardly changed in ten years.
*
* Note that certain places of the hash table may be empty,
* so we want also a stop condition about the number of
* buckets that we scanned. However scanning for free buckets
* is very fast: we are in the cache line scanning a sequential
* array of NULL pointers, so we can scan a lot more buckets
* than keys in the same time. */
long max_buckets = num*20;
long checked_buckets = 0;
while (num--) {
dictEntry *de;
long long ttl;
while (sampled < num && checked_buckets < max_buckets) {
for (int table = 0; table < 2; table++) {
if (table == 1 && !dictIsRehashing(db->expires)) break;
unsigned long idx = db->expires_cursor;
idx &= db->expires->ht[table].sizemask;
dictEntry *de = db->expires->ht[table].table[idx];
long long ttl;
/* Scan the current bucket of the current table. */
checked_buckets++;
while(de) {
/* Get the next entry now since this entry may get
* deleted. */
dictEntry *e = de;
de = de->next;
ttl = dictGetSignedIntegerVal(e)-now;
if (activeExpireCycleTryExpire(db,e,now)) expired++;
if (ttl > 0) {
/* We want the average TTL of keys yet
* not expired. */
ttl_sum += ttl;
ttl_samples++;
}
sampled++;
}
if ((de = dictGetRandomKey(db->expires)) == NULL) break;
ttl = dictGetSignedIntegerVal(de)-now;
if (activeExpireCycleTryExpire(db,de,now)) expired++;
if (ttl > 0) {
/* We want the average TTL of keys yet not expired. */
ttl_sum += ttl;
ttl_samples++;
}
db->expires_cursor++;
total_sampled++;
}
total_expired += expired;
total_sampled += sampled;
/* Update the average TTL stats for this database. */
if (ttl_samples) {
@@ -305,15 +225,12 @@ void activeExpireCycle(int type) {
break;
}
}
/* We don't repeat the cycle for the current database if there are
* an acceptable amount of stale keys (logically expired but yet
* not reclaimed). */
} while (sampled == 0 ||
(expired*100/sampled) > config_cycle_acceptable_stale);
/* We don't repeat the cycle if there are less than 25% of keys
* found expired in the current DB. */
} while (expired > ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP/4);
}
elapsed = ustime()-start;
server.stat_expire_cycle_time_used += elapsed;
latencyAddSampleIfNeeded("expire-cycle",elapsed/1000);
/* Update our estimate of keys existing but yet to be expired.
+5 -13
View File
@@ -734,22 +734,14 @@ void geohashCommand(client *c) {
r[1].max = 90;
geohashEncode(&r[0],&r[1],xy[0],xy[1],26,&hash);
char buf[12];
char buf[11];
int i;
for (i = 0; i < 11; i++) {
int idx;
if (i == 10) {
/* We have just 52 bits, but the API used to output
* an 11 bytes geohash. For compatibility we assume
* zero. */
idx = 0;
} else {
idx = (hash.bits >> (52-((i+1)*5))) & 0x1f;
}
for (i = 0; i < 10; i++) {
int idx = (hash.bits >> (52-((i+1)*5))) & 0x1f;
buf[i] = geoalphabet[idx];
}
buf[11] = '\0';
addReplyBulkCBuffer(c,buf,11);
buf[10] = '\0';
addReplyBulkCBuffer(c,buf,10);
}
}
}
-1
View File
@@ -1535,7 +1535,6 @@ void pfdebugCommand(client *c) {
sds decoded = sdsempty();
if (hdr->encoding != HLL_SPARSE) {
sdsfree(decoded);
addReplyError(c,"HLL encoding is not sparse");
return;
}
+2 -2
View File
@@ -52,8 +52,8 @@
static int is_leap_year(time_t year) {
if (year % 4) return 0; /* A year not divisible by 4 is not leap. */
else if (year % 100) return 1; /* If div by 4 and not 100 is surely leap. */
else if (year % 400) return 0; /* If div by 100 *and* not by 400 is not leap. */
else return 1; /* If div by 100 and 400 is leap. */
else if (year % 400) return 0; /* If div by 100 *and* 400 is not leap. */
else return 1; /* If div by 100 and not by 400 is leap. */
}
void nolocks_localtime(struct tm *tmp, time_t t, time_t tz, int dst) {
+1 -1
View File
@@ -3,7 +3,7 @@ GIT_SHA1=`(git show-ref --head --hash=8 2> /dev/null || echo 00000000) | head -n
GIT_DIRTY=`git diff --no-ext-diff 2> /dev/null | wc -l`
BUILD_ID=`uname -n`"-"`date +%s`
if [ -n "$SOURCE_DATE_EPOCH" ]; then
BUILD_ID=$(date -u -d "@$SOURCE_DATE_EPOCH" +%s 2>/dev/null || date -u -r "$SOURCE_DATE_EPOCH" +%s 2>/dev/null || date -u +%s)
BUILD_ID=$(date -u -d "@$SOURCE_DATE_EPOCH" +%s 2>/dev/null || date -u -r "$SOURCE_DATE_EPOCH" +%s 2>/dev/null || date -u %s)
fi
test -f release.h || touch release.h
(cat release.h | grep SHA1 | grep $GIT_SHA1) && \
+214 -837
View File
File diff suppressed because it is too large Load Diff
+34 -23
View File
@@ -1,5 +1,4 @@
/* ACL API example - An example for performing custom synchronous and
* asynchronous password authentication.
/* ACL API example - An example of performing custom password authentication
*
* -----------------------------------------------------------------------------
*
@@ -33,12 +32,17 @@
#define REDISMODULE_EXPERIMENTAL_API
#include "../redismodule.h"
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <string.h>
#include <strings.h>
#include <pthread.h>
#include <unistd.h>
// A simple global user
static RedisModuleUser *global;
static uint64_t global_auth_client_id = 0;
static RedisModuleAuthCtx *global_auth_ctx;
/* HELLOACL.REVOKE
* Synchronously revoke access from a user. */
@@ -46,8 +50,8 @@ int RevokeCommand_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, in
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (global_auth_client_id) {
RedisModule_DeauthenticateAndCloseClient(ctx, global_auth_client_id);
if (global_auth_ctx) {
RedisModule_RevokeAuthentication(global_auth_ctx);
return RedisModule_ReplyWithSimpleString(ctx, "OK");
} else {
return RedisModule_ReplyWithError(ctx, "Global user currently not used");
@@ -71,23 +75,29 @@ int ResetCommand_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int
/* Callback handler for user changes, use this to notify a module of
* changes to users authenticated by the module */
void HelloACL_UserChanged(uint64_t client_id, void *privdata) {
void HelloACL_UserChanged(RedisModuleCtx *ctx, void *privdata) {
REDISMODULE_NOT_USED(privdata);
REDISMODULE_NOT_USED(client_id);
global_auth_client_id = 0;
REDISMODULE_NOT_USED(ctx);
global_auth_ctx = NULL;
}
/* HELLOACL.AUTHGLOBAL
* Synchronously assigns a module user to the current context. */
* Synchronously assigns a module user to the current context client. */
int AuthGlobalCommand_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (global_auth_client_id) {
if (global_auth_ctx) {
return RedisModule_ReplyWithError(ctx, "Global user currently used");
}
RedisModule_AuthenticateClientWithUser(ctx, global, HelloACL_UserChanged, NULL, &global_auth_client_id);
RedisModuleAuthCtx *auth_ctx = RedisModule_CreateAuthCtx(HelloACL_UserChanged, NULL);
if (RedisModule_AuthClientWithUser(ctx, global, auth_ctx) ==
REDISMODULE_ERR) {
return RedisModule_ReplyWithError(ctx, "Couldn't authenticate client");
}
global_auth_ctx = auth_ctx;
return RedisModule_ReplyWithSimpleString(ctx, "OK");
}
@@ -103,7 +113,7 @@ int HelloACL_Reply(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModuleString *user_string = RedisModule_GetBlockedClientPrivateData(ctx);
const char *name = RedisModule_StringPtrLen(user_string, &length);
if (RedisModule_AuthenticateClientWithACLUser(ctx, name, length, NULL, NULL, NULL) ==
if (RedisModule_AuthClientWithACLUser(ctx, name, length, NULL) ==
REDISMODULE_ERR) {
return RedisModule_ReplyWithError(ctx, "Invalid Username or password");
}
@@ -117,7 +127,7 @@ int HelloACL_Timeout(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return RedisModule_ReplyWithSimpleString(ctx, "Request timedout");
}
/* Private data frees data for HELLOACL.AUTHASYNC command. */
/* FreeData callback frees private data for HELLOACL.AUTHASYNC command. */
void HelloACL_FreeData(RedisModuleCtx *ctx, void *privdata) {
REDISMODULE_NOT_USED(ctx);
RedisModule_FreeString(NULL, privdata);
@@ -140,7 +150,8 @@ int AuthAsyncCommand_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv,
if (argc != 2) return RedisModule_WrongArity(ctx);
pthread_t tid;
RedisModuleBlockedClient *bc = RedisModule_BlockClient(ctx, HelloACL_Reply, HelloACL_Timeout, HelloACL_FreeData, TIMEOUT_TIME);
RedisModuleBlockedClient *bc = RedisModule_BlockClient(ctx, HelloACL_Reply,
HelloACL_Timeout, HelloACL_FreeData, TIMEOUT_TIME);
void **targs = RedisModule_Alloc(sizeof(void*)*2);
@@ -161,23 +172,23 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"helloacl",1,REDISMODULE_APIVER_1)
if (RedisModule_Init(ctx, "helloacl", 1, REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"helloacl.reset",
ResetCommand_RedisCommand,"",0,0,0) == REDISMODULE_ERR)
if (RedisModule_CreateCommand(ctx, "helloacl.reset",
ResetCommand_RedisCommand, "", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"helloacl.revoke",
RevokeCommand_RedisCommand,"",0,0,0) == REDISMODULE_ERR)
if (RedisModule_CreateCommand(ctx, "helloacl.revoke",
RevokeCommand_RedisCommand, "", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"helloacl.authglobal",
AuthGlobalCommand_RedisCommand,"no-auth",0,0,0) == REDISMODULE_ERR)
if (RedisModule_CreateCommand(ctx, "helloacl.authglobal",
AuthGlobalCommand_RedisCommand, "no-auth", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"helloacl.authasync",
AuthAsyncCommand_RedisCommand,"no-auth",0,0,0) == REDISMODULE_ERR)
AuthAsyncCommand_RedisCommand, "no-auth", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
global = RedisModule_CreateModuleUser("global");
@@ -185,7 +196,7 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
RedisModule_SetModuleUserACL(global, "allkeys");
RedisModule_SetModuleUserACL(global, "on");
global_auth_client_id = 0;
global_auth_ctx = NULL;
return REDISMODULE_OK;
}
+4 -8
View File
@@ -106,13 +106,11 @@ void discardCommand(client *c) {
/* Send a MULTI command to all the slaves and AOF file. Check the execCommand
* implementation for more information. */
void execCommandPropagateMulti(client *c) {
propagate(server.multiCommand,c->db->id,&shared.multi,1,
PROPAGATE_AOF|PROPAGATE_REPL);
}
robj *multistring = createStringObject("MULTI",5);
void execCommandPropagateExec(client *c) {
propagate(server.execCommand,c->db->id,&shared.exec,1,
propagate(server.multiCommand,c->db->id,&multistring,1,
PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(multistring);
}
void execCommand(client *c) {
@@ -177,10 +175,8 @@ void execCommand(client *c) {
must_propagate = 1;
}
int acl_keypos;
int acl_retval = ACLCheckCommandPerm(c,&acl_keypos);
int acl_retval = ACLCheckCommandPerm(c);
if (acl_retval != ACL_OK) {
addACLLogEntry(c,acl_retval,acl_keypos,NULL);
addReplyErrorFormat(c,
"-NOPERM ACLs rules changed between the moment the "
"transaction was accumulated and the EXEC call. "
+44 -109
View File
@@ -154,10 +154,7 @@ client *createClient(connection *conn) {
c->peerid = NULL;
c->client_list_node = NULL;
c->client_tracking_redirection = 0;
c->client_tracking_prefixes = NULL;
c->auth_callback = NULL;
c->auth_callback_privdata = NULL;
c->auth_module = NULL;
c->auth_ctx = NULL;
listSetFreeMethod(c->pubsub_patterns,decrRefCountVoid);
listSetMatchMethod(c->pubsub_patterns,listMatchObjects);
if (conn) linkClient(c);
@@ -370,10 +367,9 @@ void addReplyErrorLength(client *c, const char *s, size_t len) {
* Where the master must propagate the first change even if the second
* will produce an error. However it is useful to log such events since
* they are rare and may hint at errors in a script or a bug in Redis. */
int ctype = getClientType(c);
if (ctype == CLIENT_TYPE_MASTER || ctype == CLIENT_TYPE_SLAVE) {
char* to = ctype == CLIENT_TYPE_MASTER? "master": "replica";
char* from = ctype == CLIENT_TYPE_MASTER? "replica": "master";
if (c->flags & (CLIENT_MASTER|CLIENT_SLAVE) && !(c->flags & CLIENT_MONITOR)) {
char* to = c->flags & CLIENT_MASTER? "master": "replica";
char* from = c->flags & CLIENT_MASTER? "replica": "master";
char *cmdname = c->lastcmd ? c->lastcmd->name : "<unknown>";
serverLog(LL_WARNING,"== CRITICAL == This %s is sending an error "
"to its %s: '%s' after processing the command "
@@ -1076,7 +1072,7 @@ void freeClient(client *c) {
}
/* Log link disconnection with slave */
if (getClientType(c) == CLIENT_TYPE_SLAVE) {
if ((c->flags & CLIENT_SLAVE) && !(c->flags & CLIENT_MONITOR)) {
serverLog(LL_WARNING,"Connection with replica %s lost.",
replicationGetSlaveName(c));
}
@@ -1123,7 +1119,7 @@ void freeClient(client *c) {
/* We need to remember the time when we started to have zero
* attached slaves, as after some time we'll free the replication
* backlog. */
if (getClientType(c) == CLIENT_TYPE_SLAVE && listLength(server.slaves) == 0)
if (c->flags & CLIENT_SLAVE && listLength(server.slaves) == 0)
server.repl_no_slaves_since = server.unixtime;
refreshGoodSlavesCount();
/* Fire the replica change modules event. */
@@ -1164,14 +1160,9 @@ void freeClientAsync(client *c) {
* may access the list while Redis uses I/O threads. All the other accesses
* are in the context of the main thread while the other threads are
* idle. */
static pthread_mutex_t async_free_queue_mutex = PTHREAD_MUTEX_INITIALIZER;
if (c->flags & CLIENT_CLOSE_ASAP || c->flags & CLIENT_LUA) return;
c->flags |= CLIENT_CLOSE_ASAP;
if (server.io_threads_num == 1) {
/* no need to bother with locking if there's just one thread (the main thread) */
listAddNodeTail(server.clients_to_close,c);
return;
}
static pthread_mutex_t async_free_queue_mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_lock(&async_free_queue_mutex);
listAddNodeTail(server.clients_to_close,c);
pthread_mutex_unlock(&async_free_queue_mutex);
@@ -1259,8 +1250,8 @@ int writeToClient(client *c, int handler_installed) {
* just deliver as much data as it is possible to deliver.
*
* Moreover, we also send as much as possible if the client is
* a slave or a monitor (otherwise, on high-speed traffic, the
* replication/output buffer will grow indefinitely) */
* a slave (otherwise, on high-speed traffic, the replication
* buffer will grow indefinitely) */
if (totwritten > NET_MAX_WRITES_PER_EVENT &&
(server.maxmemory == 0 ||
zmalloc_used_memory() < server.maxmemory) &&
@@ -1446,7 +1437,7 @@ int processInlineBuffer(client *c) {
/* Newline from slaves can be used to refresh the last ACK time.
* This is useful for a slave to ping back while loading a big
* RDB file. */
if (querylen == 0 && getClientType(c) == CLIENT_TYPE_SLAVE)
if (querylen == 0 && c->flags & CLIENT_SLAVE)
c->repl_ack_time = server.unixtime;
/* Move querybuffer position to the next query in the buffer. */
@@ -1460,8 +1451,12 @@ int processInlineBuffer(client *c) {
/* Create redis objects for all arguments. */
for (c->argc = 0, j = 0; j < argc; j++) {
c->argv[c->argc] = createObject(OBJ_STRING,argv[j]);
c->argc++;
if (sdslen(argv[j])) {
c->argv[c->argc] = createObject(OBJ_STRING,argv[j]);
c->argc++;
} else {
sdsfree(argv[j]);
}
}
zfree(argv);
return C_OK;
@@ -2005,6 +2000,7 @@ int clientSetNameOrReply(client *c, robj *name) {
if (len == 0) {
if (c->name) decrRefCount(c->name);
c->name = NULL;
addReply(c,shared.ok);
return C_OK;
}
@@ -2028,6 +2024,7 @@ int clientSetNameOrReply(client *c, robj *name) {
void clientCommand(client *c) {
listNode *ln;
listIter li;
client *client;
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = {
@@ -2141,7 +2138,7 @@ NULL
/* Iterate clients killing all the matching clients. */
listRewind(server.clients,&li);
while ((ln = listNext(&li)) != NULL) {
client *client = listNodeValue(ln);
client = listNodeValue(ln);
if (addr && strcmp(getClientPeerId(client),addr) != 0) continue;
if (type != -1 && getClientType(client) != type) continue;
if (id != 0 && client->id != id) continue;
@@ -2219,74 +2216,38 @@ NULL
UNIT_MILLISECONDS) != C_OK) return;
pauseClients(duration);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"tracking") && c->argc >= 3) {
/* CLIENT TRACKING (on|off) [REDIRECT <id>] [BCAST] [PREFIX first]
* [PREFIX second] ... */
} else if (!strcasecmp(c->argv[1]->ptr,"tracking") &&
(c->argc == 3 || c->argc == 5))
{
/* CLIENT TRACKING (on|off) [REDIRECT <id>] */
long long redir = 0;
int bcast = 0;
robj **prefix = NULL;
size_t numprefix = 0;
/* Parse the options. */
for (int j = 3; j < c->argc; j++) {
int moreargs = (c->argc-1) - j;
if (!strcasecmp(c->argv[j]->ptr,"redirect") && moreargs) {
j++;
if (getLongLongFromObjectOrReply(c,c->argv[j],&redir,NULL) !=
/* Parse the redirection option: we'll require the client with
* the specified ID to exist right now, even if it is possible
* it will get disconnected later. */
if (c->argc == 5) {
if (strcasecmp(c->argv[3]->ptr,"redirect") != 0) {
addReply(c,shared.syntaxerr);
return;
} else {
if (getLongLongFromObjectOrReply(c,c->argv[4],&redir,NULL) !=
C_OK) return;
/* We will require the client with the specified ID to exist
* right now, even if it is possible that it gets disconnected
* later. Still a valid sanity check. */
if (lookupClientByID(redir) == NULL) {
addReplyError(c,"The client ID you want redirect to "
"does not exist");
return;
}
} else if (!strcasecmp(c->argv[j]->ptr,"bcast")) {
bcast++;
} else if (!strcasecmp(c->argv[j]->ptr,"prefix") && moreargs) {
j++;
prefix = zrealloc(prefix,sizeof(robj*)*(numprefix+1));
prefix[numprefix++] = c->argv[j];
} else {
zfree(prefix);
addReply(c,shared.syntaxerr);
return;
}
}
/* Options are ok: enable or disable the tracking for this client. */
if (!strcasecmp(c->argv[2]->ptr,"on")) {
/* Before enabling tracking, make sure options are compatible
* among each other and with the current state of the client. */
if (!bcast && numprefix) {
addReplyError(c,
"PREFIX option requires BCAST mode to be enabled");
zfree(prefix);
return;
}
if (c->flags & CLIENT_TRACKING) {
int oldbcast = !!(c->flags & CLIENT_TRACKING_BCAST);
if (oldbcast != bcast) {
addReplyError(c,
"You can't switch BCAST mode on/off before disabling "
"tracking for this client, and then re-enabling it with "
"a different mode.");
zfree(prefix);
return;
}
}
enableTracking(c,redir,bcast,prefix,numprefix);
enableTracking(c,redir);
} else if (!strcasecmp(c->argv[2]->ptr,"off")) {
disableTracking(c);
} else {
zfree(prefix);
addReply(c,shared.syntaxerr);
return;
}
zfree(prefix);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"getredir") && c->argc == 2) {
/* CLIENT GETREDIR */
@@ -2475,14 +2436,12 @@ unsigned long getClientOutputBufferMemoryUsage(client *c) {
*
* The function will return one of the following:
* CLIENT_TYPE_NORMAL -> Normal client
* CLIENT_TYPE_SLAVE -> Slave
* CLIENT_TYPE_SLAVE -> Slave or client executing MONITOR command
* CLIENT_TYPE_PUBSUB -> Client subscribed to Pub/Sub channels
* CLIENT_TYPE_MASTER -> The client representing our replication master.
*/
int getClientType(client *c) {
if (c->flags & CLIENT_MASTER) return CLIENT_TYPE_MASTER;
/* Even though MONITOR clients are marked as replicas, we
* want the expose them as normal clients. */
if ((c->flags & CLIENT_SLAVE) && !(c->flags & CLIENT_MONITOR))
return CLIENT_TYPE_SLAVE;
if (c->flags & CLIENT_PUBSUB) return CLIENT_TYPE_PUBSUB;
@@ -2700,10 +2659,6 @@ pthread_mutex_t io_threads_mutex[IO_THREADS_MAX_NUM];
_Atomic unsigned long io_threads_pending[IO_THREADS_MAX_NUM];
int io_threads_active; /* Are the threads currently spinning waiting I/O? */
int io_threads_op; /* IO_THREADS_OP_WRITE or IO_THREADS_OP_READ. */
/* This is the list of clients each thread will serve when threaded I/O is
* used. We spawn io_threads_num-1 threads, since one is the main thread
* itself. */
list *io_threads_list[IO_THREADS_MAX_NUM];
void *IOThreadMain(void *myid) {
@@ -2764,16 +2719,12 @@ void initThreadedIO(void) {
exit(1);
}
/* Spawn and initialize the I/O threads. */
/* Spawn the I/O threads. */
for (int i = 0; i < server.io_threads_num; i++) {
/* Things we do for all the threads including the main thread. */
io_threads_list[i] = listCreate();
if (i == 0) continue; /* Thread 0 is the main thread. */
/* Things we do only for the additional threads. */
pthread_t tid;
pthread_mutex_init(&io_threads_mutex[i],NULL);
io_threads_pending[i] = 0;
io_threads_list[i] = listCreate();
pthread_mutex_lock(&io_threads_mutex[i]); /* Thread will be stopped. */
if (pthread_create(&tid,NULL,IOThreadMain,(void*)(long)i) != 0) {
serverLog(LL_WARNING,"Fatal: Can't initialize IO thread.");
@@ -2787,7 +2738,7 @@ void startThreadedIO(void) {
if (tio_debug) { printf("S"); fflush(stdout); }
if (tio_debug) printf("--- STARTING THREADED IO ---\n");
serverAssert(io_threads_active == 0);
for (int j = 1; j < server.io_threads_num; j++)
for (int j = 0; j < server.io_threads_num; j++)
pthread_mutex_unlock(&io_threads_mutex[j]);
io_threads_active = 1;
}
@@ -2801,7 +2752,7 @@ void stopThreadedIO(void) {
(int) listLength(server.clients_pending_read),
(int) listLength(server.clients_pending_write));
serverAssert(io_threads_active == 1);
for (int j = 1; j < server.io_threads_num; j++)
for (int j = 0; j < server.io_threads_num; j++)
pthread_mutex_lock(&io_threads_mutex[j]);
io_threads_active = 0;
}
@@ -2860,23 +2811,15 @@ int handleClientsWithPendingWritesUsingThreads(void) {
/* Give the start condition to the waiting threads, by setting the
* start condition atomic var. */
io_threads_op = IO_THREADS_OP_WRITE;
for (int j = 1; j < server.io_threads_num; j++) {
for (int j = 0; j < server.io_threads_num; j++) {
int count = listLength(io_threads_list[j]);
io_threads_pending[j] = count;
}
/* Also use the main thread to process a slice of clients. */
listRewind(io_threads_list[0],&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
writeToClient(c,0);
}
listEmpty(io_threads_list[0]);
/* Wait for all the other threads to end their work. */
/* Wait for all threads to end their work. */
while(1) {
unsigned long pending = 0;
for (int j = 1; j < server.io_threads_num; j++)
for (int j = 0; j < server.io_threads_num; j++)
pending += io_threads_pending[j];
if (pending == 0) break;
}
@@ -2945,23 +2888,15 @@ int handleClientsWithPendingReadsUsingThreads(void) {
/* Give the start condition to the waiting threads, by setting the
* start condition atomic var. */
io_threads_op = IO_THREADS_OP_READ;
for (int j = 1; j < server.io_threads_num; j++) {
for (int j = 0; j < server.io_threads_num; j++) {
int count = listLength(io_threads_list[j]);
io_threads_pending[j] = count;
}
/* Also use the main thread to process a slice of clients. */
listRewind(io_threads_list[0],&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
readQueryFromClient(c->conn);
}
listEmpty(io_threads_list[0]);
/* Wait for all the other threads to end their work. */
/* Wait for all threads to end their work. */
while(1) {
unsigned long pending = 0;
for (int j = 1; j < server.io_threads_num; j++)
for (int j = 0; j < server.io_threads_num; j++)
pending += io_threads_pending[j];
if (pending == 0) break;
}
+1 -1
View File
@@ -82,10 +82,10 @@ sds keyspaceEventsFlagsToString(int flags) {
if (flags & NOTIFY_EXPIRED) res = sdscatlen(res,"x",1);
if (flags & NOTIFY_EVICTED) res = sdscatlen(res,"e",1);
if (flags & NOTIFY_STREAM) res = sdscatlen(res,"t",1);
if (flags & NOTIFY_KEY_MISS) res = sdscatlen(res,"m",1);
}
if (flags & NOTIFY_KEYSPACE) res = sdscatlen(res,"K",1);
if (flags & NOTIFY_KEYEVENT) res = sdscatlen(res,"E",1);
if (flags & NOTIFY_KEY_MISS) res = sdscatlen(res,"m",1);
return res;
}
+42 -17
View File
@@ -606,13 +606,21 @@ size_t stringObjectLen(robj *o) {
int getDoubleFromObject(const robj *o, double *target) {
double value;
char *eptr;
if (o == NULL) {
value = 0;
} else {
serverAssertWithInfo(NULL,o,o->type == OBJ_STRING);
if (sdsEncodedObject(o)) {
if (!string2d(o->ptr, sdslen(o->ptr), &value))
errno = 0;
value = strtod(o->ptr, &eptr);
if (sdslen(o->ptr) == 0 ||
isspace(((const char*)o->ptr)[0]) ||
(size_t)(eptr-(char*)o->ptr) != sdslen(o->ptr) ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
isnan(value))
return C_ERR;
} else if (o->encoding == OBJ_ENCODING_INT) {
value = (long)o->ptr;
@@ -640,13 +648,21 @@ int getDoubleFromObjectOrReply(client *c, robj *o, double *target, const char *m
int getLongDoubleFromObject(robj *o, long double *target) {
long double value;
char *eptr;
if (o == NULL) {
value = 0;
} else {
serverAssertWithInfo(NULL,o,o->type == OBJ_STRING);
if (sdsEncodedObject(o)) {
if (!string2ld(o->ptr, sdslen(o->ptr), &value))
errno = 0;
value = strtold(o->ptr, &eptr);
if (sdslen(o->ptr) == 0 ||
isspace(((const char*)o->ptr)[0]) ||
(size_t)(eptr-(char*)o->ptr) != sdslen(o->ptr) ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
isnan(value))
return C_ERR;
} else if (o->encoding == OBJ_ENCODING_INT) {
value = (long)o->ptr;
@@ -974,29 +990,38 @@ struct redisMemOverhead *getMemoryOverheadData(void) {
mh->repl_backlog = mem;
mem_total += mem;
mem = 0;
if (listLength(server.slaves)) {
listIter li;
listNode *ln;
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
mem += getClientOutputBufferMemoryUsage(c);
mem += sdsAllocSize(c->querybuf);
mem += sizeof(client);
}
}
mh->clients_slaves = mem;
mem_total+=mem;
mem = 0;
if (listLength(server.clients)) {
listIter li;
listNode *ln;
size_t mem_normal = 0, mem_slaves = 0;
listRewind(server.clients,&li);
while((ln = listNext(&li))) {
size_t mem_curr = 0;
client *c = listNodeValue(ln);
int type = getClientType(c);
mem_curr += getClientOutputBufferMemoryUsage(c);
mem_curr += sdsAllocSize(c->querybuf);
mem_curr += sizeof(client);
if (type == CLIENT_TYPE_SLAVE)
mem_slaves += mem_curr;
else
mem_normal += mem_curr;
if (c->flags & CLIENT_SLAVE && !(c->flags & CLIENT_MONITOR))
continue;
mem += getClientOutputBufferMemoryUsage(c);
mem += sdsAllocSize(c->querybuf);
mem += sizeof(client);
}
mh->clients_slaves = mem_slaves;
mh->clients_normal = mem_normal;
mem = mem_slaves + mem_normal;
}
mh->clients_normal = mem;
mem_total+=mem;
mem = 0;
@@ -1185,7 +1210,7 @@ sds getMemoryDoctorReport(void) {
* is MAXMEMORY_FLAG_LRU.
* Either or both of them may be <0, in that case, nothing is set. */
int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
long long lru_clock, int lru_multiplier) {
long long lru_clock) {
if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
if (lfu_freq >= 0) {
serverAssert(lfu_freq <= 255);
@@ -1197,7 +1222,7 @@ int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
* according to the LRU clock resolution this Redis
* instance was compiled with (normally 1000 ms, so the
* below statement will expand to lru_idle*1000/1000. */
lru_idle = lru_idle*lru_multiplier/LRU_CLOCK_RESOLUTION;
lru_idle = lru_idle*1000/LRU_CLOCK_RESOLUTION;
long lru_abs = lru_clock - lru_idle; /* Absolute access time. */
/* If the LRU field underflows (since LRU it is a wrapping
* clock), the best we can do is to provide a large enough LRU
+2 -6
View File
@@ -35,11 +35,7 @@ int clientSubscriptionsCount(client *c);
* Pubsub client replies API
*----------------------------------------------------------------------------*/
/* Send a pubsub message of type "message" to the client.
* Normally 'msg' is a Redis object containing the string to send as
* message. However if the caller sets 'msg' as NULL, it will be able
* to send a special message (for instance an Array type) by using the
* addReply*() API family. */
/* Send a pubsub message of type "message" to the client. */
void addReplyPubsubMessage(client *c, robj *channel, robj *msg) {
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
@@ -47,7 +43,7 @@ void addReplyPubsubMessage(client *c, robj *channel, robj *msg) {
addReplyPushLen(c,3);
addReply(c,shared.messagebulk);
addReplyBulk(c,channel);
if (msg) addReplyBulk(c,msg);
addReplyBulk(c,msg);
}
/* Send a pubsub message of type "pmessage" to the client. The difference
+1 -3
View File
@@ -1673,7 +1673,6 @@ int raxSeek(raxIterator *it, const char *op, unsigned char *ele, size_t len) {
* node, but will be our match, representing the key "f".
*
* So in that case, we don't seek backward. */
it->data = raxGetData(it->node);
} else {
if (gt && !raxIteratorNextStep(it,0)) return 0;
if (lt && !raxIteratorPrevStep(it,0)) return 0;
@@ -1766,7 +1765,6 @@ int raxRandomWalk(raxIterator *it, size_t steps) {
if (n->iskey) steps--;
}
it->node = n;
it->data = raxGetData(it->node);
return 1;
}
@@ -1793,7 +1791,7 @@ int raxCompare(raxIterator *iter, const char *op, unsigned char *key, size_t key
if (eq && key_len == iter->key_len) return 1;
else if (lt) return iter->key_len < key_len;
else if (gt) return iter->key_len > key_len;
else return 0; /* Avoid warning, just 'eq' is handled before. */
return 0;
} else if (cmp > 0) {
return gt ? 1 : 0;
} else /* (cmp < 0) */ {
+3 -6
View File
@@ -1871,10 +1871,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, robj *key) {
}
} else if (rdbtype == RDB_TYPE_MODULE || rdbtype == RDB_TYPE_MODULE_2) {
uint64_t moduleid = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) {
rdbReportReadError("Short read module id");
return NULL;
}
if (rioGetReadError(rdb)) return NULL;
moduleType *mt = moduleTypeLookupModuleByID(moduleid);
char name[10];
@@ -2009,7 +2006,7 @@ void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
/* The DB can take some non trivial amount of time to load. Update
* our cached time since it is used to create and update the last
* interaction time with clients and for other important things. */
updateCachedTime(0);
updateCachedTime();
if (server.masterhost && server.repl_state == REPL_STATE_TRANSFER)
replicationSendNewlineToMaster();
loadingProgress(r->processed_bytes);
@@ -2242,7 +2239,7 @@ int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi) {
if (expiretime != -1) setExpire(NULL,db,key,expiretime);
/* Set usage information (for eviction). */
objectSetLRUOrLFU(val,lfu_freq,lru_idle,lru_clock,1000);
objectSetLRUOrLFU(val,lfu_freq,lru_idle,lru_clock);
/* Decrement the key refcount since dbAdd() will take its
* own reference. */
+11 -33
View File
@@ -49,7 +49,6 @@
#include <hiredis.h>
#ifdef USE_OPENSSL
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <hiredis_ssl.h>
#endif
#include <sds.h> /* use sds.h from hiredis, so that only one set of sds functions will be present in the binary */
@@ -1595,15 +1594,15 @@ static int parseOptions(int argc, char **argv) {
#ifdef USE_OPENSSL
} else if (!strcmp(argv[i],"--tls")) {
config.tls = 1;
} else if (!strcmp(argv[i],"--sni") && !lastarg) {
} else if (!strcmp(argv[i],"--sni")) {
config.sni = argv[++i];
} else if (!strcmp(argv[i],"--cacertdir") && !lastarg) {
} else if (!strcmp(argv[i],"--cacertdir")) {
config.cacertdir = argv[++i];
} else if (!strcmp(argv[i],"--cacert") && !lastarg) {
} else if (!strcmp(argv[i],"--cacert")) {
config.cacert = argv[++i];
} else if (!strcmp(argv[i],"--cert") && !lastarg) {
} else if (!strcmp(argv[i],"--cert")) {
config.cert = argv[++i];
} else if (!strcmp(argv[i],"--key") && !lastarg) {
} else if (!strcmp(argv[i],"--key")) {
config.key = argv[++i];
#endif
} else if (!strcmp(argv[i],"-v") || !strcmp(argv[i], "--version")) {
@@ -1701,13 +1700,12 @@ static void usage(void) {
" -c Enable cluster mode (follow -ASK and -MOVED redirections).\n"
#ifdef USE_OPENSSL
" --tls Establish a secure TLS connection.\n"
" --sni <host> Server name indication for TLS.\n"
" --cacert <file> CA Certificate file to verify with.\n"
" --cacertdir <dir> Directory where trusted CA certificates are stored.\n"
" --cacert CA Certificate file to verify with.\n"
" --cacertdir Directory where trusted CA certificates are stored.\n"
" If neither cacert nor cacertdir are specified, the default\n"
" system-wide trusted root certs configuration will apply.\n"
" --cert <file> Client certificate to authenticate with.\n"
" --key <file> Private key file to authenticate with.\n"
" --cert Client certificate to authenticate with.\n"
" --key Private key file to authenticate with.\n"
#endif
" --raw Use raw formatting for replies (default when STDOUT is\n"
" not a tty).\n"
@@ -1947,20 +1945,8 @@ static void repl(void) {
}
}
/* Won't save auth or acl setuser commands in history file */
int dangerous = 0;
if (argv && argc > 0) {
if (!strcasecmp(argv[skipargs], "auth")) {
dangerous = 1;
} else if (skipargs+1 < argc &&
!strcasecmp(argv[skipargs], "acl") &&
!strcasecmp(argv[skipargs+1], "setuser"))
{
dangerous = 1;
}
}
if (!dangerous) {
/* Won't save auth command in history file */
if (!(argv && argc > 0 && !strcasecmp(argv[0+skipargs], "auth"))) {
if (history) linenoiseHistoryAdd(line);
if (historyfile) linenoiseHistorySave(historyfile);
}
@@ -7935,14 +7921,6 @@ int main(int argc, char **argv) {
parseEnv();
#ifdef USE_OPENSSL
if (config.tls) {
ERR_load_crypto_strings();
SSL_load_error_strings();
SSL_library_init();
}
#endif
/* Cluster Manager mode */
if (CLUSTER_MANAGER_MODE()) {
clusterManagerCommandProc *proc = validateClusterManagerCommand();
+18 -66
View File
@@ -33,7 +33,6 @@
#define REDISMODULE_KEYTYPE_SET 4
#define REDISMODULE_KEYTYPE_ZSET 5
#define REDISMODULE_KEYTYPE_MODULE 6
#define REDISMODULE_KEYTYPE_STREAM 7
/* Reply types. */
#define REDISMODULE_REPLY_UNKNOWN -1
@@ -110,8 +109,6 @@
#define REDISMODULE_CTX_FLAGS_REPLICA_IS_ONLINE (1<<17)
/* There is currently some background process active. */
#define REDISMODULE_CTX_FLAGS_ACTIVE_CHILD (1<<18)
/* The next EXEC will fail due to dirty CAS (touched keys). */
#define REDISMODULE_CTX_FLAGS_MULTI_DIRTY (1<<19)
/* Keyspace changes notification classes. Every class is associated with a
* character for configuration purposes.
@@ -127,8 +124,8 @@
#define REDISMODULE_NOTIFY_EXPIRED (1<<8) /* x */
#define REDISMODULE_NOTIFY_EVICTED (1<<9) /* e */
#define REDISMODULE_NOTIFY_STREAM (1<<10) /* t */
#define REDISMODULE_NOTIFY_KEY_MISS (1<<11) /* m (Note: This one is excluded from REDISMODULE_NOTIFY_ALL on purpose) */
#define REDISMODULE_NOTIFY_ALL (REDISMODULE_NOTIFY_GENERIC | REDISMODULE_NOTIFY_STRING | REDISMODULE_NOTIFY_LIST | REDISMODULE_NOTIFY_SET | REDISMODULE_NOTIFY_HASH | REDISMODULE_NOTIFY_ZSET | REDISMODULE_NOTIFY_EXPIRED | REDISMODULE_NOTIFY_EVICTED | REDISMODULE_NOTIFY_STREAM) /* A */
#define REDISMODULE_NOTIFY_KEY_MISS (1<<11) /* m */
#define REDISMODULE_NOTIFY_ALL (REDISMODULE_NOTIFY_GENERIC | REDISMODULE_NOTIFY_STRING | REDISMODULE_NOTIFY_LIST | REDISMODULE_NOTIFY_SET | REDISMODULE_NOTIFY_HASH | REDISMODULE_NOTIFY_ZSET | REDISMODULE_NOTIFY_EXPIRED | REDISMODULE_NOTIFY_EVICTED | REDISMODULE_NOTIFY_STREAM | REDISMODULE_NOTIFY_KEY_MISS) /* A */
/* A special pointer that we can use between the core and the module to signal
@@ -395,9 +392,8 @@ typedef struct RedisModuleDictIter RedisModuleDictIter;
typedef struct RedisModuleCommandFilterCtx RedisModuleCommandFilterCtx;
typedef struct RedisModuleCommandFilter RedisModuleCommandFilter;
typedef struct RedisModuleInfoCtx RedisModuleInfoCtx;
typedef struct RedisModuleServerInfoData RedisModuleServerInfoData;
typedef struct RedisModuleScanCursor RedisModuleScanCursor;
typedef struct RedisModuleUser RedisModuleUser;
typedef struct RedisModuleAuthCtx RedisModuleAuthCtx;
typedef int (*RedisModuleCmdFunc)(RedisModuleCtx *ctx, RedisModuleString **argv, int argc);
typedef void (*RedisModuleDisconnectFunc)(RedisModuleCtx *ctx, RedisModuleBlockedClient *bc);
@@ -415,9 +411,7 @@ typedef void (*RedisModuleTimerProc)(RedisModuleCtx *ctx, void *data);
typedef void (*RedisModuleCommandFilterFunc) (RedisModuleCommandFilterCtx *filter);
typedef void (*RedisModuleForkDoneHandler) (int exitcode, int bysignal, void *user_data);
typedef void (*RedisModuleInfoFunc)(RedisModuleInfoCtx *ctx, int for_crash_report);
typedef void (*RedisModuleScanCB)(RedisModuleCtx *ctx, RedisModuleString *keyname, RedisModuleKey *key, void *privdata);
typedef void (*RedisModuleScanKeyCB)(RedisModuleKey *key, RedisModuleString *field, RedisModuleString *value, void *privdata);
typedef void (*RedisModuleUserChangedFunc) (uint64_t client_id, void *privdata);
typedef void (*RedisModuleUserChangedFunc)(RedisModuleCtx *ctx, void *privdata);
#define REDISMODULE_TYPE_METHOD_VERSION 2
typedef struct RedisModuleTypeMethods {
@@ -467,7 +461,6 @@ size_t REDISMODULE_API_FUNC(RedisModule_CallReplyLength)(RedisModuleCallReply *r
RedisModuleCallReply *REDISMODULE_API_FUNC(RedisModule_CallReplyArrayElement)(RedisModuleCallReply *reply, size_t idx);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateString)(RedisModuleCtx *ctx, const char *ptr, size_t len);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringFromLongLong)(RedisModuleCtx *ctx, long long ll);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringFromLongDouble)(RedisModuleCtx *ctx, long double ld, int humanfriendly);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringFromString)(RedisModuleCtx *ctx, const RedisModuleString *str);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringPrintf)(RedisModuleCtx *ctx, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule_FreeString)(RedisModuleCtx *ctx, RedisModuleString *str);
@@ -485,11 +478,9 @@ int REDISMODULE_API_FUNC(RedisModule_ReplyWithEmptyString)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithVerbatimString)(RedisModuleCtx *ctx, const char *buf, size_t len);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithNull)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithDouble)(RedisModuleCtx *ctx, double d);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithLongDouble)(RedisModuleCtx *ctx, long double d);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithCallReply)(RedisModuleCtx *ctx, RedisModuleCallReply *reply);
int REDISMODULE_API_FUNC(RedisModule_StringToLongLong)(const RedisModuleString *str, long long *ll);
int REDISMODULE_API_FUNC(RedisModule_StringToDouble)(const RedisModuleString *str, double *d);
int REDISMODULE_API_FUNC(RedisModule_StringToLongDouble)(const RedisModuleString *str, long double *d);
void REDISMODULE_API_FUNC(RedisModule_AutoMemory)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_Replicate)(RedisModuleCtx *ctx, const char *cmdname, const char *fmt, ...);
int REDISMODULE_API_FUNC(RedisModule_ReplicateVerbatim)(RedisModuleCtx *ctx);
@@ -526,11 +517,9 @@ unsigned long long REDISMODULE_API_FUNC(RedisModule_GetClientId)(RedisModuleCtx
int REDISMODULE_API_FUNC(RedisModule_GetClientInfoById)(void *ci, uint64_t id);
int REDISMODULE_API_FUNC(RedisModule_PublishMessage)(RedisModuleCtx *ctx, RedisModuleString *channel, RedisModuleString *message);
int REDISMODULE_API_FUNC(RedisModule_GetContextFlags)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_AvoidReplicaTraffic)();
void *REDISMODULE_API_FUNC(RedisModule_PoolAlloc)(RedisModuleCtx *ctx, size_t bytes);
RedisModuleType *REDISMODULE_API_FUNC(RedisModule_CreateDataType)(RedisModuleCtx *ctx, const char *name, int encver, RedisModuleTypeMethods *typemethods);
int REDISMODULE_API_FUNC(RedisModule_ModuleTypeSetValue)(RedisModuleKey *key, RedisModuleType *mt, void *value);
int REDISMODULE_API_FUNC(RedisModule_ModuleTypeReplaceValue)(RedisModuleKey *key, RedisModuleType *mt, void *new_value, void **old_value);
RedisModuleType *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetType)(RedisModuleKey *key);
void *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetValue)(RedisModuleKey *key);
int REDISMODULE_API_FUNC(RedisModule_IsIOError)(RedisModuleIO *io);
@@ -551,8 +540,6 @@ void REDISMODULE_API_FUNC(RedisModule_SaveFloat)(RedisModuleIO *io, float value)
float REDISMODULE_API_FUNC(RedisModule_LoadFloat)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_SaveLongDouble)(RedisModuleIO *io, long double value);
long double REDISMODULE_API_FUNC(RedisModule_LoadLongDouble)(RedisModuleIO *io);
void *REDISMODULE_API_FUNC(RedisModule_LoadDataTypeFromString)(const RedisModuleString *str, const RedisModuleType *mt);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_SaveDataTypeToString)(RedisModuleCtx *ctx, void *data, const RedisModuleType *mt);
void REDISMODULE_API_FUNC(RedisModule_Log)(RedisModuleCtx *ctx, const char *level, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule_LogIOError)(RedisModuleIO *io, const char *levelstr, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule__Assert)(const char *estr, const char *file, int line);
@@ -598,26 +585,13 @@ int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldCString)(RedisModuleInfoCtx *ct
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldDouble)(RedisModuleInfoCtx *ctx, char *field, double value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldLongLong)(RedisModuleInfoCtx *ctx, char *field, long long value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldULongLong)(RedisModuleInfoCtx *ctx, char *field, unsigned long long value);
RedisModuleServerInfoData *REDISMODULE_API_FUNC(RedisModule_GetServerInfo)(RedisModuleCtx *ctx, const char *section);
void REDISMODULE_API_FUNC(RedisModule_FreeServerInfo)(RedisModuleCtx *ctx, RedisModuleServerInfoData *data);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_ServerInfoGetField)(RedisModuleCtx *ctx, RedisModuleServerInfoData *data, const char* field);
const char *REDISMODULE_API_FUNC(RedisModule_ServerInfoGetFieldC)(RedisModuleServerInfoData *data, const char* field);
long long REDISMODULE_API_FUNC(RedisModule_ServerInfoGetFieldSigned)(RedisModuleServerInfoData *data, const char* field, int *out_err);
unsigned long long REDISMODULE_API_FUNC(RedisModule_ServerInfoGetFieldUnsigned)(RedisModuleServerInfoData *data, const char* field, int *out_err);
double REDISMODULE_API_FUNC(RedisModule_ServerInfoGetFieldDouble)(RedisModuleServerInfoData *data, const char* field, int *out_err);
int REDISMODULE_API_FUNC(RedisModule_SubscribeToServerEvent)(RedisModuleCtx *ctx, RedisModuleEvent event, RedisModuleEventCallback callback);
int REDISMODULE_API_FUNC(RedisModule_SetLRU)(RedisModuleKey *key, mstime_t lru_idle);
int REDISMODULE_API_FUNC(RedisModule_GetLRU)(RedisModuleKey *key, mstime_t *lru_idle);
int REDISMODULE_API_FUNC(RedisModule_SetLFU)(RedisModuleKey *key, long long lfu_freq);
int REDISMODULE_API_FUNC(RedisModule_GetLFU)(RedisModuleKey *key, long long *lfu_freq);
int REDISMODULE_API_FUNC(RedisModule_SetLRUOrLFU)(RedisModuleKey *key, long long lfu_freq, long long lru_idle);
int REDISMODULE_API_FUNC(RedisModule_GetLRUOrLFU)(RedisModuleKey *key, long long *lfu_freq, long long *lru_idle);
RedisModuleBlockedClient *REDISMODULE_API_FUNC(RedisModule_BlockClientOnKeys)(RedisModuleCtx *ctx, RedisModuleCmdFunc reply_callback, RedisModuleCmdFunc timeout_callback, void (*free_privdata)(RedisModuleCtx*,void*), long long timeout_ms, RedisModuleString **keys, int numkeys, void *privdata);
void REDISMODULE_API_FUNC(RedisModule_SignalKeyAsReady)(RedisModuleCtx *ctx, RedisModuleString *key);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_GetBlockedClientReadyKey)(RedisModuleCtx *ctx);
RedisModuleScanCursor *REDISMODULE_API_FUNC(RedisModule_ScanCursorCreate)();
void REDISMODULE_API_FUNC(RedisModule_ScanCursorRestart)(RedisModuleScanCursor *cursor);
void REDISMODULE_API_FUNC(RedisModule_ScanCursorDestroy)(RedisModuleScanCursor *cursor);
int REDISMODULE_API_FUNC(RedisModule_Scan)(RedisModuleCtx *ctx, RedisModuleScanCursor *cursor, RedisModuleScanCB fn, void *privdata);
int REDISMODULE_API_FUNC(RedisModule_ScanKey)(RedisModuleKey *key, RedisModuleScanCursor *cursor, RedisModuleScanKeyCB fn, void *privdata);
/* Experimental APIs */
#ifdef REDISMODULE_EXPERIMENTAL_API
#define REDISMODULE_EXPERIMENTAL_API_VERSION 3
@@ -662,14 +636,13 @@ int REDISMODULE_API_FUNC(RedisModule_CommandFilterArgDelete)(RedisModuleCommandF
int REDISMODULE_API_FUNC(RedisModule_Fork)(RedisModuleForkDoneHandler cb, void *user_data);
int REDISMODULE_API_FUNC(RedisModule_ExitFromChild)(int retcode);
int REDISMODULE_API_FUNC(RedisModule_KillForkChild)(int child_pid);
float REDISMODULE_API_FUNC(RedisModule_GetUsedMemoryRatio)();
size_t REDISMODULE_API_FUNC(RedisModule_MallocSize)(void* ptr);
RedisModuleUser *REDISMODULE_API_FUNC(RedisModule_CreateModuleUser)(const char *name);
void REDISMODULE_API_FUNC(RedisModule_FreeModuleUser)(RedisModuleUser *user);
int REDISMODULE_API_FUNC(RedisModule_SetModuleUserACL)(RedisModuleUser *user, const char* acl);
int REDISMODULE_API_FUNC(RedisModule_AuthenticateClientWithACLUser)(RedisModuleCtx *ctx, const char *name, size_t len, RedisModuleUserChangedFunc callback, void *privdata, uint64_t *client_id);
int REDISMODULE_API_FUNC(RedisModule_AuthenticateClientWithUser)(RedisModuleCtx *ctx, RedisModuleUser *user, RedisModuleUserChangedFunc callback, void *privdata, uint64_t *client_id);
void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);
int REDISMODULE_API_FUNC(RedisModule_AuthClientWithACLUser)(RedisModuleCtx *ctx, const char* name, size_t len, RedisModuleAuthCtx *auth_ctx);
int REDISMODULE_API_FUNC(RedisModule_AuthClientWithUser)(RedisModuleCtx *ctx, RedisModuleUser *module_user, RedisModuleAuthCtx *auth_ctx);
void REDISMODULE_API_FUNC(RedisModule_RevokeAuthentication)(RedisModuleAuthCtx *ctx);
RedisModuleAuthCtx *REDISMODULE_API_FUNC(RedisModule_CreateAuthCtx)(RedisModuleUserChangedFunc callback, void *privdata);
#endif
#define RedisModule_IsAOFClient(id) ((id) == UINT64_MAX)
@@ -703,7 +676,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(ReplyWithNull);
REDISMODULE_GET_API(ReplyWithCallReply);
REDISMODULE_GET_API(ReplyWithDouble);
REDISMODULE_GET_API(ReplyWithLongDouble);
REDISMODULE_GET_API(GetSelectedDb);
REDISMODULE_GET_API(SelectDb);
REDISMODULE_GET_API(OpenKey);
@@ -714,7 +686,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(ListPop);
REDISMODULE_GET_API(StringToLongLong);
REDISMODULE_GET_API(StringToDouble);
REDISMODULE_GET_API(StringToLongDouble);
REDISMODULE_GET_API(Call);
REDISMODULE_GET_API(CallReplyProto);
REDISMODULE_GET_API(FreeCallReply);
@@ -726,7 +697,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(CreateStringFromCallReply);
REDISMODULE_GET_API(CreateString);
REDISMODULE_GET_API(CreateStringFromLongLong);
REDISMODULE_GET_API(CreateStringFromLongDouble);
REDISMODULE_GET_API(CreateStringFromString);
REDISMODULE_GET_API(CreateStringPrintf);
REDISMODULE_GET_API(FreeString);
@@ -763,11 +733,9 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(KeyAtPos);
REDISMODULE_GET_API(GetClientId);
REDISMODULE_GET_API(GetContextFlags);
REDISMODULE_GET_API(AvoidReplicaTraffic);
REDISMODULE_GET_API(PoolAlloc);
REDISMODULE_GET_API(CreateDataType);
REDISMODULE_GET_API(ModuleTypeSetValue);
REDISMODULE_GET_API(ModuleTypeReplaceValue);
REDISMODULE_GET_API(ModuleTypeGetType);
REDISMODULE_GET_API(ModuleTypeGetValue);
REDISMODULE_GET_API(IsIOError);
@@ -787,8 +755,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(LoadFloat);
REDISMODULE_GET_API(SaveLongDouble);
REDISMODULE_GET_API(LoadLongDouble);
REDISMODULE_GET_API(SaveDataTypeToString);
REDISMODULE_GET_API(LoadDataTypeFromString);
REDISMODULE_GET_API(EmitAOF);
REDISMODULE_GET_API(Log);
REDISMODULE_GET_API(LogIOError);
@@ -835,28 +801,14 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(InfoAddFieldDouble);
REDISMODULE_GET_API(InfoAddFieldLongLong);
REDISMODULE_GET_API(InfoAddFieldULongLong);
REDISMODULE_GET_API(GetServerInfo);
REDISMODULE_GET_API(FreeServerInfo);
REDISMODULE_GET_API(ServerInfoGetField);
REDISMODULE_GET_API(ServerInfoGetFieldC);
REDISMODULE_GET_API(ServerInfoGetFieldSigned);
REDISMODULE_GET_API(ServerInfoGetFieldUnsigned);
REDISMODULE_GET_API(ServerInfoGetFieldDouble);
REDISMODULE_GET_API(GetClientInfoById);
REDISMODULE_GET_API(PublishMessage);
REDISMODULE_GET_API(SubscribeToServerEvent);
REDISMODULE_GET_API(SetLRU);
REDISMODULE_GET_API(GetLRU);
REDISMODULE_GET_API(SetLFU);
REDISMODULE_GET_API(GetLFU);
REDISMODULE_GET_API(SetLRUOrLFU);
REDISMODULE_GET_API(GetLRUOrLFU);
REDISMODULE_GET_API(BlockClientOnKeys);
REDISMODULE_GET_API(SignalKeyAsReady);
REDISMODULE_GET_API(GetBlockedClientReadyKey);
REDISMODULE_GET_API(ScanCursorCreate);
REDISMODULE_GET_API(ScanCursorRestart);
REDISMODULE_GET_API(ScanCursorDestroy);
REDISMODULE_GET_API(Scan);
REDISMODULE_GET_API(ScanKey);
#ifdef REDISMODULE_EXPERIMENTAL_API
REDISMODULE_GET_API(GetThreadSafeContext);
@@ -900,14 +852,14 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(Fork);
REDISMODULE_GET_API(ExitFromChild);
REDISMODULE_GET_API(KillForkChild);
REDISMODULE_GET_API(GetUsedMemoryRatio);
REDISMODULE_GET_API(MallocSize);
REDISMODULE_GET_API(CreateModuleUser);
REDISMODULE_GET_API(FreeModuleUser);
REDISMODULE_GET_API(SetModuleUserACL);
REDISMODULE_GET_API(DeauthenticateAndCloseClient);
REDISMODULE_GET_API(AuthenticateClientWithACLUser);
REDISMODULE_GET_API(AuthenticateClientWithUser);
REDISMODULE_GET_API(RevokeAuthentication);
REDISMODULE_GET_API(CreateAuthCtx);
REDISMODULE_GET_API(AuthClientWithACLUser);
REDISMODULE_GET_API(AuthClientWithUser);
#endif
if (RedisModule_IsModuleNameBusy && RedisModule_IsModuleNameBusy(name)) return REDISMODULE_ERR;
+12 -28
View File
@@ -31,7 +31,6 @@
#include "server.h"
#include "cluster.h"
#include "bio.h"
#include <sys/time.h>
#include <unistd.h>
@@ -1339,8 +1338,8 @@ void disklessLoadRestoreBackups(redisDb *backup, int restore, int empty_db_flags
server.db[i] = backup[i];
}
} else {
/* Delete (Pass EMPTYDB_BACKUP in order to avoid firing module events) . */
emptyDbGeneric(backup,-1,empty_db_flags|EMPTYDB_BACKUP,replicationEmptyDbCallback);
/* Delete. */
emptyDbGeneric(backup,-1,empty_db_flags,replicationEmptyDbCallback);
for (int i=0; i<server.dbnum; i++) {
dictRelease(backup[i].dict);
dictRelease(backup[i].expires);
@@ -1354,7 +1353,7 @@ void disklessLoadRestoreBackups(redisDb *backup, int restore, int empty_db_flags
void readSyncBulkPayload(connection *conn) {
char buf[4096];
ssize_t nread, readlen, nwritten;
int use_diskless_load = useDisklessLoad();
int use_diskless_load;
redisDb *diskless_load_backup = NULL;
int empty_db_flags = server.repl_slave_lazy_flush ? EMPTYDB_ASYNC :
EMPTYDB_NO_FLAGS;
@@ -1411,18 +1410,19 @@ void readSyncBulkPayload(connection *conn) {
server.repl_transfer_size = 0;
serverLog(LL_NOTICE,
"MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF %s",
use_diskless_load? "to parser":"to disk");
useDisklessLoad()? "to parser":"to disk");
} else {
usemark = 0;
server.repl_transfer_size = strtol(buf+1,NULL,10);
serverLog(LL_NOTICE,
"MASTER <-> REPLICA sync: receiving %lld bytes from master %s",
(long long) server.repl_transfer_size,
use_diskless_load? "to parser":"to disk");
useDisklessLoad()? "to parser":"to disk");
}
return;
}
use_diskless_load = useDisklessLoad();
if (!use_diskless_load) {
/* Read the data from the socket, store it to a file and search
* for the EOF. */
@@ -1524,6 +1524,7 @@ void readSyncBulkPayload(connection *conn) {
/* We need to stop any AOF rewriting child before flusing and parsing
* the RDB, otherwise we'll create a copy-on-write disaster. */
if (server.aof_state != AOF_OFF) stopAppendOnly();
signalFlushedDb(-1);
/* When diskless RDB loading is used by replicas, it may be configured
* in order to save the current DB instead of throwing it away,
@@ -1531,15 +1532,10 @@ void readSyncBulkPayload(connection *conn) {
if (use_diskless_load &&
server.repl_diskless_load == REPL_DISKLESS_LOAD_SWAPDB)
{
/* Create a backup of server.db[] and initialize to empty
* dictionaries */
diskless_load_backup = disklessLoadMakeBackups();
} else {
emptyDb(-1,empty_db_flags,replicationEmptyDbCallback);
}
/* We call to emptyDb even in case of REPL_DISKLESS_LOAD_SWAPDB
* (Where disklessLoadMakeBackups left server.db empty) because we
* want to execute all the auxiliary logic of emptyDb (Namely,
* fire module events) */
emptyDb(-1,empty_db_flags,replicationEmptyDbCallback);
/* Before loading the DB into memory we need to delete the readable
* handler, otherwise it will get called recursively since
@@ -1620,20 +1616,14 @@ void readSyncBulkPayload(connection *conn) {
killRDBChild();
}
/* Rename rdb like renaming rewrite aof asynchronously. */
int old_rdb_fd = open(server.rdb_filename,O_RDONLY|O_NONBLOCK);
if (rename(server.repl_transfer_tmpfile,server.rdb_filename) == -1) {
serverLog(LL_WARNING,
"Failed trying to rename the temp DB into %s in "
"MASTER <-> REPLICA synchronization: %s",
server.rdb_filename, strerror(errno));
cancelReplicationHandshake();
if (old_rdb_fd != -1) close(old_rdb_fd);
return;
}
/* Close old rdb asynchronously. */
if (old_rdb_fd != -1) bioCreateBackgroundJob(BIO_CLOSE_FILE,(void*)(long)old_rdb_fd,NULL,NULL);
if (rdbLoad(server.rdb_filename,&rsi,RDBFLAGS_REPLICATION) != C_OK) {
serverLog(LL_WARNING,
"Failed trying to load the MASTER synchronization "
@@ -1675,11 +1665,6 @@ void readSyncBulkPayload(connection *conn) {
if (server.repl_backlog == NULL) createReplicationBacklog();
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Finished with success");
if (server.supervised_mode == SUPERVISED_SYSTEMD) {
redisCommunicateSystemd("STATUS=MASTER <-> REPLICA sync: Finished with success. Ready to accept connections.\n");
redisCommunicateSystemd("READY=1\n");
}
/* Restart the AOF subsystem now that we finished the sync. This
* will trigger an AOF rewrite, and when done will start appending
* to the new file. */
@@ -2402,10 +2387,6 @@ void replicationUnsetMaster(void) {
moduleFireServerEvent(REDISMODULE_EVENT_REPLICATION_ROLE_CHANGED,
REDISMODULE_EVENT_REPLROLECHANGED_NOW_MASTER,
NULL);
/* Restart the AOF subsystem in case we shut it down during a sync when
* we were still a slave. */
if (server.aof_enabled && server.aof_state == AOF_OFF) restartAOFAfterSYNC();
}
/* This function is called when the slave lose the connection with the
@@ -2443,6 +2424,9 @@ void replicaofCommand(client *c) {
serverLog(LL_NOTICE,"MASTER MODE enabled (user request from '%s')",
client);
sdsfree(client);
/* Restart the AOF subsystem in case we shut it down during a sync when
* we were still a slave. */
if (server.aof_enabled && server.aof_state == AOF_OFF) restartAOFAfterSYNC();
}
} else {
long port;
+21 -44
View File
@@ -485,6 +485,13 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
static size_t cached_objects_len[LUA_CMD_OBJCACHE_SIZE];
static int inuse = 0; /* Recursive calls detection. */
/* Reflect MULTI state */
if (server.lua_multi_emitted || (server.lua_caller->flags & CLIENT_MULTI)) {
c->flags |= CLIENT_MULTI;
} else {
c->flags &= ~CLIENT_MULTI;
}
/* By using Lua debug hooks it is possible to trigger a recursive call
* to luaRedisGenericCommand(), which normally should never happen.
* To make this function reentrant is futile and makes it slower, but
@@ -606,10 +613,8 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
}
/* Check the ACLs. */
int acl_keypos;
int acl_retval = ACLCheckCommandPerm(c,&acl_keypos);
int acl_retval = ACLCheckCommandPerm(c);
if (acl_retval != ACL_OK) {
addACLLogEntry(c,acl_retval,acl_keypos,NULL);
if (acl_retval == ACL_DENIED_CMD)
luaPushError(lua, "The user executing the script can't run this "
"command or subcommand");
@@ -674,27 +679,15 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
if (server.cluster_enabled && !server.loading &&
!(server.lua_caller->flags & CLIENT_MASTER))
{
int error_code;
/* Duplicate relevant flags in the lua client. */
c->flags &= ~(CLIENT_READONLY|CLIENT_ASKING);
c->flags |= server.lua_caller->flags & (CLIENT_READONLY|CLIENT_ASKING);
if (getNodeByQuery(c,c->cmd,c->argv,c->argc,NULL,&error_code) !=
if (getNodeByQuery(c,c->cmd,c->argv,c->argc,NULL,NULL) !=
server.cluster->myself)
{
if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) {
luaPushError(lua,
"Lua script attempted to execute a write command while the "
"cluster is down and readonly");
} else if (error_code == CLUSTER_REDIR_DOWN_STATE) {
luaPushError(lua,
"Lua script attempted to execute a command while the "
"cluster is down");
} else {
luaPushError(lua,
"Lua script attempted to access a non local key in a "
"cluster node");
}
luaPushError(lua,
"Lua script attempted to access a non local key in a "
"cluster node");
goto cleanup;
}
}
@@ -710,9 +703,6 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
{
execCommandPropagateMulti(server.lua_caller);
server.lua_multi_emitted = 1;
/* Now we are in the MULTI context, the lua_client should be
* flag as CLIENT_MULTI. */
c->flags |= CLIENT_MULTI;
}
/* Run the command */
@@ -1439,22 +1429,6 @@ void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
}
}
void prepareLuaClient(void) {
/* Select the right DB in the context of the Lua client */
selectDb(server.lua_client,server.lua_caller->db->id);
server.lua_client->resp = 2; /* Default is RESP2, scripts can change it. */
/* If we are in MULTI context, flag Lua client as CLIENT_MULTI. */
if (server.lua_caller->flags & CLIENT_MULTI) {
server.lua_client->flags |= CLIENT_MULTI;
}
}
void resetLuaClient(void) {
/* After the script done, remove the MULTI state. */
server.lua_client->flags &= ~CLIENT_MULTI;
}
void evalGenericCommand(client *c, int evalsha) {
lua_State *lua = server.lua;
char funcname[43];
@@ -1543,6 +1517,10 @@ void evalGenericCommand(client *c, int evalsha) {
luaSetGlobalArray(lua,"KEYS",c->argv+3,numkeys);
luaSetGlobalArray(lua,"ARGV",c->argv+3+numkeys,c->argc-3-numkeys);
/* Set the Lua client database and protocol. */
selectDb(server.lua_client,c->db->id);
server.lua_client->resp = 2; /* Default is RESP2, scripts can change it. */
/* Set a hook in order to be able to stop the script execution if it
* is running for too much time.
* We set the hook only if the time limit is enabled as the hook will
@@ -1562,15 +1540,11 @@ void evalGenericCommand(client *c, int evalsha) {
delhook = 1;
}
prepareLuaClient();
/* At this point whether this script was never seen before or if it was
* already defined, we can call it. We have zero arguments and expect
* a single return value. */
err = lua_pcall(lua,0,1,-2);
resetLuaClient();
/* Perform some cleanup that we need to do both on error and success. */
if (delhook) lua_sethook(lua,NULL,0,0); /* Disable hook */
if (server.lua_timedout) {
@@ -1617,7 +1591,11 @@ void evalGenericCommand(client *c, int evalsha) {
if (server.lua_replicate_commands) {
preventCommandPropagation(c);
if (server.lua_multi_emitted) {
execCommandPropagateExec(c);
robj *propargv[1];
propargv[0] = createStringObject("EXEC",4);
alsoPropagate(server.execCommand,c->db->id,propargv,1,
PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(propargv[0]);
}
}
@@ -2475,7 +2453,6 @@ void ldbEval(lua_State *lua, sds *argv, int argc) {
ldbLog(sdscatfmt(sdsempty(),"<error> %s",lua_tostring(lua,-1)));
lua_pop(lua,1);
sdsfree(code);
sdsfree(expr);
return;
}
}
+6 -9
View File
@@ -3993,14 +3993,11 @@ int sentinelSendSlaveOf(sentinelRedisInstance *ri, char *host, int port) {
* an issue because CLIENT is variadic command, so Redis will not
* recognized as a syntax error, and the transaction will not fail (but
* only the unsupported command will fail). */
for (int type = 0; type < 2; type++) {
retval = redisAsyncCommand(ri->link->cc,
sentinelDiscardReplyCallback, ri, "%s KILL TYPE %s",
sentinelInstanceMapCommand(ri,"CLIENT"),
type == 0 ? "normal" : "pubsub");
if (retval == C_ERR) return retval;
ri->link->pending_commands++;
}
retval = redisAsyncCommand(ri->link->cc,
sentinelDiscardReplyCallback, ri, "%s KILL TYPE normal",
sentinelInstanceMapCommand(ri,"CLIENT"));
if (retval == C_ERR) return retval;
ri->link->pending_commands++;
retval = redisAsyncCommand(ri->link->cc,
sentinelDiscardReplyCallback, ri, "%s",
@@ -4308,7 +4305,7 @@ void sentinelFailoverDetectEnd(sentinelRedisInstance *master) {
sentinelRedisInstance *slave = dictGetVal(de);
int retval;
if (slave->flags & (SRI_PROMOTED|SRI_RECONF_DONE|SRI_RECONF_SENT)) continue;
if (slave->flags & (SRI_RECONF_DONE|SRI_RECONF_SENT)) continue;
if (slave->link->disconnected) continue;
retval = sentinelSendSlaveOf(slave,
+187 -121
View File
@@ -579,7 +579,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"select",selectCommand,2,
"ok-loading fast ok-stale @keyspace",
"ok-loading fast @keyspace",
0,NULL,0,0,0,0,0,0},
{"swapdb",swapdbCommand,3,
@@ -660,7 +660,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"lastsave",lastsaveCommand,1,
"read-only random fast ok-loading ok-stale @admin @dangerous",
"read-only random fast @admin @dangerous",
0,NULL,0,0,0,0,0,0},
{"type",typeCommand,2,
@@ -708,7 +708,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"monitor",monitorCommand,1,
"admin no-script ok-loading ok-stale",
"admin no-script",
0,NULL,0,0,0,0,0,0},
{"ttl",ttlCommand,2,
@@ -740,7 +740,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"debug",debugCommand,-2,
"admin no-script ok-loading ok-stale",
"admin no-script",
0,NULL,0,0,0,0,0,0},
{"config",configCommand,-2,
@@ -817,14 +817,14 @@ struct redisCommand redisCommandTable[] = {
{"memory",memoryCommand,-2,
"random read-only",
0,memoryGetKeys,0,0,0,0,0,0},
0,NULL,0,0,0,0,0,0},
{"client",clientCommand,-2,
"admin no-script random ok-loading ok-stale @connection",
"admin no-script random @connection",
0,NULL,0,0,0,0,0,0},
{"hello",helloCommand,-2,
"no-auth no-script fast no-monitor ok-loading ok-stale no-slowlog @connection",
"no-auth no-script fast no-monitor no-slowlog @connection",
0,NULL,0,0,0,0,0,0},
/* EVAL can modify the dataset, however it is not flagged as a write
@@ -838,7 +838,7 @@ struct redisCommand redisCommandTable[] = {
0,evalGetKeys,0,0,0,0,0,0},
{"slowlog",slowlogCommand,-2,
"admin random ok-loading ok-stale",
"admin random",
0,NULL,0,0,0,0,0,0},
{"script",scriptCommand,-2,
@@ -846,7 +846,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"time",timeCommand,1,
"read-only random fast ok-loading ok-stale",
"read-only random fast",
0,NULL,0,0,0,0,0,0},
{"bitop",bitopCommand,-4,
@@ -999,7 +999,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"acl",aclCommand,-2,
"admin no-script no-slowlog ok-loading ok-stale",
"admin no-script ok-loading ok-stale",
0,NULL,0,0,0,0,0,0}
};
@@ -1498,7 +1498,7 @@ int clientsCronHandleTimeout(client *c, mstime_t now_ms) {
time_t now = now_ms/1000;
if (server.maxidletime &&
!(c->flags & CLIENT_SLAVE) && /* no timeout for slaves and monitors */
!(c->flags & CLIENT_SLAVE) && /* no timeout for slaves */
!(c->flags & CLIENT_MASTER) && /* no timeout for masters */
!(c->flags & CLIENT_BLOCKED) && /* no timeout for BLPOP */
!(c->flags & CLIENT_PUBSUB) && /* no timeout for Pub/Sub clients */
@@ -1691,7 +1691,8 @@ void databasesCron(void) {
}
/* Defrag keys gradually. */
activeDefragCycle();
if (server.active_defrag_enabled)
activeDefragCycle();
/* Perform hash tables rehashing if needed, but only if there are no
* other processes saving the DB on disk. Otherwise rehashing is bad
@@ -1735,29 +1736,20 @@ void databasesCron(void) {
/* We take a cached value of the unix time in the global state because with
* virtual memory and aging there is to store the current time in objects at
* every object access, and accuracy is not needed. To access a global var is
* a lot faster than calling time(NULL).
*
* This function should be fast because it is called at every command execution
* in call(), so it is possible to decide if to update the daylight saving
* info or not using the 'update_daylight_info' argument. Normally we update
* such info only when calling this function from serverCron() but not when
* calling it from call(). */
void updateCachedTime(int update_daylight_info) {
server.ustime = ustime();
server.mstime = server.ustime / 1000;
server.unixtime = server.mstime / 1000;
* a lot faster than calling time(NULL) */
void updateCachedTime(void) {
server.unixtime = time(NULL);
server.mstime = mstime();
/* To get information about daylight saving time, we need to call
* localtime_r and cache the result. However calling localtime_r in this
* context is safe since we will never fork() while here, in the main
* thread. The logging function will call a thread safe version of
* localtime that has no locks. */
if (update_daylight_info) {
struct tm tm;
time_t ut = server.unixtime;
localtime_r(&ut,&tm);
server.daylight_active = tm.tm_isdst;
}
struct tm tm;
time_t ut = server.unixtime;
localtime_r(&ut,&tm);
server.daylight_active = tm.tm_isdst;
}
void checkChildrenDone(void) {
@@ -1846,7 +1838,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
if (server.watchdog_period) watchdogScheduleSignal(server.watchdog_period);
/* Update the time cache. */
updateCachedTime(1);
updateCachedTime();
server.hz = server.config_hz;
/* Adapt the server.hz value to the number of configured clients. If we have
@@ -1942,7 +1934,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Show information about connected clients */
if (!server.sentinel_mode) {
run_with_period(5000) {
serverLog(LL_DEBUG,
serverLog(LL_VERBOSE,
"%lu clients connected (%lu replicas), %zu bytes in use",
listLength(server.clients)-listLength(server.slaves),
listLength(server.slaves),
@@ -2124,10 +2116,6 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
if (listLength(server.unblocked_clients))
processUnblockedClients();
/* Send the invalidation messages to clients participating to the
* client side caching protocol in broadcasting (BCAST) mode. */
trackingBroadcastInvalidationMessages();
/* Write the AOF buffer on disk */
flushAppendOnlyFile(0);
@@ -2248,8 +2236,6 @@ void createSharedObjects(void) {
shared.rpoplpush = createStringObject("RPOPLPUSH",9);
shared.zpopmin = createStringObject("ZPOPMIN",7);
shared.zpopmax = createStringObject("ZPOPMAX",7);
shared.multi = createStringObject("MULTI",5);
shared.exec = createStringObject("EXEC",4);
for (j = 0; j < OBJ_SHARED_INTEGERS; j++) {
shared.integers[j] =
makeObjectShared(createObject(OBJ_STRING,(void*)(long)j));
@@ -2272,30 +2258,57 @@ void createSharedObjects(void) {
void initServerConfig(void) {
int j;
updateCachedTime(1);
updateCachedTime();
getRandomHexChars(server.runid,CONFIG_RUN_ID_SIZE);
server.runid[CONFIG_RUN_ID_SIZE] = '\0';
changeReplicationId();
clearReplicationId2();
server.hz = CONFIG_DEFAULT_HZ; /* Initialize it ASAP, even if it may get
updated later after loading the config.
This value may be used before the server
is initialized. */
server.timezone = getTimeZone(); /* Initialized by tzset(). */
server.configfile = NULL;
server.executable = NULL;
server.hz = server.config_hz = CONFIG_DEFAULT_HZ;
server.dynamic_hz = CONFIG_DEFAULT_DYNAMIC_HZ;
server.arch_bits = (sizeof(long) == 8) ? 64 : 32;
server.port = CONFIG_DEFAULT_SERVER_PORT;
server.tls_port = CONFIG_DEFAULT_SERVER_TLS_PORT;
server.tcp_backlog = CONFIG_DEFAULT_TCP_BACKLOG;
server.bindaddr_count = 0;
server.unixsocket = NULL;
server.unixsocketperm = CONFIG_DEFAULT_UNIX_SOCKET_PERM;
server.ipfd_count = 0;
server.tlsfd_count = 0;
server.sofd = -1;
server.protected_mode = CONFIG_DEFAULT_PROTECTED_MODE;
server.gopher_enabled = CONFIG_DEFAULT_GOPHER_ENABLED;
server.dbnum = CONFIG_DEFAULT_DBNUM;
server.verbosity = CONFIG_DEFAULT_VERBOSITY;
server.maxidletime = CONFIG_DEFAULT_CLIENT_TIMEOUT;
server.tcpkeepalive = CONFIG_DEFAULT_TCP_KEEPALIVE;
server.active_expire_enabled = 1;
server.jemalloc_bg_thread = 1;
server.active_defrag_enabled = CONFIG_DEFAULT_ACTIVE_DEFRAG;
server.active_defrag_ignore_bytes = CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES;
server.active_defrag_threshold_lower = CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER;
server.active_defrag_threshold_upper = CONFIG_DEFAULT_DEFRAG_THRESHOLD_UPPER;
server.active_defrag_cycle_min = CONFIG_DEFAULT_DEFRAG_CYCLE_MIN;
server.active_defrag_cycle_max = CONFIG_DEFAULT_DEFRAG_CYCLE_MAX;
server.active_defrag_max_scan_fields = CONFIG_DEFAULT_DEFRAG_MAX_SCAN_FIELDS;
server.proto_max_bulk_len = CONFIG_DEFAULT_PROTO_MAX_BULK_LEN;
server.client_max_querybuf_len = PROTO_MAX_QUERYBUF_LEN;
server.saveparams = NULL;
server.loading = 0;
server.logfile = zstrdup(CONFIG_DEFAULT_LOGFILE);
server.syslog_enabled = CONFIG_DEFAULT_SYSLOG_ENABLED;
server.syslog_ident = zstrdup(CONFIG_DEFAULT_SYSLOG_IDENT);
server.syslog_facility = LOG_LOCAL0;
server.daemonize = CONFIG_DEFAULT_DAEMONIZE;
server.supervised = 0;
server.supervised_mode = SUPERVISED_NONE;
server.aof_state = AOF_OFF;
server.aof_fsync = CONFIG_DEFAULT_AOF_FSYNC;
server.aof_no_fsync_on_rewrite = CONFIG_DEFAULT_AOF_NO_FSYNC_ON_REWRITE;
server.aof_rewrite_perc = AOF_REWRITE_PERC;
server.aof_rewrite_min_size = AOF_REWRITE_MIN_SIZE;
server.aof_rewrite_base_size = 0;
server.aof_rewrite_scheduled = 0;
server.aof_flush_sleep = 0;
@@ -2307,18 +2320,63 @@ void initServerConfig(void) {
server.aof_fd = -1;
server.aof_selected_db = -1; /* Make sure the first time will not match */
server.aof_flush_postponed_start = 0;
server.aof_rewrite_incremental_fsync = CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC;
server.rdb_save_incremental_fsync = CONFIG_DEFAULT_RDB_SAVE_INCREMENTAL_FSYNC;
server.rdb_key_save_delay = CONFIG_DEFAULT_RDB_KEY_SAVE_DELAY;
server.key_load_delay = CONFIG_DEFAULT_KEY_LOAD_DELAY;
server.aof_load_truncated = CONFIG_DEFAULT_AOF_LOAD_TRUNCATED;
server.aof_use_rdb_preamble = CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE;
server.pidfile = NULL;
server.rdb_filename = zstrdup(CONFIG_DEFAULT_RDB_FILENAME);
server.aof_filename = zstrdup(CONFIG_DEFAULT_AOF_FILENAME);
server.acl_filename = zstrdup(CONFIG_DEFAULT_ACL_FILENAME);
server.rdb_compression = CONFIG_DEFAULT_RDB_COMPRESSION;
server.rdb_checksum = CONFIG_DEFAULT_RDB_CHECKSUM;
server.stop_writes_on_bgsave_err = CONFIG_DEFAULT_STOP_WRITES_ON_BGSAVE_ERROR;
server.activerehashing = CONFIG_DEFAULT_ACTIVE_REHASHING;
server.active_defrag_running = 0;
server.notify_keyspace_events = 0;
server.maxclients = CONFIG_DEFAULT_MAX_CLIENTS;
server.blocked_clients = 0;
memset(server.blocked_clients_by_type,0,
sizeof(server.blocked_clients_by_type));
server.maxmemory = CONFIG_DEFAULT_MAXMEMORY;
server.maxmemory_policy = CONFIG_DEFAULT_MAXMEMORY_POLICY;
server.maxmemory_samples = CONFIG_DEFAULT_MAXMEMORY_SAMPLES;
server.lfu_log_factor = CONFIG_DEFAULT_LFU_LOG_FACTOR;
server.lfu_decay_time = CONFIG_DEFAULT_LFU_DECAY_TIME;
server.hash_max_ziplist_entries = OBJ_HASH_MAX_ZIPLIST_ENTRIES;
server.hash_max_ziplist_value = OBJ_HASH_MAX_ZIPLIST_VALUE;
server.list_max_ziplist_size = OBJ_LIST_MAX_ZIPLIST_SIZE;
server.list_compress_depth = OBJ_LIST_COMPRESS_DEPTH;
server.set_max_intset_entries = OBJ_SET_MAX_INTSET_ENTRIES;
server.zset_max_ziplist_entries = OBJ_ZSET_MAX_ZIPLIST_ENTRIES;
server.zset_max_ziplist_value = OBJ_ZSET_MAX_ZIPLIST_VALUE;
server.hll_sparse_max_bytes = CONFIG_DEFAULT_HLL_SPARSE_MAX_BYTES;
server.stream_node_max_bytes = OBJ_STREAM_NODE_MAX_BYTES;
server.stream_node_max_entries = OBJ_STREAM_NODE_MAX_ENTRIES;
server.shutdown_asap = 0;
server.cluster_enabled = 0;
server.cluster_node_timeout = CLUSTER_DEFAULT_NODE_TIMEOUT;
server.cluster_migration_barrier = CLUSTER_DEFAULT_MIGRATION_BARRIER;
server.cluster_slave_validity_factor = CLUSTER_DEFAULT_SLAVE_VALIDITY;
server.cluster_require_full_coverage = CLUSTER_DEFAULT_REQUIRE_FULL_COVERAGE;
server.cluster_slave_no_failover = CLUSTER_DEFAULT_SLAVE_NO_FAILOVER;
server.cluster_configfile = zstrdup(CONFIG_DEFAULT_CLUSTER_CONFIG_FILE);
server.cluster_announce_ip = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_IP;
server.cluster_announce_port = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_PORT;
server.cluster_announce_bus_port = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_BUS_PORT;
server.cluster_module_flags = CLUSTER_MODULE_FLAG_NONE;
server.migrate_cached_sockets = dictCreate(&migrateCacheDictType,NULL);
server.next_client_id = 1; /* Client IDs, start from 1 .*/
server.loading_process_events_interval_bytes = (1024*1024*2);
server.lazyfree_lazy_eviction = CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION;
server.lazyfree_lazy_expire = CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE;
server.lazyfree_lazy_server_del = CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL;
server.always_show_logo = CONFIG_DEFAULT_ALWAYS_SHOW_LOGO;
server.lua_time_limit = LUA_SCRIPT_TIME_LIMIT;
server.io_threads_num = CONFIG_DEFAULT_IO_THREADS_NUM;
server.io_threads_do_reads = CONFIG_DEFAULT_IO_THREADS_DO_READS;
server.lruclock = getLRUClock();
resetServerSaveParams();
@@ -2339,14 +2397,31 @@ void initServerConfig(void) {
server.repl_transfer_fd = -1;
server.repl_transfer_s = NULL;
server.repl_syncio_timeout = CONFIG_REPL_SYNCIO_TIMEOUT;
server.repl_serve_stale_data = CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA;
server.repl_slave_ro = CONFIG_DEFAULT_SLAVE_READ_ONLY;
server.repl_slave_ignore_maxmemory = CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY;
server.repl_slave_lazy_flush = CONFIG_DEFAULT_SLAVE_LAZY_FLUSH;
server.repl_down_since = 0; /* Never connected, repl is down since EVER. */
server.repl_disable_tcp_nodelay = CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY;
server.repl_diskless_sync = CONFIG_DEFAULT_REPL_DISKLESS_SYNC;
server.repl_diskless_load = CONFIG_DEFAULT_REPL_DISKLESS_LOAD;
server.repl_diskless_sync_delay = CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY;
server.repl_ping_slave_period = CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD;
server.repl_timeout = CONFIG_DEFAULT_REPL_TIMEOUT;
server.repl_min_slaves_to_write = CONFIG_DEFAULT_MIN_SLAVES_TO_WRITE;
server.repl_min_slaves_max_lag = CONFIG_DEFAULT_MIN_SLAVES_MAX_LAG;
server.slave_priority = CONFIG_DEFAULT_SLAVE_PRIORITY;
server.slave_announce_ip = CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP;
server.slave_announce_port = CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT;
server.master_repl_offset = 0;
/* Replication partial resync backlog */
server.repl_backlog = NULL;
server.repl_backlog_size = CONFIG_DEFAULT_REPL_BACKLOG_SIZE;
server.repl_backlog_histlen = 0;
server.repl_backlog_idx = 0;
server.repl_backlog_off = 0;
server.repl_backlog_time_limit = CONFIG_DEFAULT_REPL_BACKLOG_TIME_LIMIT;
server.repl_no_slaves_since = time(NULL);
/* Client output buffer limits */
@@ -2378,7 +2453,16 @@ void initServerConfig(void) {
server.pexpireCommand = lookupCommandByCString("pexpire");
server.xclaimCommand = lookupCommandByCString("xclaim");
server.xgroupCommand = lookupCommandByCString("xgroup");
server.rpoplpushCommand = lookupCommandByCString("rpoplpush");
/* Slow log */
server.slowlog_log_slower_than = CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN;
server.slowlog_max_len = CONFIG_DEFAULT_SLOWLOG_MAX_LEN;
/* Latency monitor */
server.latency_monitor_threshold = CONFIG_DEFAULT_LATENCY_MONITOR_THRESHOLD;
/* Tracking. */
server.tracking_table_max_fill = CONFIG_DEFAULT_TRACKING_TABLE_MAX_FILL;
/* Debugging */
server.assert_failed = "<no assertion failed>";
@@ -2392,8 +2476,6 @@ void initServerConfig(void) {
* script to the slave / AOF. This is the new way starting from
* Redis 5. However it is possible to revert it via redis.conf. */
server.lua_always_replicate_commands = 1;
initConfigValues();
}
extern char **environ;
@@ -2654,7 +2736,6 @@ void resetServerStats(void) {
server.stat_expiredkeys = 0;
server.stat_expired_stale_perc = 0;
server.stat_expired_time_cap_reached_count = 0;
server.stat_expire_cycle_time_used = 0;
server.stat_evictedkeys = 0;
server.stat_keyspace_misses = 0;
server.stat_keyspace_hits = 0;
@@ -2693,12 +2774,9 @@ void initServer(void) {
server.syslog_facility);
}
/* Initialization after setting defaults from the config system. */
server.aof_state = server.aof_enabled ? AOF_ON : AOF_OFF;
server.hz = server.config_hz;
server.pid = getpid();
server.current_client = NULL;
server.fixed_time_expire = 0;
server.clients = listCreate();
server.clients_index = raxNew();
server.clients_to_close = listCreate();
@@ -2760,14 +2838,12 @@ void initServer(void) {
for (j = 0; j < server.dbnum; j++) {
server.db[j].dict = dictCreate(&dbDictType,NULL);
server.db[j].expires = dictCreate(&keyptrDictType,NULL);
server.db[j].expires_cursor = 0;
server.db[j].blocking_keys = dictCreate(&keylistDictType,NULL);
server.db[j].ready_keys = dictCreate(&objectKeyPointerValueDictType,NULL);
server.db[j].watched_keys = dictCreate(&keylistDictType,NULL);
server.db[j].id = j;
server.db[j].avg_ttl = 0;
server.db[j].defrag_later = listCreate();
listSetFreeMethod(server.db[j].defrag_later,(void (*)(void*))sdsfree);
}
evictionPoolAlloc(); /* Initialize the LRU keys pool. */
server.pubsub_channels = dictCreate(&keylistDictType,NULL);
@@ -3170,13 +3246,10 @@ void preventCommandReplication(client *c) {
*
*/
void call(client *c, int flags) {
long long dirty;
ustime_t start, duration;
long long dirty, start, duration;
int client_old_flags = c->flags;
struct redisCommand *real_cmd = c->cmd;
server.fixed_time_expire++;
/* Sent the command to clients in MONITOR mode, only if the commands are
* not generated from reading an AOF. */
if (listLength(server.monitors) &&
@@ -3194,8 +3267,7 @@ void call(client *c, int flags) {
/* Call the command. */
dirty = server.dirty;
updateCachedTime(0);
start = server.ustime;
start = ustime();
c->cmd->proc(c);
duration = ustime()-start;
dirty = server.dirty-dirty;
@@ -3279,18 +3351,6 @@ void call(client *c, int flags) {
redisOp *rop;
if (flags & CMD_CALL_PROPAGATE) {
int multi_emitted = 0;
/* Wrap the commands in server.also_propagate array,
* but don't wrap it if we are already in MULIT context,
* in case the nested MULIT/EXEC.
*
* And if the array contains only one command, no need to
* wrap it, since the single command is atomic. */
if (server.also_propagate.numops > 1 && !(c->flags & CLIENT_MULTI)) {
execCommandPropagateMulti(c);
multi_emitted = 1;
}
for (j = 0; j < server.also_propagate.numops; j++) {
rop = &server.also_propagate.ops[j];
int target = rop->target;
@@ -3300,10 +3360,6 @@ void call(client *c, int flags) {
if (target)
propagate(rop->cmd,rop->dbid,rop->argv,rop->argc,target);
}
if (multi_emitted) {
execCommandPropagateExec(c);
}
}
redisOpArrayFree(&server.also_propagate);
}
@@ -3314,14 +3370,10 @@ void call(client *c, int flags) {
if (c->cmd->flags & CMD_READONLY) {
client *caller = (c->flags & CLIENT_LUA && server.lua_caller) ?
server.lua_caller : c;
if (caller->flags & CLIENT_TRACKING &&
!(caller->flags & CLIENT_TRACKING_BCAST))
{
if (caller->flags & CLIENT_TRACKING)
trackingRememberKeys(caller);
}
}
server.fixed_time_expire--;
server.stat_numcommands++;
}
@@ -3373,7 +3425,7 @@ int processCommand(client *c) {
DefaultUser->flags & USER_FLAG_DISABLED) &&
!c->authenticated;
if (auth_required) {
/* AUTH and HELLO and no auth modules are valid even in
/* AUTH, HELLO, and no-auth modules are valid even in
* non-authenticated state. */
if (!(c->cmd->flags & CMD_NO_AUTH)) {
flagTransaction(c);
@@ -3384,10 +3436,8 @@ int processCommand(client *c) {
/* Check if the user can run this command according to the current
* ACLs. */
int acl_keypos;
int acl_retval = ACLCheckCommandPerm(c,&acl_keypos);
int acl_retval = ACLCheckCommandPerm(c);
if (acl_retval != ACL_OK) {
addACLLogEntry(c,acl_retval,acl_keypos,NULL);
flagTransaction(c);
if (acl_retval == ACL_DENIED_CMD)
addReplyErrorFormat(c,
@@ -3507,10 +3557,7 @@ int processCommand(client *c) {
c->cmd->proc != unsubscribeCommand &&
c->cmd->proc != psubscribeCommand &&
c->cmd->proc != punsubscribeCommand) {
addReplyErrorFormat(c,
"Can't execute '%s': only (P)SUBSCRIBE / "
"(P)UNSUBSCRIBE / PING / QUIT are allowed in this context",
c->cmd->name);
addReplyError(c,"only (P)SUBSCRIBE / (P)UNSUBSCRIBE / PING / QUIT allowed in this context");
return C_OK;
}
@@ -3589,8 +3636,6 @@ int prepareForShutdown(int flags) {
int nosave = flags & SHUTDOWN_NOSAVE;
serverLog(LL_WARNING,"User requested shutdown...");
if (server.supervised_mode == SUPERVISED_SYSTEMD)
redisCommunicateSystemd("STOPPING=1\n");
/* Kill all the Lua debugger forked sessions. */
ldbKillForkedSessions();
@@ -3632,8 +3677,6 @@ int prepareForShutdown(int flags) {
/* Create a new RDB file before exiting. */
if ((server.saveparamslen > 0 && !nosave) || save) {
serverLog(LL_NOTICE,"Saving the final RDB snapshot before exiting.");
if (server.supervised_mode == SUPERVISED_SYSTEMD)
redisCommunicateSystemd("STATUS=Saving the final RDB snapshot\n");
/* Snapshotting. Perform a SYNC SAVE and exit */
rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi);
@@ -3644,8 +3687,6 @@ int prepareForShutdown(int flags) {
* saving aborted, handling special stuff like slaves pending for
* synchronization... */
serverLog(LL_WARNING,"Error trying to save the DB, can't exit.");
if (server.supervised_mode == SUPERVISED_SYSTEMD)
redisCommunicateSystemd("STATUS=Error trying to save the DB, can't exit.\n");
return C_ERR;
}
}
@@ -3880,7 +3921,7 @@ void bytesToHuman(char *s, unsigned long long n) {
/* Create the string returned by the INFO command. This is decoupled
* by the INFO command itself as we need to report the same information
* on memory corruption problems. */
sds genRedisInfoString(const char *section) {
sds genRedisInfoString(char *section) {
sds info = sdsempty();
time_t uptime = server.unixtime-server.stat_starttime;
int j;
@@ -4215,7 +4256,6 @@ sds genRedisInfoString(const char *section) {
"expired_keys:%lld\r\n"
"expired_stale_perc:%.2f\r\n"
"expired_time_cap_reached_count:%lld\r\n"
"expire_cycle_cpu_milliseconds:%lld\r\n"
"evicted_keys:%lld\r\n"
"keyspace_hits:%lld\r\n"
"keyspace_misses:%lld\r\n"
@@ -4228,8 +4268,7 @@ sds genRedisInfoString(const char *section) {
"active_defrag_misses:%lld\r\n"
"active_defrag_key_hits:%lld\r\n"
"active_defrag_key_misses:%lld\r\n"
"tracking_total_keys:%lld\r\n"
"tracking_total_items:%lld\r\n",
"tracking_used_slots:%lld\r\n",
server.stat_numconnections,
server.stat_numcommands,
getInstantaneousMetric(STATS_METRIC_COMMAND),
@@ -4244,7 +4283,6 @@ sds genRedisInfoString(const char *section) {
server.stat_expiredkeys,
server.stat_expired_stale_perc*100,
server.stat_expired_time_cap_reached_count,
server.stat_expire_cycle_time_used/1000,
server.stat_evictedkeys,
server.stat_keyspace_hits,
server.stat_keyspace_misses,
@@ -4257,8 +4295,7 @@ sds genRedisInfoString(const char *section) {
server.stat_active_defrag_misses,
server.stat_active_defrag_key_hits,
server.stat_active_defrag_key_misses,
(unsigned long long) trackingGetTotalKeys(),
(unsigned long long) trackingGetTotalItems());
trackingGetUsedSlots());
}
/* Replication */
@@ -4813,19 +4850,61 @@ int redisSupervisedUpstart(void) {
return 1;
}
int redisCommunicateSystemd(const char *sd_notify_msg) {
int redisSupervisedSystemd(void) {
const char *notify_socket = getenv("NOTIFY_SOCKET");
int fd = 1;
struct sockaddr_un su;
struct iovec iov;
struct msghdr hdr;
int sendto_flags = 0;
if (!notify_socket) {
serverLog(LL_WARNING,
"systemd supervision requested, but NOTIFY_SOCKET not found");
return 0;
}
#ifdef HAVE_LIBSYSTEMD
(void) sd_notify(0, sd_notify_msg);
#else
UNUSED(sd_notify_msg);
#endif
return 0;
if ((strchr("@/", notify_socket[0])) == NULL || strlen(notify_socket) < 2) {
return 0;
}
serverLog(LL_NOTICE, "supervised by systemd, will signal readiness");
if ((fd = socket(AF_UNIX, SOCK_DGRAM, 0)) == -1) {
serverLog(LL_WARNING,
"Can't connect to systemd socket %s", notify_socket);
return 0;
}
memset(&su, 0, sizeof(su));
su.sun_family = AF_UNIX;
strncpy (su.sun_path, notify_socket, sizeof(su.sun_path) -1);
su.sun_path[sizeof(su.sun_path) - 1] = '\0';
if (notify_socket[0] == '@')
su.sun_path[0] = '\0';
memset(&iov, 0, sizeof(iov));
iov.iov_base = "READY=1";
iov.iov_len = strlen("READY=1");
memset(&hdr, 0, sizeof(hdr));
hdr.msg_name = &su;
hdr.msg_namelen = offsetof(struct sockaddr_un, sun_path) +
strlen(notify_socket);
hdr.msg_iov = &iov;
hdr.msg_iovlen = 1;
unsetenv("NOTIFY_SOCKET");
#ifdef HAVE_MSG_NOSIGNAL
sendto_flags |= MSG_NOSIGNAL;
#endif
if (sendmsg(fd, &hdr, sendto_flags) < 0) {
serverLog(LL_WARNING, "Can't send notification to systemd");
close(fd);
return 0;
}
close(fd);
return 1;
}
int redisIsSupervised(int mode) {
@@ -4836,17 +4915,12 @@ int redisIsSupervised(int mode) {
if (upstart_job) {
redisSupervisedUpstart();
} else if (notify_socket) {
server.supervised_mode = SUPERVISED_SYSTEMD;
serverLog(LL_WARNING,
"WARNING auto-supervised by systemd - you MUST set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.");
return redisCommunicateSystemd("STATUS=Redis is loading...\n");
redisSupervisedSystemd();
}
} else if (mode == SUPERVISED_UPSTART) {
return redisSupervisedUpstart();
} else if (mode == SUPERVISED_SYSTEMD) {
serverLog(LL_WARNING,
"WARNING supervised by systemd - you MUST set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.");
return redisCommunicateSystemd("STATUS=Redis is loading...\n");
return redisSupervisedSystemd();
}
return 0;
@@ -5039,14 +5113,6 @@ int main(int argc, char **argv) {
serverLog(LL_NOTICE,"Ready to accept connections");
if (server.sofd > 0)
serverLog(LL_NOTICE,"The server is now ready to accept connections at %s", server.unixsocket);
if (server.supervised_mode == SUPERVISED_SYSTEMD) {
if (!server.masterhost) {
redisCommunicateSystemd("STATUS=Ready to accept connections\n");
redisCommunicateSystemd("READY=1\n");
} else {
redisCommunicateSystemd("STATUS=Waiting for MASTER <-> REPLICA sync\n");
}
}
} else {
InitServerLast();
sentinelIsRunning();
+115 -59
View File
@@ -46,16 +46,10 @@
#include <pthread.h>
#include <syslog.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <lua.h>
#include <signal.h>
#ifdef HAVE_LIBSYSTEMD
#include <systemd/sd-daemon.h>
#endif
typedef long long mstime_t; /* millisecond time type. */
typedef long long ustime_t; /* microsecond time type. */
#include "ae.h" /* Event driven programming library */
#include "sds.h" /* Dynamic safe strings */
@@ -88,10 +82,18 @@ typedef long long ustime_t; /* microsecond time type. */
#define C_ERR -1
/* Static server configuration */
#define CONFIG_DEFAULT_DYNAMIC_HZ 1 /* Adapt hz to # of clients.*/
#define CONFIG_DEFAULT_HZ 10 /* Time interrupt calls/sec. */
#define CONFIG_MIN_HZ 1
#define CONFIG_MAX_HZ 500
#define MAX_CLIENTS_PER_CLOCK_TICK 200 /* HZ is adapted based on that. */
#define CONFIG_DEFAULT_SERVER_PORT 6379 /* TCP port. */
#define CONFIG_DEFAULT_SERVER_TLS_PORT 0 /* TCP port. */
#define CONFIG_DEFAULT_TCP_BACKLOG 511 /* TCP listen backlog. */
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 0 /* Default client timeout: infinite */
#define CONFIG_DEFAULT_DBNUM 16
#define CONFIG_DEFAULT_IO_THREADS_NUM 1 /* Single threaded by default */
#define CONFIG_DEFAULT_IO_THREADS_DO_READS 0 /* Read + parse from threads? */
#define CONFIG_MAX_LINE 1024
#define CRON_DBS_PER_CALL 16
#define NET_MAX_WRITES_PER_EVENT (1024*64)
@@ -99,22 +101,87 @@ typedef long long ustime_t; /* microsecond time type. */
#define OBJ_SHARED_INTEGERS 10000
#define OBJ_SHARED_BULKHDR_LEN 32
#define LOG_MAX_LEN 1024 /* Default maximum length of syslog messages.*/
#define AOF_REWRITE_PERC 100
#define AOF_REWRITE_MIN_SIZE (64*1024*1024)
#define AOF_REWRITE_ITEMS_PER_CMD 64
#define AOF_READ_DIFF_INTERVAL_BYTES (1024*10)
#define CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN 10000
#define CONFIG_DEFAULT_SLOWLOG_MAX_LEN 128
#define CONFIG_DEFAULT_MAX_CLIENTS 10000
#define CONFIG_AUTHPASS_MAX_LEN 512
#define CONFIG_DEFAULT_SLAVE_PRIORITY 100
#define CONFIG_DEFAULT_REPL_TIMEOUT 60
#define CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD 10
#define CONFIG_RUN_ID_SIZE 40
#define RDB_EOF_MARK_SIZE 40
#define CONFIG_DEFAULT_REPL_BACKLOG_SIZE (1024*1024) /* 1mb */
#define CONFIG_DEFAULT_REPL_BACKLOG_TIME_LIMIT (60*60) /* 1 hour */
#define CONFIG_REPL_BACKLOG_MIN_SIZE (1024*16) /* 16k */
#define CONFIG_BGSAVE_RETRY_DELAY 5 /* Wait a few secs before trying again. */
#define CONFIG_DEFAULT_PID_FILE "/var/run/redis.pid"
#define CONFIG_DEFAULT_SYSLOG_IDENT "redis"
#define CONFIG_DEFAULT_CLUSTER_CONFIG_FILE "nodes.conf"
#define CONFIG_DEFAULT_CLUSTER_ANNOUNCE_IP NULL /* Auto detect. */
#define CONFIG_DEFAULT_CLUSTER_ANNOUNCE_PORT 0 /* Use server.port */
#define CONFIG_DEFAULT_CLUSTER_ANNOUNCE_BUS_PORT 0 /* Use +10000 offset. */
#define CONFIG_DEFAULT_DAEMONIZE 0
#define CONFIG_DEFAULT_UNIX_SOCKET_PERM 0
#define CONFIG_DEFAULT_TCP_KEEPALIVE 300
#define CONFIG_DEFAULT_PROTECTED_MODE 1
#define CONFIG_DEFAULT_GOPHER_ENABLED 0
#define CONFIG_DEFAULT_LOGFILE ""
#define CONFIG_DEFAULT_SYSLOG_ENABLED 0
#define CONFIG_DEFAULT_STOP_WRITES_ON_BGSAVE_ERROR 1
#define CONFIG_DEFAULT_RDB_COMPRESSION 1
#define CONFIG_DEFAULT_RDB_CHECKSUM 1
#define CONFIG_DEFAULT_RDB_FILENAME "dump.rdb"
#define CONFIG_DEFAULT_REPL_DISKLESS_SYNC 0
#define CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY 5
#define CONFIG_DEFAULT_RDB_KEY_SAVE_DELAY 0
#define CONFIG_DEFAULT_KEY_LOAD_DELAY 0
#define CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA 1
#define CONFIG_DEFAULT_SLAVE_READ_ONLY 1
#define CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY 1
#define CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP NULL
#define CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT 0
#define CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY 0
#define CONFIG_DEFAULT_MAXMEMORY 0
#define CONFIG_DEFAULT_MAXMEMORY_SAMPLES 5
#define CONFIG_DEFAULT_LFU_LOG_FACTOR 10
#define CONFIG_DEFAULT_LFU_DECAY_TIME 1
#define CONFIG_DEFAULT_AOF_FILENAME "appendonly.aof"
#define CONFIG_DEFAULT_AOF_NO_FSYNC_ON_REWRITE 0
#define CONFIG_DEFAULT_AOF_LOAD_TRUNCATED 1
#define CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE 1
#define CONFIG_DEFAULT_ACTIVE_REHASHING 1
#define CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC 1
#define CONFIG_DEFAULT_RDB_SAVE_INCREMENTAL_FSYNC 1
#define CONFIG_DEFAULT_MIN_SLAVES_TO_WRITE 0
#define CONFIG_DEFAULT_MIN_SLAVES_MAX_LAG 10
#define CONFIG_DEFAULT_ACL_FILENAME ""
#define NET_IP_STR_LEN 46 /* INET6_ADDRSTRLEN is 46, but we need to be sure */
#define NET_PEER_ID_LEN (NET_IP_STR_LEN+32) /* Must be enough for ip:port */
#define CONFIG_BINDADDR_MAX 16
#define CONFIG_MIN_RESERVED_FDS 32
#define CONFIG_DEFAULT_LATENCY_MONITOR_THRESHOLD 0
#define CONFIG_DEFAULT_SLAVE_LAZY_FLUSH 0
#define CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION 0
#define CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE 0
#define CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL 0
#define CONFIG_DEFAULT_ALWAYS_SHOW_LOGO 0
#define CONFIG_DEFAULT_ACTIVE_DEFRAG 0
#define CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER 10 /* don't defrag when fragmentation is below 10% */
#define CONFIG_DEFAULT_DEFRAG_THRESHOLD_UPPER 100 /* maximum defrag force at 100% fragmentation */
#define CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES (100<<20) /* don't defrag if frag overhead is below 100mb */
#define CONFIG_DEFAULT_DEFRAG_CYCLE_MIN 5 /* 5% CPU min (at lower threshold) */
#define CONFIG_DEFAULT_DEFRAG_CYCLE_MAX 75 /* 75% CPU max (at upper threshold) */
#define CONFIG_DEFAULT_DEFRAG_MAX_SCAN_FIELDS 1000 /* keys with more than 1000 fields will be processed separately */
#define CONFIG_DEFAULT_PROTO_MAX_BULK_LEN (512ll*1024*1024) /* Bulk request max size */
#define CONFIG_DEFAULT_TRACKING_TABLE_MAX_FILL 10 /* 10% tracking table max fill. */
#define ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP 20 /* Loopkups per loop. */
#define ACTIVE_EXPIRE_CYCLE_FAST_DURATION 1000 /* Microseconds */
#define ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC 25 /* CPU max % for keys collection */
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
@@ -244,10 +311,9 @@ typedef long long ustime_t; /* microsecond time type. */
we return single threaded that the
client has already pending commands
to be executed. */
#define CLIENT_TRACKING (1ULL<<31) /* Client enabled keys tracking in order to
#define CLIENT_TRACKING (1<<31) /* Client enabled keys tracking in order to
perform client side caching. */
#define CLIENT_TRACKING_BROKEN_REDIR (1ULL<<32) /* Target client is invalid. */
#define CLIENT_TRACKING_BCAST (1ULL<<33) /* Tracking in BCAST mode. */
/* Client block type (btype field in client structure)
* if CLIENT_BLOCKED flag is set. */
@@ -326,6 +392,7 @@ typedef long long ustime_t; /* microsecond time type. */
#define LL_NOTICE 2
#define LL_WARNING 3
#define LL_RAW (1<<10) /* Modifier to log without timestamp */
#define CONFIG_DEFAULT_VERBOSITY LL_NOTICE
/* Supervision options */
#define SUPERVISED_NONE 0
@@ -336,18 +403,36 @@ typedef long long ustime_t; /* microsecond time type. */
/* Anti-warning macro... */
#define UNUSED(V) ((void) V)
#define ZSKIPLIST_MAXLEVEL 32 /* Should be enough for 2^64 elements */
#define ZSKIPLIST_MAXLEVEL 64 /* Should be enough for 2^64 elements */
#define ZSKIPLIST_P 0.25 /* Skiplist P = 1/4 */
/* Append only defines */
#define AOF_FSYNC_NO 0
#define AOF_FSYNC_ALWAYS 1
#define AOF_FSYNC_EVERYSEC 2
#define CONFIG_DEFAULT_AOF_FSYNC AOF_FSYNC_EVERYSEC
/* Replication diskless load defines */
#define REPL_DISKLESS_LOAD_DISABLED 0
#define REPL_DISKLESS_LOAD_WHEN_DB_EMPTY 1
#define REPL_DISKLESS_LOAD_SWAPDB 2
#define CONFIG_DEFAULT_REPL_DISKLESS_LOAD REPL_DISKLESS_LOAD_DISABLED
/* Zipped structures related defaults */
#define OBJ_HASH_MAX_ZIPLIST_ENTRIES 512
#define OBJ_HASH_MAX_ZIPLIST_VALUE 64
#define OBJ_SET_MAX_INTSET_ENTRIES 512
#define OBJ_ZSET_MAX_ZIPLIST_ENTRIES 128
#define OBJ_ZSET_MAX_ZIPLIST_VALUE 64
#define OBJ_STREAM_NODE_MAX_BYTES 4096
#define OBJ_STREAM_NODE_MAX_ENTRIES 100
/* List defaults */
#define OBJ_LIST_MAX_ZIPLIST_SIZE -2
#define OBJ_LIST_COMPRESS_DEPTH 0
/* HyperLogLog defines */
#define CONFIG_DEFAULT_HLL_SPARSE_MAX_BYTES 3000
/* Sets operations codes */
#define SET_OP_UNION 0
@@ -372,6 +457,11 @@ typedef long long ustime_t; /* microsecond time type. */
#define MAXMEMORY_ALLKEYS_RANDOM ((6<<8)|MAXMEMORY_FLAG_ALLKEYS)
#define MAXMEMORY_NO_EVICTION (7<<8)
#define CONFIG_DEFAULT_MAXMEMORY_POLICY MAXMEMORY_NO_EVICTION
/* Scripting */
#define LUA_SCRIPT_TIME_LIMIT 5000 /* milliseconds */
/* Units */
#define UNIT_SECONDS 0
#define UNIT_MILLISECONDS 1
@@ -414,8 +504,8 @@ typedef long long ustime_t; /* microsecond time type. */
#define NOTIFY_EXPIRED (1<<8) /* x */
#define NOTIFY_EVICTED (1<<9) /* e */
#define NOTIFY_STREAM (1<<10) /* t */
#define NOTIFY_KEY_MISS (1<<11) /* m (Note: This one is excluded from NOTIFY_ALL on purpose) */
#define NOTIFY_ALL (NOTIFY_GENERIC | NOTIFY_STRING | NOTIFY_LIST | NOTIFY_SET | NOTIFY_HASH | NOTIFY_ZSET | NOTIFY_EXPIRED | NOTIFY_EVICTED | NOTIFY_STREAM) /* A flag */
#define NOTIFY_KEY_MISS (1<<11) /* m */
#define NOTIFY_ALL (NOTIFY_GENERIC | NOTIFY_STRING | NOTIFY_LIST | NOTIFY_SET | NOTIFY_HASH | NOTIFY_ZSET | NOTIFY_EXPIRED | NOTIFY_EVICTED | NOTIFY_STREAM | NOTIFY_KEY_MISS) /* A flag */
/* Get the first bind addr or NULL */
#define NET_FIRST_BIND_ADDR (server.bindaddr_count ? server.bindaddr[0] : NULL)
@@ -486,11 +576,6 @@ typedef void (*moduleTypeDigestFunc)(struct RedisModuleDigest *digest, void *val
typedef size_t (*moduleTypeMemUsageFunc)(const void *value);
typedef void (*moduleTypeFreeFunc)(void *value);
/* A callback that is called when the client authentication changes. This
* needs to be exposed since you can't cast a function pointer to (void *) */
typedef void (*RedisModuleUserChangedFunc) (uint64_t client_id, void *privdata);
/* The module type, which is referenced in each value of a given type, defines
* the methods and links to the module exporting the type. */
typedef struct RedisModuleType {
@@ -636,7 +721,6 @@ typedef struct redisDb {
dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */
int id; /* Database ID */
long long avg_ttl; /* Average TTL, just for stats */
unsigned long expires_cursor; /* Cursor of the active expire cycle. */
list *defrag_later; /* List of key names to attempt to defrag one by one, gradually. */
} redisDb;
@@ -809,23 +893,13 @@ typedef struct client {
list *pubsub_patterns; /* patterns a client is interested in (SUBSCRIBE) */
sds peerid; /* Cached peer ID. */
listNode *client_list_node; /* list node in client list */
RedisModuleUserChangedFunc auth_callback; /* Module callback to execute
* when the authenticated user
* changes. */
void *auth_callback_privdata; /* Private data that is passed when the auth
* changed callback is executed. Opaque for
* Redis Core. */
void *auth_module; /* The module that owns the callback, which is used
* to disconnect the client if the module is
* unloaded for cleanup. Opaque for Redis Core.*/
void *auth_ctx; /* Opaque structure to track module auth */
/* If this client is in tracking mode and this field is non zero,
* invalidation messages for keys fetched by this client will be send to
* the specified client ID. */
uint64_t client_tracking_redirection;
rax *client_tracking_prefixes; /* A dictionary of prefixes we are already
subscribed to in BCAST mode, in the
context of client side caching. */
/* Response buffer */
int bufpos;
char buf[PROTO_REPLY_CHUNK_BYTES];
@@ -851,7 +925,6 @@ struct sharedObjectsStruct {
*busykeyerr, *oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk,
*unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *unlink,
*rpop, *lpop, *lpush, *rpoplpush, *zpopmin, *zpopmax, *emptyscan,
*multi, *exec,
*select[PROTO_SHARED_SELECT_CMDS],
*integers[OBJ_SHARED_INTEGERS],
*mbulkhdr[OBJ_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
@@ -1062,8 +1135,7 @@ struct redisServer {
list *clients_pending_write; /* There is to write or install handler. */
list *clients_pending_read; /* Client has pending read socket buffers. */
list *slaves, *monitors; /* List of slaves and MONITORs */
client *current_client; /* Current client executing the command. */
long fixed_time_expire; /* If > 0, expire keys against server.mstime. */
client *current_client; /* Current client, only used on crash report */
rax *clients_index; /* Active clients dictionary by client ID. */
int clients_paused; /* True if clients are currently paused */
mstime_t clients_pause_end_time; /* Time when we undo clients_paused */
@@ -1087,7 +1159,7 @@ struct redisServer {
*lpopCommand, *rpopCommand, *zpopminCommand,
*zpopmaxCommand, *sremCommand, *execCommand,
*expireCommand, *pexpireCommand, *xclaimCommand,
*xgroupCommand, *rpoplpushCommand;
*xgroupCommand;
/* Fields used only for stats */
time_t stat_starttime; /* Server start time */
long long stat_numcommands; /* Number of processed commands */
@@ -1095,7 +1167,6 @@ struct redisServer {
long long stat_expiredkeys; /* Number of expired keys */
double stat_expired_stale_perc; /* Percentage of keys probably expired */
long long stat_expired_time_cap_reached_count; /* Early expire cylce stops.*/
long long stat_expire_cycle_time_used; /* Cumulative microseconds used. */
long long stat_evictedkeys; /* Number of evicted keys (maxmemory) */
long long stat_keyspace_hits; /* Number of successful lookups of keys */
long long stat_keyspace_misses; /* Number of failed lookups of keys */
@@ -1134,7 +1205,6 @@ struct redisServer {
int maxidletime; /* Client timeout in seconds */
int tcpkeepalive; /* Set SO_KEEPALIVE if non-zero. */
int active_expire_enabled; /* Can be disabled for testing purposes. */
int active_expire_effort; /* From 1 (default) to 10, active effort. */
int active_defrag_enabled;
int jemalloc_bg_thread; /* Enable jemalloc background thread */
size_t active_defrag_ignore_bytes; /* minimum amount of fragmentation waste to start active defrag */
@@ -1309,7 +1379,7 @@ struct redisServer {
list *ready_keys; /* List of readyList structures for BLPOP & co */
/* Client side caching. */
unsigned int tracking_clients; /* # of clients with tracking enabled.*/
int tracking_table_max_keys; /* Max number of keys in tracking table. */
int tracking_table_max_fill; /* Max fill percentage. */
/* Sort parameters - qsort_r() is only available under BSD so we
* have to take this state global, in order to pass it to sortCompare() */
int sort_desc;
@@ -1324,7 +1394,7 @@ struct redisServer {
size_t zset_max_ziplist_value;
size_t hll_sparse_max_bytes;
size_t stream_node_max_bytes;
long long stream_node_max_entries;
int64_t stream_node_max_entries;
/* List parameters */
int list_max_ziplist_size;
int list_compress_depth;
@@ -1332,8 +1402,7 @@ struct redisServer {
_Atomic time_t unixtime; /* Unix time sampled every cron cycle. */
time_t timezone; /* Cached timezone. As set by tzset(). */
int daylight_active; /* Currently in daylight saving time. */
mstime_t mstime; /* 'unixtime' in milliseconds. */
ustime_t ustime; /* 'unixtime' in microseconds. */
long long mstime; /* 'unixtime' with milliseconds resolution. */
/* Pubsub */
dict *pubsub_channels; /* Map channels to list of subscribed clients */
list *pubsub_patterns; /* A list of pubsub_patterns */
@@ -1357,8 +1426,6 @@ struct redisServer {
to set in order to suppress certain
native Redis Cluster features. Check the
REDISMODULE_CLUSTER_FLAG_*. */
int cluster_allow_reads_when_down; /* Are reads allowed when the cluster
is down? */
/* Scripting */
lua_State *lua; /* The Lua interpreter. We use just one for all clients */
client *lua_client; /* The "fake client" to query Redis from Lua */
@@ -1388,7 +1455,6 @@ struct redisServer {
dict *latency_events;
/* ACLs */
char *acl_filename; /* ACL Users file. NULL if not configured. */
unsigned long acllog_max_len; /* Maximum length of the ACL LOG list. */
/* Assert & bug reporting */
const char *assert_failed;
const char *assert_file;
@@ -1536,7 +1602,7 @@ void ModuleForkDoneHandler(int exitcode, int bysignal);
int TerminateModuleForkChild(int child_pid, int wait);
ssize_t rdbSaveModulesAux(rio *rdb, int when);
int moduleAllDatatypesHandleErrors();
sds modulesCollectInfo(sds info, const char *section, int for_crash_report, int sections);
sds modulesCollectInfo(sds info, sds section, int for_crash_report, int sections);
void moduleFireServerEvent(uint64_t eid, int subid, void *data);
void processModuleLoadingProgressEvent(int is_aof);
int moduleTryServeClientBlockedOnKey(client *c, robj *key);
@@ -1553,7 +1619,6 @@ uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
void exitFromChild(int retcode);
size_t redisPopcount(void *s, long count);
void redisSetProcTitle(char *title);
int redisCommunicateSystemd(const char *sd_notify_msg);
/* networking.c -- Networking and Client related operations */
client *createClient(connection *conn);
@@ -1651,15 +1716,13 @@ void addReplyStatusFormat(client *c, const char *fmt, ...);
#endif
/* Client side caching (tracking mode) */
void enableTracking(client *c, uint64_t redirect_to, int bcast, robj **prefix, size_t numprefix);
void enableTracking(client *c, uint64_t redirect_to);
void disableTracking(client *c);
void trackingRememberKeys(client *c);
void trackingInvalidateKey(robj *keyobj);
void trackingInvalidateKeysOnFlush(int dbid);
void trackingLimitUsedSlots(void);
uint64_t trackingGetTotalItems(void);
uint64_t trackingGetTotalKeys(void);
void trackingBroadcastInvalidationMessages(void);
unsigned long long trackingGetUsedSlots(void);
/* List data type */
void listTypeTryConversion(robj *subject, robj *value);
@@ -1687,7 +1750,6 @@ void touchWatchedKeysOnFlush(int dbid);
void discardTransaction(client *c);
void flagTransaction(client *c);
void execCommandPropagateMulti(client *c);
void execCommandPropagateExec(client *c);
/* Redis object implementation */
void decrRefCount(robj *o);
@@ -1826,12 +1888,11 @@ void ACLInit(void);
#define ACL_OK 0
#define ACL_DENIED_CMD 1
#define ACL_DENIED_KEY 2
#define ACL_DENIED_AUTH 3 /* Only used for ACL LOG entries. */
int ACLCheckUserCredentials(robj *username, robj *password);
int ACLAuthenticateUser(client *c, robj *username, robj *password);
unsigned long ACLGetCommandID(const char *cmdname);
user *ACLGetUserByName(const char *name, size_t namelen);
int ACLCheckCommandPerm(client *c, int *keyidxptr);
int ACLCheckCommandPerm(client *c);
int ACLSetUser(user *u, const char *op, ssize_t oplen);
sds ACLDefaultUserFirstPassword(void);
uint64_t ACLGetCommandCategoryFlagByName(const char *name);
@@ -1843,7 +1904,6 @@ void ACLLoadUsersAtStartup(void);
void addReplyCommandCategories(client *c, struct redisCommand *cmd);
user *ACLCreateUnlinkedUser();
void ACLFreeUserAndKillClients(user *u);
void addACLLogEntry(client *c, int reason, int keypos, sds username);
/* Sorted sets data type */
@@ -1944,7 +2004,7 @@ void populateCommandTable(void);
void resetCommandTableStats(void);
void adjustOpenFilesLimit(void);
void closeListeningSockets(int unlink_unix_socket);
void updateCachedTime(int update_daylight_info);
void updateCachedTime(void);
void resetServerStats(void);
void activeDefragCycle(void);
unsigned int getLRUClock(void);
@@ -2017,7 +2077,6 @@ void resetServerSaveParams(void);
struct rewriteConfigState; /* Forward declaration to export API. */
void rewriteConfigRewriteLine(struct rewriteConfigState *state, const char *option, sds line, int force);
int rewriteConfig(char *path);
void initConfigValues();
/* db.c -- Keyspace access API */
int removeExpire(redisDb *db, robj *key);
@@ -2035,12 +2094,11 @@ robj *lookupKeyWriteWithFlags(redisDb *db, robj *key, int flags);
robj *objectCommandLookup(client *c, robj *key);
robj *objectCommandLookupOrReply(client *c, robj *key, robj *reply);
int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
long long lru_clock, int lru_multiplier);
long long lru_clock);
#define LOOKUP_NONE 0
#define LOOKUP_NOTOUCH (1<<0)
void dbAdd(redisDb *db, robj *key, robj *val);
void dbOverwrite(redisDb *db, robj *key, robj *val);
void genericSetKey(redisDb *db, robj *key, robj *val, int keepttl);
void setKey(redisDb *db, robj *key, robj *val);
int dbExists(redisDb *db, robj *key);
robj *dbRandomKey(redisDb *db);
@@ -2050,7 +2108,6 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o);
#define EMPTYDB_NO_FLAGS 0 /* No flags. */
#define EMPTYDB_ASYNC (1<<0) /* Reclaim memory in another thread. */
#define EMPTYDB_BACKUP (1<<2) /* DB array is a backup for REPL_DISKLESS_LOAD_SWAPDB. */
long long emptyDb(int dbnum, int flags, void(callback)(void*));
long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(void*));
void flushAllDataAndResetRDB(int flags);
@@ -2083,7 +2140,6 @@ int *sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *xreadGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *memoryGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
/* Cluster */
void clusterInit(void);
@@ -2363,7 +2419,7 @@ void _serverPanic(const char *file, int line, const char *msg, ...);
void bugReportStart(void);
void serverLogObjectDebugInfo(const robj *o);
void sigsegvHandler(int sig, siginfo_t *info, void *secret);
sds genRedisInfoString(const char *section);
sds genRedisInfoString(char *section);
sds genModulesInfoString(sds info);
void enableWatchdog(int period);
void disableWatchdog(void);
-5
View File
@@ -27,9 +27,6 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __SLOWLOG_H__
#define __SLOWLOG_H__
#define SLOWLOG_ENTRY_MAX_ARGC 32
#define SLOWLOG_ENTRY_MAX_STRING 128
@@ -50,5 +47,3 @@ void slowlogPushEntryIfNeeded(client *c, robj **argv, int argc, long long durati
/* Exported commands */
void slowlogCommand(client *c);
#endif /* __SLOWLOG_H__ */
-2
View File
@@ -98,7 +98,6 @@ struct client;
stream *streamNew(void);
void freeStream(stream *s);
unsigned long streamLength(const robj *subject);
size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end, size_t count, int rev, streamCG *group, streamConsumer *consumer, int flags, streamPropInfo *spi);
void streamIteratorStart(streamIterator *si, stream *s, streamID *start, streamID *end, int rev);
int streamIteratorGetID(streamIterator *si, streamID *id, int64_t *numfields);
@@ -111,6 +110,5 @@ streamNACK *streamCreateNACK(streamConsumer *consumer);
void streamDecodeID(void *buf, streamID *id);
int streamCompareID(streamID *a, streamID *b);
void streamFreeNACK(streamNACK *na);
void streamIncrID(streamID *id);
#endif
+12 -5
View File
@@ -653,13 +653,20 @@ int serveClientBlockedOnList(client *receiver, robj *key, robj *dstkey, redisDb
if (!(dstobj &&
checkType(receiver,dstobj,OBJ_LIST)))
{
/* Propagate the RPOP operation. */
argv[0] = shared.rpop;
argv[1] = key;
propagate(server.rpopCommand,
db->id,argv,2,
PROPAGATE_AOF|
PROPAGATE_REPL);
rpoplpushHandlePush(receiver,dstkey,dstobj,
value);
/* Propagate the RPOPLPUSH operation. */
argv[0] = shared.rpoplpush;
argv[1] = key;
argv[2] = dstkey;
propagate(server.rpoplpushCommand,
/* Propagate the LPUSH operation. */
argv[0] = shared.lpush;
argv[1] = dstkey;
argv[2] = value;
propagate(server.lpushCommand,
db->id,argv,3,
PROPAGATE_AOF|
PROPAGATE_REPL);
+35 -62
View File
@@ -67,27 +67,6 @@ void freeStream(stream *s) {
zfree(s);
}
/* Return the length of a stream. */
unsigned long streamLength(const robj *subject) {
stream *s = subject->ptr;
return s->length;
}
/* Set 'id' to be its successor streamID */
void streamIncrID(streamID *id) {
if (id->seq == UINT64_MAX) {
if (id->ms == UINT64_MAX) {
/* Special case where 'id' is the last possible streamID... */
id->ms = id->seq = 0;
} else {
id->ms++;
id->seq = 0;
}
} else {
id->seq++;
}
}
/* Generate the next stream item ID given the previous one. If the current
* milliseconds Unix time is greater than the previous one, just use this
* as time part and start with sequence part of zero. Otherwise we use the
@@ -98,8 +77,8 @@ void streamNextID(streamID *last_id, streamID *new_id) {
new_id->ms = ms;
new_id->seq = 0;
} else {
*new_id = *last_id;
streamIncrID(new_id);
new_id->ms = last_id->ms;
new_id->seq = last_id->seq+1;
}
}
@@ -797,16 +776,6 @@ int streamDeleteItem(stream *s, streamID *id) {
return deleted;
}
/* Get the last valid (non-tombstone) streamID of 's'. */
void streamLastValidID(stream *s, streamID *maxid)
{
streamIterator si;
streamIteratorStart(&si,s,NULL,NULL,1);
int64_t numfields;
streamIteratorGetID(&si,maxid,&numfields);
streamIteratorStop(&si);
}
/* Emit a reply in the client output buffer by formatting a Stream ID
* in the standard <ms>-<seq> format, using the simple string protocol
* of REPL. */
@@ -1104,6 +1073,26 @@ robj *streamTypeLookupWriteOrCreate(client *c, robj *key) {
return o;
}
/* Helper function to convert a string to an unsigned long long value.
* The function attempts to use the faster string2ll() function inside
* Redis: if it fails, strtoull() is used instead. The function returns
* 1 if the conversion happened successfully or 0 if the number is
* invalid or out of range. */
int string2ull(const char *s, unsigned long long *value) {
long long ll;
if (string2ll(s,strlen(s),&ll)) {
if (ll < 0) return 0; /* Negative values are out of range. */
*value = ll;
return 1;
}
errno = 0;
char *endptr = NULL;
*value = strtoull(s,&endptr,10);
if (errno == EINVAL || errno == ERANGE || !(*s != '\0' && *endptr == '\0'))
return 0; /* strtoull() failed. */
return 1; /* Conversion done! */
}
/* Parse a stream ID in the format given by clients to Redis, that is
* <ms>-<seq>, and converts it into a streamID structure. If
* the specified ID is invalid C_ERR is returned and an error is reported
@@ -1231,27 +1220,12 @@ void xaddCommand(client *c) {
return;
}
/* Return ASAP if minimal ID (0-0) was given so we avoid possibly creating
* a new stream and have streamAppendItem fail, leaving an empty key in the
* database. */
if (id_given && id.ms == 0 && id.seq == 0) {
addReplyError(c,"The ID specified in XADD must be greater than 0-0");
return;
}
/* Lookup the stream at key. */
robj *o;
stream *s;
if ((o = streamTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
s = o->ptr;
/* Return ASAP if the stream has reached the last possible ID */
if (s->last_id.ms == UINT64_MAX && s->last_id.seq == UINT64_MAX) {
addReplyError(c,"The stream has exhausted the last possible ID, "
"unable to add more items");
return;
}
/* Append using the low level function and return the ID. */
if (streamAppendItem(s,c->argv+field_pos,(c->argc-field_pos)/2,
&id, id_given ? &id : NULL)
@@ -1516,23 +1490,20 @@ void xreadCommand(client *c) {
{
serve_synchronously = 1;
serve_history = 1;
} else if (s->length) {
} else {
/* We also want to serve a consumer in a consumer group
* synchronously in case the group top item delivered is smaller
* than what the stream has inside. */
streamID maxid, *last = &groups[i]->last_id;
streamLastValidID(s, &maxid);
if (streamCompareID(&maxid, last) > 0) {
streamID *last = &groups[i]->last_id;
if (s->length && (streamCompareID(&s->last_id, last) > 0)) {
serve_synchronously = 1;
*gt = *last;
}
}
} else if (s->length) {
} else {
/* For consumers without a group, we serve synchronously if we can
* actually provide at least one item from the stream. */
streamID maxid;
streamLastValidID(s, &maxid);
if (streamCompareID(&maxid, gt) > 0) {
if (s->length && (streamCompareID(&s->last_id, gt) > 0)) {
serve_synchronously = 1;
}
}
@@ -1544,7 +1515,7 @@ void xreadCommand(client *c) {
* so start from the next ID, since we want only messages with
* IDs greater than start. */
streamID start = *gt;
streamIncrID(&start);
start.seq++; /* uint64_t can't overflow in this context. */
/* Emit the two elements sub-array consisting of the name
* of the stream and the data we extracted from it. */
@@ -1884,7 +1855,11 @@ void xsetidCommand(client *c) {
* item, otherwise the fundamental ID monotonicity assumption is violated. */
if (s->length > 0) {
streamID maxid;
streamLastValidID(s,&maxid);
streamIterator si;
streamIteratorStart(&si,s,NULL,NULL,1);
int64_t numfields;
streamIteratorGetID(&si,&maxid,&numfields);
streamIteratorStop(&si);
if (streamCompareID(&id,&maxid) < 0) {
addReplyError(c,"The ID specified in XSETID is smaller than the "
@@ -2255,7 +2230,7 @@ void xclaimCommand(client *c) {
}
/* Do the actual claiming. */
streamConsumer *consumer = NULL;
streamConsumer *consumer = streamLookupConsumer(group,c->argv[3]->ptr,1);
void *arraylenptr = addReplyDeferredLen(c);
size_t arraylen = 0;
for (int j = 5; j <= last_id_arg; j++) {
@@ -2307,11 +2282,9 @@ void xclaimCommand(client *c) {
if (nack->consumer)
raxRemove(nack->consumer->pel,buf,sizeof(buf),NULL);
/* Update the consumer and idle time. */
if (consumer == NULL)
consumer = streamLookupConsumer(group,c->argv[3]->ptr,1);
nack->consumer = consumer;
nack->delivery_time = deliverytime;
/* Set the delivery attempts counter if given, otherwise
/* Set the delivery attempts counter if given, otherwise
* autoincrement unless JUSTID option provided */
if (retrycount >= 0) {
nack->delivery_count = retrycount;
+11 -21
View File
@@ -46,7 +46,7 @@ static int checkStringLength(client *c, long long size) {
* options and variants. This function is called in order to implement the
* following commands: SET, SETEX, PSETEX, SETNX.
*
* 'flags' changes the behavior of the command (NX or XX, see below).
* 'flags' changes the behavior of the command (NX or XX, see belove).
*
* 'expire' represents an expire to set in form of a Redis object as passed
* by the user. It is interpreted according to the specified 'unit'.
@@ -59,11 +59,10 @@ static int checkStringLength(client *c, long long size) {
* If abort_reply is NULL, "$-1" is used. */
#define OBJ_SET_NO_FLAGS 0
#define OBJ_SET_NX (1<<0) /* Set if key not exists. */
#define OBJ_SET_XX (1<<1) /* Set if key exists. */
#define OBJ_SET_EX (1<<2) /* Set if time in seconds is given */
#define OBJ_SET_PX (1<<3) /* Set if time in ms in given */
#define OBJ_SET_KEEPTTL (1<<4) /* Set and keep the ttl */
#define OBJ_SET_NX (1<<0) /* Set if key not exists. */
#define OBJ_SET_XX (1<<1) /* Set if key exists. */
#define OBJ_SET_EX (1<<2) /* Set if time in seconds is given */
#define OBJ_SET_PX (1<<3) /* Set if time in ms in given */
void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire, int unit, robj *ok_reply, robj *abort_reply) {
long long milliseconds = 0; /* initialized to avoid any harmness warning */
@@ -84,7 +83,7 @@ void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire,
addReply(c, abort_reply ? abort_reply : shared.null[c->resp]);
return;
}
genericSetKey(c->db,key,val,flags & OBJ_SET_KEEPTTL);
setKey(c->db,key,val);
server.dirty++;
if (expire) setExpire(c,c->db,key,mstime()+milliseconds);
notifyKeyspaceEvent(NOTIFY_STRING,"set",key,c->db->id);
@@ -93,7 +92,7 @@ void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire,
addReply(c, ok_reply ? ok_reply : shared.ok);
}
/* SET key value [NX] [XX] [KEEPTTL] [EX <seconds>] [PX <milliseconds>] */
/* SET key value [NX] [XX] [EX <seconds>] [PX <milliseconds>] */
void setCommand(client *c) {
int j;
robj *expire = NULL;
@@ -114,13 +113,8 @@ void setCommand(client *c) {
!(flags & OBJ_SET_NX))
{
flags |= OBJ_SET_XX;
} else if (!strcasecmp(c->argv[j]->ptr,"KEEPTTL") &&
!(flags & OBJ_SET_EX) && !(flags & OBJ_SET_PX))
{
flags |= OBJ_SET_KEEPTTL;
} else if ((a[0] == 'e' || a[0] == 'E') &&
(a[1] == 'x' || a[1] == 'X') && a[2] == '\0' &&
!(flags & OBJ_SET_KEEPTTL) &&
!(flags & OBJ_SET_PX) && next)
{
flags |= OBJ_SET_EX;
@@ -129,7 +123,6 @@ void setCommand(client *c) {
j++;
} else if ((a[0] == 'p' || a[0] == 'P') &&
(a[1] == 'x' || a[1] == 'X') && a[2] == '\0' &&
!(flags & OBJ_SET_KEEPTTL) &&
!(flags & OBJ_SET_EX) && next)
{
flags |= OBJ_SET_PX;
@@ -405,7 +398,7 @@ void decrbyCommand(client *c) {
void incrbyfloatCommand(client *c) {
long double incr, value;
robj *o, *new, *aux1, *aux2;
robj *o, *new, *aux;
o = lookupKeyWrite(c->db,c->argv[1]);
if (o != NULL && checkType(c,o,OBJ_STRING)) return;
@@ -431,13 +424,10 @@ void incrbyfloatCommand(client *c) {
/* Always replicate INCRBYFLOAT as a SET command with the final value
* in order to make sure that differences in float precision or formatting
* will not create differences in replicas or after an AOF restart. */
aux1 = createStringObject("SET",3);
rewriteClientCommandArgument(c,0,aux1);
decrRefCount(aux1);
aux = createStringObject("SET",3);
rewriteClientCommandArgument(c,0,aux);
decrRefCount(aux);
rewriteClientCommandArgument(c,2,new);
aux2 = createStringObject("KEEPTTL",7);
rewriteClientCommandArgument(c,3,aux2);
decrRefCount(aux2);
}
void appendCommand(client *c) {
+117 -266
View File
@@ -30,34 +30,39 @@
#include "server.h"
/* The tracking table is constituted by a radix tree of keys, each pointing
* to a radix tree of client IDs, used to track the clients that may have
* certain keys in their local, client side, cache.
/* The tracking table is constituted by 2^24 radix trees (each tree, and the
* table itself, are allocated in a lazy way only when needed) tracking
* clients that may have certain keys in their local, client side, cache.
*
* Keys are grouped into 2^24 slots, in a way similar to Redis Cluster hash
* slots, however here the function we use is crc64, taking the least
* significant 24 bits of the output.
*
* When a client enables tracking with "CLIENT TRACKING on", each key served to
* the client is remembered in the table mapping the keys to the client IDs.
* Later, when a key is modified, all the clients that may have local copy
* of such key will receive an invalidation message.
* the client is hashed to one of such slots, and Redis will remember what
* client may have keys about such slot. Later, when a key in a given slot is
* modified, all the clients that may have local copies of keys in that slot
* will receive an invalidation message. There is no distinction of database
* number: a single table is used.
*
* Clients will normally take frequently requested objects in memory, removing
* them when invalidation messages are received. */
rax *TrackingTable = NULL;
rax *PrefixTable = NULL;
uint64_t TrackingTableTotalItems = 0; /* Total number of IDs stored across
the whole tracking table. This givesn
an hint about the total memory we
are using server side for CSC. */
* them when invalidation messages are received. A strategy clients may use is
* to just cache objects in a dictionary, associating to each cached object
* some incremental epoch, or just a timestamp. When invalidation messages are
* received clients may store, in a different table, the timestamp (or epoch)
* of the invalidation of such given slot: later when accessing objects, the
* eviction of stale objects may be performed in a lazy way by checking if the
* cached object timestamp is older than the invalidation timestamp for such
* objects.
*
* The output of the 24 bit hash function is very large (more than 16 million
* possible slots), so clients that may want to use less resources may only
* use the most significant bits instead of the full 24 bits. */
#define TRACKING_TABLE_SIZE (1<<24)
rax **TrackingTable = NULL;
unsigned long TrackingTableUsedSlots = 0;
robj *TrackingChannelName;
/* This is the structure that we have as value of the PrefixTable, and
* represents the list of keys modified, and the list of clients that need
* to be notified, for a given prefix. */
typedef struct bcastState {
rax *keys; /* Keys modified in the current event loop cycle. */
rax *clients; /* Clients subscribed to the notification events for this
prefix. */
} bcastState;
/* Remove the tracking state from the client 'c'. Note that there is not much
* to do for us here, if not to decrement the counter of the clients in
* tracking mode, because we just store the ID of the client in the tracking
@@ -65,55 +70,9 @@ typedef struct bcastState {
* client with many entries in the table is removed, it would cost a lot of
* time to do the cleanup. */
void disableTracking(client *c) {
/* If this client is in broadcasting mode, we need to unsubscribe it
* from all the prefixes it is registered to. */
if (c->flags & CLIENT_TRACKING_BCAST) {
raxIterator ri;
raxStart(&ri,c->client_tracking_prefixes);
raxSeek(&ri,"^",NULL,0);
while(raxNext(&ri)) {
bcastState *bs = raxFind(PrefixTable,ri.key,ri.key_len);
serverAssert(bs != raxNotFound);
raxRemove(bs->clients,(unsigned char*)&c,sizeof(c),NULL);
/* Was it the last client? Remove the prefix from the
* table. */
if (raxSize(bs->clients) == 0) {
raxFree(bs->clients);
raxFree(bs->keys);
zfree(bs);
raxRemove(PrefixTable,ri.key,ri.key_len,NULL);
}
}
raxStop(&ri);
raxFree(c->client_tracking_prefixes);
c->client_tracking_prefixes = NULL;
}
/* Clear flags and adjust the count. */
if (c->flags & CLIENT_TRACKING) {
server.tracking_clients--;
c->flags &= ~(CLIENT_TRACKING|CLIENT_TRACKING_BROKEN_REDIR|
CLIENT_TRACKING_BCAST);
}
}
/* Set the client 'c' to track the prefix 'prefix'. If the client 'c' is
* already registered for the specified prefix, no operation is performed. */
void enableBcastTrackingForPrefix(client *c, char *prefix, size_t plen) {
bcastState *bs = raxFind(PrefixTable,(unsigned char*)prefix,sdslen(prefix));
/* If this is the first client subscribing to such prefix, create
* the prefix in the table. */
if (bs == raxNotFound) {
bs = zmalloc(sizeof(*bs));
bs->keys = raxNew();
bs->clients = raxNew();
raxInsert(PrefixTable,(unsigned char*)prefix,plen,bs,NULL);
}
if (raxTryInsert(bs->clients,(unsigned char*)&c,sizeof(c),NULL,NULL)) {
if (c->client_tracking_prefixes == NULL)
c->client_tracking_prefixes = raxNew();
raxInsert(c->client_tracking_prefixes,
(unsigned char*)prefix,plen,NULL,NULL);
c->flags &= ~(CLIENT_TRACKING|CLIENT_TRACKING_BROKEN_REDIR);
}
}
@@ -124,25 +83,16 @@ void enableBcastTrackingForPrefix(client *c, char *prefix, size_t plen) {
* eventually get freed, we'll send a message to the original client to
* inform it of the condition. Multiple clients can redirect the invalidation
* messages to the same client ID. */
void enableTracking(client *c, uint64_t redirect_to, int bcast, robj **prefix, size_t numprefix) {
if (!(c->flags & CLIENT_TRACKING)) server.tracking_clients++;
void enableTracking(client *c, uint64_t redirect_to) {
if (c->flags & CLIENT_TRACKING) return;
c->flags |= CLIENT_TRACKING;
c->flags &= ~(CLIENT_TRACKING_BROKEN_REDIR|CLIENT_TRACKING_BCAST);
c->flags &= ~CLIENT_TRACKING_BROKEN_REDIR;
c->client_tracking_redirection = redirect_to;
server.tracking_clients++;
if (TrackingTable == NULL) {
TrackingTable = raxNew();
PrefixTable = raxNew();
TrackingTable = zcalloc(sizeof(rax*) * TRACKING_TABLE_SIZE);
TrackingChannelName = createStringObject("__redis__:invalidate",20);
}
if (bcast) {
c->flags |= CLIENT_TRACKING_BCAST;
if (numprefix == 0) enableBcastTrackingForPrefix(c,"",0);
for (size_t j = 0; j < numprefix; j++) {
sds sdsprefix = prefix[j]->ptr;
enableBcastTrackingForPrefix(c,sdsprefix,sdslen(sdsprefix));
}
}
}
/* This function is called after the excution of a readonly command in the
@@ -158,30 +108,19 @@ void trackingRememberKeys(client *c) {
for(int j = 0; j < numkeys; j++) {
int idx = keys[j];
sds sdskey = c->argv[idx]->ptr;
rax *ids = raxFind(TrackingTable,(unsigned char*)sdskey,sdslen(sdskey));
if (ids == raxNotFound) {
ids = raxNew();
int inserted = raxTryInsert(TrackingTable,(unsigned char*)sdskey,
sdslen(sdskey),ids, NULL);
serverAssert(inserted == 1);
uint64_t hash = crc64(0,
(unsigned char*)sdskey,sdslen(sdskey))&(TRACKING_TABLE_SIZE-1);
if (TrackingTable[hash] == NULL) {
TrackingTable[hash] = raxNew();
TrackingTableUsedSlots++;
}
if (raxTryInsert(ids,(unsigned char*)&c->id,sizeof(c->id),NULL,NULL))
TrackingTableTotalItems++;
raxTryInsert(TrackingTable[hash],
(unsigned char*)&c->id,sizeof(c->id),NULL,NULL);
}
getKeysFreeResult(keys);
}
/* Given a key name, this function sends an invalidation message in the
* proper channel (depending on RESP version: PubSub or Push message) and
* to the proper client (in case fo redirection), in the context of the
* client 'c' with tracking enabled.
*
* In case the 'proto' argument is non zero, the function will assume that
* 'keyname' points to a buffer of 'keylen' bytes already expressed in the
* form of Redis RESP protocol, representing an array of keys to send
* to the client as value of the invalidation. This is used in BCAST mode
* in order to optimized the implementation to use less CPU time. */
void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
void sendTrackingMessage(client *c, long long hash) {
int using_redirection = 0;
if (c->client_tracking_redirection) {
client *redir = lookupClientByID(c->client_tracking_redirection);
@@ -207,45 +146,36 @@ void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
if (c->resp > 2) {
addReplyPushLen(c,2);
addReplyBulkCBuffer(c,"invalidate",10);
addReplyLongLong(c,hash);
} else if (using_redirection && c->flags & CLIENT_PUBSUB) {
/* We use a static object to speedup things, however we assume
* that addReplyPubsubMessage() will not take a reference. */
addReplyPubsubMessage(c,TrackingChannelName,NULL);
} else {
/* If are here, the client is not using RESP3, nor is
* redirecting to another client. We can't send anything to
* it since RESP2 does not support push messages in the same
* connection. */
return;
}
/* Send the "value" part, which is the array of keys. */
if (proto) {
addReplyProto(c,keyname,keylen);
} else {
addReplyArrayLen(c,1);
addReplyBulkCBuffer(c,keyname,keylen);
robj *msg = createStringObjectFromLongLong(hash);
addReplyPubsubMessage(c,TrackingChannelName,msg);
decrRefCount(msg);
}
}
/* This function is called when a key is modified in Redis and in the case
* we have at least one client with the BCAST mode enabled.
* Its goal is to set the key in the right broadcast state if the key
* matches one or more prefixes in the prefix table. Later when we
* return to the event loop, we'll send invalidation messages to the
* clients subscribed to each prefix. */
void trackingRememberKeyToBroadcast(char *keyname, size_t keylen) {
/* Invalidates a caching slot: this is actually the low level implementation
* of the API that Redis calls externally, that is trackingInvalidateKey(). */
void trackingInvalidateSlot(uint64_t slot) {
if (TrackingTable == NULL || TrackingTable[slot] == NULL) return;
raxIterator ri;
raxStart(&ri,PrefixTable);
raxStart(&ri,TrackingTable[slot]);
raxSeek(&ri,"^",NULL,0);
while(raxNext(&ri)) {
if (ri.key_len > keylen) continue;
if (ri.key_len != 0 && memcmp(ri.key,keyname,ri.key_len) != 0)
continue;
bcastState *bs = ri.data;
raxTryInsert(bs->keys,(unsigned char*)keyname,keylen,NULL,NULL);
uint64_t id;
memcpy(&id,ri.key,ri.key_len);
client *c = lookupClientByID(id);
if (c == NULL || !(c->flags & CLIENT_TRACKING)) continue;
sendTrackingMessage(c,slot);
}
raxStop(&ri);
/* Free the tracking table: we'll create the radix tree and populate it
* again if more keys will be modified in this caching slot. */
raxFree(TrackingTable[slot]);
TrackingTable[slot] = NULL;
TrackingTableUsedSlots--;
}
/* This function is called from signalModifiedKey() or other places in Redis
@@ -253,42 +183,12 @@ void trackingRememberKeyToBroadcast(char *keyname, size_t keylen) {
* to send a notification to every client that may have keys about such caching
* slot. */
void trackingInvalidateKey(robj *keyobj) {
if (TrackingTable == NULL) return;
if (TrackingTable == NULL || TrackingTableUsedSlots == 0) return;
sds sdskey = keyobj->ptr;
if (raxSize(PrefixTable) > 0)
trackingRememberKeyToBroadcast(sdskey,sdslen(sdskey));
rax *ids = raxFind(TrackingTable,(unsigned char*)sdskey,sdslen(sdskey));
if (ids == raxNotFound) return;;
raxIterator ri;
raxStart(&ri,ids);
raxSeek(&ri,"^",NULL,0);
while(raxNext(&ri)) {
uint64_t id;
memcpy(&id,ri.key,sizeof(id));
client *c = lookupClientByID(id);
/* Note that if the client is in BCAST mode, we don't want to
* send invalidation messages that were pending in the case
* previously the client was not in BCAST mode. This can happen if
* TRACKING is enabled normally, and then the client switches to
* BCAST mode. */
if (c == NULL ||
!(c->flags & CLIENT_TRACKING)||
c->flags & CLIENT_TRACKING_BCAST)
{
continue;
}
sendTrackingMessage(c,sdskey,sdslen(sdskey),0);
}
raxStop(&ri);
/* Free the tracking table: we'll create the radix tree and populate it
* again if more keys will be modified in this caching slot. */
TrackingTableTotalItems -= raxSize(ids);
raxFree(ids);
raxRemove(TrackingTable,(unsigned char*)sdskey,sdslen(sdskey),NULL);
uint64_t hash = crc64(0,
(unsigned char*)sdskey,sdslen(sdskey))&(TRACKING_TABLE_SIZE-1);
trackingInvalidateSlot(hash);
}
/* This function is called when one or all the Redis databases are flushed
@@ -305,10 +205,6 @@ void trackingInvalidateKey(robj *keyobj) {
* we just send the invalidation message to all the clients, but don't
* flush the table: it will slowly get garbage collected as more keys
* are modified in the used caching slots. */
void freeTrackingRadixTree(void *rt) {
raxFree(rt);
}
void trackingInvalidateKeysOnFlush(int dbid) {
if (server.tracking_clients) {
listNode *ln;
@@ -317,129 +213,84 @@ void trackingInvalidateKeysOnFlush(int dbid) {
while ((ln = listNext(&li)) != NULL) {
client *c = listNodeValue(ln);
if (c->flags & CLIENT_TRACKING) {
sendTrackingMessage(c,"",1,0);
sendTrackingMessage(c,-1);
}
}
}
/* In case of FLUSHALL, reclaim all the memory used by tracking. */
if (dbid == -1 && TrackingTable) {
raxFreeWithCallback(TrackingTable,freeTrackingRadixTree);
TrackingTableTotalItems = 0;
for (int j = 0; j < TRACKING_TABLE_SIZE && TrackingTableUsedSlots > 0; j++) {
if (TrackingTable[j] != NULL) {
raxFree(TrackingTable[j]);
TrackingTable[j] = NULL;
TrackingTableUsedSlots--;
}
}
/* If there are no clients with tracking enabled, we can even
* reclaim the memory used by the table itself. The code assumes
* the table is allocated only if there is at least one client alive
* with tracking enabled. */
if (server.tracking_clients == 0) {
zfree(TrackingTable);
TrackingTable = NULL;
}
}
}
/* Tracking forces Redis to remember information about which client may have
* certain keys. In workloads where there are a lot of reads, but keys are
* hardly modified, the amount of information we have to remember server side
* could be a lot, with the number of keys being totally not bound.
* keys about certian caching slots. In workloads where there are a lot of
* reads, but keys are hardly modified, the amount of information we have
* to remember server side could be a lot: for each 16 millions of caching
* slots we may end with a radix tree containing many entries.
*
* So Redis allows the user to configure a maximum number of keys for the
* So Redis allows the user to configure a maximum fill rate for the
* invalidation table. This function makes sure that we don't go over the
* specified fill rate: if we are over, we can just evict informations about
* a random key, and send invalidation messages to clients like if the key was
* modified. */
* random caching slots, and send invalidation messages to clients like if
* the key was modified. */
void trackingLimitUsedSlots(void) {
static unsigned int timeout_counter = 0;
if (TrackingTable == NULL) return;
if (server.tracking_table_max_keys == 0) return; /* No limits set. */
size_t max_keys = server.tracking_table_max_keys;
if (raxSize(TrackingTable) <= max_keys) {
if (server.tracking_table_max_fill == 0) return; /* No limits set. */
unsigned int max_slots =
(TRACKING_TABLE_SIZE/100) * server.tracking_table_max_fill;
if (TrackingTableUsedSlots <= max_slots) {
timeout_counter = 0;
return; /* Limit not reached. */
}
/* We have to invalidate a few keys to reach the limit again. The effort
/* We have to invalidate a few slots to reach the limit again. The effort
* we do here is proportional to the number of times we entered this
* function and found that we are still over the limit. */
int effort = 100 * (timeout_counter+1);
/* We just remove one key after another by using a random walk. */
raxIterator ri;
raxStart(&ri,TrackingTable);
/* Let's start at a random position, and perform linear probing, in order
* to improve cache locality. However once we are able to find an used
* slot, jump again randomly, in order to avoid creating big holes in the
* table (that will make this funciton use more resourced later). */
while(effort > 0) {
effort--;
raxSeek(&ri,"^",NULL,0);
raxRandomWalk(&ri,0);
rax *ids = ri.data;
TrackingTableTotalItems -= raxSize(ids);
raxFree(ids);
raxRemove(TrackingTable,ri.key,ri.key_len,NULL);
if (raxSize(TrackingTable) <= max_keys) {
timeout_counter = 0;
raxStop(&ri);
return; /* Return ASAP: we are again under the limit. */
}
unsigned int idx = rand() % TRACKING_TABLE_SIZE;
do {
effort--;
idx = (idx+1) % TRACKING_TABLE_SIZE;
if (TrackingTable[idx] != NULL) {
trackingInvalidateSlot(idx);
if (TrackingTableUsedSlots <= max_slots) {
timeout_counter = 0;
return; /* Return ASAP: we are again under the limit. */
} else {
break; /* Jump to next random position. */
}
}
} while(effort > 0);
}
/* If we reach this point, we were not able to go under the configured
* limit using the maximum effort we had for this run. */
raxStop(&ri);
timeout_counter++;
}
/* This function will run the prefixes of clients in BCAST mode and
* keys that were modified about each prefix, and will send the
* notifications to each client in each prefix. */
void trackingBroadcastInvalidationMessages(void) {
raxIterator ri, ri2;
/* Return ASAP if there is nothing to do here. */
if (TrackingTable == NULL || !server.tracking_clients) return;
raxStart(&ri,PrefixTable);
raxSeek(&ri,"^",NULL,0);
while(raxNext(&ri)) {
bcastState *bs = ri.data;
if (raxSize(bs->keys)) {
/* Create the array reply with the list of keys once, then send
* it to all the clients subscribed to this prefix. */
char buf[32];
size_t len = ll2string(buf,sizeof(buf),raxSize(bs->keys));
sds proto = sdsempty();
proto = sdsMakeRoomFor(proto,raxSize(bs->keys)*15);
proto = sdscatlen(proto,"*",1);
proto = sdscatlen(proto,buf,len);
proto = sdscatlen(proto,"\r\n",2);
raxStart(&ri2,bs->keys);
raxSeek(&ri2,"^",NULL,0);
while(raxNext(&ri2)) {
len = ll2string(buf,sizeof(buf),ri2.key_len);
proto = sdscatlen(proto,"$",1);
proto = sdscatlen(proto,buf,len);
proto = sdscatlen(proto,"\r\n",2);
proto = sdscatlen(proto,ri2.key,ri2.key_len);
proto = sdscatlen(proto,"\r\n",2);
}
raxStop(&ri2);
/* Send this array of keys to every client in the list. */
raxStart(&ri2,bs->clients);
raxSeek(&ri2,"^",NULL,0);
while(raxNext(&ri2)) {
client *c;
memcpy(&c,ri2.key,sizeof(c));
sendTrackingMessage(c,proto,sdslen(proto),1);
}
raxStop(&ri2);
/* Clean up: we can remove everything from this state, because we
* want to only track the new keys that will be accumulated starting
* from now. */
sdsfree(proto);
}
raxFree(bs->keys);
bs->keys = raxNew();
}
raxStop(&ri);
}
/* This is just used in order to access the amount of used slots in the
* tracking table. */
uint64_t trackingGetTotalItems(void) {
return TrackingTableTotalItems;
}
uint64_t trackingGetTotalKeys(void) {
return raxSize(TrackingTable);
unsigned long long trackingGetUsedSlots(void) {
return TrackingTableUsedSlots;
}
+1 -43
View File
@@ -423,26 +423,6 @@ int string2ll(const char *s, size_t slen, long long *value) {
return 1;
}
/* Helper function to convert a string to an unsigned long long value.
* The function attempts to use the faster string2ll() function inside
* Redis: if it fails, strtoull() is used instead. The function returns
* 1 if the conversion happened successfully or 0 if the number is
* invalid or out of range. */
int string2ull(const char *s, unsigned long long *value) {
long long ll;
if (string2ll(s,strlen(s),&ll)) {
if (ll < 0) return 0; /* Negative values are out of range. */
*value = ll;
return 1;
}
errno = 0;
char *endptr = NULL;
*value = strtoull(s,&endptr,10);
if (errno == EINVAL || errno == ERANGE || !(*s != '\0' && *endptr == '\0'))
return 0; /* strtoull() failed. */
return 1; /* Conversion done! */
}
/* Convert a string into a long. Returns 1 if the string could be parsed into a
* (non-overflowing) long, 0 otherwise. The value will be set to the parsed
* value when appropriate. */
@@ -471,14 +451,13 @@ int string2ld(const char *s, size_t slen, long double *dp) {
long double value;
char *eptr;
if (slen == 0 || slen >= sizeof(buf)) return 0;
if (slen >= sizeof(buf)) return 0;
memcpy(buf,s,slen);
buf[slen] = '\0';
errno = 0;
value = strtold(buf, &eptr);
if (isspace(buf[0]) || eptr[0] != '\0' ||
(size_t)(eptr-buf) != slen ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
errno == EINVAL ||
@@ -489,27 +468,6 @@ int string2ld(const char *s, size_t slen, long double *dp) {
return 1;
}
/* Convert a string into a double. Returns 1 if the string could be parsed
* into a (non-overflowing) double, 0 otherwise. The value will be set to
* the parsed value when appropriate.
*
* Note that this function demands that the string strictly represents
* a double: no spaces or other characters before or after the string
* representing the number are accepted. */
int string2d(const char *s, size_t slen, double *dp) {
errno = 0;
char *eptr;
*dp = strtod(s, &eptr);
if (slen == 0 ||
isspace(((const char*)s)[0]) ||
(size_t)(eptr-(char*)s) != slen ||
(errno == ERANGE &&
(*dp == HUGE_VAL || *dp == -HUGE_VAL || *dp == 0)) ||
isnan(*dp))
return 0;
return 1;
}
/* Convert a double to a string representation. Returns the number of bytes
* required. The representation should always be parsable by strtod(3).
* This function does not support human-friendly formatting like ld2string
-2
View File
@@ -53,10 +53,8 @@ uint32_t digits10(uint64_t v);
uint32_t sdigits10(int64_t v);
int ll2string(char *s, size_t len, long long value);
int string2ll(const char *s, size_t slen, long long *value);
int string2ull(const char *s, unsigned long long *value);
int string2l(const char *s, size_t slen, long *value);
int string2ld(const char *s, size_t slen, long double *dp);
int string2d(const char *s, size_t slen, double *dp);
int d2string(char *buf, size_t len, double value);
int ld2string(char *buf, size_t len, long double value, ld2string_mode mode);
sds getAbsolutePath(char *filename);
-29
View File
@@ -388,14 +388,6 @@ int jemalloc_purge() {
#endif
#if defined(__APPLE__)
/* For proc_pidinfo() used later in zmalloc_get_smap_bytes_by_field().
* Note that this file cannot be included in zmalloc.h because it includes
* a Darwin queue.h file where there is a "LIST_HEAD" macro (!) defined
* conficting with Redis user code. */
#include <libproc.h>
#endif
/* Get the sum of the specified field (converted form kb to bytes) in
* /proc/self/smaps. The field must be specified with trailing ":" as it
* apperas in the smaps output.
@@ -435,28 +427,7 @@ size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
return bytes;
}
#else
/* Get sum of the specified field from libproc api call.
* As there are per page value basis we need to convert
* them accordingly.
*
* Note that AnonHugePages is a no-op as THP feature
* is not supported in this platform
*/
size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
#if defined(__APPLE__)
struct proc_regioninfo pri;
if (proc_pidinfo(pid, PROC_PIDREGIONINFO, 0, &pri, PROC_PIDREGIONINFO_SIZE) ==
PROC_PIDREGIONINFO_SIZE) {
if (!strcmp(field, "Private_Dirty:")) {
return (size_t)pri.pri_pages_dirtied * 4096;
} else if (!strcmp(field, "Rss:")) {
return (size_t)pri.pri_pages_resident * 4096;
} else if (!strcmp(field, "AnonHugePages:")) {
return 0;
}
}
return 0;
#endif
((void) field);
((void) pid);
return 0;
+1 -1
View File
@@ -4,7 +4,7 @@ set ::tlsdir "tests/tls"
proc gen_write_load {host port seconds tls} {
set start_time [clock seconds]
set r [redis $host $port 1 $tls]
set r [redis $host $port 0 $tls]
$r select 9
while 1 {
$r set [expr rand()] [expr rand()]
-7
View File
@@ -70,13 +70,6 @@ start_server {tags {"repl"}} {
}
}
test {INCRBYFLOAT replication, should not remove expire} {
r set test 1 EX 100
r incrbyfloat test 0.1
after 1000
assert_equal [$A debug digest] [$B debug digest]
}
test {BRPOPLPUSH replication, when blocking against empty list} {
set rd [redis_deferring_client]
$rd brpoplpush a b 5
+1 -4
View File
@@ -19,10 +19,7 @@ TEST_MODULES = \
propagate.so \
misc.so \
hooks.so \
blockonkeys.so \
scan.so \
datatype.so \
auth.so
blockonkeys.so
.PHONY: all
-90
View File
@@ -1,90 +0,0 @@
#define REDISMODULE_EXPERIMENTAL_API
#include "redismodule.h"
// A simple global user
static RedisModuleUser *global = NULL;
static long long client_change_delta = 0;
void UserChangedCallback(uint64_t client_id, void *privdata) {
REDISMODULE_NOT_USED(privdata);
REDISMODULE_NOT_USED(client_id);
client_change_delta++;
}
int Auth_CreateModuleUser(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (global) {
RedisModule_FreeModuleUser(global);
}
global = RedisModule_CreateModuleUser("global");
RedisModule_SetModuleUserACL(global, "allcommands");
RedisModule_SetModuleUserACL(global, "allkeys");
RedisModule_SetModuleUserACL(global, "on");
return RedisModule_ReplyWithSimpleString(ctx, "OK");
}
int Auth_AuthModuleUser(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
uint64_t client_id;
RedisModule_AuthenticateClientWithUser(ctx, global, UserChangedCallback, NULL, &client_id);
return RedisModule_ReplyWithLongLong(ctx, (uint64_t) client_id);
}
int Auth_AuthRealUser(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 2) return RedisModule_WrongArity(ctx);
size_t length;
uint64_t client_id;
RedisModuleString *user_string = argv[1];
const char *name = RedisModule_StringPtrLen(user_string, &length);
if (RedisModule_AuthenticateClientWithACLUser(ctx, name, length,
UserChangedCallback, NULL, &client_id) == REDISMODULE_ERR) {
return RedisModule_ReplyWithError(ctx, "Invalid user");
}
return RedisModule_ReplyWithLongLong(ctx, (uint64_t) client_id);
}
int Auth_ChangeCount(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
long long result = client_change_delta;
client_change_delta = 0;
return RedisModule_ReplyWithLongLong(ctx, result);
}
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"testacl",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"auth.authrealuser",
Auth_AuthRealUser,"no-auth",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"auth.createmoduleuser",
Auth_CreateModuleUser,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"auth.authmoduleuser",
Auth_AuthModuleUser,"no-auth",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"auth.changecount",
Auth_ChangeCount,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
+6 -7
View File
@@ -172,13 +172,13 @@ int bpopgt_reply_callback(RedisModuleCtx *ctx, RedisModuleString **argv, int arg
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
RedisModuleString *keyname = RedisModule_GetBlockedClientReadyKey(ctx);
long long *pgt = RedisModule_GetBlockedClientPrivateData(ctx);
long long gt = (long long)RedisModule_GetBlockedClientPrivateData(ctx);
fsl_t *fsl;
if (!get_fsl(ctx, keyname, REDISMODULE_READ, 0, &fsl, 0))
return REDISMODULE_ERR;
if (!fsl || fsl->list[fsl->length-1] <= *pgt)
if (!fsl || fsl->list[fsl->length-1] <= gt)
return REDISMODULE_ERR;
RedisModule_ReplyWithLongLong(ctx, fsl->list[--fsl->length]);
@@ -192,8 +192,10 @@ int bpopgt_timeout_callback(RedisModuleCtx *ctx, RedisModuleString **argv, int a
}
void bpopgt_free_privdata(RedisModuleCtx *ctx, void *privdata) {
/* Nothing to do because privdata is actually a 'long long',
* not a pointer to the heap */
REDISMODULE_NOT_USED(ctx);
RedisModule_Free(privdata);
REDISMODULE_NOT_USED(privdata);
}
/* FSL.BPOPGT <key> <gt> <timeout> - Block clients until list has an element greater than <gt>.
@@ -215,12 +217,9 @@ int fsl_bpopgt(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return REDISMODULE_OK;
if (!fsl || fsl->list[fsl->length-1] <= gt) {
/* We use malloc so the tests in blockedonkeys.tcl can check for memory leaks */
long long *pgt = RedisModule_Alloc(sizeof(long long));
*pgt = gt;
/* Key is empty or has <2 elements, we must block */
RedisModule_BlockClientOnKeys(ctx, bpopgt_reply_callback, bpopgt_timeout_callback,
bpopgt_free_privdata, timeout, &argv[1], 1, pgt);
bpopgt_free_privdata, timeout, &argv[1], 1, (void*)gt);
} else {
RedisModule_ReplyWithLongLong(ctx, fsl->list[--fsl->length]);
}
-186
View File
@@ -1,186 +0,0 @@
/* This module current tests a small subset but should be extended in the future
* for general ModuleDataType coverage.
*/
#include "redismodule.h"
static RedisModuleType *datatype = NULL;
typedef struct {
long long intval;
RedisModuleString *strval;
} DataType;
static void *datatype_load(RedisModuleIO *io, int encver) {
(void) encver;
int intval = RedisModule_LoadSigned(io);
if (RedisModule_IsIOError(io)) return NULL;
RedisModuleString *strval = RedisModule_LoadString(io);
if (RedisModule_IsIOError(io)) return NULL;
DataType *dt = (DataType *) RedisModule_Alloc(sizeof(DataType));
dt->intval = intval;
dt->strval = strval;
return dt;
}
static void datatype_save(RedisModuleIO *io, void *value) {
DataType *dt = (DataType *) value;
RedisModule_SaveSigned(io, dt->intval);
RedisModule_SaveString(io, dt->strval);
}
static void datatype_free(void *value) {
if (value) {
DataType *dt = (DataType *) value;
if (dt->strval) RedisModule_FreeString(NULL, dt->strval);
RedisModule_Free(dt);
}
}
static int datatype_set(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 4) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
long long intval;
if (RedisModule_StringToLongLong(argv[2], &intval) != REDISMODULE_OK) {
RedisModule_ReplyWithError(ctx, "Invalid integr value");
return REDISMODULE_OK;
}
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_WRITE);
DataType *dt = RedisModule_Calloc(sizeof(DataType), 1);
dt->intval = intval;
dt->strval = argv[3];
RedisModule_RetainString(ctx, dt->strval);
RedisModule_ModuleTypeSetValue(key, datatype, dt);
RedisModule_CloseKey(key);
RedisModule_ReplyWithSimpleString(ctx, "OK");
return REDISMODULE_OK;
}
static int datatype_restore(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 3) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
DataType *dt = RedisModule_LoadDataTypeFromString(argv[2], datatype);
if (!dt) {
RedisModule_ReplyWithError(ctx, "Invalid data");
return REDISMODULE_OK;
}
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_WRITE);
RedisModule_ModuleTypeSetValue(key, datatype, dt);
RedisModule_CloseKey(key);
RedisModule_ReplyWithSimpleString(ctx, "OK");
return REDISMODULE_OK;
}
static int datatype_get(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_READ);
DataType *dt = RedisModule_ModuleTypeGetValue(key);
RedisModule_CloseKey(key);
RedisModule_ReplyWithArray(ctx, 2);
RedisModule_ReplyWithLongLong(ctx, dt->intval);
RedisModule_ReplyWithString(ctx, dt->strval);
return REDISMODULE_OK;
}
static int datatype_dump(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_READ);
DataType *dt = RedisModule_ModuleTypeGetValue(key);
RedisModule_CloseKey(key);
RedisModuleString *reply = RedisModule_SaveDataTypeToString(ctx, dt, datatype);
if (!reply) {
RedisModule_ReplyWithError(ctx, "Failed to save");
return REDISMODULE_OK;
}
RedisModule_ReplyWithString(ctx, reply);
RedisModule_FreeString(ctx, reply);
return REDISMODULE_OK;
}
static int datatype_swap(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 3) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *a = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_WRITE);
RedisModuleKey *b = RedisModule_OpenKey(ctx, argv[2], REDISMODULE_WRITE);
void *val = RedisModule_ModuleTypeGetValue(a);
int error = (RedisModule_ModuleTypeReplaceValue(b, datatype, val, &val) == REDISMODULE_ERR ||
RedisModule_ModuleTypeReplaceValue(a, datatype, val, NULL) == REDISMODULE_ERR);
if (!error)
RedisModule_ReplyWithSimpleString(ctx, "OK");
else
RedisModule_ReplyWithError(ctx, "ERR failed");
RedisModule_CloseKey(a);
RedisModule_CloseKey(b);
return REDISMODULE_OK;
}
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"datatype",1,REDISMODULE_APIVER_1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
RedisModule_SetModuleOptions(ctx, REDISMODULE_OPTIONS_HANDLE_IO_ERRORS);
RedisModuleTypeMethods datatype_methods = {
.version = REDISMODULE_TYPE_METHOD_VERSION,
.rdb_load = datatype_load,
.rdb_save = datatype_save,
.free = datatype_free,
};
datatype = RedisModule_CreateDataType(ctx, "test___dt", 1, &datatype_methods);
if (datatype == NULL)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"datatype.set", datatype_set,"deny-oom",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"datatype.get", datatype_get,"",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"datatype.restore", datatype_restore,"deny-oom",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"datatype.dump", datatype_dump,"",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"datatype.swap", datatype_swap,"",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
-73
View File
@@ -5,7 +5,6 @@
void InfoFunc(RedisModuleInfoCtx *ctx, int for_crash_report) {
RedisModule_InfoAddSection(ctx, "");
RedisModule_InfoAddFieldLongLong(ctx, "global", -2);
RedisModule_InfoAddFieldULongLong(ctx, "uglobal", (unsigned long long)-2);
RedisModule_InfoAddSection(ctx, "Spanish");
RedisModule_InfoAddFieldCString(ctx, "uno", "one");
@@ -30,67 +29,6 @@ void InfoFunc(RedisModuleInfoCtx *ctx, int for_crash_report) {
}
int info_get(RedisModuleCtx *ctx, RedisModuleString **argv, int argc, char field_type)
{
if (argc != 3 && argc != 4) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
int err = REDISMODULE_OK;
const char *section, *field;
section = RedisModule_StringPtrLen(argv[1], NULL);
field = RedisModule_StringPtrLen(argv[2], NULL);
RedisModuleServerInfoData *info = RedisModule_GetServerInfo(ctx, section);
if (field_type=='i') {
long long ll = RedisModule_ServerInfoGetFieldSigned(info, field, &err);
if (err==REDISMODULE_OK)
RedisModule_ReplyWithLongLong(ctx, ll);
} else if (field_type=='u') {
unsigned long long ll = (unsigned long long)RedisModule_ServerInfoGetFieldUnsigned(info, field, &err);
if (err==REDISMODULE_OK)
RedisModule_ReplyWithLongLong(ctx, ll);
} else if (field_type=='d') {
double d = RedisModule_ServerInfoGetFieldDouble(info, field, &err);
if (err==REDISMODULE_OK)
RedisModule_ReplyWithDouble(ctx, d);
} else if (field_type=='c') {
const char *str = RedisModule_ServerInfoGetFieldC(info, field);
if (str)
RedisModule_ReplyWithCString(ctx, str);
} else {
RedisModuleString *str = RedisModule_ServerInfoGetField(ctx, info, field);
if (str) {
RedisModule_ReplyWithString(ctx, str);
RedisModule_FreeString(ctx, str);
} else
err=REDISMODULE_ERR;
}
if (err!=REDISMODULE_OK)
RedisModule_ReplyWithError(ctx, "not found");
RedisModule_FreeServerInfo(ctx, info);
return REDISMODULE_OK;
}
int info_gets(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return info_get(ctx, argv, argc, 's');
}
int info_getc(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return info_get(ctx, argv, argc, 'c');
}
int info_geti(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return info_get(ctx, argv, argc, 'i');
}
int info_getu(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return info_get(ctx, argv, argc, 'u');
}
int info_getd(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return info_get(ctx, argv, argc, 'd');
}
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
@@ -99,16 +37,5 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
if (RedisModule_RegisterInfoFunc(ctx, InfoFunc) == REDISMODULE_ERR) return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"info.gets", info_gets,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"info.getc", info_getc,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"info.geti", info_geti,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"info.getu", info_getu,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"info.getd", info_getd,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
+10 -108
View File
@@ -6,8 +6,6 @@
#include <unistd.h>
#include <errno.h>
#define UNUSED(x) (void)(x)
int test_call_generic(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc<2) {
@@ -42,54 +40,6 @@ int test_call_info(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_OK;
}
int test_ld_conv(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
UNUSED(argv);
UNUSED(argc);
long double ld = 0.00000000000000001L;
const char *ldstr = "0.00000000000000001";
RedisModuleString *s1 = RedisModule_CreateStringFromLongDouble(ctx, ld, 1);
RedisModuleString *s2 =
RedisModule_CreateString(ctx, ldstr, strlen(ldstr));
if (RedisModule_StringCompare(s1, s2) != 0) {
char err[4096];
snprintf(err, 4096,
"Failed to convert long double to string ('%s' != '%s')",
RedisModule_StringPtrLen(s1, NULL),
RedisModule_StringPtrLen(s2, NULL));
RedisModule_ReplyWithError(ctx, err);
goto final;
}
long double ld2 = 0;
if (RedisModule_StringToLongDouble(s2, &ld2) == REDISMODULE_ERR) {
RedisModule_ReplyWithError(ctx,
"Failed to convert string to long double");
goto final;
}
if (ld2 != ld) {
char err[4096];
snprintf(err, 4096,
"Failed to convert string to long double (%.40Lf != %.40Lf)",
ld2,
ld);
RedisModule_ReplyWithError(ctx, err);
goto final;
}
/* Make sure we can't convert a string that has \0 in it */
char buf[4] = "123";
buf[1] = '\0';
RedisModuleString *s3 = RedisModule_CreateString(ctx, buf, 3);
long double ld3;
if (RedisModule_StringToLongDouble(s3, &ld3) == REDISMODULE_OK) {
RedisModule_ReplyWithError(ctx, "Invalid string successfully converted to long double");
goto final;
}
RedisModule_ReplyWithLongDouble(ctx, ld2);
final:
RedisModule_FreeString(ctx, s1);
RedisModule_FreeString(ctx, s2);
return REDISMODULE_OK;
}
int test_flushall(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
REDISMODULE_NOT_USED(argv);
@@ -118,24 +68,16 @@ int test_randomkey(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_OK;
}
RedisModuleKey *open_key_or_reply(RedisModuleCtx *ctx, RedisModuleString *keyname, int mode) {
RedisModuleKey *key = RedisModule_OpenKey(ctx, keyname, mode);
if (!key) {
RedisModule_ReplyWithError(ctx, "key not found");
return NULL;
}
return key;
}
int test_getlru(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc<2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = open_key_or_reply(ctx, argv[1], REDISMODULE_READ|REDISMODULE_OPEN_KEY_NOTOUCH);
mstime_t lru;
RedisModule_GetLRU(key, &lru);
RedisModuleString *keyname = argv[1];
RedisModuleKey *key = RedisModule_OpenKey(ctx, keyname, REDISMODULE_READ|REDISMODULE_OPEN_KEY_NOTOUCH);
long long lru, lfu;
RedisModule_GetLRUOrLFU(key, &lfu, &lru);
RedisModule_ReplyWithLongLong(ctx, lru);
RedisModule_CloseKey(key);
return REDISMODULE_OK;
@@ -147,46 +89,12 @@ int test_setlru(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = open_key_or_reply(ctx, argv[1], REDISMODULE_READ|REDISMODULE_OPEN_KEY_NOTOUCH);
mstime_t lru;
if (RedisModule_StringToLongLong(argv[2], &lru) != REDISMODULE_OK) {
RedisModule_ReplyWithError(ctx, "invalid idle time");
return REDISMODULE_OK;
}
int was_set = RedisModule_SetLRU(key, lru)==REDISMODULE_OK;
RedisModule_ReplyWithLongLong(ctx, was_set);
RedisModule_CloseKey(key);
return REDISMODULE_OK;
}
int test_getlfu(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc<2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = open_key_or_reply(ctx, argv[1], REDISMODULE_READ|REDISMODULE_OPEN_KEY_NOTOUCH);
mstime_t lfu;
RedisModule_GetLFU(key, &lfu);
RedisModule_ReplyWithLongLong(ctx, lfu);
RedisModule_CloseKey(key);
return REDISMODULE_OK;
}
int test_setlfu(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc<3) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModuleKey *key = open_key_or_reply(ctx, argv[1], REDISMODULE_READ|REDISMODULE_OPEN_KEY_NOTOUCH);
mstime_t lfu;
if (RedisModule_StringToLongLong(argv[2], &lfu) != REDISMODULE_OK) {
RedisModule_ReplyWithError(ctx, "invalid freq");
return REDISMODULE_OK;
}
int was_set = RedisModule_SetLFU(key, lfu)==REDISMODULE_OK;
RedisModule_ReplyWithLongLong(ctx, was_set);
RedisModuleString *keyname = argv[1];
RedisModuleKey *key = RedisModule_OpenKey(ctx, keyname, REDISMODULE_WRITE|REDISMODULE_OPEN_KEY_NOTOUCH);
long long lru;
RedisModule_StringToLongLong(argv[2], &lru);
RedisModule_SetLRUOrLFU(key, -1, lru);
RedisModule_ReplyWithCString(ctx, "Ok");
RedisModule_CloseKey(key);
return REDISMODULE_OK;
}
@@ -201,8 +109,6 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.call_info", test_call_info,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.ld_conversion", test_ld_conv, "",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.flushall", test_flushall,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.dbsize", test_dbsize,"",0,0,0) == REDISMODULE_ERR)
@@ -213,10 +119,6 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.getlru", test_getlru,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.setlfu", test_setlfu,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.getlfu", test_getlfu,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
-109
View File
@@ -1,109 +0,0 @@
#include "redismodule.h"
#include <string.h>
#include <assert.h>
#include <unistd.h>
typedef struct {
size_t nkeys;
} scan_strings_pd;
void scan_strings_callback(RedisModuleCtx *ctx, RedisModuleString* keyname, RedisModuleKey* key, void *privdata) {
scan_strings_pd* pd = privdata;
int was_opened = 0;
if (!key) {
key = RedisModule_OpenKey(ctx, keyname, REDISMODULE_READ);
was_opened = 1;
}
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_STRING) {
size_t len;
char * data = RedisModule_StringDMA(key, &len, REDISMODULE_READ);
RedisModule_ReplyWithArray(ctx, 2);
RedisModule_ReplyWithString(ctx, keyname);
RedisModule_ReplyWithStringBuffer(ctx, data, len);
pd->nkeys++;
}
if (was_opened)
RedisModule_CloseKey(key);
}
int scan_strings(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
scan_strings_pd pd = {
.nkeys = 0,
};
RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_ARRAY_LEN);
RedisModuleScanCursor* cursor = RedisModule_ScanCursorCreate();
while(RedisModule_Scan(ctx, cursor, scan_strings_callback, &pd));
RedisModule_ScanCursorDestroy(cursor);
RedisModule_ReplySetArrayLength(ctx, pd.nkeys);
return REDISMODULE_OK;
}
typedef struct {
RedisModuleCtx *ctx;
size_t nreplies;
} scan_key_pd;
void scan_key_callback(RedisModuleKey *key, RedisModuleString* field, RedisModuleString* value, void *privdata) {
REDISMODULE_NOT_USED(key);
scan_key_pd* pd = privdata;
RedisModule_ReplyWithArray(pd->ctx, 2);
RedisModule_ReplyWithString(pd->ctx, field);
if (value)
RedisModule_ReplyWithString(pd->ctx, value);
else
RedisModule_ReplyWithNull(pd->ctx);
pd->nreplies++;
}
int scan_key(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc != 2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
scan_key_pd pd = {
.ctx = ctx,
.nreplies = 0,
};
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_READ);
if (!key) {
RedisModule_ReplyWithError(ctx, "not found");
return REDISMODULE_OK;
}
RedisModule_ReplyWithArray(ctx, REDISMODULE_POSTPONED_ARRAY_LEN);
RedisModuleScanCursor* cursor = RedisModule_ScanCursorCreate();
while(RedisModule_ScanKey(key, cursor, scan_key_callback, &pd));
RedisModule_ScanCursorDestroy(cursor);
RedisModule_ReplySetArrayLength(ctx, pd.nreplies);
RedisModule_CloseKey(key);
return REDISMODULE_OK;
}
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx, "scan", 1, REDISMODULE_APIVER_1)== REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx, "scan.scan_strings", scan_strings, "", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx, "scan.scan_key", scan_key, "", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
+1 -5
View File
@@ -18,12 +18,8 @@ void *testrdb_type_load(RedisModuleIO *rdb, int encver) {
RedisModuleString *str = RedisModule_LoadString(rdb);
float f = RedisModule_LoadFloat(rdb);
long double ld = RedisModule_LoadLongDouble(rdb);
if (RedisModule_IsIOError(rdb)) {
RedisModuleCtx *ctx = RedisModule_GetContextFromIO(rdb);
if (str)
RedisModule_FreeString(ctx, str);
if (RedisModule_IsIOError(rdb))
return NULL;
}
/* Using the values only after checking for io errors. */
assert(count==1);
assert(encver==1);
+6 -33
View File
@@ -11,55 +11,28 @@ proc fail {msg} {
proc assert {condition} {
if {![uplevel 1 [list expr $condition]]} {
set context "(context: [info frame -1])"
error "assertion:Expected [uplevel 1 [list subst -nocommands $condition]] $context"
error "assertion:Expected condition '$condition' to be true ([uplevel 1 [list subst -nocommands $condition]])"
}
}
proc assert_no_match {pattern value} {
if {[string match $pattern $value]} {
set context "(context: [info frame -1])"
error "assertion:Expected '$value' to not match '$pattern' $context"
error "assertion:Expected '$value' to not match '$pattern'"
}
}
proc assert_match {pattern value} {
if {![string match $pattern $value]} {
set context "(context: [info frame -1])"
error "assertion:Expected '$value' to match '$pattern' $context"
error "assertion:Expected '$value' to match '$pattern'"
}
}
proc assert_equal {value expected {detail ""}} {
proc assert_equal {expected value {detail ""}} {
if {$expected ne $value} {
if {$detail ne ""} {
set detail "(detail: $detail)"
} else {
set detail "(context: [info frame -1])"
set detail " (detail: $detail)"
}
error "assertion:Expected '$value' to be equal to '$expected' $detail"
}
}
proc assert_lessthan {value expected {detail ""}} {
if {!($value < $expected)} {
if {$detail ne ""} {
set detail "(detail: $detail)"
} else {
set detail "(context: [info frame -1])"
}
error "assertion:Expected '$value' to be lessthan to '$expected' $detail"
}
}
proc assert_range {value min max {detail ""}} {
if {!($value <= $max && $value >= $min)} {
if {$detail ne ""} {
set detail "(detail: $detail)"
} else {
set detail "(context: [info frame -1])"
}
error "assertion:Expected '$value' to be between to '$min' and '$max' $detail"
error "assertion:Expected '$value' to be equal to '$expected'$detail"
}
}
-107
View File
@@ -141,111 +141,4 @@ start_server {tags {"acl"}} {
r ACL setuser newuser -debug
# The test framework will detect a leak if any.
}
test {ACL LOG shows failed command executions at toplevel} {
r ACL LOG RESET
r ACL setuser antirez >foo on +set ~object:1234
r ACL setuser antirez +eval +multi +exec
r AUTH antirez foo
catch {r GET foo}
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry username] eq {antirez}}
assert {[dict get $entry context] eq {toplevel}}
assert {[dict get $entry reason] eq {command}}
assert {[dict get $entry object] eq {get}}
}
test {ACL LOG is able to test similar events} {
r AUTH antirez foo
catch {r GET foo}
catch {r GET foo}
catch {r GET foo}
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry count] == 4}
}
test {ACL LOG is able to log keys access violations and key name} {
r AUTH antirez foo
catch {r SET somekeynotallowed 1234}
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry reason] eq {key}}
assert {[dict get $entry object] eq {somekeynotallowed}}
}
test {ACL LOG RESET is able to flush the entries in the log} {
r ACL LOG RESET
assert {[llength [r ACL LOG]] == 0}
}
test {ACL LOG can distinguish the transaction context (1)} {
r AUTH antirez foo
r MULTI
catch {r INCR foo}
catch {r EXEC}
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry context] eq {multi}}
assert {[dict get $entry object] eq {incr}}
}
test {ACL LOG can distinguish the transaction context (2)} {
set rd1 [redis_deferring_client]
r ACL SETUSER antirez +incr
r AUTH antirez foo
r MULTI
r INCR object:1234
$rd1 ACL SETUSER antirez -incr
$rd1 read
catch {r EXEC}
$rd1 close
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry context] eq {multi}}
assert {[dict get $entry object] eq {incr}}
r ACL SETUSER antirez -incr
}
test {ACL can log errors in the context of Lua scripting} {
r AUTH antirez foo
catch {r EVAL {redis.call('incr','foo')} 0}
r AUTH default ""
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry context] eq {lua}}
assert {[dict get $entry object] eq {incr}}
}
test {ACL LOG can accept a numerical argument to show less entries} {
r AUTH antirez foo
catch {r INCR foo}
catch {r INCR foo}
catch {r INCR foo}
catch {r INCR foo}
r AUTH default ""
assert {[llength [r ACL LOG]] > 1}
assert {[llength [r ACL LOG 2]] == 2}
}
test {ACL LOG can log failed auth attempts} {
catch {r AUTH antirez wrong-password}
set entry [lindex [r ACL LOG] 0]
assert {[dict get $entry context] eq {toplevel}}
assert {[dict get $entry reason] eq {auth}}
assert {[dict get $entry object] eq {AUTH}}
assert {[dict get $entry username] eq {antirez}}
}
test {ACL LOG entries are limited to a maximum amount} {
r ACL LOG RESET
r CONFIG SET acllog-max-len 5
r AUTH antirez foo
for {set j 0} {$j < 10} {incr j} {
catch {r SET obj:$j 123}
}
r AUTH default ""
assert {[llength [r ACL LOG]] == 5}
}
}
-13
View File
@@ -219,17 +219,4 @@ start_server {tags {"expire"}} {
set ttl [r ttl foo]
assert {$ttl <= 98 && $ttl > 90}
}
test {SET command will remove expire} {
r set foo bar EX 100
r set foo bar
r ttl foo
} {-1}
test {SET - use KEEPTTL option, TTL should not be removed} {
r set foo bar EX 100
r set foo bar KEEPTTL
set ttl [r ttl foo]
assert {$ttl <= 100 && $ttl > 90}
}
}
+1 -1
View File
@@ -129,7 +129,7 @@ start_server {tags {"geo"}} {
r del points
r geoadd points -5.6 42.6 test
lindex [r geohash points test] 0
} {ezs42e44yx0}
} {ezs42e44yx}
test {GEOPOS simple} {
r del points
-65
View File
@@ -57,69 +57,4 @@ start_server {tags {"introspection"}} {
fail "Client still listed in CLIENT LIST after SETNAME."
}
}
test {CONFIG sanity} {
# Do CONFIG GET, CONFIG SET and then CONFIG GET again
# Skip immutable configs, one with no get, and other complicated configs
set skip_configs {
rdbchecksum
daemonize
io-threads-do-reads
tcp-backlog
always-show-logo
syslog-enabled
cluster-enabled
aclfile
unixsocket
pidfile
syslog-ident
appendfilename
supervised
syslog-facility
databases
port
io-threads
tls-port
tls-prefer-server-ciphers
tls-cert-file
tls-key-file
tls-dh-params-file
tls-ca-cert-file
tls-ca-cert-dir
tls-protocols
tls-ciphers
tls-ciphersuites
logfile
unixsocketperm
slaveof
bind
requirepass
}
set configs {}
foreach {k v} [r config get *] {
if {[lsearch $skip_configs $k] != -1} {
continue
}
dict set configs $k $v
# try to set the config to the same value it already has
r config set $k $v
}
set newconfigs {}
foreach {k v} [r config get *] {
if {[lsearch $skip_configs $k] != -1} {
continue
}
dict set newconfigs $k $v
}
dict for {k v} $configs {
set vv [dict get $newconfigs $k]
if {$v != $vv} {
fail "config $k mismatch, expecting $v but got $vv"
}
}
}
}
-71
View File
@@ -1,71 +0,0 @@
set testmodule [file normalize tests/modules/auth.so]
start_server {tags {"modules"}} {
r module load $testmodule
test {Modules can create a user that can be authenticated} {
# Make sure we start authenticated with default user
r auth default ""
assert_equal [r acl whoami] "default"
r auth.createmoduleuser
set id [r auth.authmoduleuser]
assert_equal [r client id] $id
# Verify returned id is the same as our current id and
# we are authenticated with the specified user
assert_equal [r acl whoami] "global"
}
test {De-authenticating clients is tracked and kills clients} {
assert_equal [r auth.changecount] 0
r auth.createmoduleuser
# Catch the I/O exception that was thrown when Redis
# disconnected with us.
catch { [r ping] } e
assert_match {*I/O*} $e
# Check that a user change was registered
assert_equal [r auth.changecount] 1
}
test {Modules cant authenticate with ACLs users that dont exist} {
catch { [r auth.authrealuser auth-module-test-fake] } e
assert_match {*Invalid user*} $e
}
test {Modules can authenticate with ACL users} {
assert_equal [r acl whoami] "default"
# Create user to auth into
r acl setuser auth-module-test on allkeys allcommands
set id [r auth.authrealuser auth-module-test]
# Verify returned id is the same as our current id and
# we are authenticated with the specified user
assert_equal [r client id] $id
assert_equal [r acl whoami] "auth-module-test"
}
test {Client callback is called on user switch} {
assert_equal [r auth.changecount] 0
# Auth again and validate change count
r auth.authrealuser auth-module-test
assert_equal [r auth.changecount] 1
# Re-auth with the default user
r auth default ""
assert_equal [r auth.changecount] 1
assert_equal [r acl whoami] "default"
# Re-auth with the default user again, to
# verify the callback isn't fired again
r auth default ""
assert_equal [r auth.changecount] 0
assert_equal [r acl whoami] "default"
}
}
-35
View File
@@ -45,24 +45,18 @@ start_server {tags {"modules"}} {
test {Module client blocked on keys (with metadata): Timeout} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
r fsl.push k 33
$rd fsl.bpopgt k 35 1
assert_equal {Request timedout} [$rd read]
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, case 1} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
r fsl.push k 33
$rd fsl.bpopgt k 33 0
r fsl.push k 34
assert_equal {34} [$rd read]
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, case 2} {
@@ -76,35 +70,6 @@ start_server {tags {"modules"}} {
assert_equal {36} [$rd read]
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT KILL} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
r client kill id $cid ;# try to smoke-out client-related memory leak
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT UNBLOCK TIMEOUT} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
r client unblock $cid timeout ;# try to smoke-out client-related memory leak
assert_equal {Request timedout} [$rd read]
}
test {Module client blocked on keys (with metadata): Blocked, CLIENT UNBLOCK ERROR} {
r del k
set rd [redis_deferring_client]
$rd client id
set cid [$rd read]
$rd fsl.bpopgt k 35 0
r client unblock $cid error ;# try to smoke-out client-related memory leak
assert_error "*unblocked*" {$rd read}
}
test {Module client blocked on keys does not wake up on wrong type} {
r del k
set rd [redis_deferring_client]
-44
View File
@@ -1,44 +0,0 @@
set testmodule [file normalize tests/modules/datatype.so]
start_server {tags {"modules"}} {
r module load $testmodule
test {DataType: Test module is sane, GET/SET work.} {
r datatype.set dtkey 100 stringval
assert {[r datatype.get dtkey] eq {100 stringval}}
}
test {DataType: RM_SaveDataTypeToString(), RM_LoadDataTypeFromString() work} {
r datatype.set dtkey -1111 MyString
set encoded [r datatype.dump dtkey]
r datatype.restore dtkeycopy $encoded
assert {[r datatype.get dtkeycopy] eq {-1111 MyString}}
}
test {DataType: Handle truncated RM_LoadDataTypeFromString()} {
r datatype.set dtkey -1111 MyString
set encoded [r datatype.dump dtkey]
set truncated [string range $encoded 0 end-1]
catch {r datatype.restore dtkeycopy $truncated} e
set e
} {*Invalid*}
test {DataType: ModuleTypeReplaceValue() happy path works} {
r datatype.set key-a 1 AAA
r datatype.set key-b 2 BBB
assert {[r datatype.swap key-a key-b] eq {OK}}
assert {[r datatype.get key-a] eq {2 BBB}}
assert {[r datatype.get key-b] eq {1 AAA}}
}
test {DataType: ModuleTypeReplaceValue() fails on non-module keys} {
r datatype.set key-a 1 AAA
r set key-b RedisString
catch {r datatype.swap key-a key-b} e
set e
} {*ERR*}
}
+6 -8
View File
@@ -1,7 +1,9 @@
set testmodule [file normalize tests/modules/hooks.so]
tags "modules" {
start_server [list overrides [list loadmodule "$testmodule" appendonly yes]] {
start_server {} {
r module load $testmodule
r config set appendonly yes
test {Test clients connection / disconnection hooks} {
for {set j 0} {$j < 2} {incr j} {
@@ -42,19 +44,15 @@ tags "modules" {
r set "bar$j" x
}
# set some configs that will cause many loading progress events during aof loading
r config set key-load-delay 500
r config set key-load-delay 1
r config set dynamic-hz no
r config set hz 500
r DEBUG LOADAOF
assert_equal [r hooks.event_last loading-aof-start] 0
assert_equal [r hooks.event_last loading-end] 0
assert {[r hooks.event_count loading-rdb-start] == 0}
assert_lessthan 2 [r hooks.event_count loading-progress-rdb] ;# comes from the preamble section
assert_lessthan 2 [r hooks.event_count loading-progress-aof]
if {$::verbose} {
puts "rdb progress events [r hooks.event_count loading-progress-rdb]"
puts "aof progress events [r hooks.event_count loading-progress-aof]"
}
assert {[r hooks.event_count loading-progress-rdb] >= 2} ;# comes from the preamble section
assert {[r hooks.event_count loading-progress-aof] >= 2}
}
# undo configs before next test
r config set dynamic-hz yes
-26
View File
@@ -10,32 +10,6 @@ proc field {info property} {
start_server {tags {"modules"}} {
r module load $testmodule log-key 0
test {module reading info} {
# check string, integer and float fields
assert_equal [r info.gets replication role] "master"
assert_equal [r info.getc replication role] "master"
assert_equal [r info.geti stats expired_keys] 0
assert_equal [r info.getd stats expired_stale_perc] 0
# check signed and unsigned
assert_equal [r info.geti infotest infotest_global] -2
assert_equal [r info.getu infotest infotest_uglobal] -2
# the above are always 0, try module info that is non-zero
assert_equal [r info.geti infotest_italian infotest_due] 2
set tre [r info.getd infotest_italian infotest_tre]
assert {$tre > 3.2 && $tre < 3.4 }
# search using the wrong section
catch { [r info.gets badname redis_version] } e
assert_match {*not found*} $e
# check that section filter works
assert { [string match "*usec_per_call*" [r info.gets all cmdstat_info.gets] ] }
catch { [r info.gets default cmdstat_info.gets] ] } e
assert_match {*not found*} $e
}
test {module info all} {
set info [r info all]
# info all does not contain modules
+3 -35
View File
@@ -16,11 +16,6 @@ start_server {tags {"modules"}} {
assert { [string match "*cmdstat_module*" $info] }
}
test {test long double conversions} {
set ld [r test.ld_conversion]
assert {[string match $ld "0.00000000000000001"]}
}
test {test module db commands} {
r set x foo
set key [r test.randomkey]
@@ -31,40 +26,13 @@ start_server {tags {"modules"}} {
}
test {test modle lru api} {
r config set maxmemory-policy allkeys-lru
r set x foo
set lru [r test.getlru x]
assert { $lru <= 1000 }
set was_set [r test.setlru x 100000]
assert { $was_set == 1 }
assert { $lru <= 1 }
r test.setlru x 100
set idle [r object idletime x]
assert { $idle >= 100 }
set lru [r test.getlru x]
assert { $lru >= 100000 }
r config set maxmemory-policy allkeys-lfu
set lru [r test.getlru x]
assert { $lru == -1 }
set was_set [r test.setlru x 100000]
assert { $was_set == 0 }
assert { $lru >= 100 }
}
r config set maxmemory-policy allkeys-lru
test {test modle lfu api} {
r config set maxmemory-policy allkeys-lfu
r set x foo
set lfu [r test.getlfu x]
assert { $lfu >= 1 }
set was_set [r test.setlfu x 100]
assert { $was_set == 1 }
set freq [r object freq x]
assert { $freq <= 100 }
set lfu [r test.getlfu x]
assert { $lfu <= 100 }
r config set maxmemory-policy allkeys-lru
set lfu [r test.getlfu x]
assert { $lfu == -1 }
set was_set [r test.setlfu x 100]
assert { $was_set == 0 }
}
}
-47
View File
@@ -1,47 +0,0 @@
set testmodule [file normalize tests/modules/scan.so]
start_server {tags {"modules"}} {
r module load $testmodule
test {Module scan keyspace} {
# the module create a scan command with filtering which also return values
r set x 1
r set y 2
r set z 3
r hset h f v
lsort [r scan.scan_strings]
} {{x 1} {y 2} {z 3}}
test {Module scan hash ziplist} {
r hmset hh f1 v1 f2 v2
lsort [r scan.scan_key hh]
} {{f1 v1} {f2 v2}}
test {Module scan hash dict} {
r config set hash-max-ziplist-entries 2
r hmset hh f3 v3
lsort [r scan.scan_key hh]
} {{f1 v1} {f2 v2} {f3 v3}}
test {Module scan zset ziplist} {
r zadd zz 1 f1 2 f2
lsort [r scan.scan_key zz]
} {{f1 1} {f2 2}}
test {Module scan zset dict} {
r config set zset-max-ziplist-entries 2
r zadd zz 3 f3
lsort [r scan.scan_key zz]
} {{f1 1} {f2 2} {f3 3}}
test {Module scan set intset} {
r sadd ss 1 2
lsort [r scan.scan_key ss]
} {{1 {}} {2 {}}}
test {Module scan set dict} {
r config set set-max-intset-entries 2
r sadd ss 3
lsort [r scan.scan_key ss]
} {{1 {}} {2 {}} {3 {}}}
}
+1 -6
View File
@@ -536,7 +536,7 @@ foreach cmdrepl {0 1} {
start_server {tags {"scripting repl"}} {
start_server {} {
if {$cmdrepl == 1} {
set rt "(commands replication)"
set rt "(commmands replication)"
} else {
set rt "(scripts replication)"
r debug lua-always-replicate-commands 1
@@ -741,8 +741,3 @@ start_server {tags {"scripting repl"}} {
}
}
start_server {tags {"scripting"}} {
r script debug sync
r eval {return 'hello'} 0
r eval {return 'hello'} 0
}
-66
View File
@@ -1,66 +0,0 @@
start_server {tags {"tracking"}} {
# Create a deferred client we'll use to redirect invalidation
# messages to.
set rd1 [redis_deferring_client]
$rd1 client id
set redir [$rd1 read]
$rd1 subscribe __redis__:invalidate
$rd1 read ; # Consume the SUBSCRIBE reply.
test {Clients are able to enable tracking and redirect it} {
r CLIENT TRACKING on REDIRECT $redir
} {*OK}
test {The other connection is able to get invalidations} {
r SET a 1
r GET a
r INCR a
r INCR b ; # This key should not be notified, since it wasn't fetched.
set keys [lindex [$rd1 read] 2]
assert {[llength $keys] == 1}
assert {[lindex $keys 0] eq {a}}
}
test {The client is now able to disable tracking} {
# Make sure to add a few more keys in the tracking list
# so that we can check for leaks, as a side effect.
r MGET a b c d e f g
r CLIENT TRACKING off
}
test {Clients can enable the BCAST mode with the empty prefix} {
r CLIENT TRACKING on BCAST REDIRECT $redir
} {*OK*}
test {The connection gets invalidation messages about all the keys} {
r MSET a 1 b 2 c 3
set keys [lsort [lindex [$rd1 read] 2]]
assert {$keys eq {a b c}}
}
test {Clients can enable the BCAST mode with prefixes} {
r CLIENT TRACKING off
r CLIENT TRACKING on BCAST REDIRECT $redir PREFIX a: PREFIX b:
r MULTI
r INCR a:1
r INCR a:2
r INCR b:1
r INCR b:2
r EXEC
# Because of the internals, we know we are going to receive
# two separated notifications for the two different prefixes.
set keys1 [lsort [lindex [$rd1 read] 2]]
set keys2 [lsort [lindex [$rd1 read] 2]]
set keys [lsort [list {*}$keys1 {*}$keys2]]
assert {$keys eq {a:1 a:2 b:1 b:2}}
}
test {Adding prefixes to BCAST mode works} {
r CLIENT TRACKING on BCAST REDIRECT $redir PREFIX c:
r INCR c:1234
set keys [lsort [lindex [$rd1 read] 2]]
assert {$keys eq {c:1234}}
}
$rd1 close
}
-7
View File
@@ -390,13 +390,6 @@ start_server {tags {"hash"}} {
lappend rv [string match "ERR*not*float*" $bigerr]
} {1 1}
test {HINCRBYFLOAT fails against hash value that contains a null-terminator in the middle} {
r hset h f "1\x002"
catch {r hincrbyfloat h f 1} err
set rv {}
lappend rv [string match "ERR*not*float*" $err]
} {1}
test {HSTRLEN against the small hash} {
set err {}
foreach k [array names smallhash *] {
-14
View File
@@ -147,20 +147,6 @@ start_server {
assert {[lindex $res 0 1 1] == {2-0 {field1 B}}}
}
test {Blocking XREADGROUP will not reply with an empty array} {
r del mystream
r XGROUP CREATE mystream mygroup $ MKSTREAM
r XADD mystream 666 f v
set res [r XREADGROUP GROUP mygroup Alice BLOCK 10 STREAMS mystream ">"]
assert {[lindex $res 0 1 0] == {666-0 {f v}}}
r XADD mystream 667 f2 v2
r XDEL mystream 667
set rd [redis_deferring_client]
$rd XREADGROUP GROUP mygroup Alice BLOCK 10 STREAMS mystream ">"
after 20
assert {[$rd read] == {}} ;# before the fix, client didn't even block, but was served synchronously with {mystream {}}
}
test {XCLAIM can claim PEL items from another consumer} {
# Add 3 items into the stream, and create a consumer group
r del mystream
-44
View File
@@ -123,12 +123,6 @@ start_server {
assert {[r xlen mystream] == $j}
}
test {XADD with ID 0-0} {
r DEL otherstream
catch {r XADD otherstream 0-0 k v} err
assert {[r EXISTS otherstream] == 0}
}
test {XRANGE COUNT works as expected} {
assert {[llength [r xrange mystream - + COUNT 10]] == 10}
}
@@ -191,17 +185,6 @@ start_server {
assert {[lindex $res 0 1 0 1] eq {old abcd1234}}
}
test {Blocking XREAD will not reply with an empty array} {
r del s1
r XADD s1 666 f v
r XADD s1 667 f2 v2
r XDEL s1 667
set rd [redis_deferring_client]
$rd XREAD BLOCK 10 STREAMS s1 666
after 20
assert {[$rd read] == {}} ;# before the fix, client didn't even block, but was served synchronously with {s1 {}}
}
test "XREAD: XADD + DEL should not awake client" {
set rd [redis_deferring_client]
r del s1
@@ -339,33 +322,6 @@ start_server {
assert_equal [r xrevrange teststream2 1234567891245 -] {{1234567891240-0 {key1 value2}} {1234567891230-0 {key1 value1}}}
}
test {XREAD streamID edge (no-blocking)} {
r del x
r XADD x 1-1 f v
r XADD x 1-18446744073709551615 f v
r XADD x 2-1 f v
set res [r XREAD BLOCK 0 STREAMS x 1-18446744073709551615]
assert {[lindex $res 0 1 0] == {2-1 {f v}}}
}
test {XREAD streamID edge (blocking)} {
r del x
set rd [redis_deferring_client]
$rd XREAD BLOCK 0 STREAMS x 1-18446744073709551615
r XADD x 1-1 f v
r XADD x 1-18446744073709551615 f v
r XADD x 2-1 f v
set res [$rd read]
assert {[lindex $res 0 1 0] == {2-1 {f v}}}
}
test {XADD streamID edge} {
r del x
r XADD x 2577343934890-18446744073709551615 f v ;# we need the timestamp to be in the future
r XADD x * f2 v2
assert_equal [r XRANGE x - +] {{2577343934890-18446744073709551615 {f v}} {2577343934891-0 {f2 v2}}}
}
}
start_server {tags {"stream"} overrides {appendonly yes}} {
+1 -9
View File
@@ -7,7 +7,6 @@ TIMEOUT=2000
NODES=6
REPLICAS=1
PROTECTED_MODE=yes
ADDITIONAL_OPTIONS=""
# You may want to put the above config parameters into config.sh in order to
# override the defaults without modifying this script.
@@ -25,7 +24,7 @@ then
while [ $((PORT < ENDPORT)) != "0" ]; do
PORT=$((PORT+1))
echo "Starting $PORT"
../../src/redis-server --port $PORT --protected-mode $PROTECTED_MODE --cluster-enabled yes --cluster-config-file nodes-${PORT}.conf --cluster-node-timeout $TIMEOUT --appendonly yes --appendfilename appendonly-${PORT}.aof --dbfilename dump-${PORT}.rdb --logfile ${PORT}.log --daemonize yes ${ADDITIONAL_OPTIONS}
../../src/redis-server --port $PORT --protected-mode $PROTECTED_MODE --cluster-enabled yes --cluster-config-file nodes-${PORT}.conf --cluster-node-timeout $TIMEOUT --appendonly yes --appendfilename appendonly-${PORT}.aof --dbfilename dump-${PORT}.rdb --logfile ${PORT}.log --daemonize yes
done
exit 0
fi
@@ -71,12 +70,6 @@ then
exit 0
fi
if [ "$1" == "tailall" ]
then
tail -f *.log
exit 0
fi
if [ "$1" == "call" ]
then
while [ $((PORT < ENDPORT)) != "0" ]; do
@@ -107,6 +100,5 @@ echo "create -- Create a cluster using redis-cli --cluster create."
echo "stop -- Stop Redis Cluster instances."
echo "watch -- Show CLUSTER NODES output (first 30 lines) of first node."
echo "tail <id> -- Run tail -f of instance at base port + ID."
echo "tailall -- Run tail -f for all the log files at once."
echo "clean -- Remove all instances data, logs, configs."
echo "clean-logs -- Remove just instances logs."
-10
View File
@@ -73,16 +73,6 @@ if [ "$(id -u)" -ne 0 ] ; then
exit 1
fi
#bail if this system is managed by systemd
_pid_1_exe="$(readlink -f /proc/1/exe)"
if [ "${_pid_1_exe##*/}" = systemd ]
then
echo "This systems seems to use systemd."
echo "Please take a look at the provided example service unit files in this directory, and adapt and install them. Sorry!"
exit 1
fi
unset _pid_1_exe
if ! echo $REDIS_PORT | egrep -q '^[0-9]+$' ; then
_MANUAL_EXECUTION=true
#Read the redis port
@@ -1,37 +0,0 @@
# example systemd template service unit file for multiple redis-servers
#
# You can use this file as a blueprint for your actual template service unit
# file, if you intend to run multiple independent redis-server instances in
# parallel using systemd's "template unit files" feature. If you do, you will
# want to choose a better basename for your service unit by renaming this file
# when copying it.
#
# Please take a look at the provided "systemd-redis_server.service" example
# service unit file, too, if you choose to use this approach at managing
# multiple redis-server instances via systemd.
[Unit]
Description=Redis data structure server - instance %i
Documentation=https://redis.io/documentation
# This template unit assumes your redis-server configuration file(s)
# to live at /etc/redis/redis_server_<INSTANCE_NAME>.conf
AssertPathExists=/etc/redis/redis_server_%i.conf
#Before=your_application.service another_example_application.service
#AssertPathExists=/var/lib/redis
[Service]
ExecStart=/usr/local/bin/redis-server /etc/redis/redis_server_%i.conf
LimitNOFILE=10032
NoNewPrivileges=yes
#OOMScoreAdjust=-900
#PrivateTmp=yes
Type=notify
TimeoutStartSec=infinity
TimeoutStopSec=infinity
UMask=0077
#User=redis
#Group=redis
#WorkingDirectory=/var/lib/redis
[Install]
WantedBy=multi-user.target
-41
View File
@@ -1,41 +0,0 @@
# example systemd service unit file for redis-server
#
# In order to use this as a template for providing a redis service in your
# environment, _at the very least_ make sure to adapt the redis configuration
# file you intend to use as needed (make sure to set "supervised systemd"), and
# to set sane TimeoutStartSec and TimeoutStopSec property values in the unit's
# "[Service]" section to fit your needs.
#
# Some properties, such as User= and Group=, are highly desirable for virtually
# all deployments of redis, but cannot be provided in a manner that fits all
# expectable environments. Some of these properties have been commented out in
# this example service unit file, but you are highly encouraged to set them to
# fit your needs.
#
# Please refer to systemd.unit(5), systemd.service(5), and systemd.exec(5) for
# more information.
[Unit]
Description=Redis data structure server
Documentation=https://redis.io/documentation
#Before=your_application.service another_example_application.service
#AssertPathExists=/var/lib/redis
[Service]
ExecStart=/usr/local/bin/redis-server --supervised systemd --daemonize no
## Alternatively, have redis-server load a configuration file:
#ExecStart=/usr/local/bin/redis-server /path/to/your/redis.conf
LimitNOFILE=10032
NoNewPrivileges=yes
#OOMScoreAdjust=-900
#PrivateTmp=yes
Type=notify
TimeoutStartSec=infinity
TimeoutStopSec=infinity
UMask=0077
#User=redis
#Group=redis
#WorkingDirectory=/var/lib/redis
[Install]
WantedBy=multi-user.target