Compare commits

..
422 Commits
Author SHA1 Message Date
antirez ff6db5f176 HyperLogLog: fix the fix of a corruption bug. 2019-07-31 10:37:01 +02:00
John Sullyandantirez dce11e658c Fix HLL corruption bug 2019-07-30 10:28:14 +02:00
antirez 18572a921a Redis 4.0.14. 2019-03-18 17:23:22 +01:00
antirez 3c62ad193d HyperLogLog: handle wrong offset in the base case. 2019-03-18 11:31:10 +01:00
John Sullyandantirez cc2b3a68e6 Fix hyperloglog corruption 2019-03-18 11:29:04 +01:00
antirez e10c79d3ff redis-check-aof: fix potential overflow.
Bug signaled by @vattezhang in PR #5940 but fixed differently.
2019-03-18 11:28:59 +01:00
Guy Benoishandantirez 6fa16f585c Fix mismatching keyspace notification classes 2019-03-14 12:30:41 +01:00
Guy Benoishandantirez 643110525e Fix zlexrangespec mem-leak in genericZrangebylexCommand 2019-03-14 11:45:28 +01:00
antirez b0e863195e Make comment in #5911 stay inside 80 cols. 2019-03-10 09:52:19 +01:00
John Sullyandantirez 39ccce033a Replicas aren't allowed to run the replicaof command 2019-03-10 09:52:19 +01:00
antirez 825843db9e Redis 4.0.13. 2019-02-20 17:26:31 +01:00
Salvatore SanfilippoandGitHub 77b4a0eb1a Merge pull request #5822 from soloestoy/rewrite-brpoplpush-4.0
rewrite BRPOPLPUSH as RPOPLPUSH to propagate
2019-01-31 08:20:54 +01:00
zhaozhao.zz 60b2537fe1 rewrite BRPOPLPUSH as RPOPLPUSH to propagate 2019-01-31 11:53:36 +08:00
Chris Lambandantirez afc4b36c10 Don't treat unsupported protocols as fatal errors
If we encounter an unsupported protocol in the "bind" list, don't
ipso-facto consider it a fatal error. We continue to abort startup if
there are no listening sockets at all.

This ensures that the lack of IPv6 support does not prevent Redis from
starting on Debian where we try to bind to the ::1 interface by default
(via "bind 127.0.0.1 ::1"). A machine with IPv6 disabled (such as some
container systems) would simply fail to start Redis after the initiall
call to apt(8).

This is similar to the case for where "bind" is not specified:

  https://github.com/antirez/redis/issues/3894

... and was based on the corresponding PR:

  https://github.com/antirez/redis/pull/4108

... but also adds EADDRNOTAVAIL to the list of errors to catch which I
believe is missing from there.

This issue was raised in Debian as both <https://bugs.debian.org/900284>
& <https://bugs.debian.org/914354>.
2018-12-21 12:40:28 +01:00
antirez cbcc76ac5b Redis 4.0.12 2018-12-11 18:09:17 +01:00
antirez b6cd3b3c57 asyncCloseClientOnOutputBufferLimitReached(): don't free fake clients.
Fake clients are used in special situations and are not linked to the
normal clients list, freeing them will always result in Redis crashing
in one way or the other.

It's not common to send replies to fake clients, but we have one usage
in the modules API. When a client is blocked, we associate to the
blocked client object (that is safe to manipulate in a thread), a fake
client that accumulates replies. So because of this bug there was
the problem described in issue #5443.

The fix was verified to work with the provided example module. To write
a regression is very hard and unlikely to be triggered in the future.
2018-12-11 18:05:47 +01:00
Guy Benoishandantirez e16402b0c8 Don't call sdscmp() with shared.maxstring or shared.minstring 2018-12-11 18:04:48 +01:00
antirez 49d9f4116b Fix stringmatchlen() read past buffer bug.
See #5632.
2018-12-11 18:02:30 +01:00
Salvatore SanfilippoandGitHub 97192e2d88 Merge pull request #5569 from maximebedard/backport-4497
Backport #4497 to redis 4
2018-11-22 11:12:36 +01:00
antirezandMaxime Bedard 1908aba73d add linkClient(): adds the client and caches the list node.
We have this operation in two places: when caching the master and
when linking a new client after the client creation. By having an API
for this we avoid incurring in errors when modifying one of the two
places forgetting the other. The function is also a good place where to
document why we cache the linked list node.

Related to #4497 and #4210.
2018-11-13 19:57:04 -05:00
zhaozhao.zzandMaxime Bedard 239b08574c networking: optimize unlinkClient() in freeClient() 2018-11-13 19:52:50 -05:00
antirez 54b17f985d When replica kills a pending RDB save during SYNC, log it.
This logs what happens in the context of the fix in PR #5367.
2018-10-31 11:49:54 +01:00
Andrey Bugaevskiyandantirez b31c08db8d Move child termination to readSyncBulkPayload 2018-10-31 11:49:49 +01:00
Andrey Bugaevskiyandantirez 21971ac83c Prevent RDB autosave from overwriting full resync results
During the full database resync we may still have unsaved changes
on the receiving side. This causes a race condition between
synced data rename/load and the rename of rdbSave tempfile.
2018-10-31 11:49:37 +01:00
antirez 3a91fcbcf3 aof.c: improve indentation and change warning message.
Related to #5201.

I removed the !!! Warning part since compared to the other errors, a
missing EXEC is in theory a normal happening in the AOF file, at least
in theory: may happen in a differnet number of situations, and it's
probably better to don't give the user the feeling that something really
bad happened.
2018-10-10 11:14:23 +02:00
zhaozhao.zzandantirez e6f287d59f AOF: discard if we lost EXEC when loading aof 2018-10-10 11:14:19 +02:00
antirez e0d4c66aff Fix AOF comment to report the current behavior.
Realted to #5201.
2018-10-10 11:14:15 +02:00
antirez c7613cf34e Really bump version number to 4.0.11. 2018-08-04 00:44:56 +02:00
antirez f2e9d94e0f Redis 4.0.11. 2018-08-03 17:14:20 +02:00
antirez 677f7585e0 Set repl_down_since to zero on state change.
PR #5081 fixes an "interesting" bug about Redis Cluster failover but in
general about the updating of repl_down_since, that is used in order to
count the time a slave was left disconnected from its master.

While the fix provided resolves the specific issue, in general the
validity of repl_down_since is limited to states that are different
than the state CONNECTED, and the disconnected time is set when the
state is DISCONNECTED. However from CONNECTED to other states, the state
machine must always go to DISCONNECTED first. So it makes sense to set
the field to zero (since it is meaningless in that context) when the
state is set to CONNECTED.
2018-08-03 17:07:28 +02:00
WuYunlongandantirez 8c6223f91d fix server.repl_down_since resetting, so that slaves could failover
automatically as expected.
2018-08-03 17:07:23 +02:00
Oran Agraandantirez 9535c21515 fix rare replication stream corruption with disk-based replication
The slave sends \n keepalive messages to the master while parsing the rdb,
and later sends REPLCONF ACK once a second. rarely, the master recives both
a linefeed char and a REPLCONF in the same read, \n*3\r\n$8\r\nREPLCONF\r\n...
and it tries to trim two chars (\r\n) from the query buffer,
trimming the '*' from *3\r\n$8\r\nREPLCONF\r\n...

then the master tries to process a command starting with '3' and replies to
the slave a bunch of -ERR and one +OK.
although the slave silently ignores these (prints a log message), this corrupts
the replication offset at the slave since the slave increases the replication
offset, and the master did not.

other than the fix in processInlineBuffer, i did several other improvments
while hunting this very rare bug.

- when redis replies with "unknown command" it includes a portion of the
  arguments, not just the command name. so it would be easier to understand
  what was recived, in my case, on the slave side,  it was -ERR, but
  the "arguments" were the interesting part (containing info on the error).
- about a year ago i added code in addReplyErrorLength to print the error to
  the log in case of a reply to master (since this string isn't actually
  trasmitted to the master), now changed that block to print a similar log
  message to indicate an error being sent from the master to the slave.
  note that the slave is marked as CLIENT_SLAVE only after PSYNC was received,
  so this will not cause any harm for REPLCONF, and will only indicate problems
  that are gonna corrupt the replication stream anyway.
- two places were c->reply was emptied, and i wanted to reset sentlen
  this is a precaution (i did not actually see such a problem), since a
  non-zero sentlen will cause corruption to be transmitted on the socket.
2018-07-23 18:16:22 +02:00
zhaozhao.zzandantirez 5f1fcc599a fix exists command on slave 2018-06-29 13:29:41 +02:00
antirez ab145a9f15 Fix infinite loop in dbRandomKey().
Thanks to @kevinmcgehee for signaling the issue and reasoning about the
consequences and potential fixes.

Issue #5015.
2018-06-29 13:29:33 +02:00
antirez 2fa43ece12 Sentinel: add an option to deny online script reconfiguration.
The ability of "SENTINEL SET" to change the reconfiguration script at
runtime is a problem even in the security model of Redis: any client
inside the network may set any executable to be ran once a failover is
triggered.

This option adds protection for this problem: by default the two
SENTINEL SET subcommands modifying scripts paths are denied. However the
user is still able to rever that using the Sentinel configuration file
in order to allow such a feature.
2018-06-29 13:27:49 +02:00
antirez 556b2d2bee Redis 4.0.10. 2018-06-13 13:02:07 +02:00
antirez 9fdcc15962 Security: fix redis-cli buffer overflow.
Thanks to Fakhri Zulkifli for reporting it.

The fix switched to dynamic allocation, copying the final prompt in the
static buffer only at the end.
2018-06-13 12:40:47 +02:00
antirez cf7600714f Security: fix Lua struct package offset handling.
After the first fix to the struct package I found another similar
problem, which is fixed by this patch. It could be reproduced easily by
running the following script:

    return struct.unpack('f', "xxxxxxxxxxxxx",-3)

The above will access bytes before the 'data' pointer.
2018-06-13 12:40:46 +02:00
antirez a57595cade Security: more cmsgpack fixes by @soloestoy.
@soloestoy sent me this additional fixes, after searching for similar
problems to the one reported in mp_pack(). I'm committing the changes
because it was not possible during to make a public PR to protect Redis
users and give Redis providers some time to patch their systems.
2018-06-13 12:40:46 +02:00
antirez 8783fb9495 Security: update Lua struct package for security.
During an auditing Apple found that the "struct" Lua package
we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
a security problem. A bound-checking statement fails because of integer
overflow. The bug exists since we initially integrated this package with
Lua, when scripting was introduced, so every version of Redis with
EVAL/EVALSHA capabilities exposed is affected.

Instead of just fixing the bug, the library was updated to the latest
version shipped by the author.
2018-06-13 12:40:46 +02:00
antirez 8cb9344b65 Security: fix Lua cmsgpack library stack overflow.
During an auditing effort, the Apple Vulnerability Research team discovered
a critical Redis security issue affecting the Lua scripting part of Redis.

-- Description of the problem

Several years ago I merged a pull request including many small changes at
the Lua MsgPack library (that originally I authored myself). The Pull
Request entered Redis in commit 90b6337c1, in 2014.
Unfortunately one of the changes included a variadic Lua function that
lacked the check for the available Lua C stack. As a result, calling the
"pack" MsgPack library function with a large number of arguments, results
into pushing into the Lua C stack a number of new values proportional to
the number of arguments the function was called with. The pushed values,
moreover, are controlled by untrusted user input.

This in turn causes stack smashing which we believe to be exploitable,
while not very deterministic, but it is likely that an exploit could be
created targeting specific versions of Redis executables. However at its
minimum the issue results in a DoS, crashing the Redis server.

-- Versions affected

Versions greater or equal to Redis 2.8.18 are affected.

-- Reproducing

Reproduce with this (based on the original reproduction script by
Apple security team):

https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a

-- Verification of the fix

The fix was tested in the following way:

1) I checked that the problem is no longer observable running the trigger.
2) The Lua code was analyzed to understand the stack semantics, and that
actually enough stack is allocated in all the cases of mp_pack() calls.
3) The mp_pack() function was modified in order to show exactly what items
in the stack were being set, to make sure that there is no silent overflow
even after the fix.

-- Credits

Thank you to the Apple team and to the other persons that helped me
checking the patch and coordinating this communication.
2018-06-13 12:40:46 +02:00
赵磊andantirez 59080f6023 Fix dictScan(): It can't scan all buckets when dict is shrinking. 2018-06-01 16:54:30 +02:00
dejun.xdjandantirez ac2a824aa9 Fix redis-cli memory leak when sending set preference command. 2018-05-29 12:46:27 +02:00
dejun.xdjandantirez c7197ff50f Check if the repeat value is positive in while loop of cliSendCommand().
In case that the incoming repeat parameter is negative and causes a
deadless loop.
2018-05-29 12:46:27 +02:00
dejun.xdjandantirez 3f77777ffd Change the type of repeat argument to long for function cliSendCommand.
To be in consistent with the original definition.
2018-05-29 12:46:27 +02:00
dejun.xdjandantirez 7a565d7257 Fix negtive repeat command value issue.
If command like "-1 set a b" is sent with redis-cli, it will cause a deadless loop. So some repeat value checking logic is added to avoid this.
2018-05-29 12:46:27 +02:00
dejun.xdjandantirez 64bf60fb52 Detect and stop saving history for auth command with repeat option.
Put the repeat option checking code a little forward to avoid repeat logic.
2018-05-29 12:46:27 +02:00
dejun.xdjandantirez 5bed12aa03 Change the warning message a little bit to avoid trademark issuses. 2018-05-29 12:45:43 +02:00
dejun.xdjandantirez d71c49610b Stop saving auth command in redis-cli history. 2018-05-29 12:45:20 +02:00
dejun.xdjandantirez fca99e413a Add warning message when using password on command line 2018-05-29 12:45:11 +02:00
antirez 01407a3a42 Don't expire keys while loading RDB from AOF preamble.
The AOF tail of a combined RDB+AOF is based on the premise of applying
the AOF commands to the exact state that there was in the server while
the RDB was persisted. By expiring keys while loading the RDB file, we
change the state, so applying the AOF tail later may change the state.

Test case:

* Time1: SET a 10
* Time2: EXPIREAT a $time5
* Time3: INCR a
* Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
* Time5: Restart redis from the RDB+AOF: consistency violation.

Thanks to @soloestoy for providing the patch.
Thanks to @trevor211 for the original issue report and the initial fix.

Check issue #4950 for more info.
2018-05-29 12:44:09 +02:00
WuYunlongandantirez fb5408cf68 Fix rdb save by allowing dumping of expire keys, so that when
we add a new slave, and do a failover, eighter by manual or
not, other local slaves will delete the expired keys properly.
2018-05-29 12:44:05 +02:00
antirez 0b8b6df472 Backport hiredis issue 525 fix to compile on FreeBSD.
Close #4947.
2018-05-29 12:44:01 +02:00
antirez e98627c5b5 Add INIT INFO to the provided init script. 2018-05-23 17:19:20 +02:00
antirez 17f5de896a Fix ae.c when a timer finalizerProc adds an event.
While this feature is not used by Redis, ae.c implements the ability for
a timer to call a finalizer callback when an timer event is deleted.
This feature was bugged since the start, and because it was never used
we never noticed a problem. However Anthony LaTorre was using the same
library in order to implement a different system: he found a bug that he
describes as follows, and which he fixed with the patch in this commit,
sent me by private email:

    --- Anthony email ---

've found one bug in the current implementation of the timed events.
It's possible to lose track of a timed event if an event is added in
the finalizerProc of another event.

For example, suppose you start off with three timed events 1, 2, and
3. Then the linked list looks like:

3 -> 2 -> 1

Then, you run processTimeEvents and events 2 and 3 finish, so now the
list looks like:

-1 -> -1 -> 2

Now, on the next iteration of processTimeEvents it starts by deleting
the first event, and suppose this finalizerProc creates a new event,
so that the list looks like this:

4 -> -1 -> 2

On the next iteration of the while loop, when it gets to the second
event, the variable prev is still set to NULL, so that the head of the
event loop after the next event will be set to 2, i.e. after deleting
the next event the event loop will look like:

2

and the event with id 4 will be lost.

I've attached an example program to illustrate the issue. If you run
it you will see that it prints:

```
foo id = 0
spam!
```

But if you uncomment line 29 and run it again it won't print "spam!".

    --- End of email ---

Test.c source code is as follows:

    #include "ae.h"
    #include <stdio.h>

    aeEventLoop *el;

    int foo(struct aeEventLoop *el, long long id, void *data)
    {
	printf("foo id = %lld\n", id);

	return AE_NOMORE;
    }

    int spam(struct aeEventLoop *el, long long id, void *data)
    {
	printf("spam!\n");

	return AE_NOMORE;
    }

    void bar(struct aeEventLoop *el, void *data)
    {
	aeCreateTimeEvent(el, 0, spam, NULL, NULL);
    }

    int main(int argc, char **argv)
    {
	el = aeCreateEventLoop(100);

	//aeCreateTimeEvent(el, 0, foo, NULL, NULL);
	aeCreateTimeEvent(el, 0, foo, NULL, bar);

	aeMain(el);

	return 0;
    }

Anthony fixed the problem by using a linked list for the list of timers, and
sent me back this patch after he tested the code in production for some time.
The code looks sane to me, so committing it to Redis.
2018-05-23 17:19:11 +02:00
antirez 266e6423bf Sentinel: fix delay in detecting ODOWN.
See issue #2819 for details. The gist is that when we want to send INFO
because we are over the time, we used to send only INFO commands, no
longer sending PING commands. However if a master fails exactly when we
are about to send an INFO command, the PING times will result zero
because the PONG reply was already received, and we'll fail to send more
PINGs, since we try only to send INFO commands: the failure detector
will delay until the connection is closed and re-opened for "long
timeout".

This commit changes the logic so that we can send the three kind of
messages regardless of the fact we sent another one already in the same
code path. It could happen that we go over the message limit for the
link by a few messages, but this is not significant. However now we'll
not introduce delays in sending commands just because there was
something else to send at the same time.
2018-05-23 17:18:23 +02:00
zhaozhao.zzandantirez eafaf17269 AOF & RDB: be compatible with rdbchecksum no 2018-05-23 17:18:18 +02:00
huijing.whjandantirez 4630da375c fix int overflow problem in freeMemoryIfNeeded 2018-05-08 17:28:21 +02:00
antirez 3150c67244 Redis 4.0.9 2018-03-26 18:04:15 +02:00
zhaozhao.zzandantirez 5b722bd7bd fix missed call on freeaddrinfo 2018-03-26 13:04:11 +02:00
zhaozhao.zzandantirez 2551b0f696 anet: avoid double close 2018-03-26 13:04:06 +02:00
antirez 8d92885bac Cluster: add test for the nofailover flag. 2018-03-14 16:31:46 +01:00
antirez 70597a3011 Cluster: ability to prevent slaves from failing over their masters.
This commit, in some parts derived from PR #3041 which is no longer
possible to merge (because the user deleted the original branch),
implements the ability of slaves to have a special configuration
preventing that they try to start a failover when the master is failing.

There are multiple reasons for wanting this, and the feautre was
requested in issue #3021 time ago.

The differences between this patch and the original PR are the
following:

1. The flag is saved/loaded on the nodes configuration.
2. The 'myself' node is now flag-aware, the flag is updated as needed
   when the configuration is changed via CONFIG SET.
3. The flag name uses NOFAILOVER instead of NO_FAILOVER to be consistent
   with existing NOADDR.
4. The redis.conf documentation was rewritten.

Thanks to @deep011 for the original patch.
2018-03-14 16:31:46 +01:00
antirez 16cad10a0c redis-cli: fix missed unit in array. Change define name. 2018-03-02 12:37:22 +01:00
charsyamandantirez 640fa434f5 fix-out-of-index-range-for-redis-cli-findbigkey 2018-03-02 12:37:11 +01:00
antirez 83390f55e5 expireIfNeeded() needed a top comment documenting the behavior. 2018-02-28 18:09:43 +01:00
antirez 888039ca82 expireIfNeeded() comment: claim -> pretend. 2018-02-28 18:09:40 +01:00
antirez e09c8c102a Actually use ae_flags to add AE_BARRIER if needed.
Many thanks to @Plasma that spotted this problem reviewing the code.
2018-02-28 18:05:51 +01:00
charsyamandantirez fb7560bcbb refactoring-make-condition-clear-for-rdb 2018-02-27 19:17:25 +01:00
antirez 1e2f0d6940 ae.c: insetad of not firing, on AE_BARRIER invert the sequence.
AE_BARRIER was implemented like:

    - Fire the readable event.
    - Do not fire the writabel event if the readable fired.

However this may lead to the writable event to never be called if the
readable event is always fired. There is an alterantive, we can just
invert the sequence of the calls in case AE_BARRIER is set. This commit
does that.
2018-02-27 16:19:38 +01:00
antirez b2e4aad9e2 AOF: fix a bug that may prevent proper fsyncing when fsync=always.
In case the write handler is already installed, it could happen that we
serve the reply of a query in the same event loop cycle we received it,
preventing beforeSleep() from guaranteeing that we do the AOF fsync
before sending the reply to the client.

The AE_BARRIER mechanism, introduced in a previous commit, prevents this
problem. This commit makes actual use of this new feature to fix the
bug.
2018-02-27 16:19:33 +01:00
antirez 93bad8ae88 Cluster: improve crash-recovery safety after failover auth vote.
Add AE_BARRIER to the writable event loop so that slaves requesting
votes can't be served before we re-enter the event loop in the next
iteration, so clusterBeforeSleep() will fsync to disk in time.
Also add the call to explicitly fsync, given that we modified the last
vote epoch variable.
2018-02-27 16:19:26 +01:00
antirez e32752e8d0 ae.c: introduce the concept of read->write barrier.
AOF fsync=always, and certain Redis Cluster bus operations, require to
fsync data on disk before replying with an acknowledge.
In such case, in order to implement Group Commits, we want to be sure
that queries that are read in a given cycle of the event loop, are never
served to clients in the same event loop iteration. This way, by using
the event loop "before sleep" callback, we can fsync the information
just one time before returning into the event loop for the next cycle.
This is much more efficient compared to calling fsync() multiple times.

Unfortunately because of a bug, this was not always guaranteed: the
actual way the events are installed was the sole thing that could
control. Normally this problem is hard to trigger when AOF is enabled
with fsync=always, because we try to flush the output buffers to the
socekt directly in the beforeSleep() function of Redis. However if the
output buffers are full, we actually install a write event, and in such
a case, this bug could happen.

This change to ae.c modifies the event loop implementation to make this
concept explicit. Write events that are registered with:

    AE_WRITABLE|AE_BARRIER

Are guaranteed to never fire after the readable event was fired for the
same file descriptor. In this way we are sure that data is persisted to
disk before the client performing the operation receives an
acknowledged.

However note that this semantics does not provide all the guarantees
that one may believe are automatically provided. Take the example of the
blocking list operations in Redis.

With AOF and fsync=always we could have:

    Client A doing: BLPOP myqueue 0
    Client B doing: RPUSH myqueue a b c

In this scenario, Client A will get the "a" elements immediately after
the Client B RPUSH will be executed, even before the operation is persisted.
However when Client B will get the acknowledge, it can be sure that
"b,c" are already safe on disk inside the list.

What to note here is that it cannot be assumed that Client A receiving
the element is a guaranteed that the operation succeeded from the point
of view of Client B.

This is due to the fact that the barrier exists within the same socket,
and not between different sockets. However in the case above, the
element "a" was not going to be persisted regardless, so it is a pretty
synthetic argument.
2018-02-27 16:19:20 +01:00
antirez 262f403944 Fix ziplist prevlen encoding description. See #4705. 2018-02-27 16:19:17 +01:00
antirez 83923afa8c Track number of logically expired keys still in memory.
This commit adds two new fields in the INFO output, stats section:

expired_stale_perc:0.34
expired_time_cap_reached_count:58

The first field is an estimate of the number of keys that are yet in
memory but are already logically expired. They reason why those keys are
yet not reclaimed is because the active expire cycle can't spend more
time on the process of reclaiming the keys, and at the same time nobody
is accessing such keys. However as the active expire cycle runs, while
it will eventually have to return to the caller, because of time limit
or because there are less than 25% of keys logically expired in each
given database, it collects the stats in order to populate this INFO
field.

Note that expired_stale_perc is a running average, where the current
sample accounts for 5% and the history for 95%, so you'll see it
changing smoothly over time.

The other field, expired_time_cap_reached_count, counts the number
of times the expire cycle had to stop, even if still it was finding a
sizeable number of keys yet to expire, because of the time limit.
This allows people handling operations to understand if the Redis
server, during mass-expiration events, is able to collect keys fast
enough usually. It is normal for this field to increment during mass
expires, but normally it should very rarely increment. When instead it
constantly increments, it means that the current workloads is using
a very important percentage of CPU time to expire keys.

This feature was created thanks to the hints of Rashmi Ramesh and
Bart Robinson from Twitter. In private email exchanges, they noted how
it was important to improve the observability of this parameter in the
Redis server. Actually in big deployments, the amount of keys that are
yet to expire in each server, even if they are logically expired, may
account for a very big amount of wasted memory.
2018-02-19 11:22:34 +01:00
antirez 256ddbf6dc Remove non semantical spaces from module.c. 2018-02-15 21:47:50 +01:00
antirez 280c3e3987 Fix typo in notifyKeyspaceEvent() comment. 2018-02-15 21:47:42 +01:00
Dvir Volkandantirez 7c4623b0d3 Add doc comment about notification flags 2018-02-15 21:47:38 +01:00
Dvir Volkandantirez f4e7502e4f Fix indentation and comment style in testmodule 2018-02-15 21:46:44 +01:00
Dvir Volkandantirez 3c8456c641 Use one static client for all keyspace notification callbacks 2018-02-15 21:46:38 +01:00
Dvir Volkandantirez aaaff8bd1c Remove the NOTIFY_MODULE flag and simplify the module notification flow if there aren't subscribers 2018-02-15 21:46:31 +01:00
Dvir Volkandantirez 0be51b8f54 Document flags for notifications 2018-02-15 21:45:41 +01:00
Dvir Volkandantirez 3b95c89cdb removed some trailing whitespaces 2018-02-15 21:45:37 +01:00
Dvir Volkandantirez 84c6f1e3ca removed hellonotify.c 2018-02-15 21:45:32 +01:00
Dvir Volkandantirez 53b85e53e3 fixed test 2018-02-15 21:45:27 +01:00
Dvir Volkandantirez b43f66c9d4 finished implementation of notifications. Tests unfinished 2018-02-15 21:45:22 +01:00
antirez eddf5deb38 More verbose logging when slave sends errors to master.
See #3832.
2018-02-15 21:43:23 +01:00
oranagraandantirez c09cc0a9b7 when a slave experiances an error on commands that come from master, print to the log
since slave isn't replying to it's master, these errors go unnoticed.
since we don't expect the master to send garbadge to the slave, this should be safe.
(as long as we don't log OOM errors there)
2018-02-15 21:43:17 +01:00
charsyamandantirez 5c374f94ef getting rid of duplicated code 2018-02-13 16:21:01 +01:00
Guy Benoishandantirez a64f36e556 enlarged buffer given to ld2string 2018-02-13 15:51:36 +01:00
antirez f170580195 Make it explicit with a comment why we kill the old AOF rewrite.
See #3858.
2018-02-13 15:46:53 +01:00
Guy Benoishandantirez 0c030dea73 rewriteAppendOnlyFileBackground() failure fix
It is possible to do BGREWRITEAOF even if appendonly=no. This is by design.
stopAppendonly() didn't turn off aof_rewrite_scheduled (it can be turned on
again by BGREWRITEAOF even while appendonly is off anyway).
After configuring `appendonly yes` it will see that the state is AOF_OFF,
there's no RDB fork, so it will do rewriteAppendOnlyFileBackground() which
will fail since the aof_child_pid is set (was scheduled and started by cron).

Solution:
stopAppendonly() will turn off the schedule flag (regardless of who asked for it).
startAppendonly() will terminate any existing fork and start a new one (so it is the most recent).
2018-02-13 15:46:50 +01:00
Oran Agraandantirez 5807397460 fix to latency monitor reporting wrong max latency
in some cases LATENCY HISTORY reported latency that was
higher than the max latency reported by LATENCY LATEST / DOCTOR
2018-02-13 15:31:43 +01:00
antirez f17d82961d Redis 4.0.8. 2018-02-02 17:39:14 +01:00
antirez f603940f7c Rax updated to latest antirez/rax commit. 2018-02-02 11:10:30 +01:00
antirez 2c1fc582c7 Redis 4.0.7. 2018-01-24 11:16:18 +01:00
jianqingduandantirez 2b99d77a57 fix not call va_end when syncWrite() failed
fix not call va_end when syncWrite() failed in sendSynchronousCommand()
2018-01-24 10:58:57 +01:00
Yusaku Kanetaandantirez 5f9b9e1194 Fix the firstkey, lastkey, and keystep of moduleCommand 2018-01-24 10:58:39 +01:00
Mark Nunbergandantirez ba2d3e8e6e redismodule.h: Check ModuleNameBusy before calling it
Older versions might not have this function.
2018-01-24 10:48:42 +01:00
antirez 05c1f18d6a Fix integration test NOREPLICAS error time dependent false positive. 2018-01-24 10:24:22 +01:00
antirez 4acd6973bf Fix migrateCommand() access of not initialized byte. 2018-01-18 12:41:23 +01:00
Guy Benoishandantirez 548e4fe088 Replication buffer fills up on high rate traffic.
When feeding the master with a high rate traffic the the slave's feed is much slower.
This causes the replication buffer to grow (indefinitely) which leads to slave disconnection.
The problem is that writeToClient() decides to stop writing after NET_MAX_WRITES_PER_EVENT
writes (In order to be fair to clients).
We should ignore this when the client is a slave.
It's better if clients wait longer, the alternative is that the slave has no chance to stay in
sync in this situation.
2018-01-18 12:16:50 +01:00
antirez efa7063c52 Cluster: improve anti-affinity algo in redis-trib.rb.
See #3462 and related PRs.

We use a simple algorithm to calculate the level of affinity violation,
and then an optimizer that performs random swaps until things improve.
2018-01-18 12:16:46 +01:00
antirez 48568ab6d7 Remove useless comment from serverCron().
The behavior is well specified by the code itself.
2018-01-18 12:16:42 +01:00
heqinandantirez 0201dea577 fixbug for #4545 dead loop aof rewrite 2018-01-18 12:16:37 +01:00
antirez 926beaa3c4 Hopefully more clear comment to explain the change in #4607. 2018-01-18 12:16:31 +01:00
qinchaoandantirez 019ad3e2e3 fix assert problem in ZIP_DECODE_PREVLENSIZE
, see issue: https://github.com/antirez/redis/issues/4587
2018-01-18 12:16:23 +01:00
Oran Agraandantirez 8d9dff84ce PSYNC2 fix - promoted slave should hold on to it's backlog
after a slave is promoted (assuming it has no slaves
and it booted over an hour ago), it will lose it's replication
backlog at the next replication cron, rather than waiting for slaves
to connect to it.
so on a simple master/slave faiover, if the new slave doesn't connect
immediately, it may be too later and PSYNC2 will fail.
2018-01-18 12:16:05 +01:00
zhaozhao.zzandantirez fba2e169f9 aof: format code and comment 2018-01-18 12:15:57 +01:00
antirez 7777be7b0f Put more details in the comment introduced by #4601. 2018-01-18 12:15:53 +01:00
zhaozhao.zzandantirez 91c1568b1a lazyfree: fix memory leak for lazyfree-lazy-server-del 2018-01-18 12:15:47 +01:00
antirez f9c2c1acc6 Fix getKeysUsingCommandTable() in the case of nagative arity.
This fixes a crash with Redis Cluster when OBJECT is mis-used, because
getKeysUsingCommandTable() will call serverPanic() detecting we are
accessing an invalid argument in the case "OBJECT foo" is called.

This bug was introduced when OBJECT HELP was introduced, because the key
argument is set fixed at index 2 in the command table, however now
OBJECT may be called with an insufficient number of arguments to extract
the key.

The "Right Thing" would be to have a specific function to extract keys
from the OBJECT command, however this is kinda of an overkill, so I
preferred to make getKeysUsingCommandTable() more robust and just return
no keys when it's not possible to honor the command table, because new
commands are often added and also there are a number with an HELP
subcommand violating the normal form, and crashing for this trivial
reason or having many command-specific key extraction functions is not
great.
2018-01-18 12:15:23 +01:00
antirez 61135f1806 Document new protocol options in #4568 into redis.conf. 2018-01-18 12:15:12 +01:00
antirez e77fba4d03 proto-max-querybuf-len -> client-query-buffer-limit.
We already had client buffer limits exported as configuration options.
Stick with the naming scheme already used.

See #4568.
2018-01-18 12:15:00 +01:00
antirez 87fe813b3a New config options about protocol prefixed with "proto".
Related to #4568.
2018-01-18 12:14:53 +01:00
gnuhpcandantirez 2e0d241420 Fix a typo(maybe instruction?) in crash log 2018-01-18 12:14:48 +01:00
Dvir Volkandantirez 9f7e214e8c Added RM_UnlinkKey - a low level analog to UNLINK command 2018-01-18 12:14:42 +01:00
zhaozhao.zzandantirez 947077bbcb redis-benchmark: bugfix - handle zero liveclients in right way 2018-01-18 12:14:22 +01:00
Oran Agraandantirez ff2e628f4e Add config options for max-bulk-len and max-querybuf-len mainly to support RESTORE of large keys 2018-01-18 12:13:50 +01:00
Oran Agraandantirez aefa9caacf fix processing of large bulks (above 2GB)
- protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
  readQueryFromClient potential overflow
- rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
- several places in sds.c that used int for string length or index.
- bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
- RM_SaveStringBuffer was limitted to 32bit length
2018-01-18 12:13:25 +01:00
heqinandantirez 896cf1a9d9 fixbug for #4545 dead loop aof rewrite 2018-01-18 12:12:58 +01:00
antirez 5abb12e04f Hyperloglog: refresh hdr variable correctly.
This is a fix for the #3819 improvements. The o->ptr may change because
of hllSparseSet() calls, so 'hdr' must be correctly re-fetched.
2018-01-09 18:51:04 +01:00
antirez c39a0f7c2a Hyperloglog: Support for PFMERGE sparse encoding as target.
This is a fix for #3819.
2018-01-09 18:51:01 +01:00
antirez 8a012df9c6 Hyperloglog: refactoring of sparse/dense add function.
The commit splits the add functions into a set() and add() set of
functions, so that it's possible to set registers in an independent way
just having the index and count.

Related to #3819, otherwise a fix is not possible.
2018-01-09 18:50:58 +01:00
antirez 549409ffa5 Test: MIGRATE AUTH test added.
Related to #2507.
2018-01-09 18:49:45 +01:00
antirez 47717222b6 Rewrite MIGRATE AUTH option.
See PR #2507. This is a reimplementation of the fix that contained
different problems.
2018-01-09 18:49:41 +01:00
heqinandantirez d8da89ea5d fixbug for #4538 Error opening /setting AOF rewrite IPC pipes: No such file or directory 2018-01-09 18:42:37 +01:00
antirez 4fcc564a97 safe_write -> aofWrite. Function commented.
Related to #4498.
2018-01-09 18:42:19 +01:00
zhaozhao.zzandantirez 27d9c729e5 aof: cast sdslen to ssize_t 2018-01-09 18:41:56 +01:00
zhaozhao.zzandantirez de4fb877e8 aof: fix the short write 2018-01-09 18:41:53 +01:00
Tomasz Poradowskiandantirez 1fade3d3b6 always enable command history in redis-cli
- when redis-cli is running in a TTY - always enable command history
buffering, regardless if history file path can be successfully
determined
2018-01-09 18:39:29 +01:00
antirez 9f4d4eef8f Cluster: allow read-only EVAL/EVALSHA in slaves.
Fix #3665.
2018-01-09 18:39:22 +01:00
nasheandantirez 8eeceabdc2 Prevent off-by-one read in stringmatchlen() (fixes #4527) 2018-01-09 18:39:08 +01:00
gnuhpcandantirez 733af14822 Fix memory usage list bug 2018-01-05 11:53:45 +01:00
zhaozhao.zzandantirez c9cb699bfb dict: fix the int problem for defrag 2017-12-05 15:38:50 +01:00
zhaozhao.zzandantirez b37099a145 dict: fix the int problem 2017-12-05 15:38:50 +01:00
zhaozhao.zzandantirez 8fe586d344 set: fix the int problem for qsort 2017-12-05 15:38:50 +01:00
zhaozhao.zzandantirez 219e29af65 set: fix the int problem for SPOP & SRANDMEMBER 2017-12-05 15:38:50 +01:00
antirez 1980bcb7f6 Redis 4.0.6. 2017-12-04 18:01:09 +01:00
zhaozhao.zzandantirez 57786b14e3 quicklist: change the len of quicklist to unsigned long 2017-12-04 17:39:28 +01:00
zhaozhao.zzandantirez 2211540d93 quicklist: fix the return value of quicklistCount 2017-12-04 17:39:19 +01:00
antirez c85c84be6b Refactoring: improve luaCreateFunction() API.
The function in its initial form, and after the fixes for the PSYNC2
bugs, required code duplication in multiple spots. This commit modifies
it in order to always compute the script name independently, and to
return the SDS of the SHA of the body: this way it can be used in all
the places, including for SCRIPT LOAD, without duplicating the code to
create the Lua function name. Note that this requires to re-compute the
body SHA1 in the case of EVAL seeing a script for the first time, but
this should not change scripting performance in any way because new
scripts definition is a rare event happening the first time a script is
seen, and the SHA1 computation is anyway not a very slow process against
the typical Redis script and compared to the actua Lua byte compiling of
the body.

Note that the function used to assert() if a duplicated script was
loaded, however actually now two times over three, we want the function
to handle duplicated scripts just fine: this happens in SCRIPT LOAD and
in RDB AUX "lua" loading. Moreover the assert was not defending against
some obvious failure mode, so now the function always tests against
already defined functions at start.
2017-12-04 15:25:11 +01:00
antirez 85b2477099 Remove useless variable check from luaCreateFunction().
The block is already inside if (allow_dup).
2017-12-04 15:25:07 +01:00
antirez a945e5c066 Fix issue #4505, Lua RDB AUX field loading of existing scripts.
Unfortunately, as outlined by @soloestoy in #4505, "lua" AUX RDB field
loading in case of duplicated script was still broken. This commit fixes
this problem and also a memory leak introduced by the past commit.

Note that now we have a regression test able to duplicate the issue, so
this commit was actually tested against the regression. The original PR
also had a valid fix, but I prefer to hide the details of scripting.c
outside scripting.c, and later "SCRIPT LOAD" should also be able to use
the function luaCreateFunction() instead of redoing the work.
2017-12-04 15:25:05 +01:00
antirez 65a2e40ae4 Regression test for #4505 (Lua AUX field loading). 2017-12-04 15:25:02 +01:00
antirez d6c70f22cd DEBUG change-repl-id implemented.
With PSYNC2 to force a full SYNC in tests is hard. With this new DEBUG
subcommand we just need to call it and then CLIENT KILL TYPE master in
the slave.
2017-12-04 15:24:59 +01:00
antirez 012bcd49ee Redis 4.0.5. 2017-12-01 16:04:41 +01:00
antirez 1198f7ceff Fix loading of RDB files lua AUX fields when the script is defined.
In the case of slaves loading the RDB from master, or in other similar
cases, the script is already defined, and the function registering the
script should not fail in the assert() call.
2017-12-01 16:03:05 +01:00
antirez cb2f001f52 Redis 4.0.4. 2017-11-30 18:44:38 +01:00
antirez 8449227f55 PSYNC2: Fix off by one buffer size in luaCreateFunction(). 2017-11-30 18:41:05 +01:00
antirez eeac1d35d9 PSYNC2: just store script bodies into RDB.
Related to #4483. As suggested by @soloestoy, we can retrieve the SHA1
from the body. Given that in the new implementation using AUX fields we
ended copying around a lot to create new objects and strings, extremize
such concept and trade CPU for space inside the RDB file.
2017-11-30 18:41:00 +01:00
antirez fb0441a8a2 PSYNC2: luaCreateFunction() should handle NULL client parameter.
See #4483. This is needed because luaCreateFunction() is now called
from RDB loading code outside a client context.
2017-11-30 18:40:53 +01:00
antirez 0429db3c65 PSYNC2: Save Lua scripts state into RDB file.
This is currently needed in order to fix #4483, but this can be
useful in other contexts, so maybe later we may want to remove the
conditionals and always save/load scripts.

Note that we are using the "lua" AUX field here, in order to guarantee
backward compatibility of the RDB file. The unknown AUX fields must be
discarded by past versions of Redis.
2017-11-30 18:40:35 +01:00
antirez d06fbbdd54 Regression test: Slave restart with EVALSHA in backlog issue #4483. 2017-11-30 18:40:27 +01:00
antirez ab3d3aca48 Prevent corruption of server.executable after DEBUG RESTART.
Doing the following ended with a broken server.executable:

1. Start Redis with src/redis-server
2. Send CONFIG SET DIR /tmp/
3. Send DEBUG RESTART

At this point we called execve with an argv[0] that is no longer related
to the new path. So after the restart the absolute path of the
executable is recomputed in the wrong way. With this fix we pass the
absolute path already computed as argv[0].
2017-11-30 18:40:20 +01:00
antirez b7c7edf912 Be more verbose when DEBUG RESTART fails. 2017-11-30 18:40:17 +01:00
antirez cbb6f45784 Redis 4.0.3. 2017-11-30 13:23:42 +01:00
antirez d766322e67 LFU: Fix LFUDecrAndReturn() to just decrement.
Splitting the popularity in half actually just needs decrementing the
counter because the counter is logarithmic.
2017-11-30 13:14:04 +01:00
zhaozhao.zzandantirez 6544796ab4 LFU: add hotkeys option to redis-cli 2017-11-30 13:13:56 +01:00
zhaozhao.zzandantirez e2355c192c LFU: do some changes about LFU to find hotkeys
Firstly, use access time to replace the decreas time of LFU.
For function LFUDecrAndReturn,
it should only try to get decremented counter,
not update LFU fields, we will update it in an explicit way.
And we will times halve the counter according to the times of
elapsed time than server.lfu_decay_time.
Everytime a key is accessed, we should update the LFU
including update access time, and increment the counter after
call function LFUDecrAndReturn.
If a key is overwritten, the LFU should be also updated.
Then we can use `OBJECT freq` command to get a key's frequence,
and LFUDecrAndReturn should be called in `OBJECT freq` command
in case of the key has not been accessed for a long time,
because we update the access time only when the key is read or
overwritten.
2017-11-30 13:13:56 +01:00
zhaozhao.zzandantirez 22969a13a0 LFU: change lfu* parameters to int 2017-11-30 13:13:56 +01:00
zhaozhao.zzandantirez 6b71f714bd LFU: fix the missing of config get and rewrite 2017-11-30 13:13:56 +01:00
Felix Krauseandantirez 2090052ef2 Update link to https and use inline link 2017-11-28 18:45:03 +01:00
Bo Caiandantirez a75f2025f2 redis-cli.c typo: Requets -> Requests.
Signed-off-by: Bo Cai <charpty@gmail.com>
2017-11-28 18:45:03 +01:00
Bo Caiandantirez 76aab08f1f redis-cli.c typo: helpe -> helper.
Signed-off-by: Bo Cai <charpty@gmail.com>
2017-11-28 18:45:03 +01:00
Sébastien Fievetandantirez b6fe5074e1 Fix some typos 2017-11-28 18:45:03 +01:00
antirez eda5cb0a04 t_hash.c: clarify calling two times the same function. 2017-11-28 18:38:49 +01:00
antirez 4a60fbd8b5 adlist: fix listJoin() in the case the second list is empty.
See #4192, the original PR removed lines of code that are actually
needed, so thanks to @chunqiulfq for reporting the problem, but merging
solution from @jeesyn after checking, together with @artix75, that the
logic covers all the cases.
2017-11-28 18:27:59 +01:00
Chris Lambandantirez 060eb3b2d0 Correct spelling of "faield". 2017-11-28 18:27:59 +01:00
antirez 3c942b1269 Improve OBJECT HELP descriptions.
See #4472.
2017-11-28 17:34:47 +01:00
antirez 6b6a83c7ab Fix entry command table entry for OBJECT for HELP option.
After #4472 the command may have just 2 arguments.
2017-11-28 17:34:47 +01:00
Itamar Haberandantirez 048097ada8 Adds OBJECT help 2017-11-28 17:34:47 +01:00
David Carlierandantirez 906134fe52 Fix undefined behavior constant defined. 2017-11-28 17:34:47 +01:00
rouzierandantirez 03657e88fe Fix file descriptor leak and error handling 2017-11-28 17:34:08 +01:00
Itamar Haberandantirez 52fda0132d Prevents OBJECT freq with noeviction
When maxmemory is set to noeviction, idletime is implicitly kept. This renders access frequency nonsensical.
2017-11-27 13:08:35 +01:00
Itamar Haberandantirez 15bc8e97a9 Adds -u <uri> option to redis-cli. 2017-11-27 12:32:58 +01:00
antirez f30454c1f6 Test: regression test for latency expire events logging bug.
Regression for #4452.
2017-11-27 12:32:41 +01:00
zhaozhao.zzandantirez 1e7227f429 expire & latency: fix the missing latency records generated by expire 2017-11-27 12:32:38 +01:00
antirez 9524fce0ac Modules: fix memory leak in RM_IsModuleNameBusy(). 2017-11-24 13:30:06 +01:00
antirez 2a27da1cf5 PSYNC2: reorganize comments related to recent fixes.
Related to PR #4412 and issue #4407.
2017-11-24 11:10:03 +01:00
zhaozhao.zzandantirez e0c2a0ecfd PSYNC2: persist cached_master's dbid inside the RDB 2017-11-24 11:09:28 +01:00
zhaozhao.zzandantirez 2eca8aed14 PSYNC2: make repl_stream_db never be -1
it means that after this change all the replication
info in RDB is valid, and it can distinguish us from
the older version.
2017-11-24 11:09:28 +01:00
zhaozhao.zzandantirez 3594238350 PSYNC2: clarify the scenario when repl_stream_db can be -1 2017-11-24 11:09:28 +01:00
zhaozhao.zzandantirez be1b9ee0d5 PSYNC2 & RDB: fix the missing rdbSaveInfo for BGSAVE 2017-11-24 11:09:18 +01:00
zhaozhao.zzandantirez 9f69e179cf PSYNC2: safe free backlog when reach the time limit
When we free the backlog, we should use a new
replication ID and clear the ID2. Since without
backlog we can not increment master_repl_offset
even do write commands, that may lead to inconsistency
when we try to connect a "slave-before" master
(if this master is our slave before, our replid
equals the master's replid2). As the master have our
history, so we can match the master's replid2 and
second_replid_offset, that make partial sync work,
but the data is inconsistent.
2017-11-24 11:09:14 +01:00
zhaozhao.zzandantirez 0205dd01e2 Modules: handle the busy module name 2017-11-24 09:38:34 +01:00
zhaozhao.zzandantirez 3cce566eb1 Modules: handle the conflict of registering commands 2017-11-24 09:38:31 +01:00
Oran Agraandantirez d01f163ce0 fix string to double conversion, stopped parsing on \0 even if the string has more data.
getLongLongFromObject calls string2ll which has this line:
/* Return if not all bytes were used. */
so if you pass an sds with 3 characters "1\01" it will fail.

but getLongDoubleFromObject calls strtold, and considers it ok if eptr[0]==`\0`
i.e. if the end of the string found by strtold ends with null terminator

127.0.0.1:6379> set a 1
OK
127.0.0.1:6379> setrange a 2 2
(integer) 3
127.0.0.1:6379> get a
"1\x002"
127.0.0.1:6379> incrbyfloat a 2
"3"
127.0.0.1:6379> get a
"3"
2017-11-24 09:00:37 +01:00
antirez 9a3e15c6a2 Modules: fix for scripting replication of modules commands.
See issue #4466 / #4467.
2017-11-23 15:15:50 +01:00
Yossi Gottliebandantirez fa87879bab Nested MULTI/EXEC may replicate in different cases.
For example:
1. A module command called within a MULTI section.
2. A Lua script with replicate_commands() called within a MULTI section.
3. A module command called from a Lua script in the above context.
2017-11-23 15:15:50 +01:00
zhaozhao.zzandantirez bc7076b090 rehash: handle one db until finished 2017-11-21 09:50:08 +01:00
kmiku7andantirez 7675b00a64 fix boundary case for _dictNextPower 2017-11-08 17:09:01 +01:00
Itamar Haberandantirez f31d9b12fd Fixes an off-by-one in argument handling of MEMORY USAGE
Fixes #4430
2017-11-08 15:50:59 +01:00
antirez 897d857115 SDS: improve sdsRemoveFreeSpace() to avoid useless data copy.
Since SDS v2, we no longer have a single header, so the function to
rewrite the SDS in terms of the minimum space required, instead of just
using realloc() and let the underlying allocator decide what to do,
was doing an allocation + copy every time the minimum possible header
needed to represent the string was different than the current one.
This could be often a bit wasteful, because if we go, for instance, from
the 32 bit fields header to the 16 bit fields header, the overhead of
the header is normally very small. With this commit we call realloc
instead, unless the change in header size is very significant in relation
to the string length.
2017-11-08 15:50:59 +01:00
antirez 1ee6af4d87 Fix saving of zero-length lists.
Normally in modern Redis you can't create zero-len lists, however it's
possible to load them from old RDB files generated, for instance, using
Redis 2.8 (see issue #4409). The "Right Thing" would be not loading such
lists at all, but this requires to hook in rdb.c random places in a not
great way, for a problem that is at this point, at best, minor.

Here in this commit instead I just fix the fact that zero length lists,
materialized as quicklists with the first node set to NULL, were
iterated in the wrong way while they are saved, leading to a crash.

The other parts of the list implementation are apparently able to deal
with empty lists correctly, even if they are no longer a thing.
2017-11-06 12:37:28 +01:00
antirez 1740300f35 Fix buffer overflows occurring reading redis.conf.
There was not enough sanity checking in the code loading the slots of
Redis Cluster from the nodes.conf file, this resulted into the
attacker's ability to write data at random addresses in the process
memory, by manipulating the index of the array. The bug seems
exploitable using the following techique: the config file may be altered so
that one of the nodes gets, as node ID (which is the first field inside the
structure) some data that is actually executable: then by writing this
address in selected places, this node ID part can be executed after a
jump. So it is mostly just a matter of effort in order to exploit the
bug. In practice however the issue is not very critical because the
bug requires an unprivileged user to be able to modify the Redis cluster
nodes configuration, and at the same time this should result in some
gain. However Redis normally is unprivileged as well. Yet much better to
have this fixed indeed.

Fix #4278.
2017-10-31 09:52:56 +01:00
antirez b25c245156 Regression test for issue #4391. 2017-10-30 13:46:11 +01:00
antirez 1847b987c6 More robust object -> double conversion.
Certain checks were useless, at the same time certain malformed inputs
were accepted without problems (emtpy strings parsed as zero).
Cases where strtod() returns ERANGE but we still want to parse the input
where ok in getDoubleFromObject() but not in the long variant.

As a side effect of these fixes, this commit fixes #4391.
2017-10-30 13:46:09 +01:00
antirez c94cd1d831 Limit statement in RM_BlockClient() to 80 cols. 2017-10-04 16:12:47 +02:00
Dvir Volkandantirez 193e4acc07 Added safety net preventing redis from crashing if a module decide to block in MULTI 2017-10-04 16:12:47 +02:00
Dvir Volkandantirez d131921c41 Renamed GetCtxFlags to GetContextFlags 2017-10-04 16:12:47 +02:00
Dvir Volkandantirez 2e71edccfc Added support for module context flags with RM_GetCtxFlags 2017-10-04 16:12:47 +02:00
antirez 44053df0a4 Redis 4.0.2. 2017-09-21 16:12:52 +02:00
antirez 1c60b7a671 Clarify comment in change fixing #4323. 2017-09-21 15:46:49 +02:00
zhaozhao.zzandantirez 368124e8fa Lazyfree: avoid memory leak when free slowlog entry 2017-09-21 15:46:46 +02:00
antirez 79567b6e66 PSYNC2: More refinements related to #4316. 2017-09-20 11:47:30 +02:00
zhaozhao.zzandantirez f119464929 PSYNC2: make persisiting replication info more solid
This commit is a reinforcement of commit c1c99e9.

1. Replication information can be stored when the RDB file is
generated by a mater using server.slaveseldb when server.repl_backlog
is not NULL, or set repl_stream_db be -1. That's safe, because
NULL server.repl_backlog will trigger full synchronization,
then master will send SELECT command to replicaiton stream.
2. Only do rdbSave* when rsiptr is not NULL,
if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
3. Save the replication informations also in the case of
SAVE command, FLUSHALL command and DEBUG reload.
2017-09-20 11:47:26 +02:00
antirez 097a555677 PSYNC2: Fix the way replication info is saved/loaded from RDB.
This commit attempts to fix a number of bugs reported in #4316.
They are related to the way replication info like replication ID,
offsets, and currently selected DB in the master client, are stored
and loaded by Redis. In order to avoid inconsistencies the changes in
this commit try to enforce that:

1. Replication information are only stored when the RDB file is
generated by a slave that has a valid 'master' client, so that we can
always extract the currently selected DB.
2. When replication informations are persisted in the RDB file, all the
info for a successful PSYNC or nothing is persisted.
3. The RDB replication informations are only loaded if the instance is
configured as a slave, otherwise a master can start with IDs that relate
to a different history of the data set, and stil retain such IDs in the
future while receiving unrelated writes.
2017-09-20 11:47:23 +02:00
antirez f1a2cbfd6e PSYNC2: Create backlog on slave partial sync as well.
A slave may be started with an RDB file able to provide enough slave to
perform a successful partial SYNC with its master. However in such a
case, how outlined in issue #4268, the slave backlog will not be
started, since it was only initialized on full syncs attempts. This
creates different problems with successive PSYNC attempts that will
always result in full synchronizations.

Thanks to @fdingiit for discovering the issue.
2017-09-20 11:47:17 +02:00
antirez 0c0b77d149 Add MEMORY DOCTOR to MEMORY HELP. 2017-09-20 11:47:13 +02:00
Motaandantirez fa6bd1b230 redis-benchmark: default value size usage update.
default size of SET/GET value in usage should be 3 bytes as in main code.
2017-09-20 11:47:08 +02:00
jybaekandantirez ad0ddcf390 Remove Duplicate Processing 2017-09-20 11:46:53 +02:00
Oran Agraandantirez 8651e5d50d Flush append only buffers before existing.
when SHUTDOWN command is recived it is possible that some of the recent
command were not yet flushed from the AOF buffer, and the server
experiences data loss at shutdown.
2017-09-18 12:04:31 +02:00
antirez f2b2897f80 Changelog: note that 4.0 CLUSTER NODES output changed. 2017-08-02 13:07:45 +02:00
Itamar Haberandantirez 363be78397 Changes command stats iteration to being dict-based
With the addition of modules, looping over the redisCommandTable
misses any added commands. By moving to dictionary iteration this
is resolved.
2017-08-02 12:52:00 +02:00
antirez 3a523ac335 Redis 4.0.1. 2017-07-24 15:58:34 +02:00
Jan-Erik Redigerandantirez a8c2ef7621 Check that the whole first argument is a number
Fixes #2258
2017-07-24 15:21:41 +02:00
WuYunlongandantirez bfe5008b17 fix rewrite config: auto-aof-rewrite-min-size 2017-07-24 15:21:05 +02:00
Chris Lambandantirez a6abc2165b Correct proceding -> proceeding typo. 2017-07-24 15:21:05 +02:00
Byron Grobeandantirez 1d901b025f Fixed issue #1996 (Missing '-' in help message for redis-benchmark) 2017-07-24 15:21:05 +02:00
Jan-Erik Redigerandantirez 19e5e5eaeb Don't use extended Regexp Syntax
It's not POSIX (BSD systems have -E instead) and we don't actually need it.

Closes #1922
2017-07-24 15:21:05 +02:00
Leon Chenandantirez 62474219d0 fix return wrong value of clusterDelNodeSlots 2017-07-24 14:18:54 +02:00
Leon Chenandantirez dc782ceb83 fix mismatch argument 2017-07-24 14:18:54 +02:00
liangsijianandantirez 07631ff18e Fix lua ldb command log 2017-07-24 14:11:33 +02:00
antirez 41e3617df9 Modules: don't crash when Lua calls a module blocking command.
Lua scripting does not support calling blocking commands, however all
the native Redis commands are flagged as "s" (no scripting flag), so
this is not possible at all. With modules there is no such mechanism in
order to flag a command as non callable by the Lua scripting engine,
moreover we cannot trust the modules users from complying all the times:
it is likely that modules will be released to have blocking commands
without such commands being flagged correctly, even if we provide a way to
signal this fact.

This commit attempts to address the problem in a short term way, by
detecting that a module is trying to block in the context of the Lua
scripting engine client, and preventing to do this. The module will
actually believe to block as usually, but what happens is that the Lua
script receives an error immediately, and the background call is ignored
by the Redis engine (if not for the cleanup callbacks, once it
unblocks).

Long term, the more likely solution, is to introduce a new call called
RedisModule_GetClientFlags(), so that a command can detect if the caller
is a Lua script, and return an error, or avoid blocking at all.

Being the blocking API experimental right now, more work is needed in
this regard in order to reach a level well blocking module commands and
all the other Redis subsystems interact peacefully.

Now the effect is like the following:

    127.0.0.1:6379> eval "redis.call('hello.block',1,5000)" 0
    (error) ERR Error running script (call to
    f_b5ba35ff97bc1ef23debc4d6e9fd802da187ed53): @user_script:1: ERR
    Blocking module command called from Lua script

This commit fixes issue #4127 in the short term.
2017-07-23 13:09:26 +02:00
antirez 10370b207a Fix typo in unblockClientFromModule() top comment. 2017-07-23 13:09:23 +02:00
antirez b6c55a8916 Make representClusterNodeFlags() more robust.
This function failed when an internal-only flag was set as an only flag
in a node: the string was trimmed expecting a final comma before
exiting the function, causing a crash. See issue #4142.
Moreover generation of flags representation only needed at DEBUG log
level was always performed: a waste of CPU time. This is fixed as well
by this commit.
2017-07-23 13:09:17 +02:00
antirez 9a4f3d7297 Fix two bugs in moduleTypeLookupModuleByID().
The function cache was not working at all, and the function returned
wrong values if there where two or more modules exporting native data
types.

See issue #4131 for more details.
2017-07-23 13:09:13 +02:00
antirez 7302e18606 Allow certain modules APIs only defining REDISMODULE_EXPERIMENTAL_API.
Those calls may be subject to changes in the future, so the user should
acknowledge it is using non stable API.
2017-07-14 18:08:01 +02:00
antirez 05b81d2b02 Redis 4.0.0 GA. 2017-07-14 13:28:42 +02:00
antirez c29852ffd2 Modules: fix thread safe context DB selection.
Before this fix the DB currenty selected by the client blocked was not
respected and operations were always performed on DB 0.
2017-07-14 13:02:53 +02:00
antirez b73f186aac Modules documentation removed from source.
Moving to redis-doc repository to publish via Redis.io.
2017-07-14 12:22:32 +02:00
antirez 09d93ec963 Markdown generation of Redis Modules API reference improved. 2017-07-14 12:22:32 +02:00
antirez 87aabb1afa Fix replication of SLAVEOF inside transaction.
In Redis 4.0 replication, with the introduction of PSYNC2, masters and
slaves replicate commands to cascading slaves and to the replication
backlog itself in a different way compared to the past.

Masters actually replicate the effects of client commands.
Slaves just propagate what they receive from masters.

This mechanism can cause problems when the configuration of an instance
is changed from master to slave inside a transaction. For instance
we could send to a master instance the following sequence:

    MULTI
    SLAVEOF 127.0.0.1 0
    EXEC
    SLAVEOF NO ONE

Before the fixes in this commit, the MULTI command used to be propagated
into the replication backlog, however after the SLAVEOF command the
instance is a slave, so the EXEC implementation failed to also propagate
the EXEC command. When the slaves of the above instance reconnected,
they were incrementally synchronized just sending a "MULTI". This put
the master client (in the slaves) into MULTI state, breaking the
replication.

Notably even Redis Sentinel uses the above approach in order to guarantee
that configuration changes are always performed together with rewrites
of the configuration and with clients disconnection. Sentiel does:

    MULTI
    SLAVEOF ...
    CONFIG REWRITE
    CLIENT KILL TYPE normal
    EXEC

So this was a really problematic issue. However even with the fix in
this commit, that will add the final EXEC to the replication stream in
case the instance was switched from master to slave during the
transaction, the result would be to increment the slave replication
offset, so a successive reconnection with the new master, will not
permit a successful partial resynchronization: no way the new master can
provide us with the backlog needed, we incremented our offset to a value
that the new master cannot have.

However the EXEC implementation waits to emit the MULTI, so that if the
commands inside the transaction actually do not need to be replicated,
no commands propagation happens at all. From multi.c:

    if (!must_propagate && !(c->cmd->flags & (CMD_READONLY|CMD_ADMIN))) {
	execCommandPropagateMulti(c);
	must_propagate = 1;
    }

The above code is already modified by this commit you are reading.
Now also ADMIN commands do not trigger the emission of MULTI. It is actually
not clear why we do not just check for CMD_WRITE... Probably I wrote it this
way in order to make the code more reliable: better to over-emit MULTI
than not emitting it in time.

So this commit should indeed fix issue #3836 (verified), however it looks
like some reconsideration of this code path is needed in the long term.

BONUS POINT: The reverse bug.

Even in a read only slave "B", in a replication setup like:

	A -> B -> C

There are commands without the READONLY nor the ADMIN flag, that are also
not flagged as WRITE commands. An example is just the PING command.

So if we send B the following sequence:

    MULTI
    PING
    SLAVEOF NO ONE
    EXEC

The result will be the reverse bug, where only EXEC is emitted, but not the
previous MULTI. However this apparently does not create problems in practice
but it is yet another acknowledge of the fact some work is needed here
in order to make this code path less surprising.

Note that there are many different approaches we could follow. For instance
MULTI/EXEC blocks containing administrative commands may be allowed ONLY
if all the commands are administrative ones, otherwise they could be
denined. When allowed, the commands could simply never be replicated at all.
2017-07-14 10:55:17 +02:00
antirez 44f89d1d98 CLUSTER GETKEYSINSLOT: avoid overallocating.
Close #3911.
2017-07-14 10:55:17 +02:00
antirez 0df24b6803 Fix isHLLObjectOrReply() to handle integer encoded strings.
Close #3766.
2017-07-14 10:55:17 +02:00
antirez 884ceb692e Clients blocked in modules: free argv/argc later.
See issue #3844 for more information.
2017-07-14 10:55:17 +02:00
antirez ccbdd762c5 Event loop: call after sleep() only from top level.
In general we do not want before/after sleep() callbacks to be called
when we re-enter the event loop, since those calls are only designed in
order to perform operations every main iteration of the event loop, and
re-entering is often just a way to incrementally serve clietns with
error messages or other auxiliary operations. However, if we call the
callbacks, we are then forced to think at before/after sleep callbacks
as re-entrant, which is much harder without any good need.

However here there was also a clear bug: beforeSleep() was actually
never called when re-entering the event loop. But the new afterSleep()
callback was. This is broken and in this instance re-entering
afterSleep() caused a modules GIL dead lock.
2017-07-14 10:55:17 +02:00
antirez 10925e46d1 redis-check-aof: tell users there is a --fix option. 2017-07-14 10:55:17 +02:00
Guy Benoishandantirez 99bb1c74e9 Modules: Fix io->bytes calculation in RDB save 2017-07-14 10:55:17 +02:00
antirez cfdcd440d7 AOF check utility: ability to check files with RDB preamble. 2017-07-14 10:55:17 +02:00
sunweinanandantirez 1cefb1c54b minor fix in listJoin(). 2017-07-06 16:10:20 +02:00
antirez db791a1eee Free IO context if any in RDB loading code.
Thanks to @oranagra for spotting this bug.
2017-07-06 16:10:20 +02:00
antirez 419dacfeaf Modules: DEBUG DIGEST interface. 2017-07-06 16:10:20 +02:00
spinlockandantirez 5d03b831d0 update Makefile for test-sds 2017-07-06 16:10:20 +02:00
spinlockandantirez ed437b82cf Optimize addReplyBulkSds for better performance 2017-07-06 16:10:07 +02:00
antirez 4ebfe2653c Avoid closing invalid FDs to make Valgrind happier. 2017-07-06 16:10:07 +02:00
antirez b6cab88c1d Modules: no MULTI/EXEC for commands replicated from async contexts.
They are technically like commands executed from external clients one
after the other, and do not constitute a single atomic entity.
2017-07-06 16:10:07 +02:00
antirez 5c5e8a500c Add symmetrical assertion to track c->reply_buffer infinite growth.
Redis clients need to have an instantaneous idea of the amount of memory
they are consuming (if the number is not exact should at least be
proportional to the actual memory usage). We do that adding and
subtracting the SDS length when pushing / popping from the client->reply
list. However it is quite simple to add bugs in such a setup, by not
taking the objects in the list and the count in sync. For such reason,
Redis has an assertion to track counts near 2^64: those are always the
result of the counter wrapping around because we subtract more than we
add. This commit adds the symmetrical assertion: when the list is empty
since we sent everything, the reply_bytes count should be zero. Thanks
to the new assertion it should be simple to also detect the other
problem, where the count slowly increases because of over-counting.
The assertion adds a conditional in the code that sends the buffer to
the socket but should not create any measurable performance slowdown,
listLength() just accesses a structure field, and this code path is
totally dominated by write(2).

Related to #4100.
2017-07-06 16:10:07 +02:00
Dvir Volkandantirez c63a97f8d2 fixed #4100 2017-07-06 16:09:45 +02:00
antirez eeb905713b Fix GEORADIUS edge case with huge radius.
This commit closes issue #3698, at least for now, since the root cause
was not fixed: the bounding box function, for huge radiuses, does not
return a correct bounding box, there are points still within the radius
that are left outside.

So when using GEORADIUS queries with radiuses in the order of 5000 km or
more, it was possible to see, at the edge of the area, certain points
not correctly reported.

Because the bounding box for now was used just as an optimization, and
such huge radiuses are not common, for now the optimization is just
switched off when the radius is near such magnitude.

Three test cases found by the Continuous Integration test were added, so
that we can easily trigger the bug again, both for regression testing
and in order to properly fix it as some point in the future.
2017-07-06 16:09:41 +02:00
antirez 670456a7b3 redis-cli --latency: ability to run non interactively.
This feature was proposed by @rosmo in PR #2643 and later redesigned
in order to fit better with the other options for non-interactive modes
of redis-cli. The idea is basically to allow to collect latency
information in scripts, cron jobs or whateever, just running for a
limited time and then producing a single output.
2017-07-06 16:09:39 +02:00
antirez 64db804415 HMSET and MSET implementations unified. HSET now variadic.
This is the first step towards getting rid of HMSET which is a command
that does not make much sense once HSET is variadic, and has a saner
return value.
2017-07-06 16:09:36 +02:00
antirez e43c890e74 Aesthetic changes to #4068 PR to conform to Redis coding standard.
1. Inline if ... statement if short.
2. No lines over 80 columns.
2017-07-06 16:09:30 +02:00
itamarandantirez 3f3dc3b85e Sets up fake client to select current db in RM_Call() 2017-07-06 16:09:27 +02:00
antirez ba7737245a Fix abort typo in Lua debugger help screen. 2017-06-30 12:12:14 +02:00
antirez bdd6de963d Added GEORADIUS(BYMEMBER)_RO variants for read-only operations.
Issue #4084 shows how for a design error, GEORADIUS is a write command
because of the STORE option. Because of this it does not work
on readonly slaves, gets redirected to masters in Redis Cluster even
when the connection is in READONLY mode and so forth.

To break backward compatibility at this stage, with Redis 4.0 to be in
advanced RC state, is problematic for the user base. The API can be
fixed into the unstable branch soon if we'll decide to do so in order to
be more consistent, and reease Redis 5.0 with this incompatibility in
the future. This is still unclear.

However, the ability to scale GEO queries in slaves easily is too
important so this commit adds two read-only variants to the GEORADIUS
and GEORADIUSBYMEMBER command: GEORADIUS_RO and GEORADIUSBYMEMBER_RO.
The commands are exactly as the original commands, but they do not
accept the STORE and STOREDIST options.
2017-06-30 11:53:43 +02:00
Suraj Narkhedeandantirez de391ff11c Fix brpop command table entry and redirect blocked clients. 2017-06-30 10:13:59 +02:00
antirez 5af0fc0c8e RDB modules values serialization format version 2.
The original RDB serialization format was not parsable without the
module loaded, becuase the structure was managed only by the module
itself. Moreover RDB is a streaming protocol in the sense that it is
both produce di an append-only fashion, and is also sometimes directly
sent to the socket (in the case of diskless replication).

The fact that modules values cannot be parsed without the relevant
module loaded is a problem in many ways: RDB checking tools must have
loaded modules even for doing things not involving the value at all,
like splitting an RDB into N RDBs by key or alike, or just checking the
RDB for sanity.

In theory module values could be just a blob of data with a prefixed
length in order for us to be able to skip it. However prefixing the values
with a length would mean one of the following:

1. To be able to write some data at a previous offset. This breaks
stremaing.
2. To bufferize values before outputting them. This breaks performances.
3. To have some chunked RDB output format. This breaks simplicity.

Moreover, the above solution, still makes module values a totally opaque
matter, with the fowllowing problems:

1. The RDB check tool can just skip the value without being able to at
least check the general structure. For datasets composed mostly of
modules values this means to just check the outer level of the RDB not
actually doing any checko on most of the data itself.
2. It is not possible to do any recovering or processing of data for which a
module no longer exists in the future, or is unknown.

So this commit implements a different solution. The modules RDB
serialization API is composed if well defined calls to store integers,
floats, doubles or strings. After this commit, the parts generated by
the module API have a one-byte prefix for each of the above emitted
parts, and there is a final EOF byte as well. So even if we don't know
exactly how to interpret a module value, we can always parse it at an
high level, check the overall structure, understand the types used to
store the information, and easily skip the whole value.

The change is backward compatible: older RDB files can be still loaded
since the new encoding has a new RDB type: MODULE_2 (of value 7).
The commit also implements the ability to check RDB files for sanity
taking advantage of the new feature.
2017-06-27 17:53:36 +02:00
antirez 6516958efb ARM: Fix stack trace generation on crash. 2017-06-27 17:53:36 +02:00
antirez 3669f96e11 Issue #4027: unify comment and modify return value in freeMemoryIfNeeded().
It looks safer to return C_OK from freeMemoryIfNeeded() when clients are
paused because returning C_ERR may prevent success of writes. It is
possible that there is no difference in practice since clients cannot
execute writes while clients are paused, but it looks more correct this
way, at least conceptually.

Related to PR #4028.
2017-06-27 17:53:36 +02:00
Suraj Narkhedeandantirez 896c4690dd Fix following issues in blocking commands:
1. brpop last key index, thus checking all keys for slots.
2. Memory leak in clusterRedirectBlockedClientIfNeeded.
3. Remove while loop in clusterRedirectBlockedClientIfNeeded.
2017-06-27 17:53:36 +02:00
Zachary Marquezandantirez deeb795acc Prevent expirations and evictions while paused
Proposed fix to https://github.com/antirez/redis/issues/4027
2017-06-27 17:53:36 +02:00
antirez a6615423e2 Upgrade 4.0 changelog with more backward incompatibilities. 2017-06-23 18:34:59 +02:00
xuzhouandantirez 0b367871c4 Optimize set command with ex/px when updating aof. 2017-06-22 11:01:27 +02:00
antirez 2ae733d924 redis-benchmark: add -t hset target. 2017-06-22 11:01:27 +02:00
xuzhouandantirez 63e1c9f224 Fix set with ex/px option when propagated to aof 2017-06-22 11:01:27 +02:00
minghang.zmhandantirez 0231156f6a fix server.stat_net_output_bytes calc bug 2017-06-20 17:03:33 +02:00
xuchengxuanandantirez e99954e4d4 Fixed comments of slowlog duration 2017-06-20 16:56:47 +02:00
cbgbtandantirez d048f9721c cli: Only print elapsed time on OUTPUT_STANDARD 2017-06-20 16:54:44 +02:00
Aric Huangandantirez b5f22939c2 (fix) Update create-cluster README
Fix a few typos/adjust wording in `create-cluster` README
2017-06-20 16:54:44 +02:00
antirez 0b7ba621b7 SLOWLOG: log offending client address and name. 2017-06-15 17:02:16 +02:00
Antonio Malliaandantirez 1fbc90fe03 Removed duplicate 'sys/socket.h' include 2017-06-15 17:02:16 +02:00
Antonio Malliaandantirez c7a6b711f3 Fixed comment in clusterMsg version field 2017-06-15 17:02:16 +02:00
Qu Chenandantirez 73d358f79f Implement getKeys procedure for georadius and georadiusbymember
commands.
2017-06-14 18:16:24 +02:00
antirez c782d1894b Fix PERSIST expired key resuscitation issue #4048. 2017-06-13 10:37:28 +02:00
antirez cb548bf3c0 More informative -MISCONF error message. 2017-05-19 12:04:11 +02:00
antirez 8cd6a2bd86 Collect fork() timing info only if fork succeeded. 2017-05-19 12:03:54 +02:00
antirez a3941aa569 redis-cli --bigkeys: show error when TYPE fails.
Close #3993.
2017-05-15 11:23:55 +02:00
antirez 6b21cebd3d Modules TSC: use atomic var for server.unixtime.
This avoids Helgrind complaining, but we are actually not using
atomicGet() to get the unixtime value for now: too many places where it
is used and given tha time_t is word-sized it should be safe in all the
archs we support as it is.

On the other hand, Helgrind, when Redis is compiled with "make helgrind"
in order to force the __sync macros, will detect the write in
updateCachedTime() as a read (because atomic functions are used) and
will not complain about races.

This commit also includes minor refactoring of mutex initializations and
a "helgrind" target in the Makefile.
2017-05-11 16:44:46 +02:00
antirez 54bd224f0e atomicvar.h: show used API in INFO. Add macro to force __sync builtin.
The __sync builtin can be correctly detected by Helgrind so to force it
is useful for testing. The API in the INFO output can be useful for
debugging after problems are reported.
2017-05-11 16:44:46 +02:00
antirez a864d25c6e zmalloc.c: remove thread safe mode, it's the default way. 2017-05-11 16:44:46 +02:00
antirez b338f2b908 Modules TSC: Add mutex for server.lruclock.
Only useful for when no atomic builtins are available.
2017-05-11 16:44:46 +02:00
antirez 7e9c658d13 Modules TSC: Improve inter-thread synchronization.
More work to do with server.unixtime and similar. Need to write Helgrind
suppression file in order to suppress the valse positives.
2017-05-11 16:44:46 +02:00
antirez e69af32fd7 Simplify atomicvar.h usage by having the mutex name implicit. 2017-05-11 16:44:46 +02:00
antirez 26e57f177d Lazyfree: fix lazyfreeGetPendingObjectsCount() race reading counter. 2017-05-11 16:44:46 +02:00
antirez 2acf003c05 Modules TSC: HELLO.KEYS reply format fixed. 2017-05-11 16:44:46 +02:00
antirez 12fd298fe7 Modules TSC: put the client in the pending write list. 2017-05-11 16:44:26 +02:00
antirez 5b1afa4a22 adlist: fix final list count in listJoin(). 2017-05-11 16:44:26 +02:00
antirez 717b2eeab3 adlist: fix listJoin() to handle empty lists. 2017-05-11 16:44:26 +02:00
antirez a839036a1d Modules: remove unused var in example module. 2017-05-11 16:44:26 +02:00
antirez eda5ee5e91 Modules TSC: HELLO.KEYS example draft finished. 2017-05-11 16:44:26 +02:00
antirez fb8734fe9c Module: fix RedisModule_Call() "l" specifier to create a raw string. 2017-05-11 16:44:26 +02:00
antirez c4b884958e Modules TSC: Release the GIL for all the time we are blocked.
Instead of giving the module background operations just a small time to
run in the beforeSleep() function, we can have the lock released for all
the time we are blocked in the multiplexing syscall.
2017-05-11 16:44:26 +02:00
antirez fcd9a07df0 Modules TSC: Export symbols of the new API. 2017-05-11 16:44:26 +02:00
antirez 8affa3e78f Modules TSC: Handling of RM_Reply* functions. 2017-05-11 16:44:03 +02:00
antirez 31b1f3c1ae Modules TSC: Basic TS context creeation and handling. 2017-05-11 16:44:03 +02:00
antirez 74f3a84390 Modules TSC: GIL and cooperative multi tasking setup. 2017-05-11 16:44:03 +02:00
antirez 5021fda2b9 Regression test for #3899 fixed. 2017-05-11 16:43:46 +02:00
antirez 166bdbda03 Regression test for PSYNC2 issue #3899 added.
Experimentally verified that it can trigger the issue reverting the fix.
At least on my system... Being the bug time/backlog dependant, it is
very hard to tell if this test will be able to trigger the problem
consistently, however even if it triggers the problem once in a while,
we'll see it in the CI environment at http://ci.redis.io.
2017-04-28 10:40:44 +02:00
antirez b506eb74ac Check event loop creation return value. Fix #3951.
Normally we never check for OOM conditions inside Redis since the
allocator will always return a pointer or abort the program on OOM
conditons. However we cannot have control on epool_create(), that may
fail for kernel OOM (according to the manual page) even if all the
parameters are correct, so the function aeCreateEventLoop() may indeed
return NULL and this condition must be checked.
2017-04-28 10:40:44 +02:00
antirez 806905627c PSYNC2: fix master cleanup when caching it.
The master client cleanup was incomplete: resetClient() was missing and
the output buffer of the client was not reset, so pending commands
related to the previous connection could be still sent.

The first problem caused the client argument vector to be, at times,
half populated, so that when the correct replication stream arrived the
protcol got mixed to the arugments creating invalid commands that nobody
called.

Thanks to @yangsiran for also investigating this problem, after
already providing important design / implementation hints for the
original PSYNC2 issues (see referenced Github issue).

Note that this commit adds a new function to the list library of Redis
in order to be able to reset a list without destroying it.

Related to issue #3899.
2017-04-27 17:08:53 +02:00
antirez 8c4b0f411f Defrag: test currently disabled, too many false positives.
Related to #3786.
2017-04-22 16:00:16 +02:00
antirez 6839c759b8 Reformat 4.0 RC3 change log. 2017-04-22 13:49:41 +02:00
antirez 51b12ed1b5 Defrag: fix test false positive.
Apparently 1.4 is too low compared to what you get in certain setups
(including mine). I raised it to 1.55 that hopefully is still enough to
test that the fragmentation went down from 1.7 but without incurring in
issues, however the test setup may be still fragile so certain times this
may lead to false positives again, it's hard to test for these things
in a determinsitic way.

Related to #3786.
2017-04-22 13:23:27 +02:00
antirez 635bbe573a Redis 4.0.0-RC3 (3.9.103). 2017-04-22 13:16:41 +02:00
oranagraandantirez 94a7090705 add test for active defrag 2017-04-22 13:16:01 +02:00
antirez 1a7a532e96 Revert "Jemalloc updated to 4.4.0."
This reverts commit 36c1acc222.
2017-04-22 13:12:42 +02:00
antirez 6bc6bd4c38 PSYNC2: discard pending transactions from cached master.
During the review of the fix for #3899, @yangsiran identified an
implementation bug: given that the offset is now relative to the applied
part of the replication log, when we cache a master, the successive
PSYNC2 request will be made in order to *include* the transaction that
was not completely processed. This means that we need to discard any
pending transaction from our replication buffer: it will be re-executed.
2017-04-20 07:58:24 +02:00
antirez a91cc5bc2d Fix PSYNC2 incomplete command bug as described in #3899.
This bug was discovered by @kevinmcgehee and constituted a major hidden
bug in the PSYNC2 implementation, caused by the propagation from the
master of incomplete commands to slaves.

The bug had several results:

1. Borrowing from Kevin text in the issue: "Given that slaves blindly
copy over their master's input into their own replication backlog over
successive read syscalls, it's possible that with large commands or
small TCP buffers, partial commands are present in this buffer. If the
master were to fail before successfully propagating the entire command
to a slave, the slaves will never execute the partial command (since the
client is invalidated) but will copy it to replication backlog which may
relay those invalid bytes to its slaves on PSYNC2, corrupting the
backlog and possibly other valid commands that follow the failover.
Simple command boundaries aren't sufficient to capture this, either,
because in the case of a MULTI/EXEC block, if the master successfully
propagates a subset of the commands but not the EXEC, then the
transaction in the backlog becomes corrupt and could corrupt other
slaves that consume this data."

2. As identified by @yangsiran later, there is another effect of the
bug. For the same mechanism of the first problem, a slave having another
slave, could receive a full resynchronization request with an already
half-applied command in the backlog. Once the RDB is ready, it will be
sent to the slave, and the replication will continue sending to the
sub-slave the other half of the command, which is not valid.

The fix, designed by @yangsiran and @antirez, and implemented by
@antirez, uses a secondary buffer in order to feed the sub-masters and
update the replication backlog and offsets, only when a given part of
the query buffer is actually *applied* to the state of the instance,
that is, when the command gets processed and the command is not pending
in the Redis transaction buffer because of CLIENT_MULTI state.

Given that now the backlog and offsets representation are in agreement
with the actual processed commands, both issue 1 and 2 should no longer
be possible.

Thanks to @kevinmcgehee, @yangsiran and @oranagra for their work in
identifying and designing a fix for this problem.
2017-04-20 07:58:22 +02:00
antirez 278972ceb1 Fix getKeysUsingCommandTable() in cluster mode.
Close #3940.
2017-04-20 07:57:44 +02:00
张文康andantirez 2028501776 update block->free after some diff data are written to the child process 2017-04-20 07:57:44 +02:00
Jan-Erik Redigerandantirez 05ac217f2f Reorder to make dict-benchmark compile on Linux
Fixes #3944
2017-04-18 16:31:57 +02:00
antirez 8d44c52ae3 Fix #3848 by closing the descriptor on error. 2017-04-18 16:24:50 +02:00
antirez 5c107c627e Clarify why we save ziplist elements in revserse order.
Also get rid of variables that are now kinda redundant, since the
dictionary iterator was removed.

This is related to PR #3949.
2017-04-18 16:18:06 +02:00
spinlockandantirez 2299641417 rdb: saving skiplist in reversed order to accelerate the deserialisation process 2017-04-18 16:18:01 +02:00
antirez d98ef35a11 Cluster: discard pong times in the future.
However we allow for 500 milliseconds of tolerance, in order to
avoid often discarding semantically valid info (the node is up)
because of natural few milliseconds desync among servers even when
NTP is used.

Note that anyway we should ping the node from time to time regardless and
discover if it's actually down from our point of view, since no update
is accepted while we have an active ping on the node.

Related to #3929.
2017-04-18 16:17:45 +02:00
antirez e47c8e3f6a Test: fix, hopefully, false PSYNC failure like in issue #2715.
And many other related Github issues... all reporting the same problem.
There was probably just not enough backlog in certain unlucky runs.
I'll ask people that can reporduce if they see now this as fixed as
well.
2017-04-18 16:17:42 +02:00
antirez 1e659a04cf Cluster: always add PFAIL nodes at end of gossip section.
To rely on the fact that nodes in PFAIL state will be shared around by
randomly adding them in the gossip section is a weak assumption,
especially after changes related to sending less ping/pong packets.

We want to always include gossip entries for all the nodes that are in
PFAIL state, so that the PFAIL -> FAIL state promotion can happen much
faster and reliably.

Related to #3929.
2017-04-18 16:17:39 +02:00
antirez 39d3448703 Cluster: fix gossip section ping/pong times encoding.
The gossip section times are 32 bit, so cannot store the milliseconds
time but just the seconds approximation, which is good enough for our
uses. At the same time however, when comparing the gossip section times
of other nodes with our node's view, we need to convert back to
milliseconds.

Related to #3929. Without this change the patch to reduce the traffic in
the bus message does not work.
2017-04-18 16:17:34 +02:00
antirez 78148d0e5a Cluster: add clean-logs command to create-cluster script. 2017-04-18 16:17:31 +02:00
antirez a5c1c77eb8 Cluster: decrease ping/pong traffic by trusting other nodes reports.
Cluster of bigger sizes tend to have a lot of traffic in the cluster bus
just for failure detection: a node will try to get a ping reply from
another node no longer than when the half the node timeout would elapsed,
in order to avoid a false positive.

However this means that if we have N nodes and the node timeout is set
to, for instance M seconds, we'll have to ping N nodes every M/2
seconds. This N*M/2 pings will receive the same number of pongs, so
a total of N*M packets per node. However given that we have a total of N
nodes doing this, the total number of messages will be N*N*M.

In a 100 nodes cluster with a timeout of 60 seconds, this translates
to a total of 100*100*30 packets per second, summing all the packets
exchanged by all the nodes.

This is, as you can guess, a lot... So this patch changes the
implementation in a very simple way in order to trust the reports of
other nodes: if a node A reports a node B as alive at least up to
a given time, we update our view accordingly.

The problem with this approach is that it could result into a subset of
nodes being able to reach a given node X, and preventing others from
detecting that is actually not reachable from the majority of nodes.
So the above algorithm is refined by trusting other nodes only if we do
not have currently a ping pending for the node X, and if there are no
failure reports for that node.

Since each node, anyway, pings 10 other nodes every second (one node
every 100 milliseconds), anyway eventually even trusting the other nodes
reports, we will detect if a given node is down from our POV.

Now to understand the number of packets that the cluster would exchange
for failure detection with the patch, we can start considering the
random PINGs that the cluster sent anyway as base line:
Each node sends 10 packets per second, so the total traffic if no
additioal packets would be sent, including PONG packets, would be:

    Total messages per second = N*10*2

However by trusting other nodes gossip sections will not AWALYS prevent
pinging nodes for the "half timeout reached" rule all the times. The
math involved in computing the actual rate as N and M change is quite
complex and depends also on another parameter, which is the number of
entries in the gossip section of PING and PONG packets. However it is
possible to compare what happens in cluster of different sizes
experimentally. After applying this patch a very important reduction in
the number of packets exchanged is trivial to observe, without apparent
impacts on the failure detection performances.

Actual numbers with different cluster sizes should be published in the
Reids Cluster documentation in the future.

Related to #3929.
2017-04-18 16:17:29 +02:00
antirez 51901396ea Cluster: collect more specific bus messages stats.
First step in order to change Cluster in order to use less messages.
Related to issue #3929.
2017-04-18 16:17:26 +02:00
antirez f7b91b6c89 Add a top comment in crucial functions inside networking.c. 2017-04-18 16:17:13 +02:00
antirez 6e1489ae50 Set lua-time-limit default value at safe place.
Otherwise, as it was, it will overwrite whatever the user set.

Close #3703.
2017-04-18 16:17:11 +02:00
antirez 5fd841c069 Fix preprocessor if/else chain broken in order to fix #3927. 2017-04-18 16:17:08 +02:00
antirez 185b361aa8 Fix typo in feedReplicationBacklog() top comment. 2017-04-18 16:16:51 +02:00
lorneliandantirez b740fc1ee3 Expire: Update comment of activeExpireCycle function
The macro REDIS_EXPIRELOOKUPS_TIME_PERC has been replaced by
ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC in commit
6500fabfb8.
2017-04-18 16:16:51 +02:00
antirez 56cafcceac Fix zmalloc_get_memory_size() ifdefs to actually use the else branch.
Close #3927.
2017-04-18 16:16:28 +02:00
antirez a5b66da883 Make more obvious why there was issue #3843. 2017-04-18 16:16:27 +02:00
antirez f60d6f09ce Fix modules blocking commands awake delay.
If a thread unblocks a client blocked in a module command, by using the
RedisMdoule_UnblockClient() API, the event loop may not be awaken until
the next timeout of the multiplexing API or the next unrelated I/O
operation on other clients. We actually want the client to be served
ASAP, so a mechanism is needed in order for the unblocking API to inform
Redis that there is a client to serve ASAP.

This commit fixes the issue using the old trick of the pipe: when a
client needs to be unblocked, a byte is written in a pipe. When we run
the list of clients blocked in modules, we consume all the bytes
written in the pipe. Writes and reads are performed inside the context
of the mutex, so no race is possible in which we consume the bytes that
are actually related to an awake request for a client that should still
be put into the list of clients to unblock.

It was verified that after the fix the server handles the blocked
clients with the expected short delay.

Thanks to @dvirsky for understanding there was such a problem and
reporting it.
2017-04-18 16:16:27 +02:00
antirez c56668c89e Rax library updated. 2017-04-18 16:16:09 +02:00
antirez c4716d3345 Cluster: hash slots tracking using a radix tree. 2017-04-18 16:16:03 +02:00
viennaandantirez a9fefbce2e fix #3847: add close socket before return ANET_ERR. 2017-04-18 16:15:58 +02:00
Dvir Volkandantirez 17250409ba fixed free of blocked client before refering to it 2017-04-18 16:15:55 +02:00
Oran Agraandantirez 8aced9e9c5 add LFU policies to the test suite, just for coverage 2017-03-22 10:14:36 +01:00
antirez 3aa656abf5 Use sha256 instead of sha1 to generate tarball hashes. 2017-03-22 10:07:07 +01:00
Salvatore Sanfilippo 42d6a6c36f Makefile: fix building with Solaris C compiler, 64 bit. 2017-03-22 10:07:00 +01:00
Salvatore Sanfilippo e082d0569b Use ARM unaligned accesses ifdefs for SPARC as well. 2017-03-22 10:07:00 +01:00
Salvatore Sanfilippo 7269d5471c Fix BITPOS unaligned memory access. 2017-03-22 10:07:00 +01:00
antirez 1552058881 Solaris fixes about tail usage and atomic vars.
Testing with Solaris C compiler (SunOS 5.11 11.2 sun4v sparc sun4v)
there were issues compiling due to atomicvar.h and running the
tests also failed because of "tail" usage not conform with Solaris
tail implementation. This commit fixes both the issues.
2017-03-22 10:07:00 +01:00
antirez 9faeed04ac Test: replication-psync, wait more to detect write load.
Slow systems like the original Raspberry PI need more time
than 5 seconds to start the script and detect writes.
After fixing the Raspberry PI can pass the unit without issues.
2017-03-22 10:07:00 +01:00
antirez b3440b3559 Test: fix conditional execution of HINCRBYFLOAT representation test. 2017-03-22 10:06:44 +01:00
antirez 5a4133034e SipHash 2-4 -> SipHash 1-2.
For performance reasons we use a reduced rounds variant of
SipHash. This should still provide enough protection and the
effects in the hash table distribution are non existing.
If some real world attack on SipHash 1-2 will be found we can
trivially switch to something more secure. Anyway it is a
big step forward from Murmurhash, for which it is trivial to
generate *seed independent* colliding keys... The speed
penatly introduced by SipHash 2-4, around 4%, was a too big
price to pay compared to the effectiveness of the HashDoS
attack against SipHash 1-2, and considering so far in the
Redis history, no such an incident ever happened even while
using trivially to collide hash functions.
2017-02-21 17:16:35 +01:00
antirez a8cbc3ec87 freeMemoryIfNeeded(): improve code and lazyfree handling.
1. Refactor memory overhead computation into a function.
2. Every 10 keys evicted, check if memory usage already reached
   the target value directly, since we otherwise don't count all
   the memory reclaimed by the background thread right now.
2017-02-21 17:16:35 +01:00
antirez 857e6d5641 Use locale agnostic tolower() in dict.c hash function. 2017-02-21 17:16:35 +01:00
antirez 34387ceae3 SipHash x86 optimizations. 2017-02-21 17:16:35 +01:00
antirez ba647598b4 Use SipHash hash function to mitigate HashDos attempts.
This change attempts to switch to an hash function which mitigates
the effects of the HashDoS attack (denial of service attack trying
to force data structures to worst case behavior) while at the same time
providing Redis with an hash function that does not expect the input
data to be word aligned, a condition no longer true now that sds.c
strings have a varialbe length header.

Note that it is possible sometimes that even using an hash function
for which collisions cannot be generated without knowing the seed,
special implementation details or the exposure of the seed in an
indirect way (for example the ability to add elements to a Set and
check the return in which Redis returns them with SMEMBERS) may
make the attacker's life simpler in the process of trying to guess
the correct seed, however the next step would be to switch to a
log(N) data structure when too many items in a single bucket are
detected: this seems like an overkill in the case of Redis.

SPEED REGRESION TESTS:

In order to verify that switching from MurmurHash to SipHash had
no impact on speed, a set of benchmarks involving fast insertion
of 5 million of keys were performed.

The result shows Redis with SipHash in high pipelining conditions
to be about 4% slower compared to using the previous hash function.
However this could partially be related to the fact that the current
implementation does not attempt to hash whole words at a time but
reads single bytes, in order to have an output which is endian-netural
and at the same time working on systems where unaligned memory accesses
are a problem.

Further X86 specific optimizations should be tested, the function
may easily get at the same level of MurMurHash2 if a few optimizations
are performed.
2017-02-21 17:16:35 +01:00
Salvatore Sanfilippo 2ee19d9805 ARM: Avoid fast path for BITOP.
GCC will produce certain unaligned multi load-store instructions
that will be trapped by the Linux kernel since ARM v6 cannot
handle them with unaligned addresses. Better to use the slower
but safer implementation instead of generating the exception which
should be anyway very slow.
2017-02-21 17:16:35 +01:00
Salvatore Sanfilippo eb62cfeadb ARM: Use libc malloc by default.
I'm not sure how much test Jemalloc gets on ARM, moreover
compiling Redis with Jemalloc support in not very powerful
devices, like most ARMs people will build Redis on, is extremely
slow. It is possible to enable Jemalloc build anyway if needed
by using "make MALLOC=jemalloc".
2017-02-21 17:16:35 +01:00
Salvatore Sanfilippo 620e48b1d7 ARM: Avoid memcpy() in MurmurHash64A() if we are using 64 bit ARM.
However note that in architectures supporting 64 bit unaligned
accesses memcpy(...,...,8) is likely translated to a simple
word memory movement anyway.
2017-02-21 17:16:35 +01:00
Salvatore Sanfilippo 980d8805da ARM: Fix 64 bit unaligned access in MurmurHash64A(). 2017-02-21 17:16:35 +01:00
John.Koepiandantirez 522b10e4f6 fix #2883, #2857 pipe fds leak when fork() failed on bg aof rw 2017-02-20 10:28:49 +01:00
antirez 03f557223b Don't leak file descriptor on syncWithMaster().
Close #3804.
2017-02-20 10:28:49 +01:00
antirez 8d55aeb5de Fix MIGRATE closing of cached socket on error.
After investigating issue #3796, it was discovered that MIGRATE
could call migrateCloseSocket() after the original MIGRATE c->argv
was already rewritten as a DEL operation. As a result the host/port
passed to migrateCloseSocket() could be anything, often a NULL pointer
that gets deferenced crashing the server.

Now the socket is closed at an earlier time when there is a socket
error in a later stage where no retry will be performed, before we
rewrite the argument vector. Moreover a check was added so that later,
in the socket_err label, there is no further attempt at closing the
socket if the argument was rewritten.

This fix should resolve the bug reported in #3796.
2017-02-09 10:15:49 +01:00
antirez 7c22d76869 Fix ziplist fix... 2017-02-01 17:01:41 +01:00
antirez 8327b8136f Ziplist: insertion bug under particular conditions fixed.
Ziplists had a bug that was discovered while investigating a different
issue, resulting in a corrupted ziplist representation, and a likely
segmentation foult and/or data corruption of the last element of the
ziplist, once the ziplist is accessed again.

The bug happens when a specific set of insertions / deletions is
performed so that an entry is encoded to have a "prevlen" field (the
length of the previous entry) of 5 bytes but with a count that could be
encoded in a "prevlen" field of a since byte. This could happen when the
"cascading update" process called by ziplistInsert()/ziplistDelete() in
certain contitious forces the prevlen to be bigger than necessary in
order to avoid too much data moving around.

Once such an entry is generated, inserting a very small entry
immediately before it will result in a resizing of the ziplist for a
count smaller than the current ziplist length (which is a violation,
inserting code expects the ziplist to get bigger actually). So an FF
byte is inserted in a misplaced position. Moreover a realloc() is
performed with a count smaller than the ziplist current length so the
final bytes could be trashed as well.

SECURITY IMPLICATIONS:

Currently it looks like an attacker can only crash a Redis server by
providing specifically choosen commands. However a FF byte is written
and there are other memory operations that depend on a wrong count, so
even if it is not immediately apparent how to mount an attack in order
to execute code remotely, it is not impossible at all that this could be
done. Attacks always get better... and we did not spent enough time in
order to think how to exploit this issue, but security researchers
or malicious attackers could.
2017-02-01 15:03:06 +01:00
antirez 1688ccff26 ziplist: better comments, some refactoring. 2017-02-01 15:03:06 +01:00
antirez 36c1acc222 Jemalloc updated to 4.4.0.
The original jemalloc source tree was modified to:

1. Remove the configure error that prevents nested builds.
2. Insert the Redis private Jemalloc API in order to allow the
Redis fragmentation function to work.
2017-01-30 10:09:48 +01:00
Jan-Erik Redigerandantirez 37b4c954a9 Don't divide by zero
Previously Redis crashed on `MEMORY DOCTOR` when it has no slaves attached.

Fixes #3783
2017-01-30 10:09:45 +01:00
miterandantirez aee1ddca5d Change switch statment to if statment 2017-01-30 10:09:41 +01:00
oranagraandantirez af292b54a8 fix rare assertion in DEBUG DIGEST
getExpire calls dictFind which can do rehashing.
found by calling computeDatasetDigest from serverCron and running the test suite.
2017-01-30 10:09:36 +01:00
Itamar Haberandantirez c3c2aa3bbf Verify pairs are provided after subcommands
Fixes https://github.com/antirez/redis/issues/3639
2017-01-30 10:09:33 +01:00
antirez 7c2153dafa Add panic() into redisassert.h.
This header file is for libs, like ziplist.c, that we want to leave
almost separted from the core. The panic() calls will be easy to delete
in order to use such files outside, but the debugging info we gain are
very valuable compared to simple assertions where it is not possible to
print debugging info.
2017-01-27 10:49:25 +01:00
antirez dc83ddf068 serverPanic(): allow printf() alike formatting.
This is of great interest because allows us to print debugging
informations that could be of useful when debugging, like in the
following example:

    serverPanic("Unexpected encoding for object %d, %d",
        obj->type, obj->encoding);
2017-01-27 10:49:22 +01:00
antirez 3ef81eb301 Ziplist: remove static from functions, they prevent good crash reports. 2017-01-13 11:47:14 +01:00
Jan-Erik Redigerandantirez 96f75faac6 Initialize help only in repl mode 2017-01-13 11:34:55 +01:00
antirez bcd51a6acb Use const in modules types mem_usage method.
As suggested by @itamarhaber.
2017-01-13 09:07:37 +01:00
antirez 354ccf0ce9 Add memory defragmenting capability in 4.0 release notes. 2017-01-12 10:01:21 +01:00
antirez 57c81853b7 Defrag: don't crash when a module value is encountered. 2017-01-12 09:59:31 +01:00
antirez e36d52227c MEMORY USAGE: support for modules data types.
As a side effect of supporting it, we no longer crash when MEMORY USAGE
is called against a module data type.

Close #3637.
2017-01-12 09:59:31 +01:00
antirez 82ec0fe6fa Defrag: document the feature in redis.conf. 2017-01-12 09:59:31 +01:00
antirez 19bf0249d7 Defrag: not enabled by default. Error on CONFIG SET if not available. 2017-01-12 09:59:31 +01:00
antirez fa0d8b6204 Defrag: fix function name typo defarg -> defrag. 2017-01-12 09:59:31 +01:00
antirez ebb9a7e7e7 Defrag: do not crash on empty quicklist. 2017-01-12 09:59:31 +01:00
antirez da84b9c47a Defrag: fix comments & code to conform to the Redis code base.
Don't go over 80 cols. Start with captial letter, capital letter afer
point, end comment with a point and so forth. No actual code behavior
touched at all.
2017-01-12 09:59:31 +01:00
antirez a18f3cf389 Defrag: activate it only if running modified version of Jemalloc.
This commit also includes minor aesthetic changes like removal of
trailing spaces.
2017-01-12 09:59:31 +01:00
oranagraandantirez 1ad4883710 active defrag improvements 2017-01-12 09:59:31 +01:00
oranagraandantirez 67def2611f active memory defragmentation 2017-01-12 09:59:30 +01:00
antirez b4f3c5a499 deps/hiredis updated to latest version.
Close #3687.
2016-12-21 12:12:25 +01:00
antirez 6549c6cfaf Fix test "server is up" detection after logging changes. 2016-12-21 11:05:46 +01:00
Alexander Zhukovandantirez b87fd12075 Fix an article usage 2016-12-21 11:05:42 +01:00
whatacoldandantirez bd84549386 fix the wrong description of intsetGet(). 2016-12-21 11:05:38 +01:00
antirez 952e870696 4.0 release notes updated with API incompatibility notice about GEO. 2016-12-20 13:37:25 +01:00
antirez f3add0692f Geo: fuzzy test inconsistency report fixed to show all points.
We need to report all the points added into the set, not just the ones
matching the Tcl distance algo.
2016-12-20 13:33:46 +01:00
antirez 056c81e4a7 Geo: fix GEOHASH return value for consistency.
The same thing observed in #3551 by gnethercutt also fixed for
GEOHASH as the original PR did.
2016-12-20 13:33:46 +01:00
antirez d5036018b6 Geo: fix edge case return values for uniformity.
There were two cases outlined in issue #3512 and PR #3551 where
the Geo API returned unexpected results: empty strings where NULL
replies were expected, or a single null reply where an array was
expected. This violates the Redis principle that Redis replies for
existing keys or elements should be indistinguishable.

This is technically an API breakage so will be merged only into 4.0 and
specified in the changelog in the list of breaking compatibilities, even
if it is not very likely that actual code will be affected, hopefully,
since with the past behavior basically there was to acconut for *both*
the possibilities, and the new behavior is always one of the two, but
in a consistent way.
2016-12-20 13:33:46 +01:00
Justin Carvalhoandantirez 47b462538c Fix missing brackets around encoding variable in ZIP_DECODE_LENGTH macro 2016-12-20 13:33:46 +01:00
antirez a0e9511894 Remove first version of ASCII wave, later discarded. 2016-12-20 13:33:46 +01:00
antirez 3334a409b3 Only show Redis logo if logging to stdout / TTY.
You can still force the logo in the normal logs.
For motivations, check issue #3112. For me the reason is that actually
the logo is nice to have in interactive sessions, but inside the logs
kinda loses its usefulness, but for the ability of users to recognize
restarts easily: for this reason the new startup sequence shows a one
liner ASCII "wave" so that there is still a bit of visual clue.

Startup logging was modified in order to log events in more obvious
ways, and to log more events. Also certain important informations are
now more easy to parse/grep since they are printed in field=value style.

The option --always-show-logo in redis.conf was added, defaulting to no.
2016-12-20 13:33:45 +01:00
antirez db53c23037 adjustOpenFilesLimit() comment made hopefully more clear. 2016-12-19 08:54:46 +01:00
antirez bc00ef454c Hopefully improve code comments for issue #3616.
This commit also contains other changes in order to conform the code to
the Redis core style, specifically 80 chars max per line, smart
conditionals in the same line:

    if (that) do_this();
2016-12-19 08:54:46 +01:00
itamarandantirez 075a3381af Corrects a couple of omissions in the modules docs 2016-12-19 08:54:46 +01:00
andyliandantirez 8d82b3b166 Modify MIN->MAX 2016-12-19 08:54:46 +01:00
oranagraandantirez 69282df839 when a slave loads an RDB, stop an AOFRW fork before flusing db and parsing rdb file, to avoid a CoW disaster. 2016-12-19 08:54:46 +01:00
hylepoandantirez 869dda8494 Update redis-benchmark.c
Fixing typo in the usage of redis-benchmark
2016-12-19 08:54:46 +01:00
oranagraandantirez 7f870fadc2 fix unsigned int overflow in adjustOpenFilesLimit 2016-12-19 08:54:46 +01:00
antirez 2e375d4f42 Switch PFCOUNT to LogLog-Beta algorithm.
The new algorithm provides the same speed with a smaller error for
cardinalities in the range 0-100k. Before switching, the new and old
algorithm behavior was studied in details in the context of
issue #3677. You can find a few graphs and motivations there.
2016-12-16 11:07:42 +01:00
antirez 735b928b33 Use llroundl() before converting loglog-beta output to integer.
Otherwise for small cardinalities the algorithm will output something
like, for example, 4.99 for a candinality of 5, that will be converted
to 4 producing a huge error.
2016-12-16 11:07:42 +01:00
antirez 6cae609e8a Fix HLL gnuplot graph generator script for new redis-rb versions.
The PFADD now takes an array and has mandatory two arguments.
2016-12-16 11:07:42 +01:00
Harish Murthyandantirez 4d475e0f88 LogLog-Beta Algorithm support within HLL
Config option to use LogLog-Beta Algorithm for Cardinality
2016-12-16 11:07:42 +01:00
Dvir Volkandantirez 90d918bd6b fixed stop condition in RM_ZsetRangeNext and RM_ZsetRangePrev 2016-12-16 09:21:17 +01:00
antirez 3b19580ae8 ziplist.c explanation of format improved a bit. 2016-12-16 09:05:49 +01:00
antirez 457c6878b3 DEBUG: new "ziplist" subcommand added. Dumps a ziplist on stdout.
The commit improves ziplistRepr() and adds a new debugging subcommand so
that we can trigger the dump directly from the Redis API.
This command capability was used while investigating issue #3684.
2016-12-16 09:05:48 +01:00
antirez 17cda261a3 MIGRATE: Remove upfront ttl initialization.
After the fix for #3673 the ttl var is always initialized inside the
loop itself, so the early initialization is not needed.

Variables declaration also moved to a more local scope.
2016-12-14 12:44:35 +01:00
Jan-Erik Redigerandantirez 9515648d41 Reset the ttl for additional keys
Before, if a previous key had a TTL set but the current one didn't, the
TTL was reused and thus resulted in wrong expirations set.

This behaviour was experienced, when `MigrateDefaultPipeline` in
redis-trib was set to >1

Fixes #3655
2016-12-14 12:41:17 +01:00
antirez 1eec780e50 Writable slaves expires: unit test. 2016-12-13 18:34:04 +01:00
antirez 9a8bc6d22a Writable slaves expires: fix leak in key tracking.
We need to use a dictionary type that frees the key, since we copy the
keys in the dictionary we use to track expires created in the slave
side.
2016-12-13 18:34:04 +01:00
antirez 746d70b015 INFO: show num of slave-expires keys tracked. 2016-12-13 18:34:04 +01:00
antirez 1469c4edc1 Fix created->created typo in expire.c 2016-12-13 18:34:04 +01:00
antirez c65dfb436e Replication: fix the infamous key leakage of writable slaves + EXPIRE.
BACKGROUND AND USE CASEj

Redis slaves are normally write only, however the supprot a "writable"
mode which is very handy when scaling reads on slaves, that actually
need write operations in order to access data. For instance imagine
having slaves replicating certain Sets keys from the master. When
accessing the data on the slave, we want to peform intersections between
such Sets values. However we don't want to intersect each time: to cache
the intersection for some time often is a good idea.

To do so, it is possible to setup a slave as a writable slave, and
perform the intersection on the slave side, perhaps setting a TTL on the
resulting key so that it will expire after some time.

THE BUG

Problem: in order to have a consistent replication, expiring of keys in
Redis replication is up to the master, that synthesize DEL operations to
send in the replication stream. However slaves logically expire keys
by hiding them from read attempts from clients so that if the master did
not promptly sent a DEL, the client still see logically expired keys
as non existing.

Because slaves don't actively expire keys by actually evicting them but
just masking from the POV of read operations, if a key is created in a
writable slave, and an expire is set, the key will be leaked forever:

1. No DEL will be received from the master, which does not know about
such a key at all.

2. No eviction will be performed by the slave, since it needs to disable
eviction because it's up to masters, otherwise consistency of data is
lost.

THE FIX

In order to fix the problem, the slave should be able to tag keys that
were created in the slave side and have an expire set in some way.

My solution involved using an unique additional dictionary created by
the writable slave only if needed. The dictionary is obviously keyed by
the key name that we need to track: all the keys that are set with an
expire directly by a client writing to the slave are tracked.

The value in the dictionary is a bitmap of all the DBs where such a key
name need to be tracked, so that we can use a single dictionary to track
keys in all the DBs used by the slave (actually this limits the solution
to the first 64 DBs, but the default with Redis is to use 16 DBs).

This solution allows to pay both a small complexity and CPU penalty,
which is zero when the feature is not used, actually. The slave-side
eviction is encapsulated in code which is not coupled with the rest of
the Redis core, if not for the hook to track the keys.

TODO

I'm doing the first smoke tests to see if the feature works as expected:
so far so good. Unit tests should be added before merging into the
4.0 branch.
2016-12-13 18:34:04 +01:00
Yossi Gottliebandantirez 80944aac7f Fix redis-cli rare crash.
This happens if the server (mysteriously) returns an unexpected response
to the COMMAND command.
2016-12-12 19:37:19 +01:00
antirez 8226f2c3a7 Redis 4.0.0-RC2 (3.9.102). 2016-12-06 09:30:00 +01:00
wangshaonanandantirez 77241e86e1 Add '\n' to MEMORY DOCTOR command output message when num_reports
is 0 or empty is 1
2016-12-06 09:21:10 +01:00
Chris Lambandantirez 0ee6a23fc5 src/rdb.c: Correct "whenver" -> "whenever" typo. 2016-12-05 14:41:12 +01:00
Yossi Gottliebandantirez 2d0d2c8c6b Fix typo in RedisModuleTypeMethods declaration. 2016-12-05 14:41:07 +01:00
Dvir Volkandantirez 0fb9f341c9 fix memory corruption on RM_FreeCallReply 2016-12-05 14:40:59 +01:00
antirez 41994f2213 Geo: improve fuzz test.
The test now uses more diverse radius sizes, especially sizes near or
greater the whole earth surface are used, that are known to trigger edge
cases. Moreover the PRNG seeding was probably resulting into the same
sequence tested over and over again, now seeding unsing the current unix
time in milliseconds.

Related to #3631.
2016-12-05 14:19:11 +01:00
antirez ef9b4cf0f0 Geo: fix computation of bounding box.
A bug was reported in the context in issue #3631. The root cause of the
bug was that certain neighbor boxes were zeroed after the "inside the
bounding box or not" check, simply because the bounding box computation
function was wrong.

A few debugging infos where enhanced and moved in other parts of the
code. A check to avoid steps=0 was added, but is unrelated to this
issue and I did not verified it was an actual bug in practice.
2016-12-05 14:19:11 +01:00
antirez 2dd344d2e1 Redis 4.0.0-RC1 (3.9.101). 2016-12-02 16:36:02 +01:00
antirez 5b27e6c5c8 Modules: API doc updated (auto generated). 2016-12-02 16:35:24 +01:00
antirez a93baeafb5 Modules: types doc updated to new API. 2016-12-02 16:30:42 +01:00
1517 changed files with 74819 additions and 326060 deletions
-5
View File
@@ -1,5 +0,0 @@
[codespell]
quiet-level = 2
count =
skip = ./deps,./src/crc16_slottable.h,tmp*,./.git,./lcov-html
ignore-words = ./.codespell/wordlist.txt
-1
View File
@@ -1 +0,0 @@
codespell==2.2.5
-21
View File
@@ -1,21 +0,0 @@
ake
bale
fle
fo
gameboy
mutli
nd
nees
oll
optin
ot
smove
te
tre
cancelability
ist
statics
filetest
ro
exat
clen
-6
View File
@@ -1,6 +0,0 @@
# We set commands.c's merge driver to `binary` so when it conflicts during a
# merge git will leave the local version unmodified. This way our Makefile
# will rebuild it based on src/commands/*.json before trying to compile it.
# Otherwise the file gets modified and gets the same timestamp as the .json
# files. So the Makefile doesn't attempt to rebuild it before compiling.
src/commands.c merge=binary
-24
View File
@@ -1,24 +0,0 @@
---
name: Bug report
about: Help us improve Redis by reporting a bug
title: '[BUG]'
labels: ''
assignees: ''
---
**Describe the bug**
A short description of the bug.
**To reproduce**
Steps to reproduce the behavior and/or a minimal code sample.
**Expected behavior**
A description of what you expected to happen.
**Additional information**
Any additional information that is relevant to the problem.
-25
View File
@@ -1,25 +0,0 @@
---
name: Crash report
about: Submit a crash report
title: '[CRASH] <short description>'
labels: ''
assignees: ''
---
Notice!
- If a Redis module was involved, please open an issue in the module's repo instead!
- If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM!
**Crash report**
Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context.
```
```
**Additional information**
1. OS distribution and version
2. Steps to reproduce (if any)
-24
View File
@@ -1,24 +0,0 @@
---
name: Feature request
about: Suggest a feature for Redis
title: '[NEW]'
labels: ''
assignees: ''
---
**The problem/use-case that the feature addresses**
A description of the problem that the feature will solve, or the use-case with which the feature will be used.
**Description of the feature**
A description of what you want to happen.
**Alternatives you've considered**
Any alternative solutions or features you've considered, including references to existing open and closed feature requests in this repository.
**Additional information**
Any additional information that is relevant to the feature request.
-8
View File
@@ -1,8 +0,0 @@
---
name: Other
about: Can't find the right issue type? Use this one!
title: ''
labels: ''
assignees: ''
---
-21
View File
@@ -1,21 +0,0 @@
---
name: Question
about: Ask the Redis developers
title: '[QUESTION]'
labels: ''
assignees: ''
---
Please keep in mind that this issue tracker should be used for reporting bugs or proposing improvements to the Redis server.
Generally, questions about using Redis should be directed to the [community](https://redis.io/community):
* [the mailing list](https://groups.google.com/forum/#!forum/redis-db)
* [the `redis` tag at StackOverflow](http://stackoverflow.com/questions/tagged/redis)
* [/r/redis subreddit](http://www.reddit.com/r/redis)
* [github discussions](https://github.com/redis/redis/discussions)
It is also possible that your question was already asked here, so please do a quick issues search before submitting. Lastly, if your question is about one of Redis' [clients](https://redis.io/clients), you may to contact your client's developers for help.
That said, please feel free to replace all this with your question :)
-15
View File
@@ -1,15 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
- package-ecosystem: pip
directory: /.codespell
schedule:
interval: weekly
-102
View File
@@ -1,102 +0,0 @@
name: CI
on: [push, pull_request]
jobs:
test-ubuntu-latest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: make
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make REDIS_CFLAGS='-Werror' BUILD_TLS=yes
- name: test
run: |
sudo apt-get install tcl8.6 tclx
./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs
- name: validate commands.def up to date
run: |
touch src/commands/ping.json
make commands.def
dirty=$(git diff)
if [[ ! -z $dirty ]]; then echo $dirty; exit 1; fi
test-sanitizer-address:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: make
# build with TLS module just for compilation coverage
run: make SANITIZER=address REDIS_CFLAGS='-Werror -DDEBUG_ASSERTIONS' BUILD_TLS=module
- name: testprep
run: sudo apt-get install tcl8.6 tclx -y
- name: test
run: ./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs
build-debian-old:
runs-on: ubuntu-latest
container: debian:buster
steps:
- uses: actions/checkout@v4
- name: make
run: |
apt-get update && apt-get install -y build-essential
make REDIS_CFLAGS='-Werror'
build-macos-latest:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: make
run: make REDIS_CFLAGS='-Werror'
build-32bit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: make
run: |
sudo apt-get update && sudo apt-get install libc6-dev-i386 gcc-multilib g++-multilib
make REDIS_CFLAGS='-Werror' 32bit
build-libc-malloc:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: make
run: make REDIS_CFLAGS='-Werror' MALLOC=libc
build-centos-jemalloc:
runs-on: ubuntu-latest
container: quay.io/centos/centos:stream9
steps:
- uses: actions/checkout@v4
- name: make
run: |
dnf -y install which gcc gcc-c++ make
make REDIS_CFLAGS='-Werror'
build-old-chain-jemalloc:
runs-on: ubuntu-latest
container: ubuntu:20.04
steps:
- uses: actions/checkout@v4
- name: make
run: |
apt-get update
apt-get install -y gnupg2
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
apt-get update
apt-get install -y make gcc-4.8 g++-4.8
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 100
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 100
make CC=gcc REDIS_CFLAGS='-Werror'
-33
View File
@@ -1,33 +0,0 @@
name: "CodeQL"
on:
pull_request:
schedule:
# run weekly new vulnerability was added to the database
- cron: '0 0 * * 0'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'redis/redis'
strategy:
fail-fast: false
matrix:
language: [ 'cpp' ]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- name: Autobuild
uses: github/codeql-action/autobuild@v3
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
-32
View File
@@ -1,32 +0,0 @@
# Creates and uploads a Coverity build on a schedule
name: Coverity Scan
on:
schedule:
# Run once daily, since below 500k LOC can have 21 builds per week, per https://scan.coverity.com/faq#frequency
- cron: '0 0 * * *'
# Support manual execution
workflow_dispatch:
jobs:
coverity:
if: github.repository == 'redis/redis'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@main
- name: Download and extract the Coverity Build Tool
run: |
wget -q https://scan.coverity.com/download/cxx/linux64 --post-data "token=${{ secrets.COVERITY_SCAN_TOKEN }}&project=redis-unstable" -O cov-analysis-linux64.tar.gz
mkdir cov-analysis-linux64
tar xzf cov-analysis-linux64.tar.gz --strip 1 -C cov-analysis-linux64
- name: Install Redis dependencies
run: sudo apt install -y gcc tcl8.6 tclx procps libssl-dev
- name: Build with cov-build
run: cov-analysis-linux64/bin/cov-build --dir cov-int make
- name: Upload the result
run: |
tar czvf cov-int.tgz cov-int
curl \
--form project=redis-unstable \
--form email=${{ secrets.COVERITY_SCAN_EMAIL }} \
--form token=${{ secrets.COVERITY_SCAN_TOKEN }} \
--form file=@cov-int.tgz \
https://scan.coverity.com/builds
File diff suppressed because it is too large Load Diff
-85
View File
@@ -1,85 +0,0 @@
name: External Server Tests
on:
pull_request:
push:
schedule:
- cron: '0 0 * * *'
jobs:
test-external-standalone:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v4
- name: Build
run: make REDIS_CFLAGS=-Werror
- name: Start redis-server
run: |
./src/redis-server --daemonize yes --save "" --logfile external-redis.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Run external test
run: |
./runtest \
--host 127.0.0.1 --port 6379 \
--verbose \
--tags -slow
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-external-redis-log
path: external-redis.log
test-external-cluster:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v4
- name: Build
run: make REDIS_CFLAGS=-Werror
- name: Start redis-server
run: |
./src/redis-server --cluster-enabled yes --daemonize yes --save "" --logfile external-redis-cluster.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Create a single node cluster
run: ./src/redis-cli cluster addslots $(for slot in {0..16383}; do echo $slot; done); sleep 5
- name: Run external test
run: |
./runtest \
--host 127.0.0.1 --port 6379 \
--verbose \
--cluster-mode \
--tags -slow
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-external-cluster-log
path: external-redis-cluster.log
test-external-nodebug:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'redis/redis'
timeout-minutes: 14400
steps:
- uses: actions/checkout@v4
- name: Build
run: make REDIS_CFLAGS=-Werror
- name: Start redis-server
run: |
./src/redis-server --daemonize yes --save "" --logfile external-redis-nodebug.log
- name: Run external test
run: |
./runtest \
--host 127.0.0.1 --port 6379 \
--verbose \
--tags "-slow -needs:debug"
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-external-redis-nodebug-log
path: external-redis-nodebug.log
-35
View File
@@ -1,35 +0,0 @@
name: redis_docs_sync
on:
release:
types: [published]
jobs:
redis_docs_sync:
if: github.repository == 'redis/redis'
runs-on: ubuntu-latest
steps:
- name: Generate a token
id: generate-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.DOCS_APP_ID }}
private-key: ${{ secrets.DOCS_APP_PRIVATE_KEY }}
- name: Invoke workflow on redis/docs
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
RELEASE_NAME: ${{ github.event.release.tag_name }}
run: |
LATEST_RELEASE=$(
curl -Ls \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${GH_TOKEN}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/redis/redis/releases/latest \
| jq -r '.tag_name'
)
if [[ "${LATEST_RELEASE}" == "${RELEASE_NAME}" ]]; then
gh workflow run -R redis/docs redis_docs_sync.yaml -f release="${RELEASE_NAME}"
fi
@@ -1,22 +0,0 @@
name: Reply-schemas linter
on:
push:
paths:
- 'src/commands/*.json'
pull_request:
paths:
- 'src/commands/*.json'
jobs:
reply-schemas-linter:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup nodejs
uses: actions/setup-node@v4
- name: Install packages
run: npm install ajv
- name: linter
run: node ./utils/reply_schema_linter.js
-32
View File
@@ -1,32 +0,0 @@
# A CI action that using codespell to check spell.
# .github/.codespellrc is a config file.
# .github/wordlist.txt is a list of words that will ignore word checks.
# More details please check the following link:
# https://github.com/codespell-project/codespell
name: Spellcheck
on:
push:
pull_request:
jobs:
build:
name: Spellcheck
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: pip cache
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: ${{ runner.os }}-pip-
- name: Install prerequisites
run: sudo pip install -r ./.codespell/requirements.txt
- name: Spell check
run: codespell --config=./.codespell/.codespellrc
+1 -15
View File
@@ -1,8 +1,5 @@
.*.swp
*.o
*.xo
*.so
*.d
*.log
dump.rdb
redis-benchmark
@@ -16,8 +13,7 @@ doc-tools
release
misc/*
src/release.h
appendonly.aof*
appendonlydir
appendonly.aof
SHORT_TERM_TODO
release.h
src/transfer.sh
@@ -28,17 +24,7 @@ src/nodes.conf
deps/lua/src/lua
deps/lua/src/luac
deps/lua/src/liblua.a
deps/hdr_histogram/libhdrhistogram.a
deps/fpconv/libfpconv.a
deps/fast_float/libfast_float.a
tests/tls/*
.make-*
.prerequisites
*.dSYM
Makefile.dep
.vscode/*
.idea/*
.ccls
.ccls-cache/*
compile_commands.json
redis.code-workspace
+4794 -11
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1 +1 @@
Please check https://github.com/redis/redis/issues
Please check https://github.com/antirez/redis/issues
-96
View File
@@ -1,96 +0,0 @@
Contributor Covenant Code of Conduct
Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
this email address: redis@redis.io.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
1. Correction
Community Impact: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
Consequence: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
2. Warning
Community Impact: A violation through a single incident or series
of actions.
Consequence: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
3. Temporary Ban
Community Impact: A serious violation of community standards, including
sustained inappropriate behavior.
Consequence: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
4. Permanent Ban
Community Impact: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
Consequence: A permanent ban from any sort of public interaction within
the community.
Attribution
This Code of Conduct is adapted from the Contributor Covenant,
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by Mozilla's code of conduct
enforcement ladder.
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
+38
View File
@@ -0,0 +1,38 @@
Note: by contributing code to the Redis project in any form, including sending
a pull request via Github, a code fragment or patch via private email or
public discussion groups, you agree to release your code under the terms
of the BSD license that you can find in the COPYING file included in the Redis
source distribution. You will include BSD license in the COPYING file within
each source file that you contribute.
# IMPORTANT: HOW TO USE REDIS GITHUB ISSUES
* Github issues SHOULD ONLY BE USED to report bugs, and for DETAILED feature
requests. Everything else belongs to the Redis Google Group.
PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
bugs in the Github issues system. We'll be very happy to help you and provide
all the support at the Reddit sub:
http://reddit.com/r/redis
There is also an active community of Redis users at Stack Overflow:
http://stackoverflow.com/questions/tagged/redis
# How to provide a patch for a new feature
1. If it is a major feature or a semantical change, please post it as a new submission in r/redis on Reddit at http://reddit.com/r/redis. Try to be passionate about why the feature is needed, make users upvote your proposal to gain traction and so forth. Read feedbacks about the community. But in this first step **please don't write code yet**.
2. If in step 1 you get an acknowledgment from the project leaders, use the
following procedure to submit a patch:
a. Fork Redis on github ( http://help.github.com/fork-a-repo/ )
b. Create a topic branch (git checkout -b my_branch)
c. Push to your branch (git push origin my_branch)
d. Initiate a pull request on github ( http://help.github.com/send-pull-requests/ )
e. Done :)
For minor fixes just open a pull request on Github.
Thanks!
-118
View File
@@ -1,118 +0,0 @@
By contributing code to the Redis project in any form you agree to the Redis Software Grant and
Contributor License Agreement attached below. Only contributions made under the Redis Software Grant
and Contributor License Agreement may be accepted by Redis, and any contribution is subject to the
terms of the Redis dual-license under RSALv2/SSPLv1 as described in the LICENSE.txt file included in
the Redis source distribution.
# REDIS SOFTWARE GRANT AND CONTRIBUTOR LICENSE AGREEMENT
To specify the intellectual property license granted in any Contribution, Redis Ltd., ("**Redis**")
requires a Software Grant and Contributor License Agreement ("**Agreement**"). This Agreement is for
your protection as a contributor as well as the protection of Redis and its users; it does not
change your rights to use your own Contribution for any other purpose.
By making any Contribution, You accept and agree to the following terms and conditions for the
Contribution. Except for the license granted in this Agreement to Redis and the recipients of the
software distributed by Redis, You reserve all right, title, and interest in and to Your
Contribution.
1. **Definitions**
1.1. "**You**" (or "**Your**") means the copyright owner or legal entity authorized by the
copyright owner that is entering into this Agreement with Redis. For legal entities, the entity
making a Contribution and all other entities that Control, are Controlled by, or are under
common Control with that entity are considered to be a single contributor. For the purposes of
this definition, "**Control**" means (i) the power, direct or indirect, to cause the direction
or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty
percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
1.2. "**Contribution**" means the code, documentation, or any original work of authorship,
including any modifications or additions to an existing work described above.
2. "**Work**" means any software project stewarded by Redis.
3. **Grant of Copyright License**. Subject to the terms and conditions of this Agreement, You grant
to Redis and to the recipients of the software distributed by Redis a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare
derivative works of, publicly display, publicly perform, sublicense, and distribute Your
Contribution and such derivative works.
4. **Grant of Patent License**. Subject to the terms and conditions of this Agreement, You grant to
Redis and to the recipients of the software distributed by Redis a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent
license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable by You that are necessarily
infringed by Your Contribution alone or by a combination of Your Contribution with the Work to
which such Contribution was submitted. If any entity institutes patent litigation against You or
any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your
Contribution, or the Work to which you have contributed, constitutes a direct or contributory
patent infringement, then any patent licenses granted to the claimant entity under this Agreement
for that Contribution or Work terminate as of the date such litigation is filed.
5. **Representations and Warranties**. You represent and warrant that: (i) You are legally entitled
to grant the above licenses; and (ii) if You are an entity, each employee or agent designated by
You is authorized to submit the Contribution on behalf of You; and (iii) your Contribution is
Your original work, and that it will not infringe on any third party's intellectual property
right(s).
6. **Disclaimer**. You are not expected to provide support for Your Contribution, except to the
extent You desire to provide support. You may provide support for free, for a fee, or not at all.
Unless required by applicable law or agreed to in writing, You provide Your Contribution on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT,
MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
7. **Enforceability**. Nothing in this Agreement will be construed as creating any joint venture,
employment relationship, or partnership between You and Redis. If any provision of this Agreement
is held to be unenforceable, the remaining provisions of this Agreement will not be affected.
This represents the entire agreement between You and Redis relating to the Contribution.
# IMPORTANT: HOW TO USE REDIS GITHUB ISSUES
GitHub issues SHOULD ONLY BE USED to report bugs and for DETAILED feature
requests. Everything else should be asked on Discord:
https://discord.com/invite/redis
PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
bugs in the GitHub issues system. We'll be delighted to help you and provide
all the support on Discord.
There is also an active community of Redis users at Stack Overflow:
https://stackoverflow.com/questions/tagged/redis
Issues and pull requests for documentation belong on the redis-doc repo:
https://github.com/redis/redis-doc
If you are reporting a security bug or vulnerability, see SECURITY.md.
# How to provide a patch for a new feature
1. If it is a major feature or a semantical change, please don't start coding
straight away: if your feature is not a conceptual fit you'll lose a lot of
time writing the code without any reason. Start by posting in the mailing list
and creating an issue at Github with the description of, exactly, what you want
to accomplish and why. Use cases are important for features to be accepted.
Here you can see if there is consensus about your idea.
2. If in step 1 you get an acknowledgment from the project leaders, use the
following procedure to submit a patch:
a. Fork Redis on GitHub ( https://docs.github.com/en/github/getting-started-with-github/fork-a-repo )
b. Create a topic branch (git checkout -b my_branch)
c. Push to your branch (git push origin my_branch)
d. Initiate a pull request on GitHub ( https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request )
e. Done :)
3. Keep in mind that we are very overloaded, so issues and PRs sometimes wait
for a *very* long time. However this is not a lack of interest, as the project
gets more and more users, we find ourselves in a constant need to prioritize
certain issues/PRs over others. If you think your issue/PR is very important
try to popularize it, have other users commenting and sharing their point of
view, and so forth. This helps.
4. For minor fixes - open a pull request on GitHub.
Additional information on the RSALv2/SSPLv1 dual-license is also found in the LICENSE.txt file.
+10
View File
@@ -0,0 +1,10 @@
Copyright (c) 2006-2015, Salvatore Sanfilippo
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of Redis nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-733
View File
@@ -1,733 +0,0 @@
Starting on March 20th, 2024, Redis follows a dual-licensing model with all Redis project code
contributions under version 7.4 and subsequent releases governed by the Redis Software Grant and
Contributor License Agreement. After this date, contributions are subject to the user's choice of
the Redis Source Available License v2 (RSALv2) or the Server Side Public License v1 (SSPLv1), as
follows:
1. Redis Source Available License 2.0 (RSALv2) Agreement
========================================================
Last Update: December 30, 2023
Acceptance
----------
This Agreement sets forth the terms and conditions on which the Licensor
makes available the Software. By installing, downloading, accessing,
Using, or distributing any of the Software, You agree to all of the
terms and conditions of this Agreement.
If You are receiving the Software on behalf of Your Company, You
represent and warrant that You have the authority to agree to this
Agreement on behalf of such entity.
The Licensor reserves the right to update this Agreement from time to
time.
The terms below have the meanings set forth below for purposes of this
Agreement:
Definitions
-----------
Agreement: this Redis Source Available License 2.0 Agreement.
Control: ownership, directly or indirectly, of substantially all the
assets of an entity, or the power to direct its management and policies
by vote, contract, or otherwise.
License: the License as described in the License paragraph below.
Licensor: the entity offering these terms, which includes Redis Ltd. on
behalf of itself and its subsidiaries and affiliates worldwide.
Modify, Modified, or Modification: copy from or adapt all or part of the
work in a fashion requiring copyright permission other than making an
exact copy. The resulting work is called a Modified version of the
earlier work.
Redis: the Redis software as described in redis.com redis.io.
Software: certain Software components designed to work with Redis and
provided to You under this Agreement.
Trademark: the trademarks, service marks, and any other similar rights.
Use: anything You do with the Software requiring one of Your Licenses.
You: the recipient of the Software, the individual or entity on whose
behalf You are agreeing to this Agreement.
Your Company: any legal entity, sole proprietorship, or other kind of
organization that You work for, plus all organizations that have control
over, are under the control of, or are under common control with that
organization.
Your Licenses: means all the Licenses granted to You for the Software
under this Agreement.
License
-------
The Licensor grants You a non-exclusive, royalty-free, worldwide,
non-sublicensable, non-transferable license to use, copy, distribute,
make available, and prepare derivative works of the Software, in each
case subject to the limitations and conditions below.
Limitations
-----------
You may not make the functionality of the Software or a Modified version
available to third parties as a service or distribute the Software or a
Modified version in a manner that makes the functionality of the
Software available to third parties.
Making the functionality of the Software or Modified version available
to third parties includes, without limitation, enabling third parties to
interact with the functionality of the Software or Modified version in
distributed form or remotely through a computer network, offering a
product or service, the value of which entirely or primarily derives
from the value of the Software or Modified version, or offering a
product or service that accomplishes for users the primary purpose of
the Software or Modified version.
You may not alter, remove, or obscure any licensing, copyright, or other
notices of the Licensor in the Software. Any use of the Licensor's
Trademarks is subject to applicable law.
Patents
-------
The Licensor grants You a License, under any patent claims the Licensor
can License, or becomes able to License, to make, have made, use, sell,
offer for sale, import and have imported the Software, in each case
subject to the limitations and conditions in this License. This License
does not cover any patent claims that You cause to be infringed by
Modifications or additions to the Software. If You or Your Company make
any written claim that the Software infringes or contributes to
infringement of any patent, your patent License for the Software granted
under this Agreement ends immediately. If Your Company makes such a
claim, your patent License ends immediately for work on behalf of Your
Company.
Notices
-------
You must ensure that anyone who gets a copy of any part of the Software
from You also gets a copy of the terms and conditions in this Agreement.
If You modify the Software, You must include in any Modified copies of
the Software prominent notices stating that You have Modified the
Software.
No Other Rights
---------------
The terms and conditions of this Agreement do not imply any Licenses
other than those expressly granted in this Agreement.
Termination
-----------
If You Use the Software in violation of this Agreement, such Use is not
Licensed, and Your Licenses will automatically terminate. If the
Licensor provides You with a notice of your violation, and You cease all
violations of this License no later than 30 days after You receive that
notice, Your Licenses will be reinstated retroactively. However, if You
violate this Agreement after such reinstatement, any additional
violation of this Agreement will cause your Licenses to terminate
automatically and permanently.
No Liability
------------
As far as the law allows, the Software comes as is, without any
warranty or condition, and the Licensor will not be liable to You for
any damages arising out of this Agreement or the Use or nature of the
Software, under any kind of legal claim.
Governing Law and Jurisdiction
------------------------------
If You are located in Asia, Pacific, Americas, or other jurisdictions
not listed below, the Agreement will be construed and enforced in all
respects in accordance with the laws of the State of California, U.S.A.,
without reference to its choice of law rules. The courts located in the
County of Santa Clara, California, have exclusive jurisdiction for all
purposes relating to this Agreement.
If You are located in Israel, the Agreement will be construed and
enforced in all respects in accordance with the laws of the State of
Israel without reference to its choice of law rules. The courts located
in the Central District of the State of Israel have exclusive
jurisdiction for all purposes relating to this Agreement.
If You are located in Europe, United Kingdom, Middle East or Africa, the
Agreement will be construed and enforced in all respects in accordance
with the laws of England and Wales without reference to its choice of
law rules. The competent courts located in London, England, have
exclusive jurisdiction for all purposes relating to this Agreement.
2. Server Side Public License (SSPL)
====================================
Server Side Public License
VERSION 1, OCTOBER 16, 2018
Copyright (c) 2018 MongoDB, Inc.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to Server Side Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work in
a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based on
the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through a
computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices" to the
extent that it includes a convenient and prominently visible feature that
(1) displays an appropriate copyright notice, and (2) tells the user that
there is no warranty for the work (except to the extent that warranties
are provided), that licensees may convey the work under this License, and
how to view a copy of this License. If the interface presents a list of
user commands or options, such as a menu, a prominent item in the list
meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work for
making modifications to it. "Object code" means any non-source form of a
work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that is
widely used among developers working in that language. The "System
Libraries" of an executable work include anything, other than the work as
a whole, that (a) is included in the normal form of packaging a Major
Component, but which is not part of that Major Component, and (b) serves
only to enable use of the work with that Major Component, or to implement
a Standard Interface for which an implementation is available to the
public in source code form. A "Major Component", in this context, means a
major essential component (kernel, window system, and so on) of the
specific operating system (if any) on which the executable work runs, or
a compiler used to produce the work, or an object code interpreter used
to run it.
The "Corresponding Source" for a work in object code form means all the
source code needed to generate, install, and (for an executable work) run
the object code and to modify the work, including scripts to control
those activities. However, it does not include the work's System
Libraries, or general-purpose tools or generally available free programs
which are used unmodified in performing those activities but which are
not part of the work. For example, Corresponding Source includes
interface definition files associated with source files for the work, and
the source code for shared libraries and dynamically linked subprograms
that the work is specifically designed to require, such as by intimate
data communication or control flow between those subprograms and other
parts of the work.
The Corresponding Source need not include anything that users can
regenerate automatically from other parts of the Corresponding Source.
The Corresponding Source for a work in source code form is that same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program, subject to section 13. The
output from running a covered work is covered by this License only if the
output, given its content, constitutes a covered work. This License
acknowledges your rights of fair use or other equivalent, as provided by
copyright law. Subject to section 13, you may make, run and propagate
covered works that you do not convey, without conditions so long as your
license otherwise remains in force. You may convey covered works to
others for the sole purpose of having them make modifications exclusively
for you, or provide you with facilities for running those works, provided
that you comply with the terms of this License in conveying all
material for which you do not control copyright. Those thus making or
running the covered works for you must do so exclusively on your
behalf, under your direction and control, on terms that prohibit them
from making any copies of your copyrighted material outside their
relationship with you.
Conveying under any other circumstances is permitted solely under the
conditions stated below. Sublicensing is not allowed; section 10 makes it
unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article 11
of the WIPO copyright treaty adopted on 20 December 1996, or similar laws
prohibiting or restricting circumvention of such measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention is
effected by exercising rights under this License with respect to the
covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's users,
your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice; keep
intact all notices stating that this License and any non-permissive terms
added in accord with section 7 apply to the code; keep intact all notices
of the absence of any warranty; and give all recipients a copy of this
License along with the Program. You may charge any price or no price for
each copy that you convey, and you may offer support or warranty
protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the terms
of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified it,
and giving a relevant date.
b) The work must carry prominent notices stating that it is released
under this License and any conditions added under section 7. This
requirement modifies the requirement in section 4 to "keep intact all
notices".
c) You must license the entire work, as a whole, under this License to
anyone who comes into possession of a copy. This License will therefore
apply, along with any applicable section 7 additional terms, to the
whole of the work, and all its parts, regardless of how they are
packaged. This License gives no permission to license the work in any
other way, but it does not invalidate such permission if you have
separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your work
need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work, and
which are not combined with it such as to form a larger program, in or on
a volume of a storage or distribution medium, is called an "aggregate" if
the compilation and its resulting copyright are not used to limit the
access or legal rights of the compilation's users beyond what the
individual works permit. Inclusion of a covered work in an aggregate does
not cause this License to apply to the other parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms of
sections 4 and 5, provided that you also convey the machine-readable
Corresponding Source under the terms of this License, in one of these
ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium customarily
used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a written
offer, valid for at least three years and valid for as long as you
offer spare parts or customer support for that product model, to give
anyone who possesses the object code either (1) a copy of the
Corresponding Source for all the software in the product that is
covered by this License, on a durable physical medium customarily used
for software interchange, for a price no more than your reasonable cost
of physically performing this conveying of source, or (2) access to
copy the Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This alternative is
allowed only occasionally and noncommercially, and only if you received
the object code with such an offer, in accord with subsection 6b.
d) Convey the object code by offering access from a designated place
(gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to copy
the object code is a network server, the Corresponding Source may be on
a different server (operated by you or a third party) that supports
equivalent copying facilities, provided you maintain clear directions
next to the object code saying where to find the Corresponding Source.
Regardless of what server hosts the Corresponding Source, you remain
obligated to ensure that it is available for as long as needed to
satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided you
inform other peers where the object code and Corresponding Source of
the work are being offered to the general public at no charge under
subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be included
in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as part
of a transaction in which the right of possession and use of the User
Product is transferred to the recipient in perpetuity or for a fixed term
(regardless of how the transaction is characterized), the Corresponding
Source conveyed under this section must be accompanied by the
Installation Information. But this requirement does not apply if neither
you nor any third party retains the ability to install modified object
code on the User Product (for example, the work has been installed in
ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access
to a network may be denied when the modification itself materially
and adversely affects the operation of the network or violates the
rules and protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided, in
accord with this section must be in a format that is publicly documented
(and with an implementation available to the public in source code form),
and must require no special password or key for unpacking, reading or
copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall be
treated as though they were included in this License, to the extent that
they are valid under applicable law. If additional permissions apply only
to part of the Program, that part may be used separately under those
permissions, but the entire Program remains governed by this License
without regard to the additional permissions. When you convey a copy of
a covered work, you may at your option remove any additional permissions
from that copy, or from any part of it. (Additional permissions may be
written to require their own removal in certain cases when you modify the
work.) You may place additional permissions on material, added by you to
a covered work, for which you have or can give appropriate copyright
permission.
Notwithstanding any other provision of this License, for material you add
to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some trade
names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that material
by anyone who conveys the material (or modified versions of it) with
contractual assumptions of liability to the recipient, for any
liability that these contractual assumptions directly impose on those
licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further restriction,
you may remove that term. If a license document contains a further
restriction but permits relicensing or conveying under this License, you
may add to a covered work material governed by the terms of that license
document, provided that the further restriction does not survive such
relicensing or conveying.
If you add terms to a covered work in accord with this section, you must
place, in the relevant source files, a statement of the additional terms
that apply to those files, or a notice indicating where to find the
applicable terms. Additional terms, permissive or non-permissive, may be
stated in the form of a separately written license, or stated as
exceptions; the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or modify
it is void, and will automatically terminate your rights under this
License (including any patent licenses granted under the third paragraph
of section 11).
However, if you cease all violation of this License, then your license
from a particular copyright holder is reinstated (a) provisionally,
unless and until the copyright holder explicitly and finally terminates
your license, and (b) permanently, if the copyright holder fails to
notify you of the violation by some reasonable means prior to 60 days
after the cessation.
Moreover, your license from a particular copyright holder is reinstated
permanently if the copyright holder notifies you of the violation by some
reasonable means, this is the first time you have received notice of
violation of this License (for any work) from that copyright holder, and
you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or run a
copy of the Program. Ancillary propagation of a covered work occurring
solely as a consequence of using peer-to-peer transmission to receive a
copy likewise does not require acceptance. However, nothing other than
this License grants you permission to propagate or modify any covered
work. These actions infringe copyright if you do not accept this License.
Therefore, by modifying or propagating a covered work, you indicate your
acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically receives
a license from the original licensors, to run, modify and propagate that
work, subject to this License. You are not responsible for enforcing
compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered work
results from an entity transaction, each party to that transaction who
receives a copy of the work also receives whatever licenses to the work
the party's predecessor in interest had or could give under the previous
paragraph, plus a right to possession of the Corresponding Source of the
work from the predecessor in interest, if the predecessor has it or can
get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the rights
granted or affirmed under this License. For example, you may not impose a
license fee, royalty, or other charge for exercise of rights granted
under this License, and you may not initiate litigation (including a
cross-claim or counterclaim in a lawsuit) alleging that any patent claim
is infringed by making, using, selling, offering for sale, or importing
the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The work
thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims owned or
controlled by the contributor, whether already acquired or hereafter
acquired, that would be infringed by some manner, permitted by this
License, of making, using, or selling its contributor version, but do not
include claims that would be infringed only as a consequence of further
modification of the contributor version. For purposes of this definition,
"control" includes the right to grant patent sublicenses in a manner
consistent with the requirements of this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to make,
use, sell, offer for sale, import and otherwise run, modify and propagate
the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a party
means to make such an agreement or commitment not to enforce a patent
against the party.
If you convey a covered work, knowingly relying on a patent license, and
the Corresponding Source of the work is not available for anyone to copy,
free of charge and under the terms of this License, through a publicly
available network server or other readily accessible means, then you must
either (1) cause the Corresponding Source to be so available, or (2)
arrange to deprive yourself of the benefit of the patent license for this
particular work, or (3) arrange, in a manner consistent with the
requirements of this License, to extend the patent license to downstream
recipients. "Knowingly relying" means you have actual knowledge that, but
for the patent license, your conveying the covered work in a country, or
your recipient's use of the covered work in a country, would infringe
one or more identifiable patents in that country that you have reason
to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties receiving
the covered work authorizing them to use, propagate, modify or convey a
specific copy of the covered work, then the patent license you grant is
automatically extended to all recipients of the covered work and works
based on it.
A patent license is "discriminatory" if it does not include within the
scope of its coverage, prohibits the exercise of, or is conditioned on
the non-exercise of one or more of the rights that are specifically
granted under this License. You may not convey a covered work if you are
a party to an arrangement with a third party that is in the business of
distributing software, under which you make payment to the third party
based on the extent of your activity of conveying the work, and under
which the third party grants, to any of the parties who would receive the
covered work from you, a discriminatory patent license (a) in connection
with copies of the covered work conveyed by you (or copies made from
those copies), or (b) primarily for and in connection with specific
products or compilations that contain the covered work, unless you
entered into that arrangement, or that patent license was granted, prior
to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting any
implied license or other defenses to infringement that may otherwise be
available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot use,
propagate or convey a covered work so as to satisfy simultaneously your
obligations under this License and any other pertinent obligations, then
as a consequence you may not use, propagate or convey it at all. For
example, if you agree to terms that obligate you to collect a royalty for
further conveying from those to whom you convey the Program, the only way
you could satisfy both those terms and this License would be to refrain
entirely from conveying the Program.
13. Offering the Program as a Service.
If you make the functionality of the Program or a modified version
available to third parties as a service, you must make the Service Source
Code available via network download to everyone at no charge, under the
terms of this License. Making the functionality of the Program or
modified version available to third parties as a service includes,
without limitation, enabling third parties to interact with the
functionality of the Program or modified version remotely through a
computer network, offering a service the value of which entirely or
primarily derives from the value of the Program or modified version, or
offering a service that accomplishes for users the primary purpose of the
Program or modified version.
"Service Source Code" means the Corresponding Source for the Program or
the modified version, and the Corresponding Source for all programs that
you use to make the Program or modified version available as a service,
including, without limitation, management software, user interfaces,
application program interfaces, automation software, monitoring software,
backup software, storage software and hosting software, all such that a
user could run an instance of the service using the Service Source Code
you make available.
14. Revised Versions of this License.
MongoDB, Inc. may publish revised and/or new versions of the Server Side
Public License from time to time. Such new versions will be similar in
spirit to the present version, but may differ in detail to address new
problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies that a certain numbered version of the Server Side Public
License "or any later version" applies to it, you have the option of
following the terms and conditions either of that numbered version or of
any later version published by MongoDB, Inc. If the Program does not
specify a version number of the Server Side Public License, you may
choose any version ever published by MongoDB, Inc.
If the Program specifies that a proxy can decide which future versions of
the Server Side Public License can be used, that proxy's public statement
of acceptance of a version permanently authorizes you to choose that
version for the Program.
Later license versions may give you additional or different permissions.
However, no additional obligations are imposed on any author or copyright
holder as a result of your choosing to follow a later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING
ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF
THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO
LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU
OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided above
cannot be given local legal effect according to their terms, reviewing
courts shall apply local law that most closely approximates an absolute
waiver of all civil liability in connection with the Program, unless a
warranty or assumption of liability accompanies a copy of the Program in
return for a fee.
END OF TERMS AND CONDITIONS
+7 -46
View File
@@ -34,21 +34,7 @@ Redis Manifesto
so that the complexity is obvious and more complex operations can be
performed as the sum of the basic operations.
4 - We believe in code efficiency. Computers get faster and faster, yet we
believe that abusing computing capabilities is not wise: the amount of
operations you can do for a given amount of energy remains anyway a
significant parameter: it allows to do more with less computers and, at
the same time, having a smaller environmental impact. Similarly Redis is
able to "scale down" to smaller devices. It is perfectly usable in a
Raspberry Pi and other small ARM based computers. Faster code having
just the layers of abstractions that are really needed will also result,
often, in more predictable performances. We think likewise about memory
usage, one of the fundamental goals of the Redis project is to
incrementally build more and more memory efficient data structures, so that
problems that were not approachable in RAM in the past will be perfectly
fine to handle in the future.
5 - Code is like a poem; it's not just something we write to reach some
4 - Code is like a poem; it's not just something we write to reach some
practical result. Sometimes people that are far from the Redis philosophy
suggest using other code written by other authors (frequently in other
languages) in order to implement something Redis currently lacks. But to us
@@ -59,48 +45,23 @@ Redis Manifesto
when needed. At the same time, when writing the Redis story we're trying to
write smaller stories that will fit in to other code.
6 - We're against complexity. We believe designing systems is a fight against
5 - We're against complexity. We believe designing systems is a fight against
complexity. We'll accept to fight the complexity when it's worthwhile but
we'll try hard to recognize when a small feature is not worth 1000s of lines
of code. Most of the time the best way to fight complexity is by not
creating it at all. Complexity is also a form of lock-in: code that is
very hard to understand cannot be modified by users in an independent way
regardless of the license. One of the main Redis goals is to remain
understandable, enough for a single programmer to have a clear idea of how
it works in detail just reading the source code for a couple of weeks.
creating it at all.
7 - Threading is not a silver bullet. Instead of making Redis threaded we
believe on the idea of an efficient (mostly) single threaded Redis core.
Multiple of such cores, that may run in the same computer or may run
in multiple computers, are abstracted away as a single big system by
higher order protocols and features: Redis Cluster and the upcoming
Redis Proxy are our main goals. A shared nothing approach is not just
much simpler (see the previous point in this document), is also optimal
in NUMA systems. In the specific case of Redis it allows for each instance
to have a more limited amount of data, making the Redis persist-by-fork
approach more sounding. In the future we may explore parallelism only for
I/O, which is the low hanging fruit: minimal complexity could provide an
improved single process experience.
8 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
naturally into a distributed version of Redis and 2) a more complex API that
supports multi-key operations. Both are useful if used judiciously but
there's no way to make the more complex multi-keys API distributed in an
opaque way without violating our other principles. We don't want to provide
the illusion of something that will work magically when actually it can't in
all cases. Instead we'll provide commands to quickly migrate keys from one
instance to another to perform multi-key operations and expose the
trade-offs to the user.
instance to another to perform multi-key operations and expose the tradeoffs
to the user.
9 - We optimize for joy. We believe writing code is a lot of hard work, and the
7 - We optimize for joy. We believe writing code is a lot of hard work, and the
only way it can be worth is by enjoying it. When there is no longer joy in
writing code, the best thing to do is stop. To prevent this, we'll avoid
taking paths that will make Redis less of a joy to develop.
10 - All the above points are put together in what we call opportunistic
programming: trying to get the most for the user with minimal increases
in complexity (hanging fruits). Solve 95% of the problem with 5% of the
code when it is acceptable. Avoid a fixed schedule but follow the flow of
user requests, inspiration, Redis internal readiness for certain features
(sometimes many past changes reach a critical point making a previously
complex feature very easy to obtain).
+3 -8
View File
@@ -1,16 +1,11 @@
# Top level makefile, the real stuff is at ./src/Makefile and in ./modules/Makefile
SUBDIRS = src
ifeq ($(BUILD_WITH_MODULES), yes)
SUBDIRS += modules
endif
# Top level makefile, the real shit is at src/Makefile
default: all
.DEFAULT:
for dir in $(SUBDIRS); do $(MAKE) -C $$dir $@; done
cd src && $(MAKE) $@
install:
for dir in $(SUBDIRS); do $(MAKE) -C $$dir $@; done
cd src && $(MAKE) $@
.PHONY: install
+108 -187
View File
@@ -1,36 +1,27 @@
This README is just a fast *quick start* document. You can find more detailed documentation at [redis.io](https://redis.io).
What is Redis?
---
--------------
Redis is often referred to as a *data structures* server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a *server-client* model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
Redis is often referred as a *data structures* server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a *server-client* model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
Data structures implemented into Redis have a few special properties:
* Redis cares to store them on disk, even if they are always served and modified into the server memory. This means that Redis is fast, but that it is also non-volatile.
* The implementation of data structures emphasizes memory efficiency, so data structures inside Redis will likely use less memory compared to the same data structure modelled using a high-level programming language.
* Redis offers a number of features that are natural to find in a database, like replication, tunable levels of durability, clustering, and high availability.
* Redis cares to store them on disk, even if they are always served and modified into the server memory. This means that Redis is fast, but that is also non-volatile.
* Implementation of data structures stress on memory efficiency, so data structures inside Redis will likely use less memory compared to the same data structure modeled using an high level programming language.
* Redis offers a number of features that are natural to find in a database, like replication, tunable levels of durability, cluster, high availability.
Another good example is to think of Redis as a more complex version of memcached, where the operations are not just SETs and GETs, but operations that work with complex data types like Lists, Sets, ordered data structures, and so forth.
Another good example is to think of Redis as a more complex version of memcached, where the operations are not just SETs and GETs, but operations to work with complex data types like Lists, Sets, ordered data structures, and so forth.
If you want to know more, this is a list of selected starting points:
* Introduction to Redis data types. https://redis.io/topics/data-types-intro
* The full list of Redis commands. https://redis.io/commands
* There is much more inside the official Redis documentation. https://redis.io/documentation
What is Redis Community Edition?
---
Redis OSS was renamed Redis Community Edition (CE) with the v7.4 release.
Redis Ltd. also offers [Redis Software](https://redis.io/enterprise/), a self-managed software with additional compliance, reliability, and resiliency for enterprise scaling,
and [Redis Cloud](https://redis.io/cloud/), a fully managed service integrated with Google Cloud, Azure, and AWS for production-ready apps.
Read more about the differences between Redis Community Edition and Redis [here](https://redis.io/comparisons/oss-vs-enterprise/).
* Introduction to Redis data types. http://redis.io/topics/data-types-intro
* Try Redis directly inside your browser. http://try.redis.io
* The full list of Redis commands. http://redis.io/commands
* There is much more inside the Redis official documentation. http://redis.io/documentation
Building Redis
---
--------------
Redis can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
We support big endian and little endian architectures, and both 32 bit
@@ -38,27 +29,13 @@ and 64 bit systems.
It may compile on Solaris derived systems (for instance SmartOS) but our
support for this platform is *best effort* and Redis is not guaranteed to
work as well as in Linux, OSX, and \*BSD.
work as well as in Linux, OSX, and \*BSD there.
It is as simple as:
% make
To build with TLS support, you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu) and run:
% make BUILD_TLS=yes
To build with systemd support, you'll need systemd development libraries (such
as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS) and run:
% make USE_SYSTEMD=yes
To append a suffix to Redis program names, use:
% make PROG_SUFFIX="-alt"
You can build a 32 bit Redis binary using:
You can run a 32 bit Redis binary using:
% make 32bit
@@ -66,17 +43,10 @@ After building Redis, it is a good idea to test it using:
% make test
If TLS is built, running the tests with TLS enabled (you will need `tcl-tls`
installed):
% ./utils/gen-test-certs.sh
% ./runtest --tls
Fixing build problems with dependencies or cached build options
---
---------
Redis has some dependencies which are included in the `deps` directory.
Redis has some dependencies which are included into the `deps` directory.
`make` does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
@@ -84,9 +54,9 @@ When you update the source code with `git pull` or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
% make distclean
make distclean
This will clean: jemalloc, lua, hiredis, linenoise and other dependencies.
This will clean: jemalloc, lua, hiredis, linenoise.
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
@@ -94,7 +64,7 @@ those options are cached indefinitely until you issue a `make distclean`
command.
Fixing problems building 32 bit binaries
---
---------
If after building Redis with a 32 bit target you need to rebuild it
with a 64 bit target, or the other way around, you need to perform a
@@ -103,12 +73,12 @@ with a 64 bit target, or the other way around, you need to perform a
In case of build errors when trying to build a 32 bit binary of Redis, try
the following steps:
* Install the package libc6-dev-i386 (also try g++-multilib).
* Install the packages libc6-dev-i386 (also try g++-multilib).
* Try using the following command line instead of `make 32bit`:
`make CFLAGS="-m32 -march=native" LDFLAGS="-m32"`
Allocator
---
---------
Selecting a non-default memory allocator when building Redis is done by setting
the `MALLOC` environment variable. Redis is compiled and linked against libc
@@ -124,30 +94,18 @@ To compile against jemalloc on Mac OS X systems, use:
% make MALLOC=jemalloc
Monotonic clock
---
By default, Redis will build using the POSIX clock_gettime function as the
monotonic clock source. On most modern systems, the internal processor clock
can be used to improve performance. Cautions can be found here:
http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/
To build with support for the processor's internal instruction clock, use:
% make CFLAGS="-DUSE_PROCESSOR_CLOCK"
Verbose build
---
-------------
Redis will build with a user-friendly colorized output by default.
If you want to see a more verbose output, use the following:
Redis will build with a user friendly colorized output by default.
If you want to see a more verbose output use the following:
% make V=1
Running Redis
---
-------------
To run Redis with the default configuration, just type:
To run Redis with the default configuration just type:
% cd src
% ./redis-server
@@ -161,20 +119,14 @@ parameter (the path of the configuration file):
It is possible to alter the Redis configuration by passing parameters directly
as options using the command line. Examples:
% ./redis-server --port 9999 --replicaof 127.0.0.1 6379
% ./redis-server --port 9999 --slaveof 127.0.0.1 6379
% ./redis-server /etc/redis/6379.conf --loglevel debug
All the options in redis.conf are also supported as options using the command
line, with exactly the same name.
Running Redis with TLS
---
Please consult the [TLS.md](TLS.md) file for more information on
how to use Redis with TLS.
Playing with Redis
---
------------------
You can use redis-cli to play with Redis. Start a redis-server instance,
then in another terminal try the following:
@@ -193,29 +145,27 @@ then in another terminal try the following:
(integer) 2
redis>
You can find the list of all the available commands at https://redis.io/commands.
You can find the list of all the available commands at http://redis.io/commands.
Installing Redis
---
-----------------
In order to install Redis binaries into /usr/local/bin, just use:
In order to install Redis binaries into /usr/local/bin just use:
% make install
You can use `make PREFIX=/some/other/directory install` if you wish to use a
different destination.
`make install` will just install binaries in your system, but will not configure
Make install will just install binaries in your system, but will not configure
init scripts and configuration files in the appropriate place. This is not
needed if you just want to play a bit with Redis, but if you are installing
it the proper way for a production system, we have a script that does this
needed if you want just to play a bit with Redis, but if you are installing
it the proper way for a production system, we have a script doing this
for Ubuntu and Debian systems:
% cd utils
% ./install_server.sh
_Note_: `install_server.sh` will not work on Mac OSX; it is built for Linux only.
The script will ask you a few questions and will setup everything you need
to run Redis properly as a background daemon that will start again on
system reboots.
@@ -224,35 +174,24 @@ You'll be able to stop and start Redis using the script named
`/etc/init.d/redis_<portnumber>`, for instance `/etc/init.d/redis_6379`.
Code contributions
---
-----------------
By contributing code to the Redis project in any form, including sending a pull request via GitHub,
a code fragment or patch via private email or public discussion groups, you agree to release your
code under the terms of the [Redis Software Grant and Contributor License Agreement][1]. Redis software
contains contributions to the original Redis core project, which are owned by their contributors and
licensed under the 3BSD license. Any copy of that license in this repository applies only to those
contributions. Redis releases all Redis Community Edition versions from 7.4.x and thereafter under the
RSALv2/SSPL dual-license as described in the [LICENSE.txt][2] file included in the Redis Community Edition source distribution.
Note: by contributing code to the Redis project in any form, including sending
a pull request via Github, a code fragment or patch via private email or
public discussion groups, you agree to release your code under the terms
of the BSD license that you can find in the [COPYING][1] file included in the Redis
source distribution.
Please see the [CONTRIBUTING.md][1] file in this source distribution for more information. For
security bugs and vulnerabilities, please see [SECURITY.md][3].
Please see the [CONTRIBUTING][2] file in this source distribution for more
information.
[1]: https://github.com/redis/redis/blob/unstable/CONTRIBUTING.md
[2]: https://github.com/redis/redis/blob/unstable/LICENSE.txt
[3]: https://github.com/redis/redis/blob/unstable/SECURITY.md
Redis Trademarks
---
The purpose of a trademark is to identify the goods and services of a person or company without
causing confusion. As the registered owner of its name and logo, Redis accepts certain limited uses
of its trademarks but it has requirements that must be followed as described in its Trademark
Guidelines available at: https://redis.com/legal/trademark-guidelines/.
[1]: https://github.com/antirez/redis/blob/unstable/COPYING
[2]: https://github.com/antirez/redis/blob/unstable/CONTRIBUTING
Redis internals
===
If you are reading this README you are likely in front of a GitHub page
If you are reading this README you are likely in front of a Github page
or you just untarred the Redis distribution tar ball. In both the cases
you are basically one step away from the source code, so here we explain
the Redis source code layout, what is in each file as a general idea, the
@@ -268,7 +207,7 @@ Source code layout
The Redis root directory just contains this README, the Makefile which
calls the real Makefile inside the `src` directory and an example
configuration for Redis and Redis Sentinel. You can find a few shell
configuration for Redis and Sentinel. You can find a few shell
scripts that are used in order to execute the Redis, Redis Cluster and
Redis Sentinel unit tests, which are implemented inside the `tests`
directory.
@@ -277,7 +216,7 @@ Inside the root are the following important directories:
* `src`: contains the Redis implementation, written in C.
* `tests`: contains the unit tests, implemented in Tcl.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `redis/redis`.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `anitrez/redis`. An exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
There are a few more directories but they are not very important for our goals
here. We'll focus mostly on `src`, where the Redis implementation is contained,
@@ -287,8 +226,8 @@ of complexity incrementally.
Note: lately Redis was refactored quite a bit. Function names and file
names have been changed, so you may find that this documentation reflects the
`unstable` branch more closely. For instance, in Redis 3.0 the `server.c`
and `server.h` files were named `redis.c` and `redis.h`. However the overall
`unstable` branch more closely. For instance in Redis 3.0 the `server.c`
and `server.h` files were named to `redis.c` and `redis.h`. However the overall
structure is the same. Keep in mind that all the new developments and pull
requests should be performed against the `unstable` branch.
@@ -306,7 +245,7 @@ A few important fields in this structure are:
* `server.db` is an array of Redis databases, where data is stored.
* `server.commands` is the command table.
* `server.clients` is a linked list of clients connected to the server.
* `server.master` is a special client, the master, if the instance is a replica.
* `server.master` is a special client, the master, if the instance is a slave.
There are tons of other fields. Most fields are commented directly inside
the structure definition.
@@ -314,19 +253,19 @@ the structure definition.
Another important Redis data structure is the one defining a client.
In the past it was called `redisClient`, now just `client`. The structure
has many fields, here we'll just show the main ones:
```c
struct client {
int fd;
sds querybuf;
int argc;
robj **argv;
redisDb *db;
int flags;
list *reply;
// ... many other fields ...
char buf[PROTO_REPLY_CHUNK_BYTES];
}
```
struct client {
int fd;
sds querybuf;
int argc;
robj **argv;
redisDb *db;
int flags;
list *reply;
char buf[PROTO_REPLY_CHUNK_BYTES];
... many other fields ...
}
The client structure defines a *connected client*:
* The `fd` field is the client socket file descriptor.
@@ -338,17 +277,13 @@ As you can see in the client structure above, arguments in a command
are described as `robj` structures. The following is the full `robj`
structure, which defines a *Redis object*:
```c
struct redisObject {
unsigned type:4;
unsigned encoding:4;
unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
* LFU data (least significant 8 bits frequency
* and most significant 16 bits access time). */
int refcount;
void *ptr;
};
```
typedef struct redisObject {
unsigned type:4;
unsigned encoding:4;
unsigned lru:LRU_BITS; /* lru time (relative to server.lruclock) */
int refcount;
void *ptr;
} robj;
Basically this structure can represent all the basic Redis data types like
strings, lists, sets, sorted sets and so forth. The interesting thing is that
@@ -369,48 +304,42 @@ This is the entry point of the Redis server, where the `main()` function
is defined. The following are the most important steps in order to startup
the Redis server.
* `initServerConfig()` sets up the default values of the `server` structure.
* `initServerConfig()` setups the default values of the `server` structure.
* `initServer()` allocates the data structures needed to operate, setup the listening socket, and so forth.
* `aeMain()` starts the event loop which listens for new connections.
There are two special functions called periodically by the event loop:
1. `serverCron()` is called periodically (according to `server.hz` frequency), and performs tasks that must be performed from time to time, like checking for timed out clients.
1. `serverCron()` is called periodically (according to `server.hz` frequency), and performs tasks that must be performed from time to time, like checking for timedout clients.
2. `beforeSleep()` is called every time the event loop fired, Redis served a few requests, and is returning back into the event loop.
Inside server.c you can find code that handles other vital things of the Redis server:
* `call()` is used in order to call a given command in the context of a given client.
* `activeExpireCycle()` handles eviction of keys with a time to live set via the `EXPIRE` command.
* `performEvictions()` is called when a new write command should be performed but Redis is out of memory according to the `maxmemory` directive.
* `activeExpireCycle()` handles eviciton of keys with a time to live set via the `EXPIRE` command.
* `freeMemoryIfNeeded()` is called when a new write command should be performed but Redis is out of memory according to the `maxmemory` directive.
* The global variable `redisCommandTable` defines all the Redis commands, specifying the name of the command, the function implementing the command, the number of arguments required, and other properties of each command.
commands.c
---
This file is auto generated by utils/generate-command-code.py, the content is based on the JSON files in the src/commands folder.
These are meant to be the single source of truth about the Redis commands, and all the metadata about them.
These JSON files are not meant to be used by anyone directly, instead that metadata can be obtained via the `COMMAND` command.
networking.c
---
This file defines all the I/O functions with clients, masters and replicas
This file defines all the I/O functions with clients, masters and slaves
(which in Redis are just special clients):
* `createClient()` allocates and initializes a new client.
* The `addReply*()` family of functions are used by command implementations in order to append data to the client structure, that will be transmitted to the client as a reply for a given command executed.
* the `addReply*()` family of functions are used by commands implementations in order to append data to the client structure, that will be transmitted to the client as a reply for a given command executed.
* `writeToClient()` transmits the data pending in the output buffers to the client and is called by the *writable event handler* `sendReplyToClient()`.
* `readQueryFromClient()` is the *readable event handler* and accumulates data read from the client into the query buffer.
* `readQueryFromClient()` is the *readable event handler* and accumulates data from read from the client into the query buffer.
* `processInputBuffer()` is the entry point in order to parse the client query buffer according to the Redis protocol. Once commands are ready to be processed, it calls `processCommand()` which is defined inside `server.c` in order to actually execute the command.
* `freeClient()` deallocates, disconnects and removes a client.
aof.c and rdb.c
---
As you can guess from the names, these files implement the RDB and AOF
As you can guess from the names these files implement the RDB and AOF
persistence for Redis. Redis uses a persistence model based on the `fork()`
system call in order to create a process with the same (shared) memory
content of the main Redis process. This secondary process dumps the content
system call in order to create a thread with the same (shared) memory
content of the main Redis thread. This secondary thread dumps the content
of the memory on disk. This is used by `rdb.c` to create the snapshots
on disk and by `aof.c` in order to perform the AOF rewrite when the
append only file gets too big.
@@ -419,13 +348,13 @@ The implementation inside `aof.c` has additional functions in order to
implement an API that allows commands to append new commands into the AOF
file as clients execute them.
The `call()` function defined inside `server.c` is responsible for calling
The `call()` function defined inside `server.c` is responsible to call
the functions that in turn will write the commands into the AOF.
db.c
---
Certain Redis commands operate on specific data types; others are general.
Certain Redis commands operate on specific data types, others are general.
Examples of generic commands are `DEL` and `EXPIRE`. They operate on keys
and not on their values specifically. All those generic commands are
defined inside `db.c`.
@@ -433,13 +362,13 @@ defined inside `db.c`.
Moreover `db.c` implements an API in order to perform certain operations
on the Redis dataset without directly accessing the internal data structures.
The most important functions inside `db.c` which are used in many command
The most important functions inside `db.c` which are used in many commands
implementations are the following:
* `lookupKeyRead()` and `lookupKeyWrite()` are used in order to get a pointer to the value associated to a given key, or `NULL` if the key does not exist.
* `dbAdd()` and its higher level counterpart `setKey()` create a new key in a Redis database.
* `dbDelete()` removes a key and its associated value.
* `emptyData()` removes an entire single database or all the databases defined.
* `emptyDb()` removes an entire single database or all the databases defined.
The rest of the file implements the generic commands exposed to the client.
@@ -451,7 +380,7 @@ The `robj` structure defining Redis objects was already described. Inside
a basic level, like functions to allocate new objects, handle the reference
counting and so forth. Notable functions inside this file:
* `incrRefCount()` and `decrRefCount()` are used in order to increment or decrement an object reference count. When it drops to 0 the object is finally freed.
* `incrRefcount()` and `decrRefCount()` are used in order to increment or decrement an object reference count. When it drops to 0 the object is finally freed.
* `createObject()` allocates a new object. There are also specialized functions to allocate string objects having a specific content, like `createStringObjectFromLongLong()` and similar functions.
This file also implements the `OBJECT` command.
@@ -461,65 +390,57 @@ replication.c
This is one of the most complex files inside Redis, it is recommended to
approach it only after getting a bit familiar with the rest of the code base.
In this file there is the implementation of both the master and replica role
In this file there is the implementation of both the master and slave role
of Redis.
One of the most important functions inside this file is `replicationFeedSlaves()` that writes commands to the clients representing replica instances connected
to our master, so that the replicas can get the writes performed by the clients:
One of the most important functions inside this file is `replicationFeedSlaves()` that writes commands to the clients representing slave instances connected
to our master, so that the slaves can get the writes performed by the clients:
this way their data set will remain synchronized with the one in the master.
This file also implements both the `SYNC` and `PSYNC` commands that are
used in order to perform the first synchronization between masters and
replicas, or to continue the replication after a disconnection.
Script
---
The script unit is composed of 3 units:
* `script.c` - integration of scripts with Redis (commands execution, set replication/resp, ...)
* `script_lua.c` - responsible to execute Lua code, uses `script.c` to interact with Redis from within the Lua code.
* `function_lua.c` - contains the Lua engine implementation, uses `script_lua.c` to execute the Lua code.
* `functions.c` - contains Redis Functions implementation (`FUNCTION` command), uses `functions_lua.c` if the function it wants to invoke needs the Lua engine.
* `eval.c` - contains the `eval` implementation using `script_lua.c` to invoke the Lua code.
slaves, or to continue the replication after a disconnection.
Other C files
---
* `t_hash.c`, `t_list.c`, `t_set.c`, `t_string.c`, `t_zset.c` and `t_stream.c` contains the implementation of the Redis data types. They implement both an API to access a given data type, and the client command implementations for these data types.
* `t_hash.c`, `t_list.c`, `t_set.c`, `t_string.c` and `t_zset.c` contains the implementation of the Redis data types. They implement both an API to access a given data type, and the client commands implementations for these data types.
* `ae.c` implements the Redis event loop, it's a self contained library which is simple to read and understand.
* `sds.c` is the Redis string library, check https://github.com/antirez/sds for more information.
* `sds.c` is the Redis string library, check http://github.com/antirez/sds for more information.
* `anet.c` is a library to use POSIX networking in a simpler way compared to the raw interface exposed by the kernel.
* `dict.c` is an implementation of a non-blocking hash table which rehashes incrementally.
* `cluster.c` implements the Redis Cluster. Probably a good read only after being very familiar with the rest of the Redis code base. If you want to read `cluster.c` make sure to read the [Redis Cluster specification][4].
* `scripting.c` implements Lua scripting. It is completely self contained from the rest of the Redis implementation and is simple enough to understand if you are familar with the Lua API.
* `cluster.c` implements the Redis Cluster. Probably a good read only after being very familiar with the rest of the Redis code base. If you want to read `cluster.c` make sure to read the [Redis Cluster specification][3].
[4]: https://redis.io/topics/cluster-spec
[3]: http://redis.io/topics/cluster-spec
Anatomy of a Redis command
---
All the Redis commands are defined in the following way:
```c
void foobarCommand(client *c) {
printf("%s",c->argv[1]->ptr); /* Do something with the argument. */
addReply(c,shared.ok); /* Reply something to the client. */
}
```
void foobarCommand(client *c) {
printf("%s",c->argv[1]->ptr); /* Do something with the argument. */
addReply(c,shared.ok); /* Reply something to the client. */
}
The command function is referenced by a JSON file, together with its metadata, see `commands.c` described above for details.
The command flags are documented in the comment above the `struct redisCommand` in `server.h`.
For other details, please refer to the `COMMAND` command. https://redis.io/commands/command/
The command is then referenced inside `server.c` in the command table:
{"foobar",foobarCommand,2,"rtF",0,NULL,0,0,0,0,0},
In the above example `2` is the number of arguments the command takes,
while `"rtF"` are the command flags, as documented in the command table
top comment inside `server.c`.
After the command operates in some way, it returns a reply to the client,
usually using `addReply()` or a similar function defined inside `networking.c`.
There are tons of command implementations inside the Redis source code
that can serve as examples of actual commands implementations (e.g. pingCommand). Writing
a few toy commands can be a good exercise to get familiar with the code base.
There are tons of commands implementations inside th Redis source code
that can serve as examples of actual commands implementations. To write
a few toy commands can be a good exercise to familiarize with the code base.
There are also many other files not described here, but it is useless to
cover everything. We just want to help you with the first steps.
cover everything. We want to just help you with the first steps.
Eventually you'll find your way inside the Redis code base :-)
Enjoy!
-30
View File
@@ -1,30 +0,0 @@
Copyright (c) 2006-Present, Redis Ltd. and Contributors
All rights reserved.
Note: Continued Applicability of the BSD-3-Clause License
Despite the shift to the dual-licensing model with Redis Community Edition version 7.4 (RSALv2 or SSPLv1), portions of
Redis Community Edition remain available subject to the BSD-3-Clause License (BSD). See below for the full BSD
license:
Redistribution and use in source and binary forms, with or without modification, are permitted
provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions
and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions
and the following disclaimer in the documentation and/or other materials provided with the
distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse
or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-39
View File
@@ -1,39 +0,0 @@
# Security Policy
## Supported Versions
Redis is generally backward compatible with very few exceptions, so we
recommend users to always use the latest version to experience stability,
performance and security.
We generally backport security issues to a single previous major version,
unless this is not possible or feasible with a reasonable effort.
| Version | Supported |
| ------- | ------------------ |
| 7.4.x | :white_check_mark: |
| 7.2.x | :white_check_mark: |
| < 7.2.x | :x: |
| 6.2.x | :white_check_mark: |
| < 6.2.x | :x: |
## Reporting a Vulnerability
If you believe you've discovered a serious vulnerability, please contact the
Redis core team at redis@redis.io. We will evaluate your report and if
necessary issue a fix and an advisory. If the issue was previously undisclosed,
we'll also mention your name in the credits.
## Responsible Disclosure
In some cases, we may apply a responsible disclosure process to reported or
otherwise discovered vulnerabilities. We will usually do that for a critical
vulnerability, and only if we have a good reason to believe information about
it is not yet public.
This process involves providing an early notification about the vulnerability,
its impact and mitigations to a short list of vendors under a time-limited
embargo on public disclosure.
If you believe you should be on the list, please contact us and we will
consider your request based on the above criteria.
-104
View File
@@ -1,104 +0,0 @@
TLS Support
===========
Getting Started
---------------
### Building
To build with TLS support you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu).
To build TLS support as Redis built-in:
Run `make BUILD_TLS=yes`.
Or to build TLS as Redis module:
Run `make BUILD_TLS=module`.
Note that sentinel mode does not support TLS module.
### Tests
To run Redis test suite with TLS, you'll need TLS support for TCL (i.e.
`tcl-tls` package on Debian/Ubuntu).
1. Run `./utils/gen-test-certs.sh` to generate a root CA and a server
certificate.
2. Run `./runtest --tls` or `./runtest-cluster --tls` to run Redis and Redis
Cluster tests in TLS mode.
3. Run `./runtest --tls-module` or `./runtest-cluster --tls-module` to
run Redis and Redis cluster tests in TLS mode with Redis module.
### Running manually
To manually run a Redis server with TLS mode (assuming `gen-test-certs.sh` was
invoked so sample certificates/keys are available):
For TLS built-in mode:
./src/redis-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/redis.crt \
--tls-key-file ./tests/tls/redis.key \
--tls-ca-cert-file ./tests/tls/ca.crt
For TLS module mode:
./src/redis-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/redis.crt \
--tls-key-file ./tests/tls/redis.key \
--tls-ca-cert-file ./tests/tls/ca.crt \
--loadmodule src/redis-tls.so
To connect to this Redis server with `redis-cli`:
./src/redis-cli --tls \
--cert ./tests/tls/redis.crt \
--key ./tests/tls/redis.key \
--cacert ./tests/tls/ca.crt
This will disable TCP and enable TLS on port 6379. It's also possible to have
both TCP and TLS available, but you'll need to assign different ports.
To make a Replica connect to the master using TLS, use `--tls-replication yes`,
and to make Redis Cluster use TLS across nodes use `--tls-cluster yes`.
Connections
-----------
All socket operations now go through a connection abstraction layer that hides
I/O and read/write event handling from the caller.
**Multi-threading I/O is not currently supported for TLS**, as a TLS connection
needs to do its own manipulation of AE events which is not thread safe. The
solution is probably to manage independent AE loops for I/O threads and longer
term association of connections with threads. This may potentially improve
overall performance as well.
Sync IO for TLS is currently implemented in a hackish way, i.e. making the
socket blocking and configuring socket-level timeout. This means the timeout
value may not be so accurate, and there would be a lot of syscall overhead.
However I believe that getting rid of syncio completely in favor of pure async
work is probably a better move than trying to fix that. For replication it would
probably not be so hard. For cluster keys migration it might be more difficult,
but there are probably other good reasons to improve that part anyway.
To-Do List
----------
- [ ] redis-benchmark support. The current implementation is a mix of using
hiredis for parsing and basic networking (establishing connections), but
directly manipulating sockets for most actions. This will need to be cleaned
up for proper TLS support. The best approach is probably to migrate to hiredis
async mode.
- [ ] redis-cli `--slave` and `--rdb` support.
Multi-port
----------
Consider the implications of allowing TLS to be configured on a separate port,
making Redis listening on multiple ports:
1. Startup banner port notification
2. Proctitle
3. How slaves announce themselves
4. Cluster bus port calculation
+7 -49
View File
@@ -2,9 +2,6 @@
uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
LUA_DEBUG?=no
LUA_COVERAGE?=no
CCCOLOR="\033[34m"
LINKCOLOR="\033[34;1m"
SRCCOLOR="\033[33m"
@@ -40,20 +37,13 @@ distclean:
-(cd linenoise && $(MAKE) clean) > /dev/null || true
-(cd lua && $(MAKE) clean) > /dev/null || true
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
-(cd hdr_histogram && $(MAKE) clean) > /dev/null || true
-(cd fpconv && $(MAKE) clean) > /dev/null || true
-(cd fast_float && $(MAKE) clean) > /dev/null || true
-(rm -f .make-*)
.PHONY: distclean
ifneq (,$(filter $(BUILD_TLS),yes module))
HIREDIS_MAKE_FLAGS = USE_SSL=1
endif
hiredis: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd hiredis && $(MAKE) static $(HIREDIS_MAKE_FLAGS)
cd hiredis && $(MAKE) static
.PHONY: hiredis
@@ -63,46 +53,18 @@ linenoise: .make-prerequisites
.PHONY: linenoise
hdr_histogram: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd hdr_histogram && $(MAKE)
.PHONY: hdr_histogram
fpconv: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fpconv && $(MAKE)
.PHONY: fpconv
fast_float: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fast_float && $(MAKE) libfast_float
.PHONY: fast_float
ifeq ($(uname_S),SunOS)
# Make isinf() available
LUA_CFLAGS= -D__C99FEATURES__=1
endif
LUA_CFLAGS+= -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DREDIS_STATIC='' -DLUA_USE_MKSTEMP $(CFLAGS)
LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DREDIS_STATIC='' $(CFLAGS)
LUA_LDFLAGS+= $(LDFLAGS)
ifeq ($(LUA_DEBUG),yes)
LUA_CFLAGS+= -O0 -g -DLUA_USE_APICHECK
else
LUA_CFLAGS+= -O2
endif
ifeq ($(LUA_COVERAGE),yes)
LUA_CFLAGS += -fprofile-arcs -ftest-coverage
LUA_LDFLAGS += -fprofile-arcs -ftest-coverage
endif
# lua's Makefile defines AR="ar rcu", which is unusual, and makes it more
# challenging to cross-compile lua (and redis). These defines make it easier
# to fit redis into cross-compilation environments, which typically set AR.
AR=ar
ARFLAGS=rc
ARFLAGS=rcu
lua: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
@@ -110,16 +72,12 @@ lua: .make-prerequisites
.PHONY: lua
JEMALLOC_CFLAGS=$(CFLAGS)
JEMALLOC_LDFLAGS=$(LDFLAGS)
ifneq ($(DEB_HOST_GNU_TYPE),)
JEMALLOC_CONFIGURE_OPTS += --host=$(DEB_HOST_GNU_TYPE)
endif
JEMALLOC_CFLAGS= -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops $(CFLAGS)
JEMALLOC_LDFLAGS= $(LDFLAGS)
jemalloc: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd jemalloc && ./configure --disable-cxx --with-version=5.3.0-0-g0 --with-lg-quantum=3 --disable-cache-oblivious --with-jemalloc-prefix=je_ CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" $(JEMALLOC_CONFIGURE_OPTS)
cd jemalloc && $(MAKE) lib/libjemalloc.a
cd jemalloc && ./configure --with-lg-quantum=3 --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
.PHONY: jemalloc
+13 -53
View File
@@ -2,10 +2,10 @@ This directory contains all Redis dependencies, except for the libc that
should be provided by the operating system.
* **Jemalloc** is our memory allocator, used as replacement for libc malloc on Linux by default. It has good performances and excellent fragmentation behavior. This component is upgraded from time to time.
* **geohash-int** is inside the dependencies directory but is actually part of the Redis project, since it is our private fork (heavily modified) of a library initially developed for Ardb, which is in turn a fork of Redis.
* **hiredis** is the official C client library for Redis. It is used by redis-cli, redis-benchmark and Redis Sentinel. It is part of the Redis official ecosystem but is developed externally from the Redis repository, so we just upgrade it as needed.
* **linenoise** is a readline replacement. It is developed by the same authors of Redis but is managed as a separated project and updated as needed.
* **lua** is Lua 5.1 with minor changes for security and additional libraries.
* **hdr_histogram** Used for per-command latency tracking histograms.
How to upgrade the above dependencies
===
@@ -13,59 +13,27 @@ How to upgrade the above dependencies
Jemalloc
---
Jemalloc is modified with changes that allow us to implement the Redis
active defragmentation logic. However this feature of Redis is not mandatory
and Redis is able to understand if the Jemalloc version it is compiled
against supports such Redis-specific modifications. So in theory, if you
are not interested in the active defragmentation, you can replace Jemalloc
just following these steps:
Jemalloc is unmodified. We only change settings via the `configure` script of Jemalloc using the `--with-lg-quantum` option, setting it to the value of 3 instead of 4. This provides us with more size classes that better suit the Redis data structures, in order to gain memory efficiency.
So in order to upgrade jemalloc:
1. Remove the jemalloc directory.
2. Substitute it with the new jemalloc source tree.
3. Edit the Makefile located in the same directory as the README you are
reading, and change the --with-version in the Jemalloc configure script
options with the version you are using. This is required because otherwise
Jemalloc configuration script is broken and will not work nested in another
git repository.
However note that we change Jemalloc settings via the `configure` script of Jemalloc using the `--with-lg-quantum` option, setting it to the value of 3 instead of 4. This provides us with more size classes that better suit the Redis data structures, in order to gain memory efficiency.
Geohash
---
If you want to upgrade Jemalloc while also providing support for
active defragmentation, in addition to the above steps you need to perform
the following additional steps:
5. In Jemalloc tree, file `include/jemalloc/jemalloc_macros.h.in`, make sure
to add `#define JEMALLOC_FRAG_HINT`.
6. Implement the function `je_get_defrag_hint()` inside `src/jemalloc.c`. You
can see how it is implemented in the current Jemalloc source tree shipped
with Redis, and rewrite it according to the new Jemalloc internals, if they
changed, otherwise you could just copy the old implementation if you are
upgrading just to a similar version of Jemalloc.
#### Updating/upgrading jemalloc
The jemalloc directory is pulled as a subtree from the upstream jemalloc github repo. To update it you should run from the project root:
1. `git subtree pull --prefix deps/jemalloc https://github.com/jemalloc/jemalloc.git <version-tag> --squash`<br>
This should hopefully merge the local changes into the new version.
2. In case any conflicts arise (due to our changes) you'll need to resolve them and commit.
3. Reconfigure jemalloc:<br>
```sh
rm deps/jemalloc/VERSION deps/jemalloc/configure
cd deps/jemalloc
./autogen.sh --with-version=<version-tag>-0-g0
```
4. Update jemalloc's version in `deps/Makefile`: search for "`--with-version=<old-version-tag>-0-g0`" and update it accordingly.
5. Commit the changes (VERSION,configure,Makefile).
This is never upgraded since it's part of the Redis project. If there are changes to merge from Ardb there is the need to manually check differences, but at this point the source code is pretty different.
Hiredis
---
Hiredis is used by Sentinel, `redis-cli` and `redis-benchmark`. Like Redis, uses the SDS string library, but not necessarily the same version. In order to avoid conflicts, this version has all SDS identifiers prefixed by `hi`.
Hiredis uses the SDS string library, that must be the same version used inside Redis itself. Hiredis is also very critical for Sentinel. Historically Redis often used forked versions of hiredis in a way or the other. In order to upgrade it is adviced to take a lot of care:
1. `git subtree pull --prefix deps/hiredis https://github.com/redis/hiredis.git <version-tag> --squash`<br>
This should hopefully merge the local changes into the new version.
2. Conflicts will arise (due to our changes) you'll need to resolve them and commit.
1. Check with diff if hiredis API changed and what impact it could have in Redis.
2. Make sure thet the SDS library inside Hiredis and inside Redis are compatible.
3. After the upgrade, run the Redis Sentinel test.
4. Check manually that redis-cli and redis-benchmark behave as expecteed, since we have no tests for CLI utilities currently.
Linenoise
---
@@ -93,14 +61,6 @@ and our version:
1. Makefile is modified to allow a different compiler than GCC.
2. We have the implementation source code, and directly link to the following external libraries: `lua_cjson.o`, `lua_struct.o`, `lua_cmsgpack.o` and `lua_bit.o`.
3. There is a security fix in `ldo.c`, line 498: The check for `LUA_SIGNATURE[0]` is removed in order to avoid direct bytecode execution.
3. There is a security fix in `ldo.c`, line 498: The check for `LUA_SIGNATURE[0]` is removed in order toa void direct bytecode exectuion.
Hdr_Histogram
---
Updated source can be found here: https://github.com/HdrHistogram/HdrHistogram_c
We use a customized version based on master branch commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42.
1. Compare all changes under /hdr_histogram directory to upstream master commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42
2. Copy updated files from newer version onto files in /hdr_histogram.
3. Apply the changes from 1 above to the updated files.
-24
View File
@@ -1,24 +0,0 @@
# Fallback to gcc/g++ when $CC or $CXX is not in $PATH.
CC ?= gcc
CXX ?= g++
CFLAGS=-Wall -O3
# This avoids loosing the fastfloat specific compile flags when we override the CFLAGS via the main project
FASTFLOAT_CFLAGS=-std=c++11 -DFASTFLOAT_ALLOWS_LEADING_PLUS
LDFLAGS=
libfast_float: fast_float_strtod.o
$(AR) -r libfast_float.a fast_float_strtod.o
32bit: CFLAGS += -m32
32bit: LDFLAGS += -m32
32bit: libfast_float
fast_float_strtod.o: fast_float_strtod.cpp
$(CXX) $(CFLAGS) $(FASTFLOAT_CFLAGS) -c fast_float_strtod.cpp $(LDFLAGS)
clean:
rm -f *.o
rm -f *.a
rm -f *.h.gch
rm -rf *.dSYM
-21
View File
@@ -1,21 +0,0 @@
README for fast_float v6.1.4
----------------------------------------------
We're using the fast_float library[1] in our (compiled-in)
floating-point fast_float_strtod implementation for faster and more
portable parsing of 64 decimal strings.
The single file fast_float.h is an amalgamation of the entire library,
which can be (re)generated with the amalgamate.py script (from the
fast_float repository) via the command
```
git clone https://github.com/fastfloat/fast_float
cd fast_float
git checkout v6.1.4
python3 ./script/amalgamate.py --license=MIT \
> $REDIS_SRC/deps/fast_float/fast_float.h
```
[1]: https://github.com/fastfloat/fast_float
-3838
View File
File diff suppressed because it is too large Load Diff
-32
View File
@@ -1,32 +0,0 @@
#include "fast_float.h"
#include <iostream>
#include <string>
#include <system_error>
#include <cerrno>
/* Convert NPTR to a double using the fast_float library.
*
* This function behaves similarly to the standard strtod function, converting
* the initial portion of the string pointed to by `nptr` to a `double` value,
* using the fast_float library for high performance. If the conversion fails,
* errno is set to EINVAL error code.
*
* @param nptr A pointer to the null-terminated byte string to be interpreted.
* @param endptr A pointer to a pointer to character. If `endptr` is not NULL,
* it will point to the character after the last character used
* in the conversion.
* @return The converted value as a double. If no valid conversion could
* be performed, returns 0.0.
* If ENDPTR is not NULL, a pointer to the character after the last one used
* in the number is put in *ENDPTR. */
extern "C" double fast_float_strtod(const char *nptr, char **endptr) {
double result = 0.0;
auto answer = fast_float::from_chars(nptr, nptr + strlen(nptr), result);
if (answer.ec != std::errc()) {
errno = EINVAL; // Fallback to for other errors
}
if (endptr != NULL) {
*endptr = (char *)answer.ptr;
}
return result;
}
-15
View File
@@ -1,15 +0,0 @@
#ifndef __FAST_FLOAT_STRTOD_H__
#define __FAST_FLOAT_STRTOD_H__
#if defined(__cplusplus)
extern "C"
{
#endif
double fast_float_strtod(const char *in, char **out);
#if defined(__cplusplus)
}
#endif
#endif /* __FAST_FLOAT_STRTOD_H__ */
-23
View File
@@ -1,23 +0,0 @@
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
-27
View File
@@ -1,27 +0,0 @@
STD=
WARN= -Wall
OPT= -Os
R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
R_LDFLAGS= $(LDFLAGS)
DEBUG= -g
R_CC=$(CC) $(R_CFLAGS)
R_LD=$(CC) $(R_LDFLAGS)
AR= ar
ARFLAGS= rcs
libfpconv.a: fpconv_dtoa.o
$(AR) $(ARFLAGS) $@ $+
fpconv_dtoa.o: fpconv_dtoa.h fpconv_dtoa.c
.c.o:
$(R_CC) -c $<
clean:
rm -f *.o
rm -f *.a
-9
View File
@@ -1,9 +0,0 @@
libfpconv
----------------------------------------------
Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm[1].
This port contains a subset of the 'C' version of Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm available at [github.com/night-shift/fpconv](https://github.com/night-shift/fpconv)).
[1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
-373
View File
@@ -1,373 +0,0 @@
/* fpconv_dtoa.c -- floating point conversion utilities.
*
* Fast and accurate double to string conversion based on Florian Loitsch's
* Grisu-algorithm[1].
*
* [1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
* ----------------------------------------------------------------------------
*
* Copyright (c) 2013-2019, night-shift <as.smljk at gmail dot com>
* Copyright (c) 2009, Florian Loitsch < florian.loitsch at inria dot fr >
* All rights reserved.
*
* Boost Software License - Version 1.0 - August 17th, 2003
*
* Permission is hereby granted, free of charge, to any person or organization
* obtaining a copy of the software and accompanying documentation covered by
* this license (the "Software") to use, reproduce, display, distribute,
* execute, and transmit the Software, and to prepare derivative works of the
* Software, and to permit third-parties to whom the Software is furnished to
* do so, all subject to the following:
*
* The copyright notices in the Software and this entire statement, including
* the above license grant, this restriction and the following disclaimer,
* must be included in all copies of the Software, in whole or in part, and
* all derivative works of the Software, unless such copies or derivative
* works are solely in the form of machine-executable object code generated by
* a source language processor.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
* SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
* FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*/
#include "fpconv_dtoa.h"
#include "fpconv_powers.h"
#include <stdbool.h>
#include <string.h>
#define fracmask 0x000FFFFFFFFFFFFFU
#define expmask 0x7FF0000000000000U
#define hiddenbit 0x0010000000000000U
#define signmask 0x8000000000000000U
#define expbias (1023 + 52)
#define absv(n) ((n) < 0 ? -(n) : (n))
#define minv(a, b) ((a) < (b) ? (a) : (b))
static uint64_t tens[] = { 10000000000000000000U,
1000000000000000000U,
100000000000000000U,
10000000000000000U,
1000000000000000U,
100000000000000U,
10000000000000U,
1000000000000U,
100000000000U,
10000000000U,
1000000000U,
100000000U,
10000000U,
1000000U,
100000U,
10000U,
1000U,
100U,
10U,
1U };
static inline uint64_t get_dbits(double d) {
union
{
double dbl;
uint64_t i;
} dbl_bits = { d };
return dbl_bits.i;
}
static Fp build_fp(double d) {
uint64_t bits = get_dbits(d);
Fp fp;
fp.frac = bits & fracmask;
fp.exp = (bits & expmask) >> 52;
if (fp.exp) {
fp.frac += hiddenbit;
fp.exp -= expbias;
} else {
fp.exp = -expbias + 1;
}
return fp;
}
static void normalize(Fp *fp) {
while ((fp->frac & hiddenbit) == 0) {
fp->frac <<= 1;
fp->exp--;
}
int shift = 64 - 52 - 1;
fp->frac <<= shift;
fp->exp -= shift;
}
static void get_normalized_boundaries(Fp *fp, Fp *lower, Fp *upper) {
upper->frac = (fp->frac << 1) + 1;
upper->exp = fp->exp - 1;
while ((upper->frac & (hiddenbit << 1)) == 0) {
upper->frac <<= 1;
upper->exp--;
}
int u_shift = 64 - 52 - 2;
upper->frac <<= u_shift;
upper->exp = upper->exp - u_shift;
int l_shift = fp->frac == hiddenbit ? 2 : 1;
lower->frac = (fp->frac << l_shift) - 1;
lower->exp = fp->exp - l_shift;
lower->frac <<= lower->exp - upper->exp;
lower->exp = upper->exp;
}
static Fp multiply(Fp *a, Fp *b) {
const uint64_t lomask = 0x00000000FFFFFFFF;
uint64_t ah_bl = (a->frac >> 32) * (b->frac & lomask);
uint64_t al_bh = (a->frac & lomask) * (b->frac >> 32);
uint64_t al_bl = (a->frac & lomask) * (b->frac & lomask);
uint64_t ah_bh = (a->frac >> 32) * (b->frac >> 32);
uint64_t tmp = (ah_bl & lomask) + (al_bh & lomask) + (al_bl >> 32);
/* round up */
tmp += 1U << 31;
Fp fp = { ah_bh + (ah_bl >> 32) + (al_bh >> 32) + (tmp >> 32), a->exp + b->exp + 64 };
return fp;
}
static void round_digit(char *digits,
int ndigits,
uint64_t delta,
uint64_t rem,
uint64_t kappa,
uint64_t frac) {
while (rem < frac && delta - rem >= kappa &&
(rem + kappa < frac || frac - rem > rem + kappa - frac)) {
digits[ndigits - 1]--;
rem += kappa;
}
}
static int generate_digits(Fp *fp, Fp *upper, Fp *lower, char *digits, int *K) {
uint64_t wfrac = upper->frac - fp->frac;
uint64_t delta = upper->frac - lower->frac;
Fp one;
one.frac = 1ULL << -upper->exp;
one.exp = upper->exp;
uint64_t part1 = upper->frac >> -one.exp;
uint64_t part2 = upper->frac & (one.frac - 1);
int idx = 0, kappa = 10;
uint64_t *divp;
/* 1000000000 */
for (divp = tens + 10; kappa > 0; divp++) {
uint64_t div = *divp;
unsigned digit = part1 / div;
if (digit || idx) {
digits[idx++] = digit + '0';
}
part1 -= digit * div;
kappa--;
uint64_t tmp = (part1 << -one.exp) + part2;
if (tmp <= delta) {
*K += kappa;
round_digit(digits, idx, delta, tmp, div << -one.exp, wfrac);
return idx;
}
}
/* 10 */
uint64_t *unit = tens + 18;
while (true) {
part2 *= 10;
delta *= 10;
kappa--;
unsigned digit = part2 >> -one.exp;
if (digit || idx) {
digits[idx++] = digit + '0';
}
part2 &= one.frac - 1;
if (part2 < delta) {
*K += kappa;
round_digit(digits, idx, delta, part2, one.frac, wfrac * *unit);
return idx;
}
unit--;
}
}
static int grisu2(double d, char *digits, int *K) {
Fp w = build_fp(d);
Fp lower, upper;
get_normalized_boundaries(&w, &lower, &upper);
normalize(&w);
int k;
Fp cp = find_cachedpow10(upper.exp, &k);
w = multiply(&w, &cp);
upper = multiply(&upper, &cp);
lower = multiply(&lower, &cp);
lower.frac++;
upper.frac--;
*K = -k;
return generate_digits(&w, &upper, &lower, digits, K);
}
static int emit_digits(char *digits, int ndigits, char *dest, int K, bool neg) {
int exp = absv(K + ndigits - 1);
/* write plain integer */
if (K >= 0 && (exp < (ndigits + 7))) {
memcpy(dest, digits, ndigits);
memset(dest + ndigits, '0', K);
return ndigits + K;
}
/* write decimal w/o scientific notation */
if (K < 0 && (K > -7 || exp < 4)) {
int offset = ndigits - absv(K);
/* fp < 1.0 -> write leading zero */
if (offset <= 0) {
offset = -offset;
dest[0] = '0';
dest[1] = '.';
memset(dest + 2, '0', offset);
memcpy(dest + offset + 2, digits, ndigits);
return ndigits + 2 + offset;
/* fp > 1.0 */
} else {
memcpy(dest, digits, offset);
dest[offset] = '.';
memcpy(dest + offset + 1, digits + offset, ndigits - offset);
return ndigits + 1;
}
}
/* write decimal w/ scientific notation */
ndigits = minv(ndigits, 18 - neg);
int idx = 0;
dest[idx++] = digits[0];
if (ndigits > 1) {
dest[idx++] = '.';
memcpy(dest + idx, digits + 1, ndigits - 1);
idx += ndigits - 1;
}
dest[idx++] = 'e';
char sign = K + ndigits - 1 < 0 ? '-' : '+';
dest[idx++] = sign;
int cent = 0;
if (exp > 99) {
cent = exp / 100;
dest[idx++] = cent + '0';
exp -= cent * 100;
}
if (exp > 9) {
int dec = exp / 10;
dest[idx++] = dec + '0';
exp -= dec * 10;
} else if (cent) {
dest[idx++] = '0';
}
dest[idx++] = exp % 10 + '0';
return idx;
}
static int filter_special(double fp, char *dest) {
if (fp == 0.0) {
dest[0] = '0';
return 1;
}
uint64_t bits = get_dbits(fp);
bool nan = (bits & expmask) == expmask;
if (!nan) {
return 0;
}
if (bits & fracmask) {
dest[0] = 'n';
dest[1] = 'a';
dest[2] = 'n';
} else {
dest[0] = 'i';
dest[1] = 'n';
dest[2] = 'f';
}
return 3;
}
int fpconv_dtoa(double d, char dest[24]) {
char digits[18];
int str_len = 0;
bool neg = false;
if (get_dbits(d) & signmask) {
dest[0] = '-';
str_len++;
neg = true;
}
int spec = filter_special(d, dest + str_len);
if (spec) {
return str_len + spec;
}
int K = 0;
int ndigits = grisu2(d, digits, &K);
str_len += emit_digits(digits, ndigits, dest + str_len, K, neg);
return str_len;
}
-45
View File
@@ -1,45 +0,0 @@
/* fpconv_dtoa.h -- floating point conversion utilities.
*
* Fast and accurate double to string conversion based on Florian Loitsch's
* Grisu-algorithm[1].
*
* [1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
* ----------------------------------------------------------------------------
*
* Copyright (c) 2013-2019, night-shift <as.smljk at gmail dot com>
* Copyright (c) 2009, Florian Loitsch < florian.loitsch at inria dot fr >
* All rights reserved.
*
* Boost Software License - Version 1.0 - August 17th, 2003
*
* Permission is hereby granted, free of charge, to any person or organization
* obtaining a copy of the software and accompanying documentation covered by
* this license (the "Software") to use, reproduce, display, distribute,
* execute, and transmit the Software, and to prepare derivative works of the
* Software, and to permit third-parties to whom the Software is furnished to
* do so, all subject to the following:
*
* The copyright notices in the Software and this entire statement, including
* the above license grant, this restriction and the following disclaimer,
* must be included in all copies of the Software, in whole or in part, and
* all derivative works of the Software, unless such copies or derivative
* works are solely in the form of machine-executable object code generated by
* a source language processor.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
* SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
* FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*/
#ifndef FPCONV_DTOA_H
#define FPCONV_DTOA_H
int fpconv_dtoa(double fp, char dest[24]);
#endif
/* [1] http://florian.loitsch.com/publications/dtoa-pldi2010.pdf */
-133
View File
@@ -1,133 +0,0 @@
/* fpconv_powers.h -- floating point conversion utilities.
*
* Fast and accurate double to string conversion based on Florian Loitsch's
* Grisu-algorithm[1].
*
* [1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
* ----------------------------------------------------------------------------
*
* Copyright (c) 2021, Redis Labs
* Copyright (c) 2013-2019, night-shift <as.smljk at gmail dot com>
* Copyright (c) 2009, Florian Loitsch < florian.loitsch at inria dot fr >
* All rights reserved.
*
* Boost Software License - Version 1.0 - August 17th, 2003
*
* Permission is hereby granted, free of charge, to any person or organization
* obtaining a copy of the software and accompanying documentation covered by
* this license (the "Software") to use, reproduce, display, distribute,
* execute, and transmit the Software, and to prepare derivative works of the
* Software, and to permit third-parties to whom the Software is furnished to
* do so, all subject to the following:
*
* The copyright notices in the Software and this entire statement, including
* the above license grant, this restriction and the following disclaimer,
* must be included in all copies of the Software, in whole or in part, and
* all derivative works of the Software, unless such copies or derivative
* works are solely in the form of machine-executable object code generated by
* a source language processor.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
* SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
* FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*/
#include <stdint.h>
#define npowers 87
#define steppowers 8
#define firstpower -348 /* 10 ^ -348 */
#define expmax -32
#define expmin -60
typedef struct Fp {
uint64_t frac;
int exp;
} Fp;
static Fp powers_ten[] = {
{ 18054884314459144840U, -1220 }, { 13451937075301367670U, -1193 },
{ 10022474136428063862U, -1166 }, { 14934650266808366570U, -1140 },
{ 11127181549972568877U, -1113 }, { 16580792590934885855U, -1087 },
{ 12353653155963782858U, -1060 }, { 18408377700990114895U, -1034 },
{ 13715310171984221708U, -1007 }, { 10218702384817765436U, -980 },
{ 15227053142812498563U, -954 }, { 11345038669416679861U, -927 },
{ 16905424996341287883U, -901 }, { 12595523146049147757U, -874 },
{ 9384396036005875287U, -847 }, { 13983839803942852151U, -821 },
{ 10418772551374772303U, -794 }, { 15525180923007089351U, -768 },
{ 11567161174868858868U, -741 }, { 17236413322193710309U, -715 },
{ 12842128665889583758U, -688 }, { 9568131466127621947U, -661 },
{ 14257626930069360058U, -635 }, { 10622759856335341974U, -608 },
{ 15829145694278690180U, -582 }, { 11793632577567316726U, -555 },
{ 17573882009934360870U, -529 }, { 13093562431584567480U, -502 },
{ 9755464219737475723U, -475 }, { 14536774485912137811U, -449 },
{ 10830740992659433045U, -422 }, { 16139061738043178685U, -396 },
{ 12024538023802026127U, -369 }, { 17917957937422433684U, -343 },
{ 13349918974505688015U, -316 }, { 9946464728195732843U, -289 },
{ 14821387422376473014U, -263 }, { 11042794154864902060U, -236 },
{ 16455045573212060422U, -210 }, { 12259964326927110867U, -183 },
{ 18268770466636286478U, -157 }, { 13611294676837538539U, -130 },
{ 10141204801825835212U, -103 }, { 15111572745182864684U, -77 },
{ 11258999068426240000U, -50 }, { 16777216000000000000U, -24 },
{ 12500000000000000000U, 3 }, { 9313225746154785156U, 30 },
{ 13877787807814456755U, 56 }, { 10339757656912845936U, 83 },
{ 15407439555097886824U, 109 }, { 11479437019748901445U, 136 },
{ 17105694144590052135U, 162 }, { 12744735289059618216U, 189 },
{ 9495567745759798747U, 216 }, { 14149498560666738074U, 242 },
{ 10542197943230523224U, 269 }, { 15709099088952724970U, 295 },
{ 11704190886730495818U, 322 }, { 17440603504673385349U, 348 },
{ 12994262207056124023U, 375 }, { 9681479787123295682U, 402 },
{ 14426529090290212157U, 428 }, { 10748601772107342003U, 455 },
{ 16016664761464807395U, 481 }, { 11933345169920330789U, 508 },
{ 17782069995880619868U, 534 }, { 13248674568444952270U, 561 },
{ 9871031767461413346U, 588 }, { 14708983551653345445U, 614 },
{ 10959046745042015199U, 641 }, { 16330252207878254650U, 667 },
{ 12166986024289022870U, 694 }, { 18130221999122236476U, 720 },
{ 13508068024458167312U, 747 }, { 10064294952495520794U, 774 },
{ 14996968138956309548U, 800 }, { 11173611982879273257U, 827 },
{ 16649979327439178909U, 853 }, { 12405201291620119593U, 880 },
{ 9242595204427927429U, 907 }, { 13772540099066387757U, 933 },
{ 10261342003245940623U, 960 }, { 15290591125556738113U, 986 },
{ 11392378155556871081U, 1013 }, { 16975966327722178521U, 1039 },
{ 12648080533535911531U, 1066 }
};
/**
* Grisu needs a cache of precomputed powers-of-ten.
* This function, given an exponent and an integer k
* return the normalized floating-point approximation of the power of 10.
* @param exp
* @param k
* @return
*/
static Fp find_cachedpow10(int exp, int* k)
{
const double one_log_ten = 0.30102999566398114;
const int approx = -(exp + npowers) * one_log_ten;
int idx = (approx - firstpower) / steppowers;
while(1) {
int current = exp + powers_ten[idx].exp + 64;
if(current < expmin) {
idx++;
continue;
}
if(current > expmax) {
idx--;
continue;
}
*k = (firstpower + idx * steppowers);
return powers_ten[idx];
}
}
-121
View File
@@ -1,121 +0,0 @@
Creative Commons Legal Code
CC0 1.0 Universal
CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
HEREUNDER.
Statement of Purpose
The laws of most jurisdictions throughout the world automatically confer
exclusive Copyright and Related Rights (defined below) upon the creator
and subsequent owner(s) (each and all, an "owner") of an original work of
authorship and/or a database (each, a "Work").
Certain owners wish to permanently relinquish those rights to a Work for
the purpose of contributing to a commons of creative, cultural and
scientific works ("Commons") that the public can reliably and without fear
of later claims of infringement build upon, modify, incorporate in other
works, reuse and redistribute as freely as possible in any form whatsoever
and for any purposes, including without limitation commercial purposes.
These owners may contribute to the Commons to promote the ideal of a free
culture and the further production of creative, cultural and scientific
works, or to gain reputation or greater distribution for their Work in
part through the use and efforts of others.
For these and/or other purposes and motivations, and without any
expectation of additional consideration or compensation, the person
associating CC0 with a Work (the "Affirmer"), to the extent that he or she
is an owner of Copyright and Related Rights in the Work, voluntarily
elects to apply CC0 to the Work and publicly distribute the Work under its
terms, with knowledge of his or her Copyright and Related Rights in the
Work and the meaning and intended legal effect of CC0 on those rights.
1. Copyright and Related Rights. A Work made available under CC0 may be
protected by copyright and related or neighboring rights ("Copyright and
Related Rights"). Copyright and Related Rights include, but are not
limited to, the following:
i. the right to reproduce, adapt, distribute, perform, display,
communicate, and translate a Work;
ii. moral rights retained by the original author(s) and/or performer(s);
iii. publicity and privacy rights pertaining to a person's image or
likeness depicted in a Work;
iv. rights protecting against unfair competition in regards to a Work,
subject to the limitations in paragraph 4(a), below;
v. rights protecting the extraction, dissemination, use and reuse of data
in a Work;
vi. database rights (such as those arising under Directive 96/9/EC of the
European Parliament and of the Council of 11 March 1996 on the legal
protection of databases, and under any national implementation
thereof, including any amended or successor version of such
directive); and
vii. other similar, equivalent or corresponding rights throughout the
world based on applicable law or treaty, and any national
implementations thereof.
2. Waiver. To the greatest extent permitted by, but not in contravention
of, applicable law, Affirmer hereby overtly, fully, permanently,
irrevocably and unconditionally waives, abandons, and surrenders all of
Affirmer's Copyright and Related Rights and associated claims and causes
of action, whether now known or unknown (including existing as well as
future claims and causes of action), in the Work (i) in all territories
worldwide, (ii) for the maximum duration provided by applicable law or
treaty (including future time extensions), (iii) in any current or future
medium and for any number of copies, and (iv) for any purpose whatsoever,
including without limitation commercial, advertising or promotional
purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
member of the public at large and to the detriment of Affirmer's heirs and
successors, fully intending that such Waiver shall not be subject to
revocation, rescission, cancellation, termination, or any other legal or
equitable action to disrupt the quiet enjoyment of the Work by the public
as contemplated by Affirmer's express Statement of Purpose.
3. Public License Fallback. Should any part of the Waiver for any reason
be judged legally invalid or ineffective under applicable law, then the
Waiver shall be preserved to the maximum extent permitted taking into
account Affirmer's express Statement of Purpose. In addition, to the
extent the Waiver is so judged Affirmer hereby grants to each affected
person a royalty-free, non transferable, non sublicensable, non exclusive,
irrevocable and unconditional license to exercise Affirmer's Copyright and
Related Rights in the Work (i) in all territories worldwide, (ii) for the
maximum duration provided by applicable law or treaty (including future
time extensions), (iii) in any current or future medium and for any number
of copies, and (iv) for any purpose whatsoever, including without
limitation commercial, advertising or promotional purposes (the
"License"). The License shall be deemed effective as of the date CC0 was
applied by Affirmer to the Work. Should any part of the License for any
reason be judged legally invalid or ineffective under applicable law, such
partial invalidity or ineffectiveness shall not invalidate the remainder
of the License, and in such case Affirmer hereby affirms that he or she
will not (i) exercise any of his or her remaining Copyright and Related
Rights in the Work or (ii) assert any associated claims and causes of
action with respect to the Work, in either case contrary to Affirmer's
express Statement of Purpose.
4. Limitations and Disclaimers.
a. No trademark or patent rights held by Affirmer are waived, abandoned,
surrendered, licensed or otherwise affected by this document.
b. Affirmer offers the Work as-is and makes no representations or
warranties of any kind concerning the Work, express, implied,
statutory or otherwise, including without limitation warranties of
title, merchantability, fitness for a particular purpose, non
infringement, or the absence of latent or other defects, accuracy, or
the present or absence of errors, whether or not discoverable, all to
the greatest extent permissible under applicable law.
c. Affirmer disclaims responsibility for clearing rights of other persons
that may apply to the Work or any use thereof, including without
limitation any person's Copyright and Related Rights in the Work.
Further, Affirmer disclaims responsibility for obtaining any necessary
consents, permissions or other rights required for any use of the
Work.
d. Affirmer understands and acknowledges that Creative Commons is not a
party to this document and has no duty or obligation with respect to
this CC0 or use of the Work.
-41
View File
@@ -1,41 +0,0 @@
The code in this repository code was Written by Gil Tene, Michael Barker,
and Matt Warren, and released to the public domain, as explained at
http://creativecommons.org/publicdomain/zero/1.0/
For users of this code who wish to consume it under the "BSD" license
rather than under the public domain or CC0 contribution text mentioned
above, the code found under this directory is *also* provided under the
following license (commonly referred to as the BSD 2-Clause License). This
license does not detract from the above stated release of the code into
the public domain, and simply represents an additional license granted by
the Author.
-----------------------------------------------------------------------------
** Beginning of "BSD 2-Clause License" text. **
Copyright (c) 2012, 2013, 2014 Gil Tene
Copyright (c) 2014 Michael Barker
Copyright (c) 2014 Matt Warren
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.
-27
View File
@@ -1,27 +0,0 @@
STD= -std=c99
WARN= -Wall
OPT= -Os
R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) -DHDR_MALLOC_INCLUDE=\"hdr_redis_malloc.h\"
R_LDFLAGS= $(LDFLAGS)
DEBUG= -g
R_CC=$(CC) $(R_CFLAGS)
R_LD=$(CC) $(R_LDFLAGS)
AR= ar
ARFLAGS= rcs
libhdrhistogram.a: hdr_histogram.o
$(AR) $(ARFLAGS) $@ $+
hdr_histogram.o: hdr_histogram.h hdr_histogram.c
.c.o:
$(R_CC) -c $<
clean:
rm -f *.o
rm -f *.a
-78
View File
@@ -1,78 +0,0 @@
HdrHistogram_c: 'C' port of High Dynamic Range (HDR) Histogram
HdrHistogram
----------------------------------------------
[![Gitter chat](https://badges.gitter.im/HdrHistogram/HdrHistogram.png)](https://gitter.im/HdrHistogram/HdrHistogram)
This port contains a subset of the functionality supported by the Java
implementation. The current supported features are:
* Standard histogram with 64 bit counts (32/16 bit counts not supported)
* All iterator types (all values, recorded, percentiles, linear, logarithmic)
* Histogram serialisation (encoding version 1.2, decoding 1.0-1.2)
* Reader/writer phaser and interval recorder
Features not supported, but planned
* Auto-resizing of histograms
Features unlikely to be implemented
* Double histograms
* Atomic/Concurrent histograms
* 16/32 bit histograms
# Simple Tutorial
## Recording values
```C
#include <hdr_histogram.h>
struct hdr_histogram* histogram;
// Initialise the histogram
hdr_init(
1, // Minimum value
INT64_C(3600000000), // Maximum value
3, // Number of significant figures
&histogram) // Pointer to initialise
// Record value
hdr_record_value(
histogram, // Histogram to record to
value) // Value to record
// Record value n times
hdr_record_values(
histogram, // Histogram to record to
value, // Value to record
10) // Record value 10 times
// Record value with correction for co-ordinated omission.
hdr_record_corrected_value(
histogram, // Histogram to record to
value, // Value to record
1000) // Record with expected interval of 1000.
// Print out the values of the histogram
hdr_percentiles_print(
histogram,
stdout, // File to write to
5, // Granularity of printed values
1.0, // Multiplier for results
CLASSIC); // Format CLASSIC/CSV supported.
```
## More examples
For more detailed examples of recording and logging results look at the
[hdr_decoder](examples/hdr_decoder.c)
and [hiccup](examples/hiccup.c)
examples. You can run hiccup and decoder
and pipe the results of one into the other.
```
$ ./examples/hiccup | ./examples/hdr_decoder
```
-146
View File
@@ -1,146 +0,0 @@
/**
* hdr_atomic.h
* Written by Philip Orwig and released to the public domain,
* as explained at http://creativecommons.org/publicdomain/zero/1.0/
*/
#ifndef HDR_ATOMIC_H__
#define HDR_ATOMIC_H__
#if defined(_MSC_VER)
#include <stdint.h>
#include <intrin.h>
#include <stdbool.h>
static void __inline * hdr_atomic_load_pointer(void** pointer)
{
_ReadBarrier();
return *pointer;
}
static void hdr_atomic_store_pointer(void** pointer, void* value)
{
_WriteBarrier();
*pointer = value;
}
static int64_t __inline hdr_atomic_load_64(int64_t* field)
{
_ReadBarrier();
return *field;
}
static void __inline hdr_atomic_store_64(int64_t* field, int64_t value)
{
_WriteBarrier();
*field = value;
}
static int64_t __inline hdr_atomic_exchange_64(volatile int64_t* field, int64_t value)
{
#if defined(_WIN64)
return _InterlockedExchange64(field, value);
#else
int64_t comparand;
int64_t initial_value = *field;
do
{
comparand = initial_value;
initial_value = _InterlockedCompareExchange64(field, value, comparand);
}
while (comparand != initial_value);
return initial_value;
#endif
}
static int64_t __inline hdr_atomic_add_fetch_64(volatile int64_t* field, int64_t value)
{
#if defined(_WIN64)
return _InterlockedExchangeAdd64(field, value) + value;
#else
int64_t comparand;
int64_t initial_value = *field;
do
{
comparand = initial_value;
initial_value = _InterlockedCompareExchange64(field, comparand + value, comparand);
}
while (comparand != initial_value);
return initial_value + value;
#endif
}
static bool __inline hdr_atomic_compare_exchange_64(volatile int64_t* field, int64_t* expected, int64_t desired)
{
return *expected == _InterlockedCompareExchange64(field, desired, *expected);
}
#elif defined(__ATOMIC_SEQ_CST)
#define hdr_atomic_load_pointer(x) __atomic_load_n(x, __ATOMIC_SEQ_CST)
#define hdr_atomic_store_pointer(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST)
#define hdr_atomic_load_64(x) __atomic_load_n(x, __ATOMIC_SEQ_CST)
#define hdr_atomic_store_64(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST)
#define hdr_atomic_exchange_64(f,i) __atomic_exchange_n(f,i, __ATOMIC_SEQ_CST)
#define hdr_atomic_add_fetch_64(field, value) __atomic_add_fetch(field, value, __ATOMIC_SEQ_CST)
#define hdr_atomic_compare_exchange_64(field, expected, desired) __atomic_compare_exchange_n(field, expected, desired, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)
#elif defined(__x86_64__)
#include <stdint.h>
#include <stdbool.h>
static inline void* hdr_atomic_load_pointer(void** pointer)
{
void* p = *pointer;
asm volatile ("" ::: "memory");
return p;
}
static inline void hdr_atomic_store_pointer(void** pointer, void* value)
{
asm volatile ("lock; xchgq %0, %1" : "+q" (value), "+m" (*pointer));
}
static inline int64_t hdr_atomic_load_64(int64_t* field)
{
int64_t i = *field;
asm volatile ("" ::: "memory");
return i;
}
static inline void hdr_atomic_store_64(int64_t* field, int64_t value)
{
asm volatile ("lock; xchgq %0, %1" : "+q" (value), "+m" (*field));
}
static inline int64_t hdr_atomic_exchange_64(volatile int64_t* field, int64_t value)
{
int64_t result = 0;
asm volatile ("lock; xchgq %1, %2" : "=r" (result), "+q" (value), "+m" (*field));
return result;
}
static inline int64_t hdr_atomic_add_fetch_64(volatile int64_t* field, int64_t value)
{
return __sync_add_and_fetch(field, value);
}
static inline bool hdr_atomic_compare_exchange_64(volatile int64_t* field, int64_t* expected, int64_t desired)
{
int64_t original;
asm volatile( "lock; cmpxchgq %2, %1" : "=a"(original), "+m"(*field) : "q"(desired), "0"(*expected));
return original == *expected;
}
#else
#error "Unable to determine atomic operations for your platform"
#endif
#endif /* HDR_ATOMIC_H__ */
-1229
View File
File diff suppressed because it is too large Load Diff
-521
View File
@@ -1,521 +0,0 @@
/**
* hdr_histogram.h
* Written by Michael Barker and released to the public domain,
* as explained at http://creativecommons.org/publicdomain/zero/1.0/
*
* The source for the hdr_histogram utilises a few C99 constructs, specifically
* the use of stdint/stdbool and inline variable declaration.
*/
#ifndef HDR_HISTOGRAM_H
#define HDR_HISTOGRAM_H 1
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
struct hdr_histogram
{
int64_t lowest_discernible_value;
int64_t highest_trackable_value;
int32_t unit_magnitude;
int32_t significant_figures;
int32_t sub_bucket_half_count_magnitude;
int32_t sub_bucket_half_count;
int64_t sub_bucket_mask;
int32_t sub_bucket_count;
int32_t bucket_count;
int64_t min_value;
int64_t max_value;
int32_t normalizing_index_offset;
double conversion_ratio;
int32_t counts_len;
int64_t total_count;
int64_t* counts;
};
#ifdef __cplusplus
extern "C" {
#endif
/**
* Allocate the memory and initialise the hdr_histogram.
*
* Due to the size of the histogram being the result of some reasonably
* involved math on the input parameters this function it is tricky to stack allocate.
* The histogram should be released with hdr_close
*
* @param lowest_discernible_value The smallest possible value that is distinguishable from 0.
* Must be a positive integer that is >= 1. May be internally rounded down to nearest power of 2.
* @param highest_trackable_value The largest possible value to be put into the
* histogram.
* @param significant_figures The level of precision for this histogram, i.e. the number
* of figures in a decimal number that will be maintained. E.g. a value of 3 will mean
* the results from the histogram will be accurate up to the first three digits. Must
* be a value between 1 and 5 (inclusive).
* @param result Output parameter to capture allocated histogram.
* @return 0 on success, EINVAL if lowest_discernible_value is < 1 or the
* significant_figure value is outside of the allowed range, ENOMEM if malloc
* failed.
*/
int hdr_init(
int64_t lowest_discernible_value,
int64_t highest_trackable_value,
int significant_figures,
struct hdr_histogram** result);
/**
* Free the memory and close the hdr_histogram.
*
* @param h The histogram you want to close.
*/
void hdr_close(struct hdr_histogram* h);
/**
* Allocate the memory and initialise the hdr_histogram. This is the equivalent of calling
* hdr_init(1, highest_trackable_value, significant_figures, result);
*
* @deprecated use hdr_init.
*/
int hdr_alloc(int64_t highest_trackable_value, int significant_figures, struct hdr_histogram** result);
/**
* Reset a histogram to zero - empty out a histogram and re-initialise it
*
* If you want to re-use an existing histogram, but reset everything back to zero, this
* is the routine to use.
*
* @param h The histogram you want to reset to empty.
*
*/
void hdr_reset(struct hdr_histogram* h);
/**
* Get the memory size of the hdr_histogram.
*
* @param h "This" pointer
* @return The amount of memory used by the hdr_histogram in bytes
*/
size_t hdr_get_memory_size(struct hdr_histogram* h);
/**
* Records a value in the histogram, will round this value of to a precision at or better
* than the significant_figure specified at construction time.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_value(struct hdr_histogram* h, int64_t value);
/**
* Records a value in the histogram, will round this value of to a precision at or better
* than the significant_figure specified at construction time.
*
* Will record this value atomically, however the whole structure may appear inconsistent
* when read concurrently with this update. Do NOT mix calls to this method with calls
* to non-atomic updates.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_value_atomic(struct hdr_histogram* h, int64_t value);
/**
* Records count values in the histogram, will round this value of to a
* precision at or better than the significant_figure specified at construction
* time.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @param count Number of 'value's to add to the histogram
* @return false if any value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_values(struct hdr_histogram* h, int64_t value, int64_t count);
/**
* Records count values in the histogram, will round this value of to a
* precision at or better than the significant_figure specified at construction
* time.
*
* Will record this value atomically, however the whole structure may appear inconsistent
* when read concurrently with this update. Do NOT mix calls to this method with calls
* to non-atomic updates.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @param count Number of 'value's to add to the histogram
* @return false if any value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_values_atomic(struct hdr_histogram* h, int64_t value, int64_t count);
/**
* Record a value in the histogram and backfill based on an expected interval.
*
* Records a value in the histogram, will round this value of to a precision at or better
* than the significant_figure specified at construction time. This is specifically used
* for recording latency. If the value is larger than the expected_interval then the
* latency recording system has experienced co-ordinated omission. This method fills in the
* values that would have occurred had the client providing the load not been blocked.
* @param h "This" pointer
* @param value Value to add to the histogram
* @param expected_interval The delay between recording values.
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_corrected_value(struct hdr_histogram* h, int64_t value, int64_t expected_interval);
/**
* Record a value in the histogram and backfill based on an expected interval.
*
* Records a value in the histogram, will round this value of to a precision at or better
* than the significant_figure specified at construction time. This is specifically used
* for recording latency. If the value is larger than the expected_interval then the
* latency recording system has experienced co-ordinated omission. This method fills in the
* values that would have occurred had the client providing the load not been blocked.
*
* Will record this value atomically, however the whole structure may appear inconsistent
* when read concurrently with this update. Do NOT mix calls to this method with calls
* to non-atomic updates.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @param expected_interval The delay between recording values.
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_corrected_value_atomic(struct hdr_histogram* h, int64_t value, int64_t expected_interval);
/**
* Record a value in the histogram 'count' times. Applies the same correcting logic
* as 'hdr_record_corrected_value'.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @param count Number of 'value's to add to the histogram
* @param expected_interval The delay between recording values.
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_corrected_values(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval);
/**
* Record a value in the histogram 'count' times. Applies the same correcting logic
* as 'hdr_record_corrected_value'.
*
* Will record this value atomically, however the whole structure may appear inconsistent
* when read concurrently with this update. Do NOT mix calls to this method with calls
* to non-atomic updates.
*
* @param h "This" pointer
* @param value Value to add to the histogram
* @param count Number of 'value's to add to the histogram
* @param expected_interval The delay between recording values.
* @return false if the value is larger than the highest_trackable_value and can't be recorded,
* true otherwise.
*/
bool hdr_record_corrected_values_atomic(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval);
/**
* Adds all of the values from 'from' to 'this' histogram. Will return the
* number of values that are dropped when copying. Values will be dropped
* if they around outside of h.lowest_discernible_value and
* h.highest_trackable_value.
*
* @param h "This" pointer
* @param from Histogram to copy values from.
* @return The number of values dropped when copying.
*/
int64_t hdr_add(struct hdr_histogram* h, const struct hdr_histogram* from);
/**
* Adds all of the values from 'from' to 'this' histogram. Will return the
* number of values that are dropped when copying. Values will be dropped
* if they around outside of h.lowest_discernible_value and
* h.highest_trackable_value.
*
* @param h "This" pointer
* @param from Histogram to copy values from.
* @return The number of values dropped when copying.
*/
int64_t hdr_add_while_correcting_for_coordinated_omission(
struct hdr_histogram* h, struct hdr_histogram* from, int64_t expected_interval);
/**
* Get minimum value from the histogram. Will return 2^63-1 if the histogram
* is empty.
*
* @param h "This" pointer
*/
int64_t hdr_min(const struct hdr_histogram* h);
/**
* Get maximum value from the histogram. Will return 0 if the histogram
* is empty.
*
* @param h "This" pointer
*/
int64_t hdr_max(const struct hdr_histogram* h);
/**
* Get the value at a specific percentile.
*
* @param h "This" pointer.
* @param percentile The percentile to get the value for
*/
int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile);
/**
* Get the values at the given percentiles.
*
* @param h "This" pointer.
* @param percentiles The ordered percentiles array to get the values for.
* @param length Number of elements in the arrays.
* @param values Destination array containing the values at the given percentiles.
* The values array should be allocated by the caller.
* @return 0 on success, ENOMEM if the provided destination array is null.
*/
int hdr_value_at_percentiles(const struct hdr_histogram *h, const double *percentiles, int64_t *values, size_t length);
/**
* Gets the standard deviation for the values in the histogram.
*
* @param h "This" pointer
* @return The standard deviation
*/
double hdr_stddev(const struct hdr_histogram* h);
/**
* Gets the mean for the values in the histogram.
*
* @param h "This" pointer
* @return The mean
*/
double hdr_mean(const struct hdr_histogram* h);
/**
* Determine if two values are equivalent with the histogram's resolution.
* Where "equivalent" means that value samples recorded for any two
* equivalent values are counted in a common total count.
*
* @param h "This" pointer
* @param a first value to compare
* @param b second value to compare
* @return 'true' if values are equivalent with the histogram's resolution.
*/
bool hdr_values_are_equivalent(const struct hdr_histogram* h, int64_t a, int64_t b);
/**
* Get the lowest value that is equivalent to the given value within the histogram's resolution.
* Where "equivalent" means that value samples recorded for any two
* equivalent values are counted in a common total count.
*
* @param h "This" pointer
* @param value The given value
* @return The lowest value that is equivalent to the given value within the histogram's resolution.
*/
int64_t hdr_lowest_equivalent_value(const struct hdr_histogram* h, int64_t value);
/**
* Get the count of recorded values at a specific value
* (to within the histogram resolution at the value level).
*
* @param h "This" pointer
* @param value The value for which to provide the recorded count
* @return The total count of values recorded in the histogram within the value range that is
* {@literal >=} lowestEquivalentValue(<i>value</i>) and {@literal <=} highestEquivalentValue(<i>value</i>)
*/
int64_t hdr_count_at_value(const struct hdr_histogram* h, int64_t value);
int64_t hdr_count_at_index(const struct hdr_histogram* h, int32_t index);
int64_t hdr_value_at_index(const struct hdr_histogram* h, int32_t index);
struct hdr_iter_percentiles
{
bool seen_last_value;
int32_t ticks_per_half_distance;
double percentile_to_iterate_to;
double percentile;
};
struct hdr_iter_recorded
{
int64_t count_added_in_this_iteration_step;
};
struct hdr_iter_linear
{
int64_t value_units_per_bucket;
int64_t count_added_in_this_iteration_step;
int64_t next_value_reporting_level;
int64_t next_value_reporting_level_lowest_equivalent;
};
struct hdr_iter_log
{
double log_base;
int64_t count_added_in_this_iteration_step;
int64_t next_value_reporting_level;
int64_t next_value_reporting_level_lowest_equivalent;
};
/**
* The basic iterator. This is a generic structure
* that supports all of the types of iteration. Use
* the appropriate initialiser to get the desired
* iteration.
*
* @
*/
struct hdr_iter
{
const struct hdr_histogram* h;
/** raw index into the counts array */
int32_t counts_index;
/** snapshot of the length at the time the iterator is created */
int64_t total_count;
/** value directly from array for the current counts_index */
int64_t count;
/** sum of all of the counts up to and including the count at this index */
int64_t cumulative_count;
/** The current value based on counts_index */
int64_t value;
int64_t highest_equivalent_value;
int64_t lowest_equivalent_value;
int64_t median_equivalent_value;
int64_t value_iterated_from;
int64_t value_iterated_to;
union
{
struct hdr_iter_percentiles percentiles;
struct hdr_iter_recorded recorded;
struct hdr_iter_linear linear;
struct hdr_iter_log log;
} specifics;
bool (* _next_fp)(struct hdr_iter* iter);
};
/**
* Initalises the basic iterator.
*
* @param itr 'This' pointer
* @param h The histogram to iterate over
*/
void hdr_iter_init(struct hdr_iter* iter, const struct hdr_histogram* h);
/**
* Initialise the iterator for use with percentiles.
*/
void hdr_iter_percentile_init(struct hdr_iter* iter, const struct hdr_histogram* h, int32_t ticks_per_half_distance);
/**
* Initialise the iterator for use with recorded values.
*/
void hdr_iter_recorded_init(struct hdr_iter* iter, const struct hdr_histogram* h);
/**
* Initialise the iterator for use with linear values.
*/
void hdr_iter_linear_init(
struct hdr_iter* iter,
const struct hdr_histogram* h,
int64_t value_units_per_bucket);
/**
* Update the iterator value units per bucket
*/
void hdr_iter_linear_set_value_units_per_bucket(struct hdr_iter* iter, int64_t value_units_per_bucket);
/**
* Initialise the iterator for use with logarithmic values
*/
void hdr_iter_log_init(
struct hdr_iter* iter,
const struct hdr_histogram* h,
int64_t value_units_first_bucket,
double log_base);
/**
* Iterate to the next value for the iterator. If there are no more values
* available return faluse.
*
* @param itr 'This' pointer
* @return 'false' if there are no values remaining for this iterator.
*/
bool hdr_iter_next(struct hdr_iter* iter);
typedef enum
{
CLASSIC,
CSV
} format_type;
/**
* Print out a percentile based histogram to the supplied stream. Note that
* this call will not flush the FILE, this is left up to the user.
*
* @param h 'This' pointer
* @param stream The FILE to write the output to
* @param ticks_per_half_distance The number of iteration steps per half-distance to 100%
* @param value_scale Scale the output values by this amount
* @param format_type Format to use, e.g. CSV.
* @return 0 on success, error code on failure. EIO if an error occurs writing
* the output.
*/
int hdr_percentiles_print(
struct hdr_histogram* h, FILE* stream, int32_t ticks_per_half_distance,
double value_scale, format_type format);
/**
* Internal allocation methods, used by hdr_dbl_histogram.
*/
struct hdr_histogram_bucket_config
{
int64_t lowest_discernible_value;
int64_t highest_trackable_value;
int64_t unit_magnitude;
int64_t significant_figures;
int32_t sub_bucket_half_count_magnitude;
int32_t sub_bucket_half_count;
int64_t sub_bucket_mask;
int32_t sub_bucket_count;
int32_t bucket_count;
int32_t counts_len;
};
int hdr_calculate_bucket_config(
int64_t lowest_discernible_value,
int64_t highest_trackable_value,
int significant_figures,
struct hdr_histogram_bucket_config* cfg);
void hdr_init_preallocated(struct hdr_histogram* h, struct hdr_histogram_bucket_config* cfg);
int64_t hdr_size_of_equivalent_value_range(const struct hdr_histogram* h, int64_t value);
int64_t hdr_next_non_equivalent_value(const struct hdr_histogram* h, int64_t value);
int64_t hdr_median_equivalent_value(const struct hdr_histogram* h, int64_t value);
/**
* Used to reset counters after importing data manually into the histogram, used by the logging code
* and other custom serialisation tools.
*/
void hdr_reset_internal_counters(struct hdr_histogram* h);
#ifdef __cplusplus
}
#endif
#endif
-13
View File
@@ -1,13 +0,0 @@
#ifndef HDR_MALLOC_H__
#define HDR_MALLOC_H__
void *zmalloc(size_t size);
void *zcalloc_num(size_t num, size_t size);
void *zrealloc(void *ptr, size_t size);
void zfree(void *ptr);
#define hdr_malloc zmalloc
#define hdr_calloc zcalloc_num
#define hdr_realloc zrealloc
#define hdr_free zfree
#endif
-22
View File
@@ -1,22 +0,0 @@
#ifndef HDR_TESTS_H
#define HDR_TESTS_H
/* These are functions used in tests and are not intended for normal usage. */
#include "hdr_histogram.h"
#ifdef __cplusplus
extern "C" {
#endif
int32_t counts_index_for(const struct hdr_histogram* h, int64_t value);
int hdr_encode_compressed(struct hdr_histogram* h, uint8_t** compressed_histogram, size_t* compressed_len);
int hdr_decode_compressed(uint8_t* buffer, size_t length, struct hdr_histogram** histogram);
void hdr_base64_decode_block(const char* input, uint8_t* output);
void hdr_base64_encode_block(const uint8_t* input, char* output);
#ifdef __cplusplus
}
#endif
#endif
-49
View File
@@ -1,49 +0,0 @@
name-template: '$NEXT_MAJOR_VERSION'
tag-template: 'v$NEXT_MAJOR_VERSION'
autolabeler:
- label: 'maintenance'
files:
- '*.md'
- '.github/*'
- label: 'bug'
branch:
- '/bug-.+'
- label: 'maintenance'
branch:
- '/maintenance-.+'
- label: 'feature'
branch:
- '/feature-.+'
categories:
- title: 'Breaking Changes'
labels:
- 'breakingchange'
- title: '🧪 Experimental Features'
labels:
- 'experimental'
- title: '🚀 New Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
labels:
- 'fix'
- 'bugfix'
- 'bug'
- 'BUG'
- title: '🧰 Maintenance'
label: 'maintenance'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
exclude-labels:
- 'skip-changelog'
template: |
## Changes
$CHANGES
## Contributors
We'd like to thank all the contributors who worked on this release!
$CONTRIBUTORS
-177
View File
@@ -1,177 +0,0 @@
name: Build and test
on: [push, pull_request]
jobs:
ubuntu:
name: Ubuntu
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update
sudo apt-get install -y redis-server valgrind libevent-dev
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake .. && make
- name: Build using makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
# - name: Run tests under valgrind
# env:
# SKIPS_AS_FAILS: 1
# TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
# run: $GITHUB_WORKSPACE/test.sh
centos7:
name: CentOS 7
runs-on: ubuntu-latest
container: centos:7
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
yum -y --enablerepo=remi install redis
yum -y install gcc gcc-c++ make openssl openssl-devel cmake3 valgrind libevent-devel
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake3 .. && make
- name: Build using Makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
- name: Run tests under valgrind
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
run: $GITHUB_WORKSPACE/test.sh
centos8:
name: RockyLinux 8
runs-on: ubuntu-latest
container: rockylinux:8
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
dnf -y upgrade --refresh
dnf -y install https://rpms.remirepo.net/enterprise/remi-release-8.rpm
dnf -y module install redis:remi-6.0
dnf -y group install "Development Tools"
dnf -y install openssl-devel cmake valgrind libevent-devel
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake .. && make
- name: Build using Makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
- name: Run tests under valgrind
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
run: $GITHUB_WORKSPACE/test.sh
freebsd:
runs-on: macos-12
name: FreeBSD
steps:
- uses: actions/checkout@v3
- name: Build in FreeBSD
uses: vmactions/freebsd-vm@v0
with:
prepare: pkg install -y gmake cmake
run: |
mkdir build && cd build && cmake .. && make && cd ..
gmake
macos:
name: macOS
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
brew install openssl redis@7.0
brew link redis@7.0 --force
- name: Build hiredis
run: USE_SSL=1 make
- name: Run tests
env:
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
windows:
name: Windows
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
choco install -y ninja memurai-developer
- uses: ilammy/msvc-dev-cmd@v1
- name: Build hiredis
run: |
mkdir build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_EXAMPLES=ON
ninja -v
- name: Run tests
run: |
./build/hiredis-test.exe
- name: Install Cygwin Action
uses: cygwin/cygwin-install-action@v2
with:
packages: make git gcc-core
- name: Build in cygwin
env:
HIREDIS_PATH: ${{ github.workspace }}
run: |
make clean && make
-19
View File
@@ -1,19 +0,0 @@
name: Release Drafter
on:
push:
# branches to consider in the event; optional, defaults to all
branches:
- master
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
# Drafts your next Release notes as Pull Requests are merged into "master"
- uses: release-drafter/release-drafter@v5
with:
# (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
config-name: release-drafter-config.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-100
View File
@@ -1,100 +0,0 @@
name: C/C++ CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
full-build:
name: Build all, plus default examples, run tests against redis
runs-on: ubuntu-latest
env:
# the docker image used by the test.sh
REDIS_DOCKER: redis:alpine
steps:
- name: Install prerequisites
run: sudo apt-get update && sudo apt-get install -y libev-dev libevent-dev libglib2.0-dev libssl-dev valgrind
- uses: actions/checkout@v3
- name: Run make
run: make all examples
- name: Run unittests
run: make check
- name: Run tests with valgrind
env:
TEST_PREFIX: valgrind --error-exitcode=100
SKIPS_ARG: --skip-throughput
run: make check
build-32-bit:
name: Build and test minimal 32 bit linux
runs-on: ubuntu-latest
steps:
- name: Install prerequisites
run: sudo apt-get update && sudo apt-get install gcc-multilib
- uses: actions/checkout@v3
- name: Run make
run: make all
env:
PLATFORM_FLAGS: -m32
- name: Run unittests
env:
REDIS_DOCKER: redis:alpine
run: make check
build-arm:
name: Cross-compile and test arm linux with Qemu
runs-on: ubuntu-latest
strategy:
matrix:
include:
- name: arm
toolset: arm-linux-gnueabi
emulator: qemu-arm
- name: aarch64
toolset: aarch64-linux-gnu
emulator: qemu-aarch64
steps:
- name: Install qemu
if: matrix.emulator
run: sudo apt-get update && sudo apt-get install -y qemu-user
- name: Install platform toolset
if: matrix.toolset
run: sudo apt-get install -y gcc-${{matrix.toolset}}
- uses: actions/checkout@v3
- name: Run make
run: make all
env:
CC: ${{matrix.toolset}}-gcc
AR: ${{matrix.toolset}}-ar
- name: Run unittests
env:
REDIS_DOCKER: redis:alpine
TEST_PREFIX: ${{matrix.emulator}} -L /usr/${{matrix.toolset}}/
run: make check
build-windows:
name: Build and test on windows 64 bit Intel
runs-on: windows-latest
steps:
- uses: microsoft/setup-msbuild@v1.0.2
- uses: actions/checkout@v3
- name: Run CMake (shared lib)
run: cmake -Wno-dev CMakeLists.txt
- name: Build hiredis (shared lib)
run: MSBuild hiredis.vcxproj /p:Configuration=Debug
- name: Run CMake (static lib)
run: cmake -Wno-dev CMakeLists.txt -DBUILD_SHARED_LIBS=OFF
- name: Build hiredis (static lib)
run: MSBuild hiredis.vcxproj /p:Configuration=Debug
- name: Build hiredis-test
run: MSBuild hiredis-test.vcxproj /p:Configuration=Debug
# use memurai, redis compatible server, since it is easy to install. Can't
# install official redis containers on the windows runner
- name: Install Memurai redis server
run: choco install -y memurai-developer.install
- name: Run tests
run: Debug\hiredis-test.exe
-2
View File
@@ -5,5 +5,3 @@
/*.dylib
/*.a
/*.pc
*.dSYM
tags
+13 -99
View File
@@ -1,4 +1,5 @@
language: c
sudo: false
compiler:
- gcc
- clang
@@ -7,29 +8,8 @@ os:
- linux
- osx
dist: bionic
branches:
only:
- staging
- trying
- master
- /^release\/.*$/
install:
- if [ "$TRAVIS_COMPILER" != "mingw" ]; then
wget https://github.com/redis/redis/archive/6.0.6.tar.gz;
tar -xzvf 6.0.6.tar.gz;
pushd redis-6.0.6 && BUILD_TLS=yes make && export PATH=$PWD/src:$PATH && popd;
fi;
before_script:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then
curl -O https://distfiles.macports.org/MacPorts/MacPorts-2.6.2-10.13-HighSierra.pkg;
sudo installer -pkg MacPorts-2.6.2-10.13-HighSierra.pkg -target /;
export PATH=$PATH:/opt/local/bin && sudo port -v selfupdate;
sudo port -N install openssl redis;
fi;
- if [ "$TRAVIS_OS_NAME" == "osx" ] ; then brew update; brew install redis; fi
addons:
apt:
@@ -40,86 +20,20 @@ addons:
- libc6-dev-i386
- libc6-dbg:i386
- gcc-multilib
- g++-multilib
- libssl-dev
- libssl-dev:i386
- valgrind
env:
- BITS="32"
- BITS="64"
- CFLAGS="-Werror"
- PRE="valgrind --track-origins=yes --leak-check=full"
- TARGET="32bit" TARGET_VARS="32bit-vars" CFLAGS="-Werror"
- TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
script:
- EXTRA_CMAKE_OPTS="-DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON";
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
else
TEST_PREFIX="valgrind --track-origins=yes --leak-check=full";
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
fi;
export CFLAGS CXXFLAGS LDFLAGS TEST_PREFIX EXTRA_CMAKE_OPTS
- make && make clean;
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
if [ "$BITS" == "64" ]; then
OPENSSL_PREFIX="$(ls -d /usr/local/Cellar/openssl@1.1/*)" USE_SSL=1 make;
fi;
else
USE_SSL=1 make;
fi;
- mkdir build/ && cd build/
- cmake .. ${EXTRA_CMAKE_OPTS}
- make VERBOSE=1
- if [ "$BITS" == "64" ]; then
TEST_SSL=1 SKIPS_AS_FAILS=1 ctest -V;
else
SKIPS_AS_FAILS=1 ctest -V;
fi;
matrix:
exclude:
- os: osx
env: PRE="valgrind --track-origins=yes --leak-check=full"
jobs:
include:
# Windows MinGW cross compile on Linux
- os: linux
dist: xenial
compiler: mingw
addons:
apt:
packages:
- ninja-build
- gcc-mingw-w64-x86-64
- g++-mingw-w64-x86-64
script:
- mkdir build && cd build
- CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_WITH_INSTALL_RPATH=on
- ninja -v
- os: osx
env: TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
# Windows MSVC 2017
- os: windows
compiler: msvc
env:
- MATRIX_EVAL="CC=cl.exe && CXX=cl.exe"
before_install:
- eval "${MATRIX_EVAL}"
install:
- choco install ninja
- choco install -y memurai-developer
script:
- mkdir build && cd build
- cmd.exe //C 'C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat' amd64 '&&'
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_EXAMPLES=ON '&&' ninja -v
- ./hiredis-test.exe
script: make $TARGET CFLAGS="$CFLAGS" && make check PRE="$PRE" && make $TARGET_VARS hiredis-example
+11 -442
View File
@@ -1,450 +1,18 @@
## [1.2.0](https://github.com/redis/hiredis/tree/v1.2.0) - (2023-06-04)
### 1.0.0 (unreleased)
Announcing Hiredis v1.2.0 with with new adapters, and a great many bug fixes.
**Fixes**:
## 🚀 New Features
- Add sdevent adapter @Oipo (#1144)
- Allow specifying the keepalive interval @michael-grunder (#1168)
- Add RedisModule adapter @tezc (#1182)
- Helper for setting TCP_USER_TIMEOUT socket option @zuiderkwast (#1188)
## 🐛 Bug Fixes
- Fix a typo in b6a052f. @yossigo (#1190)
- Fix wincrypt symbols conflict @hudayou (#1151)
- Don't attempt to set a timeout if we are in an error state. @michael-grunder (#1180)
- Accept -nan per the RESP3 spec recommendation. @michael-grunder (#1178)
- Fix colliding option values @zuiderkwast (#1172)
- Ensure functionality without `_MSC_VER` definition @windyakin (#1194)
## 🧰 Maintenance
- Add a test for the TCP_USER_TIMEOUT option. @michael-grunder (#1192)
- Add -Werror as a default. @yossigo (#1193)
- CI: Update homebrew Redis version. @yossigo (#1191)
- Fix typo in makefile. @michael-grunder (#1179)
- Write a version file for the CMake package @Neverlord (#1165)
- CMakeLists.txt: respect BUILD_SHARED_LIBS @ffontaine (#1147)
- Cmake static or shared @autoantwort (#1160)
- fix typo @tillkruss (#1153)
- Add a test ensuring we don't clobber connection error. @michael-grunder (#1181)
- Search for openssl on macOS @michael-grunder (#1169)
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/neverlord"><img src="https://github.com/neverlord.png" width="32" height="32"></a>
<a href="https://github.com/Oipo"><img src="https://github.com/Oipo.png" width="32" height="32"></a>
<a href="https://github.com/autoantwort"><img src="https://github.com/autoantwort.png" width="32" height="32"></a>
<a href="https://github.com/ffontaine"><img src="https://github.com/ffontaine.png" width="32" height="32"></a>
<a href="https://github.com/hudayou"><img src="https://github.com/hudayou.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/postgraph"><img src="https://github.com/postgraph.png" width="32" height="32"></a>
<a href="https://github.com/tezc"><img src="https://github.com/tezc.png" width="32" height="32"></a>
<a href="https://github.com/tillkruss"><img src="https://github.com/tillkruss.png" width="32" height="32"></a>
<a href="https://github.com/vityafx"><img src="https://github.com/vityafx.png" width="32" height="32"></a>
<a href="https://github.com/windyakin"><img src="https://github.com/windyakin.png" width="32" height="32"></a>
<a href="https://github.com/yossigo"><img src="https://github.com/yossigo.png" width="32" height="32"></a>
<a href="https://github.com/zuiderkwast"><img src="https://github.com/zuiderkwast.png" width="32" height="32"></a>
## [1.1.0](https://github.com/redis/hiredis/tree/v1.1.0) - (2022-11-15)
Announcing Hiredis v1.1.0 GA with better SSL convenience, new async adapters and a great many bug fixes.
**NOTE**: Hiredis can now return `nan` in addition to `-inf` and `inf` when returning a `REDIS_REPLY_DOUBLE`.
## 🐛 Bug Fixes
- Add support for nan in RESP3 double [@filipecosta90](https://github.com/filipecosta90)
([\#1133](https://github.com/redis/hiredis/pull/1133))
## 🧰 Maintenance
- Add an example that calls redisCommandArgv [@michael-grunder](https://github.com/michael-grunder)
([\#1140](https://github.com/redis/hiredis/pull/1140))
- fix flag reference [@pata00](https://github.com/pata00) ([\#1136](https://github.com/redis/hiredis/pull/1136))
- Make freeing a NULL redisAsyncContext a no op. [@michael-grunder](https://github.com/michael-grunder)
([\#1135](https://github.com/redis/hiredis/pull/1135))
- CI updates ([@bjosv](https://github.com/redis/bjosv) ([\#1139](https://github.com/redis/hiredis/pull/1139))
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/bjsov"><img src="https://github.com/bjosv.png" width="32" height="32"></a>
<a href="https://github.com/filipecosta90"><img src="https://github.com/filipecosta90.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/pata00"><img src="https://github.com/pata00.png" width="32" height="32"></a>
## [1.1.0-rc1](https://github.com/redis/hiredis/tree/v1.1.0-rc1) - (2022-11-06)
Announcing Hiredis v1.1.0-rc1, with better SSL convenience, new async adapters, and a great many bug fixes.
## 🚀 New Features
- Add possibility to prefer IPv6, IPv4 or unspecified [@zuiderkwast](https://github.com/zuiderkwast)
([\#1096](https://github.com/redis/hiredis/pull/1096))
- Add adapters/libhv [@ithewei](https://github.com/ithewei) ([\#904](https://github.com/redis/hiredis/pull/904))
- Add timeout support to libhv adapter. [@michael-grunder](https://github.com/michael-grunder) ([\#1109](https://github.com/redis/hiredis/pull/1109))
- set default SSL verification path [@adobeturchenko](https://github.com/adobeturchenko) ([\#928](https://github.com/redis/hiredis/pull/928))
- Introduce .close method for redisContextFuncs [@pizhenwei](https://github.com/pizhenwei) ([\#1094](https://github.com/redis/hiredis/pull/1094))
- Make it possible to set SSL verify mode [@stanhu](https://github.com/stanhu) ([\#1085](https://github.com/redis/hiredis/pull/1085))
- Polling adapter and example [@kristjanvalur](https://github.com/kristjanvalur) ([\#932](https://github.com/redis/hiredis/pull/932))
- Unsubscribe handling in async [@bjosv](https://github.com/bjosv) ([\#1047](https://github.com/redis/hiredis/pull/1047))
- Add timeout support for libuv adapter [@MichaelSuen-thePointer](https://github.com/@MichaelSuenthePointer) ([\#1016](https://github.com/redis/hiredis/pull/1016))
## 🐛 Bug Fixes
- Update for MinGW cross compile [@bit0fun](https://github.com/bit0fun) ([\#1127](https://github.com/redis/hiredis/pull/1127))
- fixed CPP build error with adapters/libhv.h [@mtdxc](https://github.com/mtdxc) ([\#1125](https://github.com/redis/hiredis/pull/1125))
- Fix protocol error
[@michael-grunder](https://github.com/michael-grunder),
[@mtuleika-appcast](https://github.com/mtuleika-appcast) ([\#1106](https://github.com/redis/hiredis/pull/1106))
- Use a windows specific keepalive function. [@michael-grunder](https://github.com/michael-grunder) ([\#1104](https://github.com/redis/hiredis/pull/1104))
- Fix CMake config path on Linux. [@xkszltl](https://github.com/xkszltl) ([\#989](https://github.com/redis/hiredis/pull/989))
- Fix potential fault at createDoubleObject [@afcidk](https://github.com/afcidk) ([\#964](https://github.com/redis/hiredis/pull/964))
- Fix some undefined behavior [@jengab](https://github.com/jengab) ([\#1091](https://github.com/redis/hiredis/pull/1091))
- Copy OOM errors to redisAsyncContext when finding subscribe callback [@bjosv](https://github.com/bjosv) ([\#1090](https://github.com/redis/hiredis/pull/1090))
- Maintain backward compatibility with our onConnect callback. [@michael-grunder](https://github.com/michael-grunder) ([\#1087](https://github.com/redis/hiredis/pull/1087))
- Fix PUSH handler tests for Redis >= 7.0.5 [@michael-grunder](https://github.com/michael-grunder) ([\#1121](https://github.com/redis/hiredis/pull/1121))
- fix heap-buffer-overflow [@zhangtaoXT5](https://github.com/zhangtaoXT5) ([\#957](https://github.com/redis/hiredis/pull/957))
- Fix heap-buffer-overflow issue in redisvFormatCommad [@bjosv](https://github.com/bjosv) ([\#1097](https://github.com/redis/hiredis/pull/1097))
- Polling adapter requires sockcompat.h [@michael-grunder](https://github.com/michael-grunder) ([\#1095](https://github.com/redis/hiredis/pull/1095))
- Illumos test fixes, error message difference for bad hostname test. [@devnexen](https://github.com/devnexen) ([\#901](https://github.com/redis/hiredis/pull/901))
- Remove semicolon after do-while in \_EL\_CLEANUP [@sundb](https://github.com/sundb) ([\#905](https://github.com/redis/hiredis/pull/905))
- Stability: Support calling redisAsyncCommand and redisAsyncDisconnect from the onConnected callback [@kristjanvalur](https://github.com/kristjanvalur)
([\#931](https://github.com/redis/hiredis/pull/931))
- Fix async connect on Windows [@kristjanvalur](https://github.com/kristjanvalur) ([\#1073](https://github.com/redis/hiredis/pull/1073))
- Fix tests so they work for Redis 7.0 [@michael-grunder](https://github.com/michael-grunder) ([\#1072](https://github.com/redis/hiredis/pull/1072))
- Fix warnings on Win64 [@orgads](https://github.com/orgads) ([\#1058](https://github.com/redis/hiredis/pull/1058))
- Handle push notifications before or after reply. [@yossigo](https://github.com/yossigo) ([\#1062](https://github.com/redis/hiredis/pull/1062))
- Update hiredis sds with improvements found in redis [@bjosv](https://github.com/bjosv) ([\#1045](https://github.com/redis/hiredis/pull/1045))
- Avoid incorrect call to the previous reply's callback [@bjosv](https://github.com/bjosv) ([\#1040](https://github.com/redis/hiredis/pull/1040))
- fix building on AIX and SunOS [\#1031](https://github.com/redis/hiredis/pull/1031) ([@scddev](https://github.com/scddev))
- Allow sending commands after sending an unsubscribe [@bjosv](https://github.com/bjosv) ([\#1036](https://github.com/redis/hiredis/pull/1036))
- Correction for command timeout during pubsub [@bjosv](https://github.com/bjosv) ([\#1038](https://github.com/redis/hiredis/pull/1038))
- Fix adapters/libevent.h compilation for 64-bit Windows [@pbtummillo](https://github.com/pbtummillo) ([\#937](https://github.com/redis/hiredis/pull/937))
- Fix integer overflow when format command larger than 4GB [@sundb](https://github.com/sundb) ([\#1030](https://github.com/redis/hiredis/pull/1030))
- Handle array response during subscribe in RESP3 [@bjosv](https://github.com/bjosv) ([\#1014](https://github.com/redis/hiredis/pull/1014))
- Support PING while subscribing (RESP2) [@bjosv](https://github.com/bjosv) ([\#1027](https://github.com/redis/hiredis/pull/1027))
## 🧰 Maintenance
- CI fixes in preparation of release [@michael-grunder](https://github.com/michael-grunder) ([\#1130](https://github.com/redis/hiredis/pull/1130))
- Add do while(0) (protection for macros [@afcidk](https://github.com/afcidk) [\#959](https://github.com/redis/hiredis/pull/959))
- Fixup of PR734: Coverage of hiredis.c [@bjosv](https://github.com/bjosv) ([\#1124](https://github.com/redis/hiredis/pull/1124))
- CMake corrections for building on Windows [@bjosv](https://github.com/bjosv) ([\#1122](https://github.com/redis/hiredis/pull/1122))
- Install on windows fixes [@bjosv](https://github.com/bjosv) ([\#1117](https://github.com/redis/hiredis/pull/1117))
- Add libhv example to our standard Makefile [@michael-grunder](https://github.com/michael-grunder) ([\#1108](https://github.com/redis/hiredis/pull/1108))
- Additional include directory given by pkg-config [@bjosv](https://github.com/bjosv) ([\#1118](https://github.com/redis/hiredis/pull/1118))
- Use __attribute__ when building with Clang on Windows [@bjosv](https://github.com/bjosv) ([\#1115](https://github.com/redis/hiredis/pull/1115))
- Minor refactor [@michael-grunder](https://github.com/michael-grunder) ([\#1110](https://github.com/redis/hiredis/pull/1110))
- Fix pkgconfig result for hiredis_ssl [@bjosv](https://github.com/bjosv) ([\#1107](https://github.com/redis/hiredis/pull/1107))
- Update documentation to explain redisConnectWithOptions. [@michael-grunder](https://github.com/michael-grunder) ([\#1099](https://github.com/redis/hiredis/pull/1099))
- uvadapter: reduce number of uv_poll_start calls [@noxiouz](https://github.com/noxiouz) ([\#1098](https://github.com/redis/hiredis/pull/1098))
- Regression test for off-by-one parsing error [@bugwz](https://github.com/bugwz) ([\#1092](https://github.com/redis/hiredis/pull/1092))
- CMake: remove dict.c form hiredis_sources [@Lipraxde](https://github.com/Lipraxde) ([\#1055](https://github.com/redis/hiredis/pull/1055))
- Do store command timeout in the context for redisSetTimeout [@catterer](https://github.com/catterer) ([\#593](https://github.com/redis/hiredis/pull/593), [\#1093](https://github.com/redis/hiredis/pull/1093))
- Add GitHub Actions CI workflow for hiredis: Arm, Arm64, 386, windows. [@kristjanvalur](https://github.com/kristjanvalur) ([\#943](https://github.com/redis/hiredis/pull/943))
- CI: bump macOS runner version [@SukkaW](https://github.com/SukkaW) ([\#1079](https://github.com/redis/hiredis/pull/1079))
- Support for generating release notes [@chayim](https://github.com/chayim) ([\#1083](https://github.com/redis/hiredis/pull/1083))
- Improve example for SSL initialization in README.md [@stanhu](https://github.com/stanhu) ([\#1084](https://github.com/redis/hiredis/pull/1084))
- Fix README typos [@bjosv](https://github.com/bjosv) ([\#1080](https://github.com/redis/hiredis/pull/1080))
- fix cmake version [@smmir-cent](https://github.com/@smmircent) ([\#1050](https://github.com/redis/hiredis/pull/1050))
- Use the same name for static and shared libraries [@orgads](https://github.com/orgads) ([\#1057](https://github.com/redis/hiredis/pull/1057))
- Embed debug information in windows static .lib file [@kristjanvalur](https://github.com/kristjanvalur) ([\#1054](https://github.com/redis/hiredis/pull/1054))
- Improved async documentation [@kristjanvalur](https://github.com/kristjanvalur) ([\#1074](https://github.com/redis/hiredis/pull/1074))
- Use official repository for redis package. [@yossigo](https://github.com/yossigo) ([\#1061](https://github.com/redis/hiredis/pull/1061))
- Whitelist hiredis repo path in cygwin [@michael-grunder](https://github.com/michael-grunder) ([\#1063](https://github.com/redis/hiredis/pull/1063))
- CentOS 8 is EOL, switch to RockyLinux [@michael-grunder](https://github.com/michael-grunder) ([\#1046](https://github.com/redis/hiredis/pull/1046))
- CMakeLists.txt: allow building without a C++ compiler [@ffontaine](https://github.com/ffontaine) ([\#872](https://github.com/redis/hiredis/pull/872))
- Makefile: move SSL options into a block and refine rules [@pizhenwei](https://github.com/pizhenwei) ([\#997](https://github.com/redis/hiredis/pull/997))
- Update CMakeLists.txt for more portability [@EricDeng1001](https://github.com/EricDeng1001) ([\#1005](https://github.com/redis/hiredis/pull/1005))
- FreeBSD build fixes + CI [@michael-grunder](https://github.com/michael-grunder) ([\#1026](https://github.com/redis/hiredis/pull/1026))
- Add asynchronous test for pubsub using RESP3 [@bjosv](https://github.com/bjosv) ([\#1012](https://github.com/redis/hiredis/pull/1012))
- Trigger CI failure when Valgrind issues are found [@bjosv](https://github.com/bjosv) ([\#1011](https://github.com/redis/hiredis/pull/1011))
- Move to using make directly in Cygwin [@michael-grunder](https://github.com/michael-grunder) ([\#1020](https://github.com/redis/hiredis/pull/1020))
- Add asynchronous API tests [@bjosv](https://github.com/bjosv) ([\#1010](https://github.com/redis/hiredis/pull/1010))
- Correcting the build target `coverage` for enabled SSL [@bjosv](https://github.com/bjosv) ([\#1009](https://github.com/redis/hiredis/pull/1009))
- GH Actions: Run SSL tests during CI [@bjosv](https://github.com/bjosv) ([\#1008](https://github.com/redis/hiredis/pull/1008))
- GH: Actions - Add valgrind and CMake [@michael-grunder](https://github.com/michael-grunder) ([\#1004](https://github.com/redis/hiredis/pull/1004))
- Add Centos8 tests in GH Actions [@michael-grunder](https://github.com/michael-grunder) ([\#1001](https://github.com/redis/hiredis/pull/1001))
- We should run actions on PRs [@michael-grunder](https://github.com/michael-grunder) (([\#1000](https://github.com/redis/hiredis/pull/1000))
- Add Cygwin test in GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#999](https://github.com/redis/hiredis/pull/999))
- Add Windows tests in GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#996](https://github.com/redis/hiredis/pull/996))
- Switch to GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#995](https://github.com/redis/hiredis/pull/995))
- Minor refactor of CVE-2021-32765 fix. [@michael-grunder](https://github.com/michael-grunder) ([\#993](https://github.com/redis/hiredis/pull/993))
- Remove extra comma from CMake var. [@xkszltl](https://github.com/xkszltl) ([\#988](https://github.com/redis/hiredis/pull/988))
- Add REDIS\_OPT\_PREFER\_UNSPEC [@michael-grunder](https://github.com/michael-grunder) ([\#1101](https://github.com/redis/hiredis/pull/1101))
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/EricDeng1001"><img src="https://github.com/EricDeng1001.png" width="32" height="32"></a>
<a href="https://github.com/Lipraxde"><img src="https://github.com/Lipraxde.png" width="32" height="32"></a>
<a href="https://github.com/MichaelSuen-thePointer"><img src="https://github.com/MichaelSuen-thePointer.png" width="32" height="32"></a>
<a href="https://github.com/SukkaW"><img src="https://github.com/SukkaW.png" width="32" height="32"></a>
<a href="https://github.com/adobeturchenko"><img src="https://github.com/adobeturchenko.png" width="32" height="32"></a>
<a href="https://github.com/afcidk"><img src="https://github.com/afcidk.png" width="32" height="32"></a>
<a href="https://github.com/bit0fun"><img src="https://github.com/bit0fun.png" width="32" height="32"></a>
<a href="https://github.com/bjosv"><img src="https://github.com/bjosv.png" width="32" height="32"></a>
<a href="https://github.com/bugwz"><img src="https://github.com/bugwz.png" width="32" height="32"></a>
<a href="https://github.com/catterer"><img src="https://github.com/catterer.png" width="32" height="32"></a>
<a href="https://github.com/chayim"><img src="https://github.com/chayim.png" width="32" height="32"></a>
<a href="https://github.com/devnexen"><img src="https://github.com/devnexen.png" width="32" height="32"></a>
<a href="https://github.com/ffontaine"><img src="https://github.com/ffontaine.png" width="32" height="32"></a>
<a href="https://github.com/ithewei"><img src="https://github.com/ithewei.png" width="32" height="32"></a>
<a href="https://github.com/jengab"><img src="https://github.com/jengab.png" width="32" height="32"></a>
<a href="https://github.com/kristjanvalur"><img src="https://github.com/kristjanvalur.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/noxiouz"><img src="https://github.com/noxiouz.png" width="32" height="32"></a>
<a href="https://github.com/mtdxc"><img src="https://github.com/mtdxc.png" width="32" height="32"></a>
<a href="https://github.com/orgads"><img src="https://github.com/orgads.png" width="32" height="32"></a>
<a href="https://github.com/pbtummillo"><img src="https://github.com/pbtummillo.png" width="32" height="32"></a>
<a href="https://github.com/pizhenwei"><img src="https://github.com/pizhenwei.png" width="32" height="32"></a>
<a href="https://github.com/scddev"><img src="https://github.com/scddev.png" width="32" height="32"></a>
<a href="https://github.com/smmir-cent"><img src="https://github.com/smmir-cent.png" width="32" height="32"></a>
<a href="https://github.com/stanhu"><img src="https://github.com/stanhu.png" width="32" height="32"></a>
<a href="https://github.com/sundb"><img src="https://github.com/sundb.png" width="32" height="32"></a>
<a href="https://github.com/vturchenko"><img src="https://github.com/vturchenko.png" width="32" height="32"></a>
<a href="https://github.com/xkszltl"><img src="https://github.com/xkszltl.png" width="32" height="32"></a>
<a href="https://github.com/yossigo"><img src="https://github.com/yossigo.png" width="32" height="32"></a>
<a href="https://github.com/zhangtaoXT5"><img src="https://github.com/zhangtaoXT5.png" width="32" height="32"></a>
<a href="https://github.com/zuiderkwast"><img src="https://github.com/zuiderkwast.png" width="32" height="32"></a>
## [1.0.2](https://github.com/redis/hiredis/tree/v1.0.2) - (2021-10-07)
Announcing Hiredis v1.0.2, which fixes CVE-2021-32765 but returns the SONAME to the correct value of `1.0.0`.
- [Revert SONAME bump](https://github.com/redis/hiredis/commit/d4e6f109a064690cde64765c654e679fea1d3548)
([Michael Grunder](https://github.com/michael-grunder))
## [1.0.1](https://github.com/redis/hiredis/tree/v1.0.1) - (2021-10-04)
<span style="color:red">This release erroneously bumped the SONAME, please use [1.0.2](https://github.com/redis/hiredis/tree/v1.0.2)</span>
Announcing Hiredis v1.0.1, a security release fixing CVE-2021-32765
- Fix for [CVE-2021-32765](https://github.com/redis/hiredis/security/advisories/GHSA-hfm9-39pp-55p2)
[commit](https://github.com/redis/hiredis/commit/76a7b10005c70babee357a7d0f2becf28ec7ed1e)
([Yossi Gottlieb](https://github.com/yossigo))
_Thanks to [Yossi Gottlieb](https://github.com/yossigo) for the security fix and to [Microsoft Security Vulnerability Research](https://www.microsoft.com/en-us/msrc/msvr) for finding the bug._ :sparkling_heart:
## [1.0.0](https://github.com/redis/hiredis/tree/v1.0.0) - (2020-08-03)
Announcing Hiredis v1.0.0, which adds support for RESP3, SSL connections, allocator injection, and better Windows support! :tada:
_A big thanks to everyone who helped with this release. The following list includes everyone who contributed at least five lines, sorted by lines contributed._ :sparkling_heart:
[Michael Grunder](https://github.com/michael-grunder), [Yossi Gottlieb](https://github.com/yossigo),
[Mark Nunberg](https://github.com/mnunberg), [Marcus Geelnard](https://github.com/mbitsnbites),
[Justin Brewer](https://github.com/justinbrewer), [Valentino Geron](https://github.com/valentinogeron),
[Minun Dragonation](https://github.com/dragonation), [Omri Steiner](https://github.com/OmriSteiner),
[Sangmoon Yi](https://github.com/jman-krafton), [Jinjiazh](https://github.com/jinjiazhang),
[Odin Hultgren Van Der Horst](https://github.com/Miniwoffer), [Muhammad Zahalqa](https://github.com/tryfinally),
[Nick Rivera](https://github.com/heronr), [Qi Yang](https://github.com/movebean),
[kevin1018](https://github.com/kevin1018)
[Full Changelog](https://github.com/redis/hiredis/compare/v0.14.1...v1.0.0)
**BREAKING CHANGES**:
* `redisOptions` now has two timeout fields. One for connecting, and one for commands. If you're presently using `options->timeout` you will need to change it to use `options->connect_timeout`. (See [example](https://github.com/redis/hiredis/commit/38b5ae543f5c99eb4ccabbe277770fc6bc81226f#diff-86ba39d37aa829c8c82624cce4f049fbL36))
* Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now protocol errors. This is consistent
with the RESP specification. On 32-bit platforms, the upper bound is lowered to `SIZE_MAX`.
* `redisReplyObjectFunctions.createArray` now takes `size_t` for its length parameter.
**New features:**
- Support for RESP3
[\#697](https://github.com/redis/hiredis/pull/697),
[\#805](https://github.com/redis/hiredis/pull/805),
[\#819](https://github.com/redis/hiredis/pull/819),
[\#841](https://github.com/redis/hiredis/pull/841)
([Yossi Gottlieb](https://github.com/yossigo), [Michael Grunder](https://github.com/michael-grunder))
- Support for SSL connections
[\#645](https://github.com/redis/hiredis/pull/645),
[\#699](https://github.com/redis/hiredis/pull/699),
[\#702](https://github.com/redis/hiredis/pull/702),
[\#708](https://github.com/redis/hiredis/pull/708),
[\#711](https://github.com/redis/hiredis/pull/711),
[\#821](https://github.com/redis/hiredis/pull/821),
[more](https://github.com/redis/hiredis/pulls?q=is%3Apr+is%3Amerged+SSL)
([Mark Nunberg](https://github.com/mnunberg), [Yossi Gottlieb](https://github.com/yossigo))
- Run-time allocator injection
[\#800](https://github.com/redis/hiredis/pull/800)
([Michael Grunder](https://github.com/michael-grunder))
- Improved Windows support (including MinGW and Windows CI)
[\#652](https://github.com/redis/hiredis/pull/652),
[\#663](https://github.com/redis/hiredis/pull/663)
([Marcus Geelnard](https://www.bitsnbites.eu/author/m/))
- Adds support for distinct connect and command timeouts
[\#839](https://github.com/redis/hiredis/pull/839),
[\#829](https://github.com/redis/hiredis/pull/829)
([Valentino Geron](https://github.com/valentinogeron))
- Add generic pointer and destructor to `redisContext` that users can use for context.
[\#855](https://github.com/redis/hiredis/pull/855)
([Michael Grunder](https://github.com/michael-grunder))
**Closed issues (that involved code changes):**
- Makefile does not install TLS libraries [\#809](https://github.com/redis/hiredis/issues/809)
- redisConnectWithOptions should not set command timeout [\#722](https://github.com/redis/hiredis/issues/722), [\#829](https://github.com/redis/hiredis/pull/829) ([valentinogeron](https://github.com/valentinogeron))
- Fix integer overflow in `sdsrange` [\#827](https://github.com/redis/hiredis/issues/827)
- INFO & CLUSTER commands failed when using RESP3 [\#802](https://github.com/redis/hiredis/issues/802)
- Windows compatibility patches [\#687](https://github.com/redis/hiredis/issues/687), [\#838](https://github.com/redis/hiredis/issues/838), [\#842](https://github.com/redis/hiredis/issues/842)
- RESP3 PUSH messages incorrectly use pending callback [\#825](https://github.com/redis/hiredis/issues/825)
- Asynchronous PSUBSCRIBE command fails when using RESP3 [\#815](https://github.com/redis/hiredis/issues/815)
- New SSL API [\#804](https://github.com/redis/hiredis/issues/804), [\#813](https://github.com/redis/hiredis/issues/813)
- Hard-coded limit of nested reply depth [\#794](https://github.com/redis/hiredis/issues/794)
- Fix TCP_NODELAY in Windows/OSX [\#679](https://github.com/redis/hiredis/issues/679), [\#690](https://github.com/redis/hiredis/issues/690), [\#779](https://github.com/redis/hiredis/issues/779), [\#785](https://github.com/redis/hiredis/issues/785),
- Added timers to libev adapter. [\#778](https://github.com/redis/hiredis/issues/778), [\#795](https://github.com/redis/hiredis/pull/795)
- Initialization discards const qualifier [\#777](https://github.com/redis/hiredis/issues/777)
- \[BUG\]\[MinGW64\] Error setting socket timeout [\#775](https://github.com/redis/hiredis/issues/775)
- undefined reference to hi_malloc [\#769](https://github.com/redis/hiredis/issues/769)
- hiredis pkg-config file incorrectly ignores multiarch libdir spec'n [\#767](https://github.com/redis/hiredis/issues/767)
- Don't use -G to build shared object on Solaris [\#757](https://github.com/redis/hiredis/issues/757)
- error when make USE\_SSL=1 [\#748](https://github.com/redis/hiredis/issues/748)
- Allow to change SSL Mode [\#646](https://github.com/redis/hiredis/issues/646)
- hiredis/adapters/libevent.h memleak [\#618](https://github.com/redis/hiredis/issues/618)
- redisLibuvPoll crash when server closes the connetion [\#545](https://github.com/redis/hiredis/issues/545)
- about redisAsyncDisconnect question [\#518](https://github.com/redis/hiredis/issues/518)
- hiredis adapters libuv error for help [\#508](https://github.com/redis/hiredis/issues/508)
- API/ABI changes analysis [\#506](https://github.com/redis/hiredis/issues/506)
- Memory leak patch in Redis [\#502](https://github.com/redis/hiredis/issues/502)
- Remove the depth limitation [\#421](https://github.com/redis/hiredis/issues/421)
**Merged pull requests:**
- Move SSL management to a distinct private pointer [\#855](https://github.com/redis/hiredis/pull/855) ([michael-grunder](https://github.com/michael-grunder))
- Move include to sockcompat.h to maintain style [\#850](https://github.com/redis/hiredis/pull/850) ([michael-grunder](https://github.com/michael-grunder))
- Remove erroneous tag and add license to push example [\#849](https://github.com/redis/hiredis/pull/849) ([michael-grunder](https://github.com/michael-grunder))
- fix windows compiling with mingw [\#848](https://github.com/redis/hiredis/pull/848) ([rmalizia44](https://github.com/rmalizia44))
- Some Windows quality of life improvements. [\#846](https://github.com/redis/hiredis/pull/846) ([michael-grunder](https://github.com/michael-grunder))
- Use \_WIN32 define instead of WIN32 [\#845](https://github.com/redis/hiredis/pull/845) ([michael-grunder](https://github.com/michael-grunder))
- Non Linux CI fixes [\#844](https://github.com/redis/hiredis/pull/844) ([michael-grunder](https://github.com/michael-grunder))
- Resp3 oob push support [\#841](https://github.com/redis/hiredis/pull/841) ([michael-grunder](https://github.com/michael-grunder))
- fix \#785: defer TCP\_NODELAY in async tcp connections [\#836](https://github.com/redis/hiredis/pull/836) ([OmriSteiner](https://github.com/OmriSteiner))
- sdsrange overflow fix [\#830](https://github.com/redis/hiredis/pull/830) ([michael-grunder](https://github.com/michael-grunder))
- Use explicit pointer casting for c++ compatibility [\#826](https://github.com/redis/hiredis/pull/826) ([aureus1](https://github.com/aureus1))
- Document allocator injection and completeness fix in test.c [\#824](https://github.com/redis/hiredis/pull/824) ([michael-grunder](https://github.com/michael-grunder))
- Use unique names for allocator struct members [\#823](https://github.com/redis/hiredis/pull/823) ([michael-grunder](https://github.com/michael-grunder))
- New SSL API to replace redisSecureConnection\(\). [\#821](https://github.com/redis/hiredis/pull/821) ([yossigo](https://github.com/yossigo))
- Add logic to handle RESP3 push messages [\#819](https://github.com/redis/hiredis/pull/819) ([michael-grunder](https://github.com/michael-grunder))
- Use standrad isxdigit instead of custom helper function. [\#814](https://github.com/redis/hiredis/pull/814) ([tryfinally](https://github.com/tryfinally))
- Fix missing SSL build/install options. [\#812](https://github.com/redis/hiredis/pull/812) ([yossigo](https://github.com/yossigo))
- Add link to ABI tracker [\#808](https://github.com/redis/hiredis/pull/808) ([michael-grunder](https://github.com/michael-grunder))
- Resp3 verbatim string support [\#805](https://github.com/redis/hiredis/pull/805) ([michael-grunder](https://github.com/michael-grunder))
- Allow users to replace allocator and handle OOM everywhere. [\#800](https://github.com/redis/hiredis/pull/800) ([michael-grunder](https://github.com/michael-grunder))
- Remove nested depth limitation. [\#797](https://github.com/redis/hiredis/pull/797) ([michael-grunder](https://github.com/michael-grunder))
- Attempt to fix compilation on Solaris [\#796](https://github.com/redis/hiredis/pull/796) ([michael-grunder](https://github.com/michael-grunder))
- Support timeouts in libev adapater [\#795](https://github.com/redis/hiredis/pull/795) ([michael-grunder](https://github.com/michael-grunder))
- Fix pkgconfig when installing to a custom lib dir [\#793](https://github.com/redis/hiredis/pull/793) ([michael-grunder](https://github.com/michael-grunder))
- Fix USE\_SSL=1 make/cmake on OSX and CMake tests [\#789](https://github.com/redis/hiredis/pull/789) ([michael-grunder](https://github.com/michael-grunder))
- Use correct libuv call on Windows [\#784](https://github.com/redis/hiredis/pull/784) ([michael-grunder](https://github.com/michael-grunder))
- Added CMake package config and fixed hiredis\_ssl on Windows [\#783](https://github.com/redis/hiredis/pull/783) ([michael-grunder](https://github.com/michael-grunder))
- CMake: Set hiredis\_ssl shared object version. [\#780](https://github.com/redis/hiredis/pull/780) ([yossigo](https://github.com/yossigo))
- Win32 tests and timeout fix [\#776](https://github.com/redis/hiredis/pull/776) ([michael-grunder](https://github.com/michael-grunder))
- Provides an optional cleanup callback for async data. [\#768](https://github.com/redis/hiredis/pull/768) ([heronr](https://github.com/heronr))
- Housekeeping fixes [\#764](https://github.com/redis/hiredis/pull/764) ([michael-grunder](https://github.com/michael-grunder))
- install alloc.h [\#756](https://github.com/redis/hiredis/pull/756) ([ch1aki](https://github.com/ch1aki))
- fix spelling mistakes [\#746](https://github.com/redis/hiredis/pull/746) ([ShooterIT](https://github.com/ShooterIT))
- Free the reply in redisGetReply when passed NULL [\#741](https://github.com/redis/hiredis/pull/741) ([michael-grunder](https://github.com/michael-grunder))
- Fix dead code in sslLogCallback relating to should\_log variable. [\#737](https://github.com/redis/hiredis/pull/737) ([natoscott](https://github.com/natoscott))
- Fix typo in dict.c. [\#731](https://github.com/redis/hiredis/pull/731) ([Kevin-Xi](https://github.com/Kevin-Xi))
- Adding an option to DISABLE\_TESTS [\#727](https://github.com/redis/hiredis/pull/727) ([pbotros](https://github.com/pbotros))
- Update README with SSL support. [\#720](https://github.com/redis/hiredis/pull/720) ([yossigo](https://github.com/yossigo))
- Fixes leaks in unit tests [\#715](https://github.com/redis/hiredis/pull/715) ([michael-grunder](https://github.com/michael-grunder))
- SSL Tests [\#711](https://github.com/redis/hiredis/pull/711) ([yossigo](https://github.com/yossigo))
- SSL Reorganization [\#708](https://github.com/redis/hiredis/pull/708) ([yossigo](https://github.com/yossigo))
- Fix MSVC build. [\#706](https://github.com/redis/hiredis/pull/706) ([yossigo](https://github.com/yossigo))
- SSL: Properly report SSL\_connect\(\) errors. [\#702](https://github.com/redis/hiredis/pull/702) ([yossigo](https://github.com/yossigo))
- Silent SSL trace to stdout by default. [\#699](https://github.com/redis/hiredis/pull/699) ([yossigo](https://github.com/yossigo))
- Port RESP3 support from Redis. [\#697](https://github.com/redis/hiredis/pull/697) ([yossigo](https://github.com/yossigo))
- Removed whitespace before newline [\#691](https://github.com/redis/hiredis/pull/691) ([Miniwoffer](https://github.com/Miniwoffer))
- Add install adapters header files [\#688](https://github.com/redis/hiredis/pull/688) ([kevin1018](https://github.com/kevin1018))
- Remove unnecessary null check before free [\#684](https://github.com/redis/hiredis/pull/684) ([qlyoung](https://github.com/qlyoung))
- redisReaderGetReply leak memory [\#671](https://github.com/redis/hiredis/pull/671) ([movebean](https://github.com/movebean))
- fix timeout code in windows [\#670](https://github.com/redis/hiredis/pull/670) ([jman-krafton](https://github.com/jman-krafton))
- test: fix errstr matching for musl libc [\#665](https://github.com/redis/hiredis/pull/665) ([ghost](https://github.com/ghost))
- Windows: MinGW fixes and Windows Travis builders [\#663](https://github.com/redis/hiredis/pull/663) ([mbitsnbites](https://github.com/mbitsnbites))
- The setsockopt and getsockopt API diffs from BSD socket and WSA one [\#662](https://github.com/redis/hiredis/pull/662) ([dragonation](https://github.com/dragonation))
- Fix Compile Error On Windows \(Visual Studio\) [\#658](https://github.com/redis/hiredis/pull/658) ([jinjiazhang](https://github.com/jinjiazhang))
- Fix NXDOMAIN test case [\#653](https://github.com/redis/hiredis/pull/653) ([michael-grunder](https://github.com/michael-grunder))
- Add MinGW support [\#652](https://github.com/redis/hiredis/pull/652) ([mbitsnbites](https://github.com/mbitsnbites))
- SSL Support [\#645](https://github.com/redis/hiredis/pull/645) ([mnunberg](https://github.com/mnunberg))
- Fix Invalid argument after redisAsyncConnectUnix [\#644](https://github.com/redis/hiredis/pull/644) ([codehz](https://github.com/codehz))
- Makefile: use predefined AR [\#632](https://github.com/redis/hiredis/pull/632) ([Mic92](https://github.com/Mic92))
- FreeBSD build fix [\#628](https://github.com/redis/hiredis/pull/628) ([devnexen](https://github.com/devnexen))
- Fix errors not propagating properly with libuv.h. [\#624](https://github.com/redis/hiredis/pull/624) ([yossigo](https://github.com/yossigo))
- Update README.md [\#621](https://github.com/redis/hiredis/pull/621) ([Crunsher](https://github.com/Crunsher))
- Fix redisBufferRead documentation [\#620](https://github.com/redis/hiredis/pull/620) ([hacst](https://github.com/hacst))
- Add CPPFLAGS to REAL\_CFLAGS [\#614](https://github.com/redis/hiredis/pull/614) ([thomaslee](https://github.com/thomaslee))
- Update createArray to take size\_t [\#597](https://github.com/redis/hiredis/pull/597) ([justinbrewer](https://github.com/justinbrewer))
- fix common realloc mistake and add null check more [\#580](https://github.com/redis/hiredis/pull/580) ([charsyam](https://github.com/charsyam))
- Proper error reporting for connect failures [\#578](https://github.com/redis/hiredis/pull/578) ([mnunberg](https://github.com/mnunberg))
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
## [1.0.0-rc1](https://github.com/redis/hiredis/tree/v1.0.0-rc1) - (2020-07-29)
_Note: There were no changes to code between v1.0.0-rc1 and v1.0.0 so see v1.0.0 for changelog_
### 0.14.1 (2020-03-13)
* Adds safe allocation wrappers (CVE-2020-7105, #747, #752) (Michael Grunder)
### 0.14.0 (2018-09-25)
**BREAKING CHANGES**:
* Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well.
If it was used to compare to other values, casting might be necessary or can be removed, if casting was applied before.
* Make string2ll static to fix conflict with Redis (Tom Lee [c3188b])
* Use -dynamiclib instead of -shared for OSX (Ryan Schmidt [a65537])
* Use string2ll from Redis w/added tests (Michael Grunder [7bef04, 60f622])
* Makefile - OSX compilation fixes (Ryan Schmidt [881fcb, 0e9af8])
* Remove redundant NULL checks (Justin Brewer [54acc8, 58e6b8])
* Fix bulk and multi-bulk length truncation (Justin Brewer [109197])
* Fix SIGSEGV in OpenBSD by checking for NULL before calling freeaddrinfo (Justin Brewer [546d94])
* Several POSIX compatibility fixes (Justin Brewer [bbeab8, 49bbaa, d1c1b6])
* Makefile - Compatibility fixes (Dimitri Vorobiev [3238cf, 12a9d1])
* Makefile - Fix make install on FreeBSD (Zach Shipko [a2ef2b])
* Makefile - don't assume $(INSTALL) is cp (Igor Gnatenko [725a96])
* Separate side-effect causing function from assert and small cleanup (amallia [b46413, 3c3234])
* Don't send negative values to `__redisAsyncCommand` (Frederik Deweerdt [706129])
* Fix leak if setsockopt fails (Frederik Deweerdt [e21c9c])
* Fix libevent leak (zfz [515228])
* Clean up GCC warning (Ichito Nagata [2ec774])
* Keep track of errno in `__redisSetErrorFromErrno()` as snprintf may use it (Jin Qing [25cd88])
* Solaris compilation fix (Donald Whyte [41b07d])
* Reorder linker arguments when building examples (Tustfarm-heart [06eedd])
* Keep track of subscriptions in case of rapid subscribe/unsubscribe (Hyungjin Kim [073dc8, be76c5, d46999])
* libuv use after free fix (Paul Scott [cbb956])
* Properly close socket fd on reconnect attempt (WSL [64d1ec])
* Skip valgrind in OSX tests (Jan-Erik Rediger [9deb78])
* Various updates for Travis testing OSX (Ted Nyman [fa3774, 16a459, bc0ea5])
* Update libevent (Chris Xin [386802])
* Change sds.h for building in C++ projects (Ali Volkan ATLI [f5b32e])
* Use proper format specifier in redisFormatSdsCommandArgv (Paulino Huerta, Jan-Erik Rediger [360a06, 8655a6])
* Better handling of NULL reply in example code (Jan-Erik Rediger [1b8ed3])
* Prevent overflow when formatting an error (Jan-Erik Rediger [0335cb])
* Compatibility fix for strerror_r (Tom Lee [bb1747])
* Properly detect integer parse/overflow errors (Justin Brewer [93421f])
* Adds CI for Windows and cygwin fixes (owent, [6c53d6, 6c3e40])
* Catch a buffer overflow when formatting the error message
* Import latest upstream sds. This breaks applications that are linked against the old hiredis v0.13
* Fix warnings, when compiled with -Wshadow
* Make hiredis compile in Cygwin on Windows, now CI-tested
* Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to `SIZE_MAX`.
**BREAKING CHANGES**:
* Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well.
If it was used to compare to other values, casting might be necessary or can be removed, if casting was applied before.
* Remove backwards compatibility macro's
@@ -526,7 +94,7 @@ The parser, standalone since v0.12.0, can now be compiled on Windows
* Add IPv6 support
* Remove possibility of multiple close on same fd
* Remove possiblity of multiple close on same fd
* Add ability to bind source address on connect
@@ -578,3 +146,4 @@ The parser, standalone since v0.12.0, can now be compiled on Windows
### 0.10.0
* See commit log.
-237
View File
@@ -1,237 +0,0 @@
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
OPTION(BUILD_SHARED_LIBS "Build shared libraries" ON)
OPTION(ENABLE_SSL "Build hiredis_ssl for SSL support" OFF)
OPTION(DISABLE_TESTS "If tests should be compiled or not" OFF)
OPTION(ENABLE_SSL_TESTS "Should we test SSL connections" OFF)
OPTION(ENABLE_EXAMPLES "Enable building hiredis examples" OFF)
OPTION(ENABLE_ASYNC_TESTS "Should we run all asynchronous API tests" OFF)
MACRO(getVersionBit name)
SET(VERSION_REGEX "^#define ${name} (.+)$")
FILE(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/hiredis.h"
VERSION_BIT REGEX ${VERSION_REGEX})
STRING(REGEX REPLACE ${VERSION_REGEX} "\\1" ${name} "${VERSION_BIT}")
ENDMACRO(getVersionBit)
getVersionBit(HIREDIS_MAJOR)
getVersionBit(HIREDIS_MINOR)
getVersionBit(HIREDIS_PATCH)
getVersionBit(HIREDIS_SONAME)
SET(VERSION "${HIREDIS_MAJOR}.${HIREDIS_MINOR}.${HIREDIS_PATCH}")
MESSAGE("Detected version: ${VERSION}")
PROJECT(hiredis LANGUAGES "C" VERSION "${VERSION}")
INCLUDE(GNUInstallDirs)
# Hiredis requires C99
SET(CMAKE_C_STANDARD 99)
SET(CMAKE_DEBUG_POSTFIX d)
SET(hiredis_sources
alloc.c
async.c
hiredis.c
net.c
read.c
sds.c
sockcompat.c)
SET(hiredis_sources ${hiredis_sources})
IF(WIN32)
ADD_DEFINITIONS(-D_CRT_SECURE_NO_WARNINGS -DWIN32_LEAN_AND_MEAN)
ENDIF()
ADD_LIBRARY(hiredis ${hiredis_sources})
ADD_LIBRARY(hiredis::hiredis ALIAS hiredis)
set(hiredis_export_name hiredis CACHE STRING "Name of the exported target")
set_target_properties(hiredis PROPERTIES EXPORT_NAME ${hiredis_export_name})
SET_TARGET_PROPERTIES(hiredis
PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS TRUE
VERSION "${HIREDIS_SONAME}")
IF(MSVC)
SET_TARGET_PROPERTIES(hiredis
PROPERTIES COMPILE_FLAGS /Z7)
ENDIF()
IF(WIN32)
TARGET_LINK_LIBRARIES(hiredis PUBLIC ws2_32 crypt32)
ELSEIF(CMAKE_SYSTEM_NAME MATCHES "FreeBSD")
TARGET_LINK_LIBRARIES(hiredis PUBLIC m)
ELSEIF(CMAKE_SYSTEM_NAME MATCHES "SunOS")
TARGET_LINK_LIBRARIES(hiredis PUBLIC socket)
ENDIF()
TARGET_INCLUDE_DIRECTORIES(hiredis PUBLIC $<INSTALL_INTERFACE:include> $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>)
CONFIGURE_FILE(hiredis.pc.in hiredis.pc @ONLY)
set(CPACK_PACKAGE_VENDOR "Redis")
set(CPACK_PACKAGE_DESCRIPTION "\
Hiredis is a minimalistic C client library for the Redis database.
It is minimalistic because it just adds minimal support for the protocol, \
but at the same time it uses a high level printf-alike API in order to make \
it much higher level than otherwise suggested by its minimal code base and the \
lack of explicit bindings for every Redis command.
Apart from supporting sending commands and receiving replies, it comes with a \
reply parser that is decoupled from the I/O layer. It is a stream parser designed \
for easy reusability, which can for instance be used in higher level language bindings \
for efficient reply parsing.
Hiredis only supports the binary-safe Redis protocol, so you can use it with any Redis \
version >= 1.2.0.
The library comes with multiple APIs. There is the synchronous API, the asynchronous API \
and the reply parsing API.")
set(CPACK_PACKAGE_HOMEPAGE_URL "https://github.com/redis/hiredis")
set(CPACK_PACKAGE_CONTACT "michael dot grunder at gmail dot com")
set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
set(CPACK_RPM_PACKAGE_AUTOREQPROV ON)
include(CPack)
INSTALL(TARGETS hiredis
EXPORT hiredis-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
if (MSVC AND BUILD_SHARED_LIBS)
INSTALL(FILES $<TARGET_PDB_FILE:hiredis>
DESTINATION ${CMAKE_INSTALL_BINDIR}
CONFIGURATIONS Debug RelWithDebInfo)
endif()
# For NuGet packages
INSTALL(FILES hiredis.targets
DESTINATION build/native)
INSTALL(FILES hiredis.h read.h sds.h async.h alloc.h sockcompat.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(DIRECTORY adapters
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
export(EXPORT hiredis-targets
FILE "${CMAKE_CURRENT_BINARY_DIR}/hiredis-targets.cmake"
NAMESPACE hiredis::)
if(WIN32)
SET(CMAKE_CONF_INSTALL_DIR share/hiredis)
else()
SET(CMAKE_CONF_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR}/cmake/hiredis)
endif()
SET(INCLUDE_INSTALL_DIR include)
include(CMakePackageConfigHelpers)
write_basic_package_version_file("${CMAKE_CURRENT_BINARY_DIR}/hiredis-config-version.cmake"
COMPATIBILITY SameMajorVersion)
configure_package_config_file(hiredis-config.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/hiredis-config.cmake
INSTALL_DESTINATION ${CMAKE_CONF_INSTALL_DIR}
PATH_VARS INCLUDE_INSTALL_DIR)
INSTALL(EXPORT hiredis-targets
FILE hiredis-targets.cmake
NAMESPACE hiredis::
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis-config.cmake
${CMAKE_CURRENT_BINARY_DIR}/hiredis-config-version.cmake
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
IF(ENABLE_SSL)
IF (NOT OPENSSL_ROOT_DIR)
IF (APPLE)
SET(OPENSSL_ROOT_DIR "/usr/local/opt/openssl")
ENDIF()
ENDIF()
FIND_PACKAGE(OpenSSL REQUIRED)
SET(hiredis_ssl_sources
ssl.c)
ADD_LIBRARY(hiredis_ssl ${hiredis_ssl_sources})
ADD_LIBRARY(hiredis::hiredis_ssl ALIAS hiredis_ssl)
IF (APPLE AND BUILD_SHARED_LIBS)
SET_PROPERTY(TARGET hiredis_ssl PROPERTY LINK_FLAGS "-Wl,-undefined -Wl,dynamic_lookup")
ENDIF()
SET_TARGET_PROPERTIES(hiredis_ssl
PROPERTIES
WINDOWS_EXPORT_ALL_SYMBOLS TRUE
VERSION "${HIREDIS_SONAME}")
IF(MSVC)
SET_TARGET_PROPERTIES(hiredis_ssl
PROPERTIES COMPILE_FLAGS /Z7)
ENDIF()
TARGET_LINK_LIBRARIES(hiredis_ssl PRIVATE OpenSSL::SSL)
IF(WIN32)
TARGET_LINK_LIBRARIES(hiredis_ssl PRIVATE hiredis)
ENDIF()
CONFIGURE_FILE(hiredis_ssl.pc.in hiredis_ssl.pc @ONLY)
INSTALL(TARGETS hiredis_ssl
EXPORT hiredis_ssl-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
if (MSVC AND BUILD_SHARED_LIBS)
INSTALL(FILES $<TARGET_PDB_FILE:hiredis_ssl>
DESTINATION ${CMAKE_INSTALL_BINDIR}
CONFIGURATIONS Debug RelWithDebInfo)
endif()
INSTALL(FILES hiredis_ssl.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
export(EXPORT hiredis_ssl-targets
FILE "${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-targets.cmake"
NAMESPACE hiredis::)
if(WIN32)
SET(CMAKE_CONF_INSTALL_DIR share/hiredis_ssl)
else()
SET(CMAKE_CONF_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR}/cmake/hiredis_ssl)
endif()
configure_package_config_file(hiredis_ssl-config.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-config.cmake
INSTALL_DESTINATION ${CMAKE_CONF_INSTALL_DIR}
PATH_VARS INCLUDE_INSTALL_DIR)
INSTALL(EXPORT hiredis_ssl-targets
FILE hiredis_ssl-targets.cmake
NAMESPACE hiredis::
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-config.cmake
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
ENDIF()
IF(NOT DISABLE_TESTS)
ENABLE_TESTING()
ADD_EXECUTABLE(hiredis-test test.c)
TARGET_LINK_LIBRARIES(hiredis-test hiredis)
IF(ENABLE_SSL_TESTS)
ADD_DEFINITIONS(-DHIREDIS_TEST_SSL=1)
TARGET_LINK_LIBRARIES(hiredis-test hiredis_ssl)
ENDIF()
IF(ENABLE_ASYNC_TESTS)
ADD_DEFINITIONS(-DHIREDIS_TEST_ASYNC=1)
TARGET_LINK_LIBRARIES(hiredis-test event)
ENDIF()
ADD_TEST(NAME hiredis-test
COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/test.sh)
ENDIF()
# Add examples
IF(ENABLE_EXAMPLES)
ADD_SUBDIRECTORY(examples)
ENDIF(ENABLE_EXAMPLES)
+57 -194
View File
@@ -3,8 +3,8 @@
# Copyright (C) 2010-2011 Pieter Noordhuis <pcnoordhuis at gmail dot com>
# This file is released under the BSD license, see the COPYING file
OBJ=alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev hiredis-example-glib hiredis-example-push hiredis-example-poll
OBJ=net.o hiredis.o sds.o async.o read.o
EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev hiredis-example-glib
TESTS=hiredis-test
LIBNAME=libhiredis
PKGCONFNAME=hiredis.pc
@@ -36,174 +36,71 @@ endef
export REDIS_TEST_CONFIG
# Fallback to gcc when $CC is not in $PATH.
CC:=$(shell sh -c 'type $${CC%% *} >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
CXX:=$(shell sh -c 'type $${CXX%% *} >/dev/null 2>/dev/null && echo $(CXX) || echo g++')
CC:=$(shell sh -c 'type $(CC) >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
CXX:=$(shell sh -c 'type $(CXX) >/dev/null 2>/dev/null && echo $(CXX) || echo g++')
OPTIMIZATION?=-O3
WARNINGS=-Wall -Wextra -Werror -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers
WARNINGS=-Wall -W -Wstrict-prototypes -Wwrite-strings
DEBUG_FLAGS?= -g -ggdb
REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CPPFLAGS) $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(PLATFORM_FLAGS)
REAL_LDFLAGS=$(LDFLAGS)
REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(ARCH)
REAL_LDFLAGS=$(LDFLAGS) $(ARCH)
DYLIBSUFFIX=so
STLIBSUFFIX=a
DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
DYLIB_MAJOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
DYLIBNAME=$(LIBNAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) $(PLATFORM_FLAGS) -shared -Wl,-soname,$(DYLIB_MINOR_NAME)
DYLIB_MAKE_CMD=$(CC) -shared -Wl,-soname,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
STLIBNAME=$(LIBNAME).$(STLIBSUFFIX)
STLIB_MAKE_CMD=$(AR) rcs
#################### SSL variables start ####################
SSL_OBJ=ssl.o
SSL_LIBNAME=libhiredis_ssl
SSL_PKGCONFNAME=hiredis_ssl.pc
SSL_INSTALLNAME=install-ssl
SSL_DYLIB_MINOR_NAME=$(SSL_LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
SSL_DYLIB_MAJOR_NAME=$(SSL_LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
SSL_DYLIBNAME=$(SSL_LIBNAME).$(DYLIBSUFFIX)
SSL_STLIBNAME=$(SSL_LIBNAME).$(STLIBSUFFIX)
SSL_DYLIB_MAKE_CMD=$(CC) $(PLATFORM_FLAGS) -shared -Wl,-soname,$(SSL_DYLIB_MINOR_NAME)
USE_SSL?=0
ifeq ($(USE_SSL),1)
# This is required for test.c only
CFLAGS+=-DHIREDIS_TEST_SSL
EXAMPLES+=hiredis-example-ssl hiredis-example-libevent-ssl
SSL_STLIB=$(SSL_STLIBNAME)
SSL_DYLIB=$(SSL_DYLIBNAME)
SSL_PKGCONF=$(SSL_PKGCONFNAME)
SSL_INSTALL=$(SSL_INSTALLNAME)
else
SSL_STLIB=
SSL_DYLIB=
SSL_PKGCONF=
SSL_INSTALL=
endif
##################### SSL variables end #####################
STLIB_MAKE_CMD=ar rcs $(STLIBNAME)
# Platform-specific overrides
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
# This is required for test.c only
ifeq ($(TEST_ASYNC),1)
export CFLAGS+=-DHIREDIS_TEST_ASYNC
endif
ifeq ($(USE_SSL),1)
ifndef OPENSSL_PREFIX
ifeq ($(uname_S),Darwin)
SEARCH_PATH1=/opt/homebrew/opt/openssl
SEARCH_PATH2=/usr/local/opt/openssl
ifneq ($(wildcard $(SEARCH_PATH1)),)
OPENSSL_PREFIX=$(SEARCH_PATH1)
else ifneq ($(wildcard $(SEARCH_PATH2)),)
OPENSSL_PREFIX=$(SEARCH_PATH2)
endif
endif
endif
ifdef OPENSSL_PREFIX
CFLAGS+=-I$(OPENSSL_PREFIX)/include
SSL_LDFLAGS+=-L$(OPENSSL_PREFIX)/lib
endif
SSL_LDFLAGS+=-lssl -lcrypto
endif
ifeq ($(uname_S),FreeBSD)
LDFLAGS+=-lm
IS_GCC=$(shell sh -c '$(CC) --version 2>/dev/null |egrep -i -c "gcc"')
ifeq ($(IS_GCC),1)
REAL_CFLAGS+=-pedantic
endif
else
REAL_CFLAGS+=-pedantic
endif
ifeq ($(uname_S),SunOS)
IS_SUN_CC=$(shell sh -c '$(CC) -V 2>&1 |egrep -i -c "sun|studio"')
ifeq ($(IS_SUN_CC),1)
SUN_SHARED_FLAG=-G
else
SUN_SHARED_FLAG=-shared
endif
REAL_LDFLAGS+= -ldl -lnsl -lsocket
DYLIB_MAKE_CMD=$(CC) $(SUN_SHARED_FLAG) -o $(DYLIBNAME) -h $(DYLIB_MINOR_NAME) $(LDFLAGS)
SSL_DYLIB_MAKE_CMD=$(CC) $(SUN_SHARED_FLAG) -o $(SSL_DYLIBNAME) -h $(SSL_DYLIB_MINOR_NAME) $(LDFLAGS) $(SSL_LDFLAGS)
DYLIB_MAKE_CMD=$(CC) -G -o $(DYLIBNAME) -h $(DYLIB_MINOR_NAME) $(LDFLAGS)
INSTALL= cp -r
endif
ifeq ($(uname_S),Darwin)
DYLIBSUFFIX=dylib
DYLIB_MINOR_NAME=$(LIBNAME).$(HIREDIS_SONAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) -dynamiclib -Wl,-install_name,$(PREFIX)/$(LIBRARY_PATH)/$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
SSL_DYLIB_MAKE_CMD=$(CC) -dynamiclib -Wl,-install_name,$(PREFIX)/$(LIBRARY_PATH)/$(SSL_DYLIB_MINOR_NAME) -o $(SSL_DYLIBNAME) $(LDFLAGS) $(SSL_LDFLAGS)
DYLIB_PLUGIN=-Wl,-undefined -Wl,dynamic_lookup
DYLIB_MAKE_CMD=$(CC) -shared -Wl,-install_name,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
endif
all: dynamic static hiredis-test pkgconfig
dynamic: $(DYLIBNAME) $(SSL_DYLIB)
static: $(STLIBNAME) $(SSL_STLIB)
pkgconfig: $(PKGCONFNAME) $(SSL_PKGCONF)
all: $(DYLIBNAME) $(STLIBNAME) hiredis-test $(PKGCONFNAME)
# Deps (use make dep to generate this)
alloc.o: alloc.c fmacros.h alloc.h
async.o: async.c fmacros.h alloc.h async.h hiredis.h read.h sds.h net.h dict.c dict.h win32.h async_private.h
dict.o: dict.c fmacros.h alloc.h dict.h
hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h alloc.h net.h async.h win32.h
net.o: net.c fmacros.h net.h hiredis.h read.h sds.h alloc.h sockcompat.h win32.h
read.o: read.c fmacros.h alloc.h read.h sds.h win32.h
sds.o: sds.c sds.h sdsalloc.h alloc.h
sockcompat.o: sockcompat.c sockcompat.h
test.o: test.c fmacros.h hiredis.h read.h sds.h alloc.h net.h sockcompat.h win32.h
async.o: async.c fmacros.h async.h hiredis.h read.h sds.h net.h dict.c dict.h
dict.o: dict.c fmacros.h dict.h
hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h net.h
net.o: net.c fmacros.h net.h hiredis.h read.h sds.h
read.o: read.c fmacros.h read.h sds.h
sds.o: sds.c sds.h
test.o: test.c fmacros.h hiredis.h read.h sds.h
$(DYLIBNAME): $(OBJ)
$(DYLIB_MAKE_CMD) -o $(DYLIBNAME) $(OBJ) $(REAL_LDFLAGS)
$(DYLIB_MAKE_CMD) $(OBJ)
$(STLIBNAME): $(OBJ)
$(STLIB_MAKE_CMD) $(STLIBNAME) $(OBJ)
$(STLIB_MAKE_CMD) $(OBJ)
#################### SSL building rules start ####################
$(SSL_DYLIBNAME): $(SSL_OBJ)
$(SSL_DYLIB_MAKE_CMD) $(DYLIB_PLUGIN) -o $(SSL_DYLIBNAME) $(SSL_OBJ) $(REAL_LDFLAGS) $(LDFLAGS) $(SSL_LDFLAGS)
$(SSL_STLIBNAME): $(SSL_OBJ)
$(STLIB_MAKE_CMD) $(SSL_STLIBNAME) $(SSL_OBJ)
$(SSL_OBJ): ssl.c hiredis.h read.h sds.h alloc.h async.h win32.h async_private.h
#################### SSL building rules end ####################
dynamic: $(DYLIBNAME)
static: $(STLIBNAME)
# Binaries:
hiredis-example-libevent: examples/example-libevent.c adapters/libevent.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libevent-ssl: examples/example-libevent-ssl.c adapters/libevent.h $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -levent $(STLIBNAME)
hiredis-example-libev: examples/example-libev.c adapters/libev.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -lev $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libhv: examples/example-libhv.c adapters/libhv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -lhv $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -lev $(STLIBNAME)
hiredis-example-glib: examples/example-glib.c adapters/glib.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(shell pkg-config --cflags --libs glib-2.0) $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) $(shell pkg-config --cflags --libs glib-2.0) -I. $< $(STLIBNAME)
hiredis-example-ivykis: examples/example-ivykis.c adapters/ivykis.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -livykis $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -livykis $(STLIBNAME)
hiredis-example-macosx: examples/example-macosx.c adapters/macosx.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-ssl: examples/example-ssl.c $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
hiredis-example-poll: examples/example-poll.c adapters/poll.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME)
ifndef AE_DIR
hiredis-example-ae:
@@ -215,13 +112,12 @@ hiredis-example-ae: examples/example-ae.c adapters/ae.h $(STLIBNAME)
endif
ifndef LIBUV_DIR
# dynamic link libuv.so
hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. -I$(LIBUV_DIR)/include $< -luv -lpthread -lrt $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libuv:
@echo "Please specify LIBUV_DIR (e.g. ../libuv/)"
@false
else
# use user provided static lib
hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME)
endif
ifeq ($(and $(QT_MOC),$(QT_INCLUDE_DIR),$(QT_LIBRARY_DIR)),)
@@ -238,93 +134,61 @@ hiredis-example-qt: examples/example-qt.cpp adapters/qt.h $(STLIBNAME)
endif
hiredis-example: examples/example.c $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-push: examples/example-push.c $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< $(STLIBNAME)
examples: $(EXAMPLES)
TEST_LIBS = $(STLIBNAME) $(SSL_STLIB)
TEST_LDFLAGS = $(SSL_LDFLAGS)
ifeq ($(USE_SSL),1)
TEST_LDFLAGS += -pthread
endif
ifeq ($(TEST_ASYNC),1)
TEST_LDFLAGS += -levent
endif
hiredis-test: test.o $(TEST_LIBS)
$(CC) -o $@ $(REAL_CFLAGS) -I. $^ $(REAL_LDFLAGS) $(TEST_LDFLAGS)
hiredis-test: test.o $(STLIBNAME)
hiredis-%: %.o $(STLIBNAME)
$(CC) $(REAL_CFLAGS) -o $@ $< $(TEST_LIBS) $(REAL_LDFLAGS)
$(CC) $(REAL_CFLAGS) -o $@ $(REAL_LDFLAGS) $< $(STLIBNAME)
test: hiredis-test
./hiredis-test
check: hiredis-test
TEST_SSL=$(USE_SSL) ./test.sh
@echo "$$REDIS_TEST_CONFIG" | $(REDIS_SERVER) -
$(PRE) ./hiredis-test -h 127.0.0.1 -p $(REDIS_PORT) -s /tmp/hiredis-test-redis.sock || \
( kill `cat /tmp/hiredis-test-redis.pid` && false )
kill `cat /tmp/hiredis-test-redis.pid`
.c.o:
$(CC) -std=c99 -c $(REAL_CFLAGS) $<
$(CC) -std=c99 -pedantic -c $(REAL_CFLAGS) $<
clean:
rm -rf $(DYLIBNAME) $(STLIBNAME) $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
rm -rf $(DYLIBNAME) $(STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
dep:
$(CC) $(CPPFLAGS) $(CFLAGS) -MM *.c
$(CC) -MM *.c
INSTALL?= cp -pPR
ifeq ($(uname_S),SunOS)
INSTALL?= cp -r
endif
INSTALL?= cp -a
$(PKGCONFNAME): hiredis.h
@echo "Generating $@ for pkgconfig..."
@echo prefix=$(PREFIX) > $@
@echo exec_prefix=\$${prefix} >> $@
@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
@echo includedir=$(PREFIX)/include >> $@
@echo pkgincludedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo includedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo >> $@
@echo Name: hiredis >> $@
@echo Description: Minimalistic C client library for Redis. >> $@
@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
@echo Libs: -L\$${libdir} -lhiredis >> $@
@echo Cflags: -I\$${pkgincludedir} -I\$${includedir} -D_FILE_OFFSET_BITS=64 >> $@
@echo Cflags: -I\$${includedir} -D_FILE_OFFSET_BITS=64 >> $@
$(SSL_PKGCONFNAME): hiredis_ssl.h
@echo "Generating $@ for pkgconfig..."
@echo prefix=$(PREFIX) > $@
@echo exec_prefix=\$${prefix} >> $@
@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
@echo includedir=$(PREFIX)/include >> $@
@echo pkgincludedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo >> $@
@echo Name: hiredis_ssl >> $@
@echo Description: SSL Support for hiredis. >> $@
@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
@echo Requires: hiredis >> $@
@echo Libs: -L\$${libdir} -lhiredis_ssl >> $@
@echo Libs.private: -lssl -lcrypto >> $@
install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME) $(SSL_INSTALL)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_INCLUDE_PATH)/adapters $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis.h async.h read.h sds.h alloc.h sockcompat.h $(INSTALL_INCLUDE_PATH)
$(INSTALL) adapters/*.h $(INSTALL_INCLUDE_PATH)/adapters
install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis.h async.h read.h sds.h adapters $(INSTALL_INCLUDE_PATH)
$(INSTALL) $(DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(DYLIB_MINOR_NAME)
cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIBNAME) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIB_MAJOR_NAME)
cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIBNAME)
$(INSTALL) $(STLIBNAME) $(INSTALL_LIBRARY_PATH)
mkdir -p $(INSTALL_PKGCONF_PATH)
$(INSTALL) $(PKGCONFNAME) $(INSTALL_PKGCONF_PATH)
install-ssl: $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(SSL_PKGCONFNAME)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis_ssl.h $(INSTALL_INCLUDE_PATH)
$(INSTALL) $(SSL_DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(SSL_DYLIB_MINOR_NAME)
cd $(INSTALL_LIBRARY_PATH) && ln -sf $(SSL_DYLIB_MINOR_NAME) $(SSL_DYLIBNAME) && ln -sf $(SSL_DYLIB_MINOR_NAME) $(SSL_DYLIB_MAJOR_NAME)
$(INSTALL) $(SSL_STLIBNAME) $(INSTALL_LIBRARY_PATH)
mkdir -p $(INSTALL_PKGCONF_PATH)
$(INSTALL) $(SSL_PKGCONFNAME) $(INSTALL_PKGCONF_PATH)
32bit:
@echo ""
@echo "WARNING: if this fails under Linux you probably need to install libc6-dev-i386"
@@ -339,14 +203,13 @@ gprof:
$(MAKE) CFLAGS="-pg" LDFLAGS="-pg"
gcov:
$(MAKE) CFLAGS+="-fprofile-arcs -ftest-coverage" LDFLAGS="-fprofile-arcs"
$(MAKE) CFLAGS="-fprofile-arcs -ftest-coverage" LDFLAGS="-fprofile-arcs"
coverage: gcov
make check
mkdir -p tmp/lcov
lcov -d . -c --exclude '/usr*' -o tmp/lcov/hiredis.info
lcov -q -l tmp/lcov/hiredis.info
genhtml --legend -q -o tmp/lcov/report tmp/lcov/hiredis.info
lcov -d . -c -o tmp/lcov/hiredis.info
genhtml --legend -o tmp/lcov/report tmp/lcov/hiredis.info
noopt:
$(MAKE) OPTIMIZATION=""
+24 -434
View File
@@ -1,11 +1,10 @@
[![Build Status](https://travis-ci.org/redis/hiredis.png)](https://travis-ci.org/redis/hiredis)
[![Build Status](https://github.com/redis/hiredis/actions/workflows/build.yml/badge.svg)](https://github.com/redis/hiredis/actions/workflows/build.yml)
**This Readme reflects the latest changed in the master branch. See [v1.0.0](https://github.com/redis/hiredis/tree/v1.0.0) for the Readme and documentation for the latest release ([API/ABI history](https://abi-laboratory.pro/?view=timeline&l=hiredis)).**
**This Readme reflects the latest changed in the master branch. See [v0.13.3](https://github.com/redis/hiredis/tree/v0.13.3) for the Readme and documentation for the latest release.**
# HIREDIS
Hiredis is a minimalistic C client library for the [Redis](https://redis.io/) database.
Hiredis is a minimalistic C client library for the [Redis](http://redis.io/) database.
It is minimalistic because it just adds minimal support for the protocol, but
at the same time it uses a high level printf-alike API in order to make it
@@ -23,46 +22,14 @@ Redis version >= 1.2.0.
The library comes with multiple APIs. There is the
*synchronous API*, the *asynchronous API* and the *reply parsing API*.
## Upgrading to `1.1.0`
Almost all users will simply need to recompile their applications against the newer version of hiredis.
**NOTE**: Hiredis can now return `nan` in addition to `-inf` and `inf` in a `REDIS_REPLY_DOUBLE`.
Applications that deal with `RESP3` doubles should make sure to account for this.
## Upgrading to `1.0.2`
<span style="color:red">NOTE: v1.0.1 erroneously bumped SONAME, which is why it is skipped here.</span>
Version 1.0.2 is simply 1.0.0 with a fix for [CVE-2021-32765](https://github.com/redis/hiredis/security/advisories/GHSA-hfm9-39pp-55p2). They are otherwise identical.
## Upgrading to `1.0.0`
Version 1.0.0 marks the first stable release of Hiredis.
Version 1.0.0 marks a stable release of hiredis.
It includes some minor breaking changes, mostly to make the exposed API more uniform and self-explanatory.
It also bundles the updated `sds` library, to sync up with upstream and Redis.
For most applications a recompile against the new hiredis should be enough.
For code changes see the [Changelog](CHANGELOG.md).
_Note: As described below, a few member names have been changed but most applications should be able to upgrade with minor code changes and recompiling._
## IMPORTANT: Breaking changes from `0.14.1` -> `1.0.0`
* `redisContext` has two additional members (`free_privdata`, and `privctx`).
* `redisOptions.timeout` has been renamed to `redisOptions.connect_timeout`, and we've added `redisOptions.command_timeout`.
* `redisReplyObjectFunctions.createArray` now takes `size_t` instead of `int` for its length parameter.
## IMPORTANT: Breaking changes when upgrading from 0.13.x -> 0.14.x
Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to `SIZE_MAX`.
Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well. If it was used to
compare to other values, casting might be necessary or can be removed, if
casting was applied before.
## Upgrading from `<0.9.0`
Version 0.9.0 is a major overhaul of hiredis in every aspect. However, upgrading existing
@@ -89,7 +56,6 @@ an error state. The field `errstr` will contain a string with a description of
the error. More information on errors can be found in the **Errors** section.
After trying to connect to Redis using `redisConnect` you should
check the `err` field to see if establishing the connection was successful:
```c
redisContext *c = redisConnect("127.0.0.1", 6379);
if (c == NULL || c->err) {
@@ -102,74 +68,8 @@ if (c == NULL || c->err) {
}
```
One can also use `redisConnectWithOptions` which takes a `redisOptions` argument
that can be configured with endpoint information as well as many different flags
to change how the `redisContext` will be configured.
```c
redisOptions opt = {0};
/* One can set the endpoint with one of our helper macros */
if (tcp) {
REDIS_OPTIONS_SET_TCP(&opt, "localhost", 6379);
} else {
REDIS_OPTIONS_SET_UNIX(&opt, "/tmp/redis.sock");
}
/* And privdata can be specified with another helper */
REDIS_OPTIONS_SET_PRIVDATA(&opt, myPrivData, myPrivDataDtor);
/* Finally various options may be set via the `options` member, as described below */
opt->options |= REDIS_OPT_PREFER_IPV4;
```
If a connection is lost, `int redisReconnect(redisContext *c)` can be used to restore the connection using the same endpoint and options as the given context.
### Configurable redisOptions flags
There are several flags you may set in the `redisOptions` struct to change default behavior. You can specify the flags via the `redisOptions->options` member.
| Flag | Description |
| --- | --- |
| REDIS\_OPT\_NONBLOCK | Tells hiredis to make a non-blocking connection. |
| REDIS\_OPT\_REUSEADDR | Tells hiredis to set the [SO_REUSEADDR](https://man7.org/linux/man-pages/man7/socket.7.html) socket option |
| REDIS\_OPT\_PREFER\_IPV4<br>REDIS\_OPT\_PREFER_IPV6<br>REDIS\_OPT\_PREFER\_IP\_UNSPEC | Informs hiredis to either prefer IPv4 or IPv6 when invoking [getaddrinfo](https://man7.org/linux/man-pages/man3/gai_strerror.3.html). `REDIS_OPT_PREFER_IP_UNSPEC` will cause hiredis to specify `AF_UNSPEC` in the getaddrinfo call, which means both IPv4 and IPv6 addresses will be searched simultaneously.<br>Hiredis prefers IPv4 by default. |
| REDIS\_OPT\_NO\_PUSH\_AUTOFREE | Tells hiredis to not install the default RESP3 PUSH handler (which just intercepts and frees the replies). This is useful in situations where you want to process these messages in-band. |
| REDIS\_OPT\_NOAUTOFREEREPLIES | **ASYNC**: tells hiredis not to automatically invoke `freeReplyObject` after executing the reply callback. |
| REDIS\_OPT\_NOAUTOFREE | **ASYNC**: Tells hiredis not to automatically free the `redisAsyncContext` on connection/communication failure, but only if the user makes an explicit call to `redisAsyncDisconnect` or `redisAsyncFree` |
*Note: A `redisContext` is not thread-safe.*
### Other configuration using socket options
The following socket options are applied directly to the underlying socket.
The values are not stored in the `redisContext`, so they are not automatically applied when reconnecting using `redisReconnect()`.
These functions return `REDIS_OK` on success.
On failure, `REDIS_ERR` is returned and the underlying connection is closed.
To configure these for an asyncronous context (see *Asynchronous API* below), use `ac->c` to get the redisContext out of an asyncRedisContext.
```C
int redisEnableKeepAlive(redisContext *c);
int redisEnableKeepAliveWithInterval(redisContext *c, int interval);
```
Enables TCP keepalive by setting the following socket options (with some variations depending on OS):
* `SO_KEEPALIVE`;
* `TCP_KEEPALIVE` or `TCP_KEEPIDLE`, value configurable using the `interval` parameter, default 15 seconds;
* `TCP_KEEPINTVL` set to 1/3 of `interval`;
* `TCP_KEEPCNT` set to 3.
```C
int redisSetTcpUserTimeout(redisContext *c, unsigned int timeout);
```
Set the `TCP_USER_TIMEOUT` Linux-specific socket option which is as described in the `tcp` man page:
> When the value is greater than 0, it specifies the maximum amount of time in milliseconds that trans mitted data may remain unacknowledged before TCP will forcibly close the corresponding connection and return ETIMEDOUT to the application.
> If the option value is specified as 0, TCP will use the system default.
### Sending commands
There are several ways to issue commands to Redis. The first that will be introduced is
@@ -210,8 +110,6 @@ The standard replies that `redisCommand` are of the type `redisReply`. The
`type` field in the `redisReply` should be used to test what kind of reply
was received:
### RESP2
* **`REDIS_REPLY_STATUS`**:
* The command replied with a status reply. The status string can be accessed using `reply->str`.
The length of this string can be accessed using `reply->len`.
@@ -236,51 +134,16 @@ was received:
and can be accessed via `reply->element[..index..]`.
Redis may reply with nested arrays but this is fully supported.
### RESP3
Hiredis also supports every new `RESP3` data type which are as follows. For more information about the protocol see the `RESP3` [specification.](https://github.com/antirez/RESP3/blob/master/spec.md)
* **`REDIS_REPLY_DOUBLE`**:
* The command replied with a double-precision floating point number.
The value is stored as a string in the `str` member, and can be converted with `strtod` or similar.
* **`REDIS_REPLY_BOOL`**:
* A boolean true/false reply.
The value is stored in the `integer` member and will be either `0` or `1`.
* **`REDIS_REPLY_MAP`**:
* An array with the added invariant that there will always be an even number of elements.
The MAP is functionally equivalent to `REDIS_REPLY_ARRAY` except for the previously mentioned invariant.
* **`REDIS_REPLY_SET`**:
* An array response where each entry is unique.
Like the MAP type, the data is identical to an array response except there are no duplicate values.
* **`REDIS_REPLY_PUSH`**:
* An array that can be generated spontaneously by Redis.
This array response will always contain at least two subelements. The first contains the type of `PUSH` message (e.g. `message`, or `invalidate`), and the second being a sub-array with the `PUSH` payload itself.
* **`REDIS_REPLY_ATTR`**:
* An array structurally identical to a `MAP` but intended as meta-data about a reply.
_As of Redis 6.0.6 this reply type is not used in Redis_
* **`REDIS_REPLY_BIGNUM`**:
* A string representing an arbitrarily large signed or unsigned integer value.
The number will be encoded as a string in the `str` member of `redisReply`.
* **`REDIS_REPLY_VERB`**:
* A verbatim string, intended to be presented to the user without modification.
The string payload is stored in the `str` member, and type data is stored in the `vtype` member (e.g. `txt` for raw text or `md` for markdown).
Replies should be freed using the `freeReplyObject()` function.
Note that this function will take care of freeing sub-reply objects
contained in arrays and nested arrays, so there is no need for the user to
free the sub replies (it is actually harmful and will corrupt the memory).
**Important:** the current version of hiredis (1.0.0) frees replies when the
**Important:** the current version of hiredis (0.10.0) frees replies when the
asynchronous API is used. This means you should not call `freeReplyObject` when
you use this API. The reply is cleaned up by hiredis _after_ the callback
returns. We may introduce a flag to make this configurable in future versions of the library.
returns. This behavior will probably change in future releases, so make sure to
keep an eye on the changelog when upgrading (see issue #39).
### Cleaning up
@@ -324,7 +187,7 @@ following two execution paths:
* Read from the socket until a single reply could be parsed
The function `redisGetReply` is exported as part of the Hiredis API and can be used when a reply
is expected on the socket. To pipeline commands, the only thing that needs to be done is
is expected on the socket. To pipeline commands, the only things that needs to be done is
filling up the output buffer. For this cause, two commands can be used that are identical
to the `redisCommand` family, apart from not returning a reply:
```c
@@ -342,16 +205,16 @@ a single call to `read(2)`):
redisReply *reply;
redisAppendCommand(context,"SET foo bar");
redisAppendCommand(context,"GET foo");
redisGetReply(context,(void**)&reply); // reply for SET
redisGetReply(context,&reply); // reply for SET
freeReplyObject(reply);
redisGetReply(context,(void**)&reply); // reply for GET
redisGetReply(context,&reply); // reply for GET
freeReplyObject(reply);
```
This API can also be used to implement a blocking subscriber:
```c
reply = redisCommand(context,"SUBSCRIBE foo");
freeReplyObject(reply);
while(redisGetReply(context,(void *)&reply) == REDIS_OK) {
while(redisGetReply(context,&reply) == REDIS_OK) {
// consume message
freeReplyObject(reply);
}
@@ -394,48 +257,23 @@ Redis. It returns a pointer to the newly created `redisAsyncContext` struct. The
should be checked after creation to see if there were errors creating the connection.
Because the connection that will be created is non-blocking, the kernel is not able to
instantly return if the specified host and port is able to accept a connection.
In case of error, it is the caller's responsibility to free the context using `redisAsyncFree()`
*Note: A `redisAsyncContext` is not thread-safe.*
An application function creating a connection might look like this:
```c
void appConnect(myAppData *appData)
{
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
printf("Error: %s\n", c->errstr);
// handle error
redisAsyncFree(c);
c = NULL;
} else {
appData->context = c;
appData->connecting = 1;
c->data = appData; /* store application pointer for the callbacks */
redisAsyncSetConnectCallback(c, appOnConnect);
redisAsyncSetDisconnectCallback(c, appOnDisconnect);
}
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
printf("Error: %s\n", c->errstr);
// handle error
}
```
The asynchronous context _should_ hold a *connect* callback function that is called when the connection
attempt completes, either successfully or with an error.
It _can_ also hold a *disconnect* callback function that is called when the
connection is disconnected (either because of an error or per user request). Both callbacks should
The asynchronous context can hold a disconnect callback function that is called when the
connection is disconnected (either because of an error or per user request). This function should
have the following prototype:
```c
void(const redisAsyncContext *c, int status);
```
On a *connect*, the `status` argument is set to `REDIS_OK` if the connection attempt succeeded. In this
case, the context is ready to accept commands. If it is called with `REDIS_ERR` then the
connection attempt failed. The `err` field in the context can be accessed to find out the cause of the error.
After a failed connection attempt, the context object is automatically freed by the library after calling
the connect callback. This may be a good point to create a new context and retry the connection.
On a disconnect, the `status` argument is set to `REDIS_OK` when disconnection was initiated by the
user, or `REDIS_ERR` when the disconnection was caused by an error. When it is `REDIS_ERR`, the `err`
field in the context can be accessed to find out the cause of the error.
@@ -443,46 +281,11 @@ field in the context can be accessed to find out the cause of the error.
The context object is always freed after the disconnect callback fired. When a reconnect is needed,
the disconnect callback is a good point to do so.
Setting the connect or disconnect callbacks can only be done once per context. For subsequent calls the
api will return `REDIS_ERR`. The function to set the callbacks have the following prototype:
Setting the disconnect callback can only be done once per context. For subsequent calls it will
return `REDIS_ERR`. The function to set the disconnect callback has the following prototype:
```c
/* Alternatively you can use redisAsyncSetConnectCallbackNC which will be passed a non-const
redisAsyncContext* on invocation (e.g. allowing writes to the privdata member). */
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
```
`ac->data` may be used to pass user data to both callbacks. A typical implementation
might look something like this:
```c
void appOnConnect(redisAsyncContext *c, int status)
{
myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
appData->connecting = 0;
if (status == REDIS_OK) {
appData->connected = 1;
} else {
appData->connected = 0;
appData->err = c->err;
appData->context = NULL; /* avoid stale pointer when callback returns */
}
appAttemptReconnect();
}
void appOnDisconnect(redisAsyncContext *c, int status)
{
myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
appData->connected = 0;
appData->err = c->err;
appData->context = NULL; /* avoid stale pointer when callback returns */
if (status == REDIS_OK) {
appNotifyDisconnectCompleted(mydata);
} else {
appNotifyUnexpectedDisconnect(mydata);
appAttemptReconnect();
}
}
```
### Sending commands and their callbacks
In an asynchronous context, commands are automatically pipelined due to the nature of an event loop.
@@ -515,14 +318,6 @@ valid for the duration of the callback.
All pending callbacks are called with a `NULL` reply when the context encountered an error.
For every command issued, with the exception of **SUBSCRIBE** and **PSUBSCRIBE**, the callback is
called exactly once. Even if the context object id disconnected or deleted, every pending callback
will be called with a `NULL` reply.
For **SUBSCRIBE** and **PSUBSCRIBE**, the callbacks may be called repeatedly until an `unsubscribe`
message arrives. This will be the last invocation of the callback. In case of error, the callbacks
may receive a final `NULL` reply instead.
### Disconnecting
An asynchronous connection can be terminated using:
@@ -535,15 +330,6 @@ have been written to the socket, their respective replies have been read and the
callbacks have been executed. After this, the disconnection callback is executed with the
`REDIS_OK` status and the context object is freed.
The connection can be forcefully disconnected using
```c
void redisAsyncFree(redisAsyncContext *ac);
```
In this case, nothing more is written to the socket, all pending callbacks are called with a `NULL`
reply and the disconnection callback is called with `REDIS_OK`, after which the context object
is freed.
### Hooking it up to event library *X*
There are a few hooks that need to be set on the context object after it is created.
@@ -617,205 +403,9 @@ This should be done only in order to maximize performances when working with
large payloads. The context should be set back to `REDIS_READER_MAX_BUF` again
as soon as possible in order to prevent allocation of useless memory.
### Reader max array elements
By default the hiredis reply parser sets the maximum number of multi-bulk elements
to 2^32 - 1 or 4,294,967,295 entries. If you need to process multi-bulk replies
with more than this many elements you can set the value higher or to zero, meaning
unlimited with:
```c
context->reader->maxelements = 0;
```
## SSL/TLS Support
### Building
SSL/TLS support is not built by default and requires an explicit flag:
make USE_SSL=1
This requires OpenSSL development package (e.g. including header files to be
available.
When enabled, SSL/TLS support is built into extra `libhiredis_ssl.a` and
`libhiredis_ssl.so` static/dynamic libraries. This leaves the original libraries
unaffected so no additional dependencies are introduced.
### Using it
First, you'll need to make sure you include the SSL header file:
```c
#include <hiredis/hiredis.h>
#include <hiredis/hiredis_ssl.h>
```
You will also need to link against `libhiredis_ssl`, **in addition** to
`libhiredis` and add `-lssl -lcrypto` to satisfy its dependencies.
Hiredis implements SSL/TLS on top of its normal `redisContext` or
`redisAsyncContext`, so you will need to establish a connection first and then
initiate an SSL/TLS handshake.
#### Hiredis OpenSSL Wrappers
Before Hiredis can negotiate an SSL/TLS connection, it is necessary to
initialize OpenSSL and create a context. You can do that in two ways:
1. Work directly with the OpenSSL API to initialize the library's global context
and create `SSL_CTX *` and `SSL *` contexts. With an `SSL *` object you can
call `redisInitiateSSL()`.
2. Work with a set of Hiredis-provided wrappers around OpenSSL, create a
`redisSSLContext` object to hold configuration and use
`redisInitiateSSLWithContext()` to initiate the SSL/TLS handshake.
```c
/* An Hiredis SSL context. It holds SSL configuration and can be reused across
* many contexts.
*/
redisSSLContext *ssl_context;
/* An error variable to indicate what went wrong, if the context fails to
* initialize.
*/
redisSSLContextError ssl_error = REDIS_SSL_CTX_NONE;
/* Initialize global OpenSSL state.
*
* You should call this only once when your app initializes, and only if
* you don't explicitly or implicitly initialize OpenSSL it elsewhere.
*/
redisInitOpenSSL();
/* Create SSL context */
ssl_context = redisCreateSSLContext(
"cacertbundle.crt", /* File name of trusted CA/ca bundle file, optional */
"/path/to/certs", /* Path of trusted certificates, optional */
"client_cert.pem", /* File name of client certificate file, optional */
"client_key.pem", /* File name of client private key, optional */
"redis.mydomain.com", /* Server name to request (SNI), optional */
&ssl_error);
if(ssl_context == NULL || ssl_error != REDIS_SSL_CTX_NONE) {
/* Handle error and abort... */
/* e.g.
printf("SSL error: %s\n",
(ssl_error != REDIS_SSL_CTX_NONE) ?
redisSSLContextGetError(ssl_error) : "Unknown error");
// Abort
*/
}
/* Create Redis context and establish connection */
c = redisConnect("localhost", 6443);
if (c == NULL || c->err) {
/* Handle error and abort... */
}
/* Negotiate SSL/TLS */
if (redisInitiateSSLWithContext(c, ssl_context) != REDIS_OK) {
/* Handle error, in c->err / c->errstr */
}
```
## RESP3 PUSH replies
Redis 6.0 introduced PUSH replies with the reply-type `>`. These messages are generated spontaneously and can arrive at any time, so must be handled using callbacks.
### Default behavior
Hiredis installs handlers on `redisContext` and `redisAsyncContext` by default, which will intercept and free any PUSH replies detected. This means existing code will work as-is after upgrading to Redis 6 and switching to `RESP3`.
### Custom PUSH handler prototypes
The callback prototypes differ between `redisContext` and `redisAsyncContext`.
#### redisContext
```c
void my_push_handler(void *privdata, void *reply) {
/* Handle the reply */
/* Note: We need to free the reply in our custom handler for
blocking contexts. This lets us keep the reply if
we want. */
freeReplyObject(reply);
}
```
#### redisAsyncContext
```c
void my_async_push_handler(redisAsyncContext *ac, void *reply) {
/* Handle the reply */
/* Note: Because async hiredis always frees replies, you should
not call freeReplyObject in an async push callback. */
}
```
### Installing a custom handler
There are two ways to set your own PUSH handlers.
1. Set `push_cb` or `async_push_cb` in the `redisOptions` struct and connect with `redisConnectWithOptions` or `redisAsyncConnectWithOptions`.
```c
redisOptions = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
options->push_cb = my_push_handler;
redisContext *context = redisConnectWithOptions(&options);
```
2. Call `redisSetPushCallback` or `redisAsyncSetPushCallback` on a connected context.
```c
redisContext *context = redisConnect("127.0.0.1", 6379);
redisSetPushCallback(context, my_push_handler);
```
_Note `redisSetPushCallback` and `redisAsyncSetPushCallback` both return any currently configured handler, making it easy to override and then return to the old value._
### Specifying no handler
If you have a unique use-case where you don't want hiredis to automatically intercept and free PUSH replies, you will want to configure no handler at all. This can be done in two ways.
1. Set the `REDIS_OPT_NO_PUSH_AUTOFREE` flag in `redisOptions` and leave the callback function pointer `NULL`.
```c
redisOptions = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
options->options |= REDIS_OPT_NO_PUSH_AUTOFREE;
redisContext *context = redisConnectWithOptions(&options);
```
3. Call `redisSetPushCallback` with `NULL` once connected.
```c
redisContext *context = redisConnect("127.0.0.1", 6379);
redisSetPushCallback(context, NULL);
```
_Note: With no handler configured, calls to `redisCommand` may generate more than one reply, so this strategy is only applicable when there's some kind of blocking `redisGetReply()` loop (e.g. `MONITOR` or `SUBSCRIBE` workloads)._
## Allocator injection
Hiredis uses a pass-thru structure of function pointers defined in [alloc.h](https://github.com/redis/hiredis/blob/f5d25850/alloc.h#L41) that contain the currently configured allocation and deallocation functions. By default they just point to libc (`malloc`, `calloc`, `realloc`, etc).
### Overriding
One can override the allocators like so:
```c
hiredisAllocFuncs myfuncs = {
.mallocFn = my_malloc,
.callocFn = my_calloc,
.reallocFn = my_realloc,
.strdupFn = my_strdup,
.freeFn = my_free,
};
// Override allocators (function returns current allocators if needed)
hiredisAllocFuncs orig = hiredisSetAllocators(&myfuncs);
```
To reset the allocators to their default libc function simply call:
```c
hiredisResetAllocators();
```
## AUTHORS
Salvatore Sanfilippo (antirez at gmail),\
Pieter Noordhuis (pcnoordhuis at gmail)\
Michael Grunder (michael dot grunder at gmail)
_Hiredis is released under the BSD license._
Hiredis was written by Salvatore Sanfilippo (antirez at gmail) and
Pieter Noordhuis (pcnoordhuis at gmail) and is released under the BSD license.
Hiredis is currently maintained by Matt Stancliff (matt at genges dot com) and
Jan-Erik Rediger (janerik at fnordig dot com)
+2 -5
View File
@@ -96,7 +96,7 @@ static void redisAeCleanup(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
redisAeDelRead(privdata);
redisAeDelWrite(privdata);
hi_free(e);
free(e);
}
static int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
@@ -108,10 +108,7 @@ static int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisAeEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e = (redisAeEvents*)malloc(sizeof(*e));
e->context = ac;
e->loop = loop;
e->fd = c->fd;
-3
View File
@@ -134,9 +134,6 @@ redis_source_new (redisAsyncContext *ac)
g_return_val_if_fail(ac != NULL, NULL);
source = (RedisSource *)g_source_new(&source_funcs, sizeof *source);
if (source == NULL)
return NULL;
source->ac = ac;
source->poll_fd.fd = c->fd;
source->poll_fd.events = 0;
+2 -5
View File
@@ -43,7 +43,7 @@ static void redisIvykisCleanup(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_unregister(&e->fd);
hi_free(e);
free(e);
}
static int redisIvykisAttach(redisAsyncContext *ac) {
@@ -55,10 +55,7 @@ static int redisIvykisAttach(redisAsyncContext *ac) {
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisIvykisEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e = (redisIvykisEvents*)malloc(sizeof(*e));
e->context = ac;
/* Register functions to start/stop listening for events */
+10 -51
View File
@@ -41,12 +41,11 @@ typedef struct redisLibevEvents {
struct ev_loop *loop;
int reading, writing;
ev_io rev, wev;
ev_timer timer;
} redisLibevEvents;
static void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
((void)loop);
#endif
((void)revents);
@@ -56,7 +55,7 @@ static void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
static void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
((void)loop);
#endif
((void)revents);
@@ -66,9 +65,8 @@ static void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
static void redisLibevAddRead(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
((void)loop);
if (!e->reading) {
e->reading = 1;
ev_io_start(EV_A_ &e->rev);
@@ -77,9 +75,8 @@ static void redisLibevAddRead(void *privdata) {
static void redisLibevDelRead(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
((void)loop);
if (e->reading) {
e->reading = 0;
ev_io_stop(EV_A_ &e->rev);
@@ -88,9 +85,8 @@ static void redisLibevDelRead(void *privdata) {
static void redisLibevAddWrite(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
((void)loop);
if (!e->writing) {
e->writing = 1;
ev_io_start(EV_A_ &e->wev);
@@ -99,53 +95,19 @@ static void redisLibevAddWrite(void *privdata) {
static void redisLibevDelWrite(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
((void)loop);
if (e->writing) {
e->writing = 0;
ev_io_stop(EV_A_ &e->wev);
}
}
static void redisLibevStopTimer(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
ev_timer_stop(EV_A_ &e->timer);
}
static void redisLibevCleanup(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
redisLibevDelRead(privdata);
redisLibevDelWrite(privdata);
redisLibevStopTimer(privdata);
hi_free(e);
}
static void redisLibevTimeout(EV_P_ ev_timer *timer, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
#endif
((void)revents);
redisLibevEvents *e = (redisLibevEvents*)timer->data;
redisAsyncHandleTimeout(e->context);
}
static void redisLibevSetTimeout(void *privdata, struct timeval tv) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (!ev_is_active(&e->timer)) {
ev_init(&e->timer, redisLibevTimeout);
e->timer.data = e;
}
e->timer.repeat = tv.tv_sec + tv.tv_usec / 1000000.00;
ev_timer_again(EV_A_ &e->timer);
free(e);
}
static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
@@ -157,16 +119,14 @@ static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibevEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e = (redisLibevEvents*)malloc(sizeof(*e));
e->context = ac;
#if EV_MULTIPLICITY
e->loop = EV_A;
e->loop = loop;
#else
e->loop = NULL;
#endif
e->reading = e->writing = 0;
e->rev.data = e;
e->wev.data = e;
@@ -176,7 +136,6 @@ static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
ac->ev.addWrite = redisLibevAddWrite;
ac->ev.delWrite = redisLibevDelWrite;
ac->ev.cleanup = redisLibevCleanup;
ac->ev.scheduleTimer = redisLibevSetTimeout;
ac->ev.data = e;
/* Initialize read/write events */
+24 -91
View File
@@ -34,113 +34,48 @@
#include "../hiredis.h"
#include "../async.h"
#define REDIS_LIBEVENT_DELETED 0x01
#define REDIS_LIBEVENT_ENTERED 0x02
typedef struct redisLibeventEvents {
redisAsyncContext *context;
struct event *ev;
struct event_base *base;
struct timeval tv;
short flags;
short state;
struct event *rev, *wev;
} redisLibeventEvents;
static void redisLibeventDestroy(redisLibeventEvents *e) {
hi_free(e);
}
static void redisLibeventHandler(evutil_socket_t fd, short event, void *arg) {
((void)fd);
static void redisLibeventReadEvent(int fd, short event, void *arg) {
((void)fd); ((void)event);
redisLibeventEvents *e = (redisLibeventEvents*)arg;
e->state |= REDIS_LIBEVENT_ENTERED;
#define CHECK_DELETED() if (e->state & REDIS_LIBEVENT_DELETED) {\
redisLibeventDestroy(e);\
return; \
}
if ((event & EV_TIMEOUT) && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleTimeout(e->context);
CHECK_DELETED();
}
if ((event & EV_READ) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleRead(e->context);
CHECK_DELETED();
}
if ((event & EV_WRITE) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleWrite(e->context);
CHECK_DELETED();
}
e->state &= ~REDIS_LIBEVENT_ENTERED;
#undef CHECK_DELETED
redisAsyncHandleRead(e->context);
}
static void redisLibeventUpdate(void *privdata, short flag, int isRemove) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
const struct timeval *tv = e->tv.tv_sec || e->tv.tv_usec ? &e->tv : NULL;
if (isRemove) {
if ((e->flags & flag) == 0) {
return;
} else {
e->flags &= ~flag;
}
} else {
if (e->flags & flag) {
return;
} else {
e->flags |= flag;
}
}
event_del(e->ev);
event_assign(e->ev, e->base, e->context->c.fd, e->flags | EV_PERSIST,
redisLibeventHandler, privdata);
event_add(e->ev, tv);
static void redisLibeventWriteEvent(int fd, short event, void *arg) {
((void)fd); ((void)event);
redisLibeventEvents *e = (redisLibeventEvents*)arg;
redisAsyncHandleWrite(e->context);
}
static void redisLibeventAddRead(void *privdata) {
redisLibeventUpdate(privdata, EV_READ, 0);
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_add(e->rev,NULL);
}
static void redisLibeventDelRead(void *privdata) {
redisLibeventUpdate(privdata, EV_READ, 1);
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_del(e->rev);
}
static void redisLibeventAddWrite(void *privdata) {
redisLibeventUpdate(privdata, EV_WRITE, 0);
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_add(e->wev,NULL);
}
static void redisLibeventDelWrite(void *privdata) {
redisLibeventUpdate(privdata, EV_WRITE, 1);
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_del(e->wev);
}
static void redisLibeventCleanup(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
if (!e) {
return;
}
event_del(e->ev);
event_free(e->ev);
e->ev = NULL;
if (e->state & REDIS_LIBEVENT_ENTERED) {
e->state |= REDIS_LIBEVENT_DELETED;
} else {
redisLibeventDestroy(e);
}
}
static void redisLibeventSetTimeout(void *privdata, struct timeval tv) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
short flags = e->flags;
e->flags = 0;
e->tv = tv;
redisLibeventUpdate(e, flags, 0);
event_del(e->rev);
event_del(e->wev);
free(e);
}
static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
@@ -152,10 +87,7 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibeventEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e = (redisLibeventEvents*)malloc(sizeof(*e));
e->context = ac;
/* Register functions to start/stop listening for events */
@@ -164,12 +96,13 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
ac->ev.addWrite = redisLibeventAddWrite;
ac->ev.delWrite = redisLibeventDelWrite;
ac->ev.cleanup = redisLibeventCleanup;
ac->ev.scheduleTimer = redisLibeventSetTimeout;
ac->ev.data = e;
/* Initialize and install read/write events */
e->ev = event_new(base, c->fd, EV_READ | EV_WRITE, redisLibeventHandler, e);
e->base = base;
e->rev = event_new(base, c->fd, EV_READ, redisLibeventReadEvent, e);
e->wev = event_new(base, c->fd, EV_WRITE, redisLibeventWriteEvent, e);
event_add(e->rev, NULL);
event_add(e->wev, NULL);
return REDIS_OK;
}
#endif
-123
View File
@@ -1,123 +0,0 @@
#ifndef __HIREDIS_LIBHV_H__
#define __HIREDIS_LIBHV_H__
#include <hv/hloop.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct redisLibhvEvents {
hio_t *io;
htimer_t *timer;
} redisLibhvEvents;
static void redisLibhvHandleEvents(hio_t* io) {
redisAsyncContext* context = (redisAsyncContext*)hevent_userdata(io);
int events = hio_events(io);
int revents = hio_revents(io);
if (context && (events & HV_READ) && (revents & HV_READ)) {
redisAsyncHandleRead(context);
}
if (context && (events & HV_WRITE) && (revents & HV_WRITE)) {
redisAsyncHandleWrite(context);
}
}
static void redisLibhvAddRead(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_add(events->io, redisLibhvHandleEvents, HV_READ);
}
static void redisLibhvDelRead(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_del(events->io, HV_READ);
}
static void redisLibhvAddWrite(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_add(events->io, redisLibhvHandleEvents, HV_WRITE);
}
static void redisLibhvDelWrite(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_del(events->io, HV_WRITE);
}
static void redisLibhvCleanup(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
if (events->timer)
htimer_del(events->timer);
hio_close(events->io);
hevent_set_userdata(events->io, NULL);
hi_free(events);
}
static void redisLibhvTimeout(htimer_t* timer) {
hio_t* io = (hio_t*)hevent_userdata(timer);
redisAsyncHandleTimeout((redisAsyncContext*)hevent_userdata(io));
}
static void redisLibhvSetTimeout(void *privdata, struct timeval tv) {
redisLibhvEvents* events;
uint32_t millis;
hloop_t* loop;
events = (redisLibhvEvents*)privdata;
millis = tv.tv_sec * 1000 + tv.tv_usec / 1000;
if (millis == 0) {
/* Libhv disallows zero'd timers so treat this as a delete or NO OP */
if (events->timer) {
htimer_del(events->timer);
events->timer = NULL;
}
} else if (events->timer == NULL) {
/* Add new timer */
loop = hevent_loop(events->io);
events->timer = htimer_add(loop, redisLibhvTimeout, millis, 1);
hevent_set_userdata(events->timer, events->io);
} else {
/* Update existing timer */
htimer_reset(events->timer, millis);
}
}
static int redisLibhvAttach(redisAsyncContext* ac, hloop_t* loop) {
redisContext *c = &(ac->c);
redisLibhvEvents *events;
hio_t* io = NULL;
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
/* Create container struct to keep track of our io and any timer */
events = (redisLibhvEvents*)hi_malloc(sizeof(*events));
if (events == NULL) {
return REDIS_ERR;
}
io = hio_get(loop, c->fd);
if (io == NULL) {
hi_free(events);
return REDIS_ERR;
}
hevent_set_userdata(io, ac);
events->io = io;
events->timer = NULL;
ac->ev.addRead = redisLibhvAddRead;
ac->ev.delRead = redisLibhvDelRead;
ac->ev.addWrite = redisLibhvAddWrite;
ac->ev.delWrite = redisLibhvDelWrite;
ac->ev.cleanup = redisLibhvCleanup;
ac->ev.scheduleTimer = redisLibhvSetTimeout;
ac->ev.data = events;
return REDIS_OK;
}
#endif
-177
View File
@@ -1,177 +0,0 @@
#ifndef HIREDIS_LIBSDEVENT_H
#define HIREDIS_LIBSDEVENT_H
#include <systemd/sd-event.h>
#include "../hiredis.h"
#include "../async.h"
#define REDIS_LIBSDEVENT_DELETED 0x01
#define REDIS_LIBSDEVENT_ENTERED 0x02
typedef struct redisLibsdeventEvents {
redisAsyncContext *context;
struct sd_event *event;
struct sd_event_source *fdSource;
struct sd_event_source *timerSource;
int fd;
short flags;
short state;
} redisLibsdeventEvents;
static void redisLibsdeventDestroy(redisLibsdeventEvents *e) {
if (e->fdSource) {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
if (e->timerSource) {
e->timerSource = sd_event_source_disable_unref(e->timerSource);
}
sd_event_unref(e->event);
hi_free(e);
}
static int redisLibsdeventTimeoutHandler(sd_event_source *s, uint64_t usec, void *userdata) {
((void)s);
((void)usec);
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
redisAsyncHandleTimeout(e->context);
return 0;
}
static int redisLibsdeventHandler(sd_event_source *s, int fd, uint32_t event, void *userdata) {
((void)s);
((void)fd);
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->state |= REDIS_LIBSDEVENT_ENTERED;
#define CHECK_DELETED() if (e->state & REDIS_LIBSDEVENT_DELETED) {\
redisLibsdeventDestroy(e);\
return 0; \
}
if ((event & EPOLLIN) && e->context && (e->state & REDIS_LIBSDEVENT_DELETED) == 0) {
redisAsyncHandleRead(e->context);
CHECK_DELETED();
}
if ((event & EPOLLOUT) && e->context && (e->state & REDIS_LIBSDEVENT_DELETED) == 0) {
redisAsyncHandleWrite(e->context);
CHECK_DELETED();
}
e->state &= ~REDIS_LIBSDEVENT_ENTERED;
#undef CHECK_DELETED
return 0;
}
static void redisLibsdeventAddRead(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (e->flags & EPOLLIN) {
return;
}
e->flags |= EPOLLIN;
if (e->flags & EPOLLOUT) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
sd_event_add_io(e->event, &e->fdSource, e->fd, e->flags, redisLibsdeventHandler, e);
}
}
static void redisLibsdeventDelRead(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->flags &= ~EPOLLIN;
if (e->flags) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
}
static void redisLibsdeventAddWrite(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (e->flags & EPOLLOUT) {
return;
}
e->flags |= EPOLLOUT;
if (e->flags & EPOLLIN) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
sd_event_add_io(e->event, &e->fdSource, e->fd, e->flags, redisLibsdeventHandler, e);
}
}
static void redisLibsdeventDelWrite(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->flags &= ~EPOLLOUT;
if (e->flags) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
}
static void redisLibsdeventCleanup(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (!e) {
return;
}
if (e->state & REDIS_LIBSDEVENT_ENTERED) {
e->state |= REDIS_LIBSDEVENT_DELETED;
} else {
redisLibsdeventDestroy(e);
}
}
static void redisLibsdeventSetTimeout(void *userdata, struct timeval tv) {
redisLibsdeventEvents *e = (redisLibsdeventEvents *)userdata;
uint64_t usec = tv.tv_sec * 1000000 + tv.tv_usec;
if (!e->timerSource) {
sd_event_add_time_relative(e->event, &e->timerSource, CLOCK_MONOTONIC, usec, 1, redisLibsdeventTimeoutHandler, e);
} else {
sd_event_source_set_time_relative(e->timerSource, usec);
}
}
static int redisLibsdeventAttach(redisAsyncContext *ac, struct sd_event *event) {
redisContext *c = &(ac->c);
redisLibsdeventEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibsdeventEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return REDIS_ERR;
/* Initialize and increase event refcount */
e->context = ac;
e->event = event;
e->fd = c->fd;
sd_event_ref(event);
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisLibsdeventAddRead;
ac->ev.delRead = redisLibsdeventDelRead;
ac->ev.addWrite = redisLibsdeventAddWrite;
ac->ev.delWrite = redisLibsdeventDelWrite;
ac->ev.cleanup = redisLibsdeventCleanup;
ac->ev.scheduleTimer = redisLibsdeventSetTimeout;
ac->ev.data = e;
return REDIS_OK;
}
#endif
+64 -113
View File
@@ -7,165 +7,116 @@
#include <string.h>
typedef struct redisLibuvEvents {
redisAsyncContext* context;
uv_poll_t handle;
uv_timer_t timer;
int events;
redisAsyncContext* context;
uv_poll_t handle;
int events;
} redisLibuvEvents;
static void redisLibuvPoll(uv_poll_t* handle, int status, int events) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
int ev = (status ? p->events : events);
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
if (p->context != NULL && (ev & UV_READABLE)) {
redisAsyncHandleRead(p->context);
}
if (p->context != NULL && (ev & UV_WRITABLE)) {
redisAsyncHandleWrite(p->context);
}
if (status != 0) {
return;
}
if (p->context != NULL && (events & UV_READABLE)) {
redisAsyncHandleRead(p->context);
}
if (p->context != NULL && (events & UV_WRITABLE)) {
redisAsyncHandleWrite(p->context);
}
}
static void redisLibuvAddRead(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
if (p->events & UV_READABLE) {
return;
}
p->events |= UV_READABLE;
p->events |= UV_READABLE;
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
}
static void redisLibuvDelRead(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->events &= ~UV_READABLE;
p->events &= ~UV_READABLE;
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
}
static void redisLibuvAddWrite(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
if (p->events & UV_WRITABLE) {
return;
}
p->events |= UV_WRITABLE;
p->events |= UV_WRITABLE;
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
}
static void redisLibuvDelWrite(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->events &= ~UV_WRITABLE;
p->events &= ~UV_WRITABLE;
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
}
static void on_timer_close(uv_handle_t *handle) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
p->timer.data = NULL;
if (!p->handle.data) {
// both timer and handle are closed
hi_free(p);
}
// else, wait for `on_handle_close`
static void on_close(uv_handle_t* handle) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
free(p);
}
static void on_handle_close(uv_handle_t *handle) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
p->handle.data = NULL;
if (!p->timer.data) {
// timer never started, or timer already destroyed
hi_free(p);
}
// else, wait for `on_timer_close`
}
// libuv removed `status` parameter since v0.11.23
// see: https://github.com/libuv/libuv/blob/v0.11.23/include/uv.h
#if (UV_VERSION_MAJOR == 0 && UV_VERSION_MINOR < 11) || \
(UV_VERSION_MAJOR == 0 && UV_VERSION_MINOR == 11 && UV_VERSION_PATCH < 23)
static void redisLibuvTimeout(uv_timer_t *timer, int status) {
(void)status; // unused
#else
static void redisLibuvTimeout(uv_timer_t *timer) {
#endif
redisLibuvEvents *e = (redisLibuvEvents*)timer->data;
redisAsyncHandleTimeout(e->context);
}
static void redisLibuvSetTimeout(void *privdata, struct timeval tv) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
uint64_t millsec = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
if (!p->timer.data) {
// timer is uninitialized
if (uv_timer_init(p->handle.loop, &p->timer) != 0) {
return;
}
p->timer.data = p;
}
// updates the timeout if the timer has already started
// or start the timer
uv_timer_start(&p->timer, redisLibuvTimeout, millsec, 0);
}
static void redisLibuvCleanup(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->context = NULL; // indicate that context might no longer exist
if (p->timer.data) {
uv_close((uv_handle_t*)&p->timer, on_timer_close);
}
uv_close((uv_handle_t*)&p->handle, on_handle_close);
p->context = NULL; // indicate that context might no longer exist
uv_close((uv_handle_t*)&p->handle, on_close);
}
static int redisLibuvAttach(redisAsyncContext* ac, uv_loop_t* loop) {
redisContext *c = &(ac->c);
redisContext *c = &(ac->c);
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
ac->ev.addRead = redisLibuvAddRead;
ac->ev.delRead = redisLibuvDelRead;
ac->ev.addWrite = redisLibuvAddWrite;
ac->ev.delWrite = redisLibuvDelWrite;
ac->ev.cleanup = redisLibuvCleanup;
ac->ev.scheduleTimer = redisLibuvSetTimeout;
ac->ev.addRead = redisLibuvAddRead;
ac->ev.delRead = redisLibuvDelRead;
ac->ev.addWrite = redisLibuvAddWrite;
ac->ev.delWrite = redisLibuvDelWrite;
ac->ev.cleanup = redisLibuvCleanup;
redisLibuvEvents* p = (redisLibuvEvents*)hi_malloc(sizeof(*p));
if (p == NULL)
return REDIS_ERR;
redisLibuvEvents* p = (redisLibuvEvents*)malloc(sizeof(*p));
memset(p, 0, sizeof(*p));
if (!p) {
return REDIS_ERR;
}
if (uv_poll_init_socket(loop, &p->handle, c->fd) != 0) {
return REDIS_ERR;
}
memset(p, 0, sizeof(*p));
ac->ev.data = p;
p->handle.data = p;
p->context = ac;
if (uv_poll_init(loop, &p->handle, c->fd) != 0) {
return REDIS_ERR;
}
return REDIS_OK;
ac->ev.data = p;
p->handle.data = p;
p->context = ac;
return REDIS_OK;
}
#endif
+3 -4
View File
@@ -27,7 +27,7 @@ static int freeRedisRunLoop(RedisRunLoop* redisRunLoop) {
CFSocketInvalidate(redisRunLoop->socketRef);
CFRelease(redisRunLoop->socketRef);
}
hi_free(redisRunLoop);
free(redisRunLoop);
}
return REDIS_ERR;
}
@@ -80,9 +80,8 @@ static int redisMacOSAttach(redisAsyncContext *redisAsyncCtx, CFRunLoopRef runLo
/* Nothing should be attached when something is already attached */
if( redisAsyncCtx->ev.data != NULL ) return REDIS_ERR;
RedisRunLoop* redisRunLoop = (RedisRunLoop*) hi_calloc(1, sizeof(RedisRunLoop));
if (redisRunLoop == NULL)
return REDIS_ERR;
RedisRunLoop* redisRunLoop = (RedisRunLoop*) calloc(1, sizeof(RedisRunLoop));
if( !redisRunLoop ) return REDIS_ERR;
/* Setup redis stuff */
redisRunLoop->context = redisAsyncCtx;
-197
View File
@@ -1,197 +0,0 @@
#ifndef HIREDIS_POLL_H
#define HIREDIS_POLL_H
#include "../async.h"
#include "../sockcompat.h"
#include <string.h> // for memset
#include <errno.h>
/* Values to return from redisPollTick */
#define REDIS_POLL_HANDLED_READ 1
#define REDIS_POLL_HANDLED_WRITE 2
#define REDIS_POLL_HANDLED_TIMEOUT 4
/* An adapter to allow manual polling of the async context by checking the state
* of the underlying file descriptor. Useful in cases where there is no formal
* IO event loop but regular ticking can be used, such as in game engines. */
typedef struct redisPollEvents {
redisAsyncContext *context;
redisFD fd;
char reading, writing;
char in_tick;
char deleted;
double deadline;
} redisPollEvents;
static double redisPollTimevalToDouble(struct timeval *tv) {
if (tv == NULL)
return 0.0;
return tv->tv_sec + tv->tv_usec / 1000000.00;
}
static double redisPollGetNow(void) {
#ifndef _MSC_VER
struct timeval tv;
gettimeofday(&tv,NULL);
return redisPollTimevalToDouble(&tv);
#else
FILETIME ft;
ULARGE_INTEGER li;
GetSystemTimeAsFileTime(&ft);
li.HighPart = ft.dwHighDateTime;
li.LowPart = ft.dwLowDateTime;
return (double)li.QuadPart * 1e-7;
#endif
}
/* Poll for io, handling any pending callbacks. The timeout argument can be
* positive to wait for a maximum given time for IO, zero to poll, or negative
* to wait forever */
static int redisPollTick(redisAsyncContext *ac, double timeout) {
int reading, writing;
struct pollfd pfd;
int handled;
int ns;
int itimeout;
redisPollEvents *e = (redisPollEvents*)ac->ev.data;
if (!e)
return 0;
/* local flags, won't get changed during callbacks */
reading = e->reading;
writing = e->writing;
if (!reading && !writing)
return 0;
pfd.fd = e->fd;
pfd.events = 0;
if (reading)
pfd.events = POLLIN;
if (writing)
pfd.events |= POLLOUT;
if (timeout >= 0.0) {
itimeout = (int)(timeout * 1000.0);
} else {
itimeout = -1;
}
ns = poll(&pfd, 1, itimeout);
if (ns < 0) {
/* ignore the EINTR error */
if (errno != EINTR)
return ns;
ns = 0;
}
handled = 0;
e->in_tick = 1;
if (ns) {
if (reading && (pfd.revents & POLLIN)) {
redisAsyncHandleRead(ac);
handled |= REDIS_POLL_HANDLED_READ;
}
/* on Windows, connection failure is indicated with the Exception fdset.
* handle it the same as writable. */
if (writing && (pfd.revents & (POLLOUT | POLLERR))) {
/* context Read callback may have caused context to be deleted, e.g.
by doing an redisAsyncDisconnect() */
if (!e->deleted) {
redisAsyncHandleWrite(ac);
handled |= REDIS_POLL_HANDLED_WRITE;
}
}
}
/* perform timeouts */
if (!e->deleted && e->deadline != 0.0) {
double now = redisPollGetNow();
if (now >= e->deadline) {
/* deadline has passed. disable timeout and perform callback */
e->deadline = 0.0;
redisAsyncHandleTimeout(ac);
handled |= REDIS_POLL_HANDLED_TIMEOUT;
}
}
/* do a delayed cleanup if required */
if (e->deleted)
hi_free(e);
else
e->in_tick = 0;
return handled;
}
static void redisPollAddRead(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->reading = 1;
}
static void redisPollDelRead(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->reading = 0;
}
static void redisPollAddWrite(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->writing = 1;
}
static void redisPollDelWrite(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->writing = 0;
}
static void redisPollCleanup(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
/* if we are currently processing a tick, postpone deletion */
if (e->in_tick)
e->deleted = 1;
else
hi_free(e);
}
static void redisPollScheduleTimer(void *data, struct timeval tv)
{
redisPollEvents *e = (redisPollEvents*)data;
double now = redisPollGetNow();
e->deadline = now + redisPollTimevalToDouble(&tv);
}
static int redisPollAttach(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisPollEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisPollEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
memset(e, 0, sizeof(*e));
e->context = ac;
e->fd = c->fd;
e->reading = e->writing = 0;
e->in_tick = e->deleted = 0;
e->deadline = 0.0;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisPollAddRead;
ac->ev.delRead = redisPollDelRead;
ac->ev.addWrite = redisPollAddWrite;
ac->ev.delWrite = redisPollDelWrite;
ac->ev.scheduleTimer = redisPollScheduleTimer;
ac->ev.cleanup = redisPollCleanup;
ac->ev.data = e;
return REDIS_OK;
}
#endif /* HIREDIS_POLL_H */
-144
View File
@@ -1,144 +0,0 @@
#ifndef __HIREDIS_REDISMODULEAPI_H__
#define __HIREDIS_REDISMODULEAPI_H__
#include "redismodule.h"
#include "../async.h"
#include "../hiredis.h"
#include <sys/types.h>
typedef struct redisModuleEvents {
redisAsyncContext *context;
RedisModuleCtx *module_ctx;
int fd;
int reading, writing;
int timer_active;
RedisModuleTimerID timer_id;
} redisModuleEvents;
static inline void redisModuleReadEvent(int fd, void *privdata, int mask) {
(void) fd;
(void) mask;
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisAsyncHandleRead(e->context);
}
static inline void redisModuleWriteEvent(int fd, void *privdata, int mask) {
(void) fd;
(void) mask;
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisAsyncHandleWrite(e->context);
}
static inline void redisModuleAddRead(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (!e->reading) {
e->reading = 1;
RedisModule_EventLoopAdd(e->fd, REDISMODULE_EVENTLOOP_READABLE, redisModuleReadEvent, e);
}
}
static inline void redisModuleDelRead(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->reading) {
e->reading = 0;
RedisModule_EventLoopDel(e->fd, REDISMODULE_EVENTLOOP_READABLE);
}
}
static inline void redisModuleAddWrite(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (!e->writing) {
e->writing = 1;
RedisModule_EventLoopAdd(e->fd, REDISMODULE_EVENTLOOP_WRITABLE, redisModuleWriteEvent, e);
}
}
static inline void redisModuleDelWrite(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->writing) {
e->writing = 0;
RedisModule_EventLoopDel(e->fd, REDISMODULE_EVENTLOOP_WRITABLE);
}
}
static inline void redisModuleStopTimer(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->timer_active) {
RedisModule_StopTimer(e->module_ctx, e->timer_id, NULL);
}
e->timer_active = 0;
}
static inline void redisModuleCleanup(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisModuleDelRead(privdata);
redisModuleDelWrite(privdata);
redisModuleStopTimer(privdata);
hi_free(e);
}
static inline void redisModuleTimeout(RedisModuleCtx *ctx, void *privdata) {
(void) ctx;
redisModuleEvents *e = (redisModuleEvents*)privdata;
e->timer_active = 0;
redisAsyncHandleTimeout(e->context);
}
static inline void redisModuleSetTimeout(void *privdata, struct timeval tv) {
redisModuleEvents* e = (redisModuleEvents*)privdata;
redisModuleStopTimer(privdata);
mstime_t millis = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
e->timer_id = RedisModule_CreateTimer(e->module_ctx, millis, redisModuleTimeout, e);
e->timer_active = 1;
}
/* Check if Redis version is compatible with the adapter. */
static inline int redisModuleCompatibilityCheck(void) {
if (!RedisModule_EventLoopAdd ||
!RedisModule_EventLoopDel ||
!RedisModule_CreateTimer ||
!RedisModule_StopTimer) {
return REDIS_ERR;
}
return REDIS_OK;
}
static inline int redisModuleAttach(redisAsyncContext *ac, RedisModuleCtx *module_ctx) {
redisContext *c = &(ac->c);
redisModuleEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisModuleEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e->context = ac;
e->module_ctx = module_ctx;
e->fd = c->fd;
e->reading = e->writing = 0;
e->timer_active = 0;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisModuleAddRead;
ac->ev.delRead = redisModuleDelRead;
ac->ev.addWrite = redisModuleAddWrite;
ac->ev.delWrite = redisModuleDelWrite;
ac->ev.cleanup = redisModuleCleanup;
ac->ev.scheduleTimer = redisModuleSetTimeout;
ac->ev.data = e;
return REDIS_OK;
}
#endif
-90
View File
@@ -1,90 +0,0 @@
/*
* Copyright (c) 2020, Michael Grunder <michael dot grunder at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "fmacros.h"
#include "alloc.h"
#include <string.h>
#include <stdlib.h>
hiredisAllocFuncs hiredisAllocFns = {
.mallocFn = malloc,
.callocFn = calloc,
.reallocFn = realloc,
.strdupFn = strdup,
.freeFn = free,
};
/* Override hiredis' allocators with ones supplied by the user */
hiredisAllocFuncs hiredisSetAllocators(hiredisAllocFuncs *override) {
hiredisAllocFuncs orig = hiredisAllocFns;
hiredisAllocFns = *override;
return orig;
}
/* Reset allocators to use libc defaults */
void hiredisResetAllocators(void) {
hiredisAllocFns = (hiredisAllocFuncs) {
.mallocFn = malloc,
.callocFn = calloc,
.reallocFn = realloc,
.strdupFn = strdup,
.freeFn = free,
};
}
#ifdef _WIN32
void *hi_malloc(size_t size) {
return hiredisAllocFns.mallocFn(size);
}
void *hi_calloc(size_t nmemb, size_t size) {
/* Overflow check as the user can specify any arbitrary allocator */
if (SIZE_MAX / size < nmemb)
return NULL;
return hiredisAllocFns.callocFn(nmemb, size);
}
void *hi_realloc(void *ptr, size_t size) {
return hiredisAllocFns.reallocFn(ptr, size);
}
char *hi_strdup(const char *str) {
return hiredisAllocFns.strdupFn(str);
}
void hi_free(void *ptr) {
hiredisAllocFns.freeFn(ptr);
}
#endif
-96
View File
@@ -1,96 +0,0 @@
/*
* Copyright (c) 2020, Michael Grunder <michael dot grunder at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef HIREDIS_ALLOC_H
#define HIREDIS_ALLOC_H
#include <stddef.h> /* for size_t */
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
/* Structure pointing to our actually configured allocators */
typedef struct hiredisAllocFuncs {
void *(*mallocFn)(size_t);
void *(*callocFn)(size_t,size_t);
void *(*reallocFn)(void*,size_t);
char *(*strdupFn)(const char*);
void (*freeFn)(void*);
} hiredisAllocFuncs;
hiredisAllocFuncs hiredisSetAllocators(hiredisAllocFuncs *ha);
void hiredisResetAllocators(void);
#ifndef _WIN32
/* Hiredis' configured allocator function pointer struct */
extern hiredisAllocFuncs hiredisAllocFns;
static inline void *hi_malloc(size_t size) {
return hiredisAllocFns.mallocFn(size);
}
static inline void *hi_calloc(size_t nmemb, size_t size) {
/* Overflow check as the user can specify any arbitrary allocator */
if (SIZE_MAX / size < nmemb)
return NULL;
return hiredisAllocFns.callocFn(nmemb, size);
}
static inline void *hi_realloc(void *ptr, size_t size) {
return hiredisAllocFns.reallocFn(ptr, size);
}
static inline char *hi_strdup(const char *str) {
return hiredisAllocFns.strdupFn(str);
}
static inline void hi_free(void *ptr) {
hiredisAllocFns.freeFn(ptr);
}
#else
void *hi_malloc(size_t size);
void *hi_calloc(size_t nmemb, size_t size);
void *hi_realloc(void *ptr, size_t size);
char *hi_strdup(const char *str);
void hi_free(void *ptr);
#endif
#ifdef __cplusplus
}
#endif
#endif /* HIREDIS_ALLOC_H */
+18 -6
View File
@@ -1,14 +1,24 @@
# Appveyor configuration file for CI build of hiredis on Windows (under Cygwin)
environment:
matrix:
- CYG_BASH: C:\cygwin64\bin\bash
- CYG_ROOT: C:\cygwin64
CYG_SETUP: setup-x86_64.exe
CYG_MIRROR: http://cygwin.mirror.constant.com
CYG_CACHE: C:\cygwin64\var\cache\setup
CYG_BASH: C:\cygwin64\bin\bash
CC: gcc
- CYG_BASH: C:\cygwin\bin\bash
- CYG_ROOT: C:\cygwin
CYG_SETUP: setup-x86.exe
CYG_MIRROR: http://cygwin.mirror.constant.com
CYG_CACHE: C:\cygwin\var\cache\setup
CYG_BASH: C:\cygwin\bin\bash
CC: gcc
CFLAGS: -m32
CXXFLAGS: -m32
LDFLAGS: -m32
TARGET: 32bit
TARGET_VARS: 32bit-vars
# Cache Cygwin files to speed up build
cache:
- '%CYG_CACHE%'
clone_depth: 1
# Attempt to ensure we don't try to convert line endings to Win32 CRLF as this will cause build to fail
@@ -17,8 +27,10 @@ init:
# Install needed build dependencies
install:
- ps: 'Start-FileDownload "http://cygwin.com/$env:CYG_SETUP" -FileName "$env:CYG_SETUP"'
- '%CYG_SETUP% --quiet-mode --no-shortcuts --only-site --root "%CYG_ROOT%" --site "%CYG_MIRROR%" --local-package-dir "%CYG_CACHE%" --packages automake,bison,gcc-core,libtool,make,gettext-devel,gettext,intltool,pkg-config,clang,llvm > NUL 2>&1'
- '%CYG_BASH% -lc "cygcheck -dc cygwin"'
build_script:
- 'echo building...'
- '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; mkdir build && cd build && cmake .. -G \"Unix Makefiles\" && make VERBOSE=1"'
- '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; make LDFLAGS=$LDFLAGS CC=$CC $TARGET CFLAGS=$CFLAGS && make LDFLAGS=$LDFLAGS CC=$CC $TARGET_VARS hiredis-example"'
+180 -527
View File
File diff suppressed because it is too large Load Diff
+1 -24
View File
@@ -45,8 +45,6 @@ typedef void (redisCallbackFn)(struct redisAsyncContext*, void*, void*);
typedef struct redisCallback {
struct redisCallback *next; /* simple singly linked list */
redisCallbackFn *fn;
int pending_subs;
int unsubscribe_sent;
void *privdata;
} redisCallback;
@@ -58,8 +56,6 @@ typedef struct redisCallbackList {
/* Connection callback prototypes */
typedef void (redisDisconnectCallback)(const struct redisAsyncContext*, int status);
typedef void (redisConnectCallback)(const struct redisAsyncContext*, int status);
typedef void (redisConnectCallbackNC)(struct redisAsyncContext *, int status);
typedef void(redisTimerCallback)(void *timer, void *privdata);
/* Context for an async connection to Redis */
typedef struct redisAsyncContext {
@@ -72,7 +68,6 @@ typedef struct redisAsyncContext {
/* Not used by hiredis */
void *data;
void (*dataCleanup)(void *privdata);
/* Event library data and hooks */
struct {
@@ -85,7 +80,6 @@ typedef struct redisAsyncContext {
void (*addWrite)(void *privdata);
void (*delWrite)(void *privdata);
void (*cleanup)(void *privdata);
void (*scheduleTimer)(void *privdata, struct timeval tv);
} ev;
/* Called when either the connection is terminated due to an error or per
@@ -94,49 +88,32 @@ typedef struct redisAsyncContext {
/* Called when the first write event was received. */
redisConnectCallback *onConnect;
redisConnectCallbackNC *onConnectNC;
/* Regular command callbacks */
redisCallbackList replies;
/* Address used for connect() */
struct sockaddr *saddr;
size_t addrlen;
/* Subscription callbacks */
struct {
redisCallbackList replies;
redisCallbackList invalid;
struct dict *channels;
struct dict *patterns;
int pending_unsubs;
} sub;
/* Any configured RESP3 PUSH handler */
redisAsyncPushFn *push_cb;
} redisAsyncContext;
/* Functions that proxy to hiredis */
redisAsyncContext *redisAsyncConnectWithOptions(const redisOptions *options);
redisAsyncContext *redisAsyncConnect(const char *ip, int port);
redisAsyncContext *redisAsyncConnectBind(const char *ip, int port, const char *source_addr);
redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
const char *source_addr);
redisAsyncContext *redisAsyncConnectUnix(const char *path);
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetConnectCallbackNC(redisAsyncContext *ac, redisConnectCallbackNC *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
redisAsyncPushFn *redisAsyncSetPushCallback(redisAsyncContext *ac, redisAsyncPushFn *fn);
int redisAsyncSetTimeout(redisAsyncContext *ac, struct timeval tv);
void redisAsyncDisconnect(redisAsyncContext *ac);
void redisAsyncFree(redisAsyncContext *ac);
/* Handle read/write events */
void redisAsyncHandleRead(redisAsyncContext *ac);
void redisAsyncHandleWrite(redisAsyncContext *ac);
void redisAsyncHandleTimeout(redisAsyncContext *ac);
void redisAsyncRead(redisAsyncContext *ac);
void redisAsyncWrite(redisAsyncContext *ac);
/* Command functions for an async context. Write the command to the
* output buffer and register the provided callback. */
-75
View File
@@ -1,75 +0,0 @@
/*
* Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_ASYNC_PRIVATE_H
#define __HIREDIS_ASYNC_PRIVATE_H
#define _EL_ADD_READ(ctx) \
do { \
refreshTimeout(ctx); \
if ((ctx)->ev.addRead) (ctx)->ev.addRead((ctx)->ev.data); \
} while (0)
#define _EL_DEL_READ(ctx) do { \
if ((ctx)->ev.delRead) (ctx)->ev.delRead((ctx)->ev.data); \
} while(0)
#define _EL_ADD_WRITE(ctx) \
do { \
refreshTimeout(ctx); \
if ((ctx)->ev.addWrite) (ctx)->ev.addWrite((ctx)->ev.data); \
} while (0)
#define _EL_DEL_WRITE(ctx) do { \
if ((ctx)->ev.delWrite) (ctx)->ev.delWrite((ctx)->ev.data); \
} while(0)
#define _EL_CLEANUP(ctx) do { \
if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
ctx->ev.cleanup = NULL; \
} while(0)
static inline void refreshTimeout(redisAsyncContext *ctx) {
#define REDIS_TIMER_ISSET(tvp) \
(tvp && ((tvp)->tv_sec || (tvp)->tv_usec))
#define REDIS_EL_TIMER(ac, tvp) \
if ((ac)->ev.scheduleTimer && REDIS_TIMER_ISSET(tvp)) { \
(ac)->ev.scheduleTimer((ac)->ev.data, *(tvp)); \
}
if (ctx->c.flags & REDIS_CONNECTED) {
REDIS_EL_TIMER(ctx, ctx->c.command_timeout);
} else {
REDIS_EL_TIMER(ctx, ctx->c.connect_timeout);
}
}
void __redisAsyncDisconnect(redisAsyncContext *ac);
void redisProcessCallbacks(redisAsyncContext *ac);
#endif /* __HIREDIS_ASYNC_PRIVATE_H */
+17 -22
View File
@@ -34,7 +34,6 @@
*/
#include "fmacros.h"
#include "alloc.h"
#include <stdlib.h>
#include <assert.h>
#include <limits.h>
@@ -72,10 +71,7 @@ static void _dictReset(dict *ht) {
/* Create a new hash table */
static dict *dictCreate(dictType *type, void *privDataPtr) {
dict *ht = hi_malloc(sizeof(*ht));
if (ht == NULL)
return NULL;
dict *ht = malloc(sizeof(*ht));
_dictInit(ht,type,privDataPtr);
return ht;
}
@@ -101,9 +97,7 @@ static int dictExpand(dict *ht, unsigned long size) {
_dictInit(&n, ht->type, ht->privdata);
n.size = realsize;
n.sizemask = realsize-1;
n.table = hi_calloc(realsize,sizeof(dictEntry*));
if (n.table == NULL)
return DICT_ERR;
n.table = calloc(realsize,sizeof(dictEntry*));
/* Copy all the elements from the old to the new table:
* note that if the old hash table is empty ht->size is zero,
@@ -130,7 +124,7 @@ static int dictExpand(dict *ht, unsigned long size) {
}
}
assert(ht->used == 0);
hi_free(ht->table);
free(ht->table);
/* Remap the new hashtable in the old */
*ht = n;
@@ -148,10 +142,7 @@ static int dictAdd(dict *ht, void *key, void *val) {
return DICT_ERR;
/* Allocates the memory and stores key */
entry = hi_malloc(sizeof(*entry));
if (entry == NULL)
return DICT_ERR;
entry = malloc(sizeof(*entry));
entry->next = ht->table[index];
ht->table[index] = entry;
@@ -175,9 +166,6 @@ static int dictReplace(dict *ht, void *key, void *val) {
return 1;
/* It already exists, get the entry */
entry = dictFind(ht, key);
if (entry == NULL)
return 0;
/* Free the old value and set the new one */
/* Set the new value and free the old one. Note that it is important
* to do that in this order, as the value may just be exactly the same
@@ -211,7 +199,7 @@ static int dictDelete(dict *ht, const void *key) {
dictFreeEntryKey(ht,de);
dictFreeEntryVal(ht,de);
hi_free(de);
free(de);
ht->used--;
return DICT_OK;
}
@@ -234,13 +222,13 @@ static int _dictClear(dict *ht) {
nextHe = he->next;
dictFreeEntryKey(ht, he);
dictFreeEntryVal(ht, he);
hi_free(he);
free(he);
ht->used--;
he = nextHe;
}
}
/* Free the table and the allocated cache structure */
hi_free(ht->table);
free(ht->table);
/* Re-initialize the table */
_dictReset(ht);
return DICT_OK; /* never fails */
@@ -249,7 +237,7 @@ static int _dictClear(dict *ht) {
/* Clear & Release the hash table */
static void dictRelease(dict *ht) {
_dictClear(ht);
hi_free(ht);
free(ht);
}
static dictEntry *dictFind(dict *ht, const void *key) {
@@ -267,11 +255,14 @@ static dictEntry *dictFind(dict *ht, const void *key) {
return NULL;
}
static void dictInitIterator(dictIterator *iter, dict *ht) {
static dictIterator *dictGetIterator(dict *ht) {
dictIterator *iter = malloc(sizeof(*iter));
iter->ht = ht;
iter->index = -1;
iter->entry = NULL;
iter->nextEntry = NULL;
return iter;
}
static dictEntry *dictNext(dictIterator *iter) {
@@ -294,12 +285,16 @@ static dictEntry *dictNext(dictIterator *iter) {
return NULL;
}
static void dictReleaseIterator(dictIterator *iter) {
free(iter);
}
/* ------------------------- private functions ------------------------------ */
/* Expand the hash table if needed */
static int _dictExpandIfNeeded(dict *ht) {
/* If the hash table is empty expand it to the initial size,
* if the table is "full" double its size. */
* if the table is "full" dobule its size. */
if (ht->size == 0)
return dictExpand(ht, DICT_HT_INITIAL_SIZE);
if (ht->used == ht->size)
+2 -1
View File
@@ -119,7 +119,8 @@ static int dictReplace(dict *ht, void *key, void *val);
static int dictDelete(dict *ht, const void *key);
static void dictRelease(dict *ht);
static dictEntry * dictFind(dict *ht, const void *key);
static void dictInitIterator(dictIterator *iter, dict *ht);
static dictIterator *dictGetIterator(dict *ht);
static dictEntry *dictNext(dictIterator *iter);
static void dictReleaseIterator(dictIterator *iter);
#endif /* __DICT_H */
-61
View File
@@ -1,61 +0,0 @@
INCLUDE(FindPkgConfig)
# Check for GLib
PKG_CHECK_MODULES(GLIB2 glib-2.0)
if (GLIB2_FOUND)
INCLUDE_DIRECTORIES(${GLIB2_INCLUDE_DIRS})
LINK_DIRECTORIES(${GLIB2_LIBRARY_DIRS})
ADD_EXECUTABLE(example-glib example-glib.c)
TARGET_LINK_LIBRARIES(example-glib hiredis ${GLIB2_LIBRARIES})
ENDIF(GLIB2_FOUND)
FIND_PATH(LIBEV ev.h
HINTS /usr/local /usr/opt/local
ENV LIBEV_INCLUDE_DIR)
if (LIBEV)
# Just compile and link with libev
ADD_EXECUTABLE(example-libev example-libev.c)
TARGET_LINK_LIBRARIES(example-libev hiredis ev)
ENDIF()
FIND_PATH(LIBEVENT event.h)
if (LIBEVENT)
ADD_EXECUTABLE(example-libevent example-libevent.c)
TARGET_LINK_LIBRARIES(example-libevent hiredis event)
ENDIF()
FIND_PATH(LIBHV hv/hv.h)
IF (LIBHV)
ADD_EXECUTABLE(example-libhv example-libhv.c)
TARGET_LINK_LIBRARIES(example-libhv hiredis hv)
ENDIF()
FIND_PATH(LIBUV uv.h)
IF (LIBUV)
ADD_EXECUTABLE(example-libuv example-libuv.c)
TARGET_LINK_LIBRARIES(example-libuv hiredis uv)
ENDIF()
FIND_PATH(LIBSDEVENT systemd/sd-event.h)
IF (LIBSDEVENT)
ADD_EXECUTABLE(example-libsdevent example-libsdevent.c)
TARGET_LINK_LIBRARIES(example-libsdevent hiredis systemd)
ENDIF()
IF (APPLE)
FIND_LIBRARY(CF CoreFoundation)
ADD_EXECUTABLE(example-macosx example-macosx.c)
TARGET_LINK_LIBRARIES(example-macosx hiredis ${CF})
ENDIF()
IF (ENABLE_SSL)
ADD_EXECUTABLE(example-ssl example-ssl.c)
TARGET_LINK_LIBRARIES(example-ssl hiredis hiredis_ssl)
ENDIF()
ADD_EXECUTABLE(example example.c)
TARGET_LINK_LIBRARIES(example hiredis)
ADD_EXECUTABLE(example-push example-push.c)
TARGET_LINK_LIBRARIES(example-push hiredis)
-2
View File
@@ -33,9 +33,7 @@ void disconnectCallback(const redisAsyncContext *c, int status) {
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
iv_init();
-2
View File
@@ -33,9 +33,7 @@ void disconnectCallback(const redisAsyncContext *c, int status) {
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
-90
View File
@@ -1,90 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <hiredis.h>
#include <hiredis_ssl.h>
#include <async.h>
#include <adapters/libevent.h>
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
redisAsyncDisconnect(c);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
struct event_base *base = event_base_new();
if (argc < 5) {
fprintf(stderr,
"Usage: %s <key> <host> <port> <cert> <certKey> [ca]\n", argv[0]);
exit(1);
}
const char *value = argv[1];
size_t nvalue = strlen(value);
const char *hostname = argv[2];
int port = atoi(argv[3]);
const char *cert = argv[4];
const char *certKey = argv[5];
const char *caCert = argc > 5 ? argv[6] : NULL;
redisSSLContext *ssl;
redisSSLContextError ssl_error = REDIS_SSL_CTX_NONE;
redisInitOpenSSL();
ssl = redisCreateSSLContext(caCert, NULL,
cert, certKey, NULL, &ssl_error);
if (!ssl) {
printf("Error: %s\n", redisSSLContextGetError(ssl_error));
return 1;
}
redisAsyncContext *c = redisAsyncConnect(hostname, port);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
return 1;
}
if (redisInitiateSSLWithContext(&c->c, ssl) != REDIS_OK) {
printf("SSL Error!\n");
exit(1);
}
redisLibeventAttach(c,base);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncCommand(c, NULL, NULL, "SET key %b", value, nvalue);
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
event_base_dispatch(base);
redisFreeSSLContext(ssl);
return 0;
}
+2 -16
View File
@@ -9,12 +9,7 @@
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) {
if (c->errstr) {
printf("errstr: %s\n", c->errstr);
}
return;
}
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
@@ -38,19 +33,10 @@ void disconnectCallback(const redisAsyncContext *c, int status) {
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
struct event_base *base = event_base_new();
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
struct timeval tv = {0};
tv.tv_sec = 1;
options.connect_timeout = &tv;
redisAsyncContext *c = redisAsyncConnectWithOptions(&options);
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
-70
View File
@@ -1,70 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <hiredis.h>
#include <async.h>
#include <adapters/libhv.h>
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
redisAsyncDisconnect(c);
}
void debugCallback(redisAsyncContext *c, void *r, void *privdata) {
(void)privdata;
redisReply *reply = r;
if (reply == NULL) {
printf("`DEBUG SLEEP` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
redisAsyncDisconnect(c);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
return 1;
}
hloop_t* loop = hloop_new(HLOOP_FLAG_QUIT_WHEN_NO_ACTIVE_EVENTS);
redisLibhvAttach(c, loop);
redisAsyncSetTimeout(c, (struct timeval){.tv_sec = 0, .tv_usec = 500000});
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
redisAsyncCommand(c, debugCallback, NULL, "DEBUG SLEEP %d", 1);
hloop_run(loop);
hloop_free(&loop);
return 0;
}
-86
View File
@@ -1,86 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <hiredis.h>
#include <async.h>
#include <adapters/libsdevent.h>
void debugCallback(redisAsyncContext *c, void *r, void *privdata) {
(void)privdata;
redisReply *reply = r;
if (reply == NULL) {
/* The DEBUG SLEEP command will almost always fail, because we have set a 1 second timeout */
printf("`DEBUG SLEEP` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
/* Disconnect after receiving the reply of DEBUG SLEEP (which will not)*/
redisAsyncDisconnect(c);
}
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) {
printf("`GET key` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
printf("`GET key` result: argv[%s]: %s\n", (char*)privdata, reply->str);
/* start another request that demonstrate timeout */
redisAsyncCommand(c, debugCallback, NULL, "DEBUG SLEEP %f", 1.5);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("connect error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("disconnect because of error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
signal(SIGPIPE, SIG_IGN);
struct sd_event *event;
sd_event_default(&event);
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
printf("Error: %s\n", c->errstr);
redisAsyncFree(c);
return 1;
}
redisLibsdeventAttach(c,event);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncSetTimeout(c, (struct timeval){ .tv_sec = 1, .tv_usec = 0});
/*
In this demo, we first `set key`, then `get key` to demonstrate the basic usage of libsdevent adapter.
Then in `getCallback`, we start a `debug sleep` command to create 1.5 second long request.
Because we have set a 1 second timeout to the connection, the command will always fail with a
timeout error, which is shown in the `debugCallback`.
*/
redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
/* sd-event does not quit when there are no handlers registered. Manually exit after 1.5 seconds */
sd_event_source *s;
sd_event_add_time_relative(event, &s, CLOCK_MONOTONIC, 1500000, 1, NULL, NULL);
sd_event_loop(event);
sd_event_source_disable_unref(s);
sd_event_unref(event);
return 0;
}
+6 -34
View File
@@ -7,33 +7,18 @@
#include <async.h>
#include <adapters/libuv.h>
void debugCallback(redisAsyncContext *c, void *r, void *privdata) {
(void)privdata; //unused
redisReply *reply = r;
if (reply == NULL) {
/* The DEBUG SLEEP command will almost always fail, because we have set a 1 second timeout */
printf("`DEBUG SLEEP` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
/* Disconnect after receiving the reply of DEBUG SLEEP (which will not)*/
redisAsyncDisconnect(c);
}
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) {
printf("`GET key` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
printf("`GET key` result: argv[%s]: %s\n", (char*)privdata, reply->str);
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* start another request that demonstrate timeout */
redisAsyncCommand(c, debugCallback, NULL, "DEBUG SLEEP %f", 1.5);
/* Disconnect after receiving the reply to GET */
redisAsyncDisconnect(c);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("connect error: %s\n", c->errstr);
printf("Error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
@@ -41,17 +26,14 @@ void connectCallback(const redisAsyncContext *c, int status) {
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("disconnect because of error: %s\n", c->errstr);
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
#ifndef _WIN32
signal(SIGPIPE, SIG_IGN);
#endif
uv_loop_t* loop = uv_default_loop();
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
@@ -64,18 +46,8 @@ int main (int argc, char **argv) {
redisLibuvAttach(c,loop);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncSetTimeout(c, (struct timeval){ .tv_sec = 1, .tv_usec = 0});
/*
In this demo, we first `set key`, then `get key` to demonstrate the basic usage of libuv adapter.
Then in `getCallback`, we start a `debug sleep` command to create 1.5 second long request.
Because we have set a 1 second timeout to the connection, the command will always fail with a
timeout error, which is shown in the `debugCallback`.
*/
redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
uv_run(loop, UV_RUN_DEFAULT);
return 0;
}
-62
View File
@@ -1,62 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <async.h>
#include <adapters/poll.h>
/* Put in the global scope, so that loop can be explicitly stopped */
static int exit_loop = 0;
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
redisAsyncDisconnect(c);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
exit_loop = 1;
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
exit_loop = 1;
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
signal(SIGPIPE, SIG_IGN);
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
return 1;
}
redisPollAttach(c);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc-1], strlen(argv[argc-1]));
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
while (!exit_loop)
{
redisPollTick(c, 0.1);
}
return 0;
}
-159
View File
@@ -1,159 +0,0 @@
/*
* Copyright (c) 2020, Michael Grunder <michael dot grunder at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <hiredis.h>
#define KEY_COUNT 5
#define panicAbort(fmt, ...) \
do { \
fprintf(stderr, "%s:%d:%s(): " fmt, __FILE__, __LINE__, __func__, __VA_ARGS__); \
exit(-1); \
} while (0)
static void assertReplyAndFree(redisContext *context, redisReply *reply, int type) {
if (reply == NULL)
panicAbort("NULL reply from server (error: %s)", context->errstr);
if (reply->type != type) {
if (reply->type == REDIS_REPLY_ERROR)
fprintf(stderr, "Redis Error: %s\n", reply->str);
panicAbort("Expected reply type %d but got type %d", type, reply->type);
}
freeReplyObject(reply);
}
/* Switch to the RESP3 protocol and enable client tracking */
static void enableClientTracking(redisContext *c) {
redisReply *reply = redisCommand(c, "HELLO 3");
if (reply == NULL || c->err) {
panicAbort("NULL reply or server error (error: %s)", c->errstr);
}
if (reply->type != REDIS_REPLY_MAP) {
fprintf(stderr, "Error: Can't send HELLO 3 command. Are you sure you're ");
fprintf(stderr, "connected to redis-server >= 6.0.0?\nRedis error: %s\n",
reply->type == REDIS_REPLY_ERROR ? reply->str : "(unknown)");
exit(-1);
}
freeReplyObject(reply);
/* Enable client tracking */
reply = redisCommand(c, "CLIENT TRACKING ON");
assertReplyAndFree(c, reply, REDIS_REPLY_STATUS);
}
void pushReplyHandler(void *privdata, void *r) {
redisReply *reply = r;
int *invalidations = privdata;
/* Sanity check on the invalidation reply */
if (reply->type != REDIS_REPLY_PUSH || reply->elements != 2 ||
reply->element[1]->type != REDIS_REPLY_ARRAY ||
reply->element[1]->element[0]->type != REDIS_REPLY_STRING)
{
panicAbort("%s", "Can't parse PUSH message!");
}
/* Increment our invalidation count */
*invalidations += 1;
printf("pushReplyHandler(): INVALIDATE '%s' (invalidation count: %d)\n",
reply->element[1]->element[0]->str, *invalidations);
freeReplyObject(reply);
}
/* We aren't actually freeing anything here, but it is included to show that we can
* have hiredis call our data destructor when freeing the context */
void privdata_dtor(void *privdata) {
unsigned int *icount = privdata;
printf("privdata_dtor(): In context privdata dtor (invalidations: %u)\n", *icount);
}
int main(int argc, char **argv) {
unsigned int j, invalidations = 0;
redisContext *c;
redisReply *reply;
const char *hostname = (argc > 1) ? argv[1] : "127.0.0.1";
int port = (argc > 2) ? atoi(argv[2]) : 6379;
redisOptions o = {0};
REDIS_OPTIONS_SET_TCP(&o, hostname, port);
/* Set our context privdata to the address of our invalidation counter. Each
* time our PUSH handler is called, hiredis will pass the privdata for context.
*
* This could also be done after we create the context like so:
*
* c->privdata = &invalidations;
* c->free_privdata = privdata_dtor;
*/
REDIS_OPTIONS_SET_PRIVDATA(&o, &invalidations, privdata_dtor);
/* Set our custom PUSH message handler */
o.push_cb = pushReplyHandler;
c = redisConnectWithOptions(&o);
if (c == NULL || c->err)
panicAbort("Connection error: %s", c ? c->errstr : "OOM");
/* Enable RESP3 and turn on client tracking */
enableClientTracking(c);
/* Set some keys and then read them back. Once we do that, Redis will deliver
* invalidation push messages whenever the key is modified */
for (j = 0; j < KEY_COUNT; j++) {
reply = redisCommand(c, "SET key:%d initial:%d", j, j);
assertReplyAndFree(c, reply, REDIS_REPLY_STATUS);
reply = redisCommand(c, "GET key:%d", j);
assertReplyAndFree(c, reply, REDIS_REPLY_STRING);
}
/* Trigger invalidation messages by updating keys we just read */
for (j = 0; j < KEY_COUNT; j++) {
printf(" main(): SET key:%d update:%d\n", j, j);
reply = redisCommand(c, "SET key:%d update:%d", j, j);
assertReplyAndFree(c, reply, REDIS_REPLY_STATUS);
printf(" main(): SET REPLY OK\n");
}
printf("\nTotal detected invalidations: %d, expected: %d\n", invalidations, KEY_COUNT);
/* PING server */
redisFree(c);
}
-101
View File
@@ -1,101 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <hiredis.h>
#include <async.h>
#include <adapters/redismoduleapi.h>
void debugCallback(redisAsyncContext *c, void *r, void *privdata) {
(void)privdata; //unused
redisReply *reply = r;
if (reply == NULL) {
/* The DEBUG SLEEP command will almost always fail, because we have set a 1 second timeout */
printf("`DEBUG SLEEP` error: %s\n", c->errstr ? c->errstr : "unknown error");
return;
}
/* Disconnect after receiving the reply of DEBUG SLEEP (which will not)*/
redisAsyncDisconnect(c);
}
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) {
if (c->errstr) {
printf("errstr: %s\n", c->errstr);
}
return;
}
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* start another request that demonstrate timeout */
redisAsyncCommand(c, debugCallback, NULL, "DEBUG SLEEP %f", 1.5);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
/*
* This example requires Redis 7.0 or above.
*
* 1- Compile this file as a shared library. Directory of "redismodule.h" must
* be in the include path.
* gcc -fPIC -shared -I../../redis/src/ -I.. example-redismoduleapi.c -o example-redismoduleapi.so
*
* 2- Load module:
* redis-server --loadmodule ./example-redismoduleapi.so value
*/
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
int ret = RedisModule_Init(ctx, "example-redismoduleapi", 1, REDISMODULE_APIVER_1);
if (ret != REDISMODULE_OK) {
printf("error module init \n");
return REDISMODULE_ERR;
}
if (redisModuleCompatibilityCheck() != REDIS_OK) {
printf("Redis 7.0 or above is required! \n");
return REDISMODULE_ERR;
}
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
return 1;
}
size_t len;
const char *val = RedisModule_StringPtrLen(argv[argc-1], &len);
RedisModuleCtx *module_ctx = RedisModule_GetDetachedThreadSafeContext(ctx);
redisModuleAttach(c, module_ctx);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncSetTimeout(c, (struct timeval){ .tv_sec = 1, .tv_usec = 0});
/*
In this demo, we first `set key`, then `get key` to demonstrate the basic usage of the adapter.
Then in `getCallback`, we start a `debug sleep` command to create 1.5 second long request.
Because we have set a 1 second timeout to the connection, the command will always fail with a
timeout error, which is shown in the `debugCallback`.
*/
redisAsyncCommand(c, NULL, NULL, "SET key %b", val, len);
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
return 0;
}
-112
View File
@@ -1,112 +0,0 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <hiredis.h>
#include <hiredis_ssl.h>
#ifdef _MSC_VER
#include <winsock2.h> /* For struct timeval */
#endif
int main(int argc, char **argv) {
unsigned int j;
redisSSLContext *ssl;
redisSSLContextError ssl_error = REDIS_SSL_CTX_NONE;
redisContext *c;
redisReply *reply;
if (argc < 4) {
printf("Usage: %s <host> <port> <cert> <key> [ca]\n", argv[0]);
exit(1);
}
const char *hostname = (argc > 1) ? argv[1] : "127.0.0.1";
int port = atoi(argv[2]);
const char *cert = argv[3];
const char *key = argv[4];
const char *ca = argc > 4 ? argv[5] : NULL;
redisInitOpenSSL();
ssl = redisCreateSSLContext(ca, NULL, cert, key, NULL, &ssl_error);
if (!ssl || ssl_error != REDIS_SSL_CTX_NONE) {
printf("SSL Context error: %s\n", redisSSLContextGetError(ssl_error));
exit(1);
}
struct timeval tv = { 1, 500000 }; // 1.5 seconds
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, hostname, port);
options.connect_timeout = &tv;
c = redisConnectWithOptions(&options);
if (c == NULL || c->err) {
if (c) {
printf("Connection error: %s\n", c->errstr);
redisFree(c);
} else {
printf("Connection error: can't allocate redis context\n");
}
exit(1);
}
if (redisInitiateSSLWithContext(c, ssl) != REDIS_OK) {
printf("Couldn't initialize SSL!\n");
printf("Error: %s\n", c->errstr);
redisFree(c);
exit(1);
}
/* PING server */
reply = redisCommand(c,"PING");
printf("PING: %s\n", reply->str);
freeReplyObject(reply);
/* Set a key */
reply = redisCommand(c,"SET %s %s", "foo", "hello world");
printf("SET: %s\n", reply->str);
freeReplyObject(reply);
/* Set a key using binary safe API */
reply = redisCommand(c,"SET %b %b", "bar", (size_t) 3, "hello", (size_t) 5);
printf("SET (binary API): %s\n", reply->str);
freeReplyObject(reply);
/* Try a GET and two INCR */
reply = redisCommand(c,"GET foo");
printf("GET foo: %s\n", reply->str);
freeReplyObject(reply);
reply = redisCommand(c,"INCR counter");
printf("INCR counter: %lld\n", reply->integer);
freeReplyObject(reply);
/* again ... */
reply = redisCommand(c,"INCR counter");
printf("INCR counter: %lld\n", reply->integer);
freeReplyObject(reply);
/* Create a list of numbers, from 0 to 9 */
reply = redisCommand(c,"DEL mylist");
freeReplyObject(reply);
for (j = 0; j < 10; j++) {
char buf[64];
snprintf(buf,64,"%u",j);
reply = redisCommand(c,"LPUSH mylist element-%s", buf);
freeReplyObject(reply);
}
/* Let's check what we have inside the list */
reply = redisCommand(c,"LRANGE mylist 0 -1");
if (reply->type == REDIS_REPLY_ARRAY) {
for (j = 0; j < reply->elements; j++) {
printf("%u) %s\n", j, reply->element[j]->str);
}
}
freeReplyObject(reply);
/* Disconnects and frees the context */
redisFree(c);
redisFreeSSLContext(ssl);
return 0;
}
+3 -70
View File
@@ -1,82 +1,18 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <hiredis.h>
#ifdef _MSC_VER
#include <winsock2.h> /* For struct timeval */
#endif
static void example_argv_command(redisContext *c, size_t n) {
char **argv, tmp[42];
size_t *argvlen;
redisReply *reply;
/* We're allocating two additional elements for command and key */
argv = malloc(sizeof(*argv) * (2 + n));
argvlen = malloc(sizeof(*argvlen) * (2 + n));
/* First the command */
argv[0] = (char*)"RPUSH";
argvlen[0] = sizeof("RPUSH") - 1;
/* Now our key */
argv[1] = (char*)"argvlist";
argvlen[1] = sizeof("argvlist") - 1;
/* Now add the entries we wish to add to the list */
for (size_t i = 2; i < (n + 2); i++) {
argvlen[i] = snprintf(tmp, sizeof(tmp), "argv-element-%zu", i - 2);
argv[i] = strdup(tmp);
}
/* Execute the command using redisCommandArgv. We're sending the arguments with
* two explicit arrays. One for each argument's string, and the other for its
* length. */
reply = redisCommandArgv(c, n + 2, (const char **)argv, (const size_t*)argvlen);
if (reply == NULL || c->err) {
fprintf(stderr, "Error: Couldn't execute redisCommandArgv\n");
exit(1);
}
if (reply->type == REDIS_REPLY_INTEGER) {
printf("%s reply: %lld\n", argv[0], reply->integer);
}
freeReplyObject(reply);
/* Clean up */
for (size_t i = 2; i < (n + 2); i++) {
free(argv[i]);
}
free(argv);
free(argvlen);
}
int main(int argc, char **argv) {
unsigned int j, isunix = 0;
unsigned int j;
redisContext *c;
redisReply *reply;
const char *hostname = (argc > 1) ? argv[1] : "127.0.0.1";
if (argc > 2) {
if (*argv[2] == 'u' || *argv[2] == 'U') {
isunix = 1;
/* in this case, host is the path to the unix socket */
printf("Will connect to unix socket @%s\n", hostname);
}
}
int port = (argc > 2) ? atoi(argv[2]) : 6379;
struct timeval timeout = { 1, 500000 }; // 1.5 seconds
if (isunix) {
c = redisConnectUnixWithTimeout(hostname, timeout);
} else {
c = redisConnectWithTimeout(hostname, port, timeout);
}
c = redisConnectWithTimeout(hostname, port, timeout);
if (c == NULL || c->err) {
if (c) {
printf("Connection error: %s\n", c->errstr);
@@ -135,9 +71,6 @@ int main(int argc, char **argv) {
}
freeReplyObject(reply);
/* See function for an example of redisCommandArgv */
example_argv_command(c, 10);
/* Disconnects and frees the context */
redisFree(c);
+15 -4
View File
@@ -1,14 +1,25 @@
#ifndef __HIREDIS_FMACRO_H
#define __HIREDIS_FMACRO_H
#ifndef _AIX
#define _XOPEN_SOURCE 600
#if defined(__linux__)
#define _BSD_SOURCE
#define _DEFAULT_SOURCE
#endif
#if defined(__CYGWIN__)
#include <sys/cdefs.h>
#endif
#if defined(__sun__)
#define _POSIX_C_SOURCE 200112L
#else
#if !(defined(__APPLE__) && defined(__MACH__)) && !(defined(__FreeBSD__))
#define _XOPEN_SOURCE 600
#endif
#endif
#if defined(__APPLE__) && defined(__MACH__)
/* Enable TCP_KEEPALIVE */
#define _DARWIN_C_SOURCE
#define _OSX
#endif
#endif
-56
View File
@@ -1,56 +0,0 @@
/*
* Copyright (c) 2020, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2020, Pieter Noordhuis <pcnoordhuis at gmail dot com>
* Copyright (c) 2020, Matt Stancliff <matt at genges dot com>,
* Jan-Erik Rediger <janerik at fnordig dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdlib.h>
#include <string.h>
#include "hiredis.h"
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
char *new_str, *cmd;
if (size < 3)
return 0;
new_str = malloc(size+1);
if (new_str == NULL)
return 0;
memcpy(new_str, data, size);
new_str[size] = '\0';
if (redisFormatCommand(&cmd, new_str) != -1)
hi_free(cmd);
free(new_str);
return 0;
}
-13
View File
@@ -1,13 +0,0 @@
@PACKAGE_INIT@
set_and_check(hiredis_INCLUDEDIR "@PACKAGE_INCLUDE_INSTALL_DIR@")
IF (NOT TARGET hiredis::@hiredis_export_name@)
INCLUDE(${CMAKE_CURRENT_LIST_DIR}/hiredis-targets.cmake)
ENDIF()
SET(hiredis_LIBRARIES hiredis::@hiredis_export_name@)
SET(hiredis_INCLUDE_DIRS ${hiredis_INCLUDEDIR})
check_required_components(hiredis)
+240 -438
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More