Compare commits

..
72 Commits
Author SHA1 Message Date
Madelyn OlsonandGitHub ffcb4e222b Release notes for 7.2.5 (#322)
Add GA release notes for 7.2.5. 

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-15 21:18:47 -07:00
Ping XieandMadelyn Olson 6a5f8eaf1b Fixed url links in valkey.conf (#320)
Signed-off-by: Ping Xie <pingxie@google.com>
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-15 12:50:27 -07:00
Madelyn OlsonandGitHub 75c2746c95 Bumped version to 7.2.5 for valkey and wrote release notes (#305)
Release notes for RC2 (really 7.2.5-rc1). Based on conversation, decided
to have first release on 7.2.5 to make it clear this was a patch release
over Redis. This could be considered a minor release, but we wanted to
clearly signal compatibility with Redis OSS.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-12 12:06:53 -07:00
Madelyn OlsonandMadelyn Olson 25486f1647 Add links for security issues (#299)
Add an initial security release page. In the fullness of time I would
like to also include our version support here, but until that has been
decided I would like to keep this simple and just include links.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-11 15:58:12 -07:00
ParthandMadelyn Olson 573579acfa Fixing a lua debugger bug that prevented use of 'server' for server.call invocations. (#303)
* Tested it on local instance. This was originally part of
https://github.com/valkey-io/valkey/pull/288 but I am pushing this
separately, so that we can easily merge it into the upcoming release.

```
lua debugger> server ping
<redis> ping
<reply> "+PONG"
lua debugger> redis ping
<redis> ping
<reply> "+PONG"
```

* I also searched for lua debugger related unit tests to add coverage
for this but did not find any relevant test to modify. Leaving it at
that for now.

---------

Signed-off-by: Parth Patel <661497+parthpatel@users.noreply.github.com>
2024-04-11 15:55:32 -07:00
Madelyn Olson 78a168a538 Overwrite 7.2 README file with unstable's version
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-11 13:17:06 -07:00
Roshan KhatriandMadelyn Olson 8c7484d9bf Revert update of RedisModuleEvent_MasterLinkChange (#289)
ValkeyModuleEvent_MasterLinkChange was updated to use more inclusive
language, but it was done in the compatibility layer as well
(RedisModuleEvent_).

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2024-04-10 17:35:38 -07:00
Björn SvenssonandGitHub b61785fc37 Correcting the installed redis symlinks in 7.2.4-rc1 (#282)
This is a PR directly to the 7.2 branch.

The make variable `ENGINE_NAME` (from
7d5fe95851) was lost during branching of
`7.2` and tag `7.2.4-rc1`
resulting in the creation of faulty symlinks:
```
      INSTALL SYMLINK valkey-serverredis -> valkey-server
      INSTALL SYMLINK valkey-cliredis -> valkey-cli
      INSTALL SYMLINK valkey-benchmarkredis -> valkey-benchmark
      INSTALL SYMLINK valkey-check-rdbredis -> valkey-check-rdb
      INSTALL SYMLINK valkey-check-aofredis -> valkey-check-aof
      INSTALL SYMLINK valkey-sentinelredis -> valkey-sentinel
```

By just adding the variable we get a minimal diff compared to unstable.

Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
2024-04-10 16:55:27 -07:00
ShivshankarandViktor Söderqvist babf19e1f1 Rename redis in valkey-cli file comments and prints (#222)
Updated to Valkey in valkey-cli.c file's comments and prints.

* The output of valkey-cli --help
* The output of the cli built-in HELP command
* The prompt in interactive valkey-cli -s unixsocket
* The history file and the default rc file (changed filename)

---------

Signed-off-by: Shivshankar-Reddy <shiva.sheri.github@gmail.com>
2024-04-10 11:03:38 +02:00
Madelyn OlsonandMadelyn Olson 4fcde04dcc Revert the default PID file back to the real default (#275)
The default pid file is created at /var/run/redis.pid based on the code
at
https://github.com/valkey-io/valkey/blob/c65a6fecba8e6bb09b9f349f96b63f7b693ae252/src/server.h#L132.
Until we update it, we should reflect that in the conf file.

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-09 08:22:18 -07:00
Madelyn OlsonandGitHub 94fcf4318e Initial draft for 7.2.4.Rc1 release notes (#247)
Add release nodes for 7.2.4-rc1. 

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-09 08:13:22 -07:00
Lipeng ZhuandViktor Söderqvist fb75e2f9df Changes references to redis binaries in output of "--help", "--version" (#113)
Rename output from redis-* to valkey-* for binaries:

1. `valkey-benchmark`
2. `valkey-cli`
3. `valkey-server`
4. `valkey-sentinel`
5. `valkey-check-rdb`
6. `valkey-check-aof`

"--help" "--version" option.

Signed-off-by: Lipeng Zhu <lipeng.zhu@intel.com>
2024-04-09 09:49:33 +02:00
Vitah LinandMadelyn Olson d27f439e67 Fix rename redis to valkey to pass reply-schemas-validator job (#133)
Signed-off-by: Vitah Lin <vitahlin@gmail.com>
2024-04-08 17:11:02 -07:00
Madelyn OlsonandMadelyn Olson 885f969d50 Update crash wording to include our repo (#263)
Update the wording in the crash log to point to Valkey repo instead of Redis repo.

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-08 10:09:57 -07:00
Harkrishn PatroandMadelyn Olson 13332e28c3 Pass extensions to node if extension processing is handled by it (#52)
Ref: https://github.com/redis/redis/pull/12760

enabled by default) with older Redis cluster (< 7.0 - extensions not
handled) .

With some of the extensions enabled by default in 7.2 version, new nodes
running 7.2 and above start sending out larger clusterbus message
payload including the ping extensions. This caused an incompatibility
with node running engine versions < 7.0. Old nodes (< 7.0) would receive
the payload from new nodes (> 7.2) would observe a payload length
(totlen) > (estlen) and would perform an early exit and won't process
the message.

This fix introduces a flag `extensions_supported` on the clusterMsg
indicating the sender node can handle extensions parsing. Once, a
receiver nodes receives a message with this flag set to 1, it would
update clusterNode new field extensions_supported and start sending out
extensions if it has any.

This PR also introduces a DEBUG sub command to enable/disable cluster
message extensions `process-clustermsg-extensions` feature.

Note: A successful `PING`/`PONG` is required as a sender for a given
node to be marked as `extensions_supported` and then extensions message
will be sent to it. This could cause a slight delay in receiving the
extensions message(s).

TCL test verifying the cluster state is healthy irrespective of
enabling/disabling cluster message extensions feature.

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2024-04-08 09:26:13 -07:00
BanyandMadelyn Olson b3d4653ccb Update ValkeyModuleEvent_MasterLinkChange to ValkeyModuleEvent_PrimaryLinkChange (#262)
Update ValkeyModuleEvent_MasterLinkChange to ValkeyModuleEvent_PrimaryLinkChange

Signed-off-by: 0del <bany.y0599@gmail.com>
2024-04-08 09:16:45 -07:00
Viktor SöderqvistandMadelyn Olson af5784c031 Log branding (#252)
Small changes to the log messages printed during startup and shutdown,
for Valkey branding.

SERVER_NAME is replaced by verbatim "Valkey" in one place, because
SERVER_NAME expands to "valkey" in lowercase. (Should we introduce
another macro that expands to "Valkey"?)

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2024-04-07 17:32:22 -07:00
Madelyn OlsonandMadelyn Olson f5d8b9f536 Madelyn's attempt as a logo (#251)
Apply new logo at startup.

It is one character wider and 2 characters taller than the original
Redis logo.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-07 17:32:10 -07:00
Madelyn OlsonandMadelyn Olson 3cfeaea24c Fix two typos that were flagged in the 7.2 build (#248)
These were flagged on the 7.2 build system, which is using the old spell
check. I think we should consider re-adding it as it missed some typos.

Relevant: https://github.com/valkey-io/valkey/pull/72

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-07 00:08:13 -07:00
f598645e7e The usage (--help) message now refers to valkey (#189)
Fixing redis -> valkey in the output of valkey-server --help.

Signed-off-by: Daniel House <daniel.house@huawei.com>
Co-authored-by: Daniel House <daniel.house@huawei.com>
2024-04-06 23:51:02 -07:00
Madelyn Olson b2ff9407e2 Apply missing 7.2 changes
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 23:50:57 -07:00
Wen HuiandMadelyn Olson 2872565dd1 Update Valkey keyword in sentinel.conf (#171)
Mostly comments, but one pre-filled config in this template config file
is changed:

    pidfile /var/run/valkey-sentinel.pid

---------

Signed-off-by: hwware <wen.hui.ware@gmail.com>
2024-04-06 22:50:16 -07:00
Harkrishn PatroandMadelyn Olson 1ba9929633 Remove trademarked wording on configuration file and individual configs (#29)
Remove trademarked wording on configuration layer.

Following changes for release notes:

1. Rename redis.conf to valkey.conf
2. Pre-filled config in the template config file: Changing pidfile to `/var/run/valkey_6379.pid`

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
2024-04-06 22:49:27 -07:00
d3972ad5a8 Add redis symlinks at the same place as the installed binaries (#193)
Adds a new make variable called `USE_REDIS_SYMLINKS`, with default value
`yes`. If yes, then `make install` creates additional symlinks to the
installed binaries:

* `valkey-server`
* `valkey-cli`
* `valkey-benchmark`
* `valkey-check-rdb`
* `valkey-check-aof`
* `valkey-sentinel`

The names of the symlinks are the legacy redis binary names
(`redis-server`, etc.). The purpose is to provide backward compatibility
for scripts expecting the these filenames. The symlinks are installed in
the same directory as the binaries (typically `/usr/local/bin/` or
similar).

Similarly, `make uninstall` removes these symlinks if
`USE_REDIS_SYMLINKS` is `yes`.

This is described in a note in README.md.

Fixes #147

---------

Signed-off-by: Vitah Lin <vitahlin@gmail.com>
Co-authored-by: Madelyn Olson <34459052+madolson@users.noreply.github.com>
2024-04-06 22:44:42 -07:00
bacdcbe18b Rename ValkeyModule_DefragModuleString to ValkeyModule_DefragValkeyModuleString (#243)
fixes: #242

---------

Signed-off-by: 0del <bany.y0599@gmail.com>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2024-04-06 22:41:55 -07:00
Madelyn Olson 8da5b6333f Fix merge conflicts
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 22:41:50 -07:00
debing.sunandMadelyn Olson ef746b0dff Check user's oom_score_adj write permission for oom-score-adj test (#13111)
`CONFIG SET oom-score-adj handles configuration failures` test failed in
some CI jobs today.
Failed CI: https://github.com/redis/redis/actions/runs/8152519326

Not sure why the github action's docker image perssions have changed,
but the issue is similar to #12887,
where we can't assume the range of oom_score_adj that a user can change.

## Solution:
Modify the way of determining whether the current user has no privileges
or not,
instead of relying on whether the user id is 0 or not.
2024-04-06 22:24:22 -07:00
04c2d0dfe9 Fix oom-score-adj test due to no permission (#12887)
Fix #12792

On ubuntu 23(lunar), non-root users will not be allowed to change the
oom_score_adj of a process to a value that is too low.
Since terminal's default oom_score_adj is 200, if we run the test on
terminal, we won't be able to set the oom_score_adj of the redis process
to 9 or 22, which is too low.

Reproduction on ubuntu 23(lunar) terminal:
```sh
$ cat /proc/`pgrep redis-server`/oom_score_adj
200
$ echo 100 > /proc/`pgrep redis-server`/oom_score_adj
# success without error
$ echo 99 > /proc/`pgrep redis-server`/oom_score_adj
echo: write error: Permission denied
```

As from the output above, we can only set the minimum oom score of redis
processes to 100.
By modifying the test, make oom_score_adj only increase upwards and not
decrease.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2024-04-06 22:08:51 -07:00
Ping XieandMadelyn Olson c79091d72e Remove REDISMODULE_ prefixes and introduce compatibility header (#194)
Fix #146

Removed REDISMODULE_ prefixes from the core source code to align with
the new SERVERMODULE_ naming convention. Added a new 'redismodule.h'
header file to ensure full backward compatibility with existing modules.
This compatibility layer maps all legacy REDISMODULE_ prefixed
identifiers to their new SERVERMODULE_ equivalents, allowing existing
Redis modules to function without modification.

---------

Signed-off-by: Ping Xie <pingxie@google.com>
2024-04-06 21:59:15 -07:00
ParthandMadelyn Olson f30986f299 Adding server.call/pcall option to LUA scripting. (#136) (#213)
This commit does not remove redis.call/pcall just yet. It also does not
rename Redis in error messages such as "Please specify at least one
argument for this redis lib call". This allows users to maintain full
backwards compatibility while introducing an option to use server.call
for new code.

I verified that the unit tests pass. Also manually verified that the
redis-server responds to server.call invocations within lua scripting.
Also verified that function registration works as expected.

```
[ok]: EVAL - is Lua able to call Redis API? (0 ms)
[ok]: EVAL - is Lua able to call Server API? (1 ms)
[ok]: EVAL - No arguments to redis.call/pcall is considered an error (0 ms)
[ok]: EVAL - No arguments to server.call/pcall is considered an error (1 ms)
```

---------

Signed-off-by: Parth Patel <661497+parthpatel@users.noreply.github.com>
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 20:43:56 -07:00
Roshan KhatriandMadelyn Olson 1abedeeb7e For additional compatibility this adds REDIS_CFLAGS and REDIS_LDFLAGS support to MAKEFILE (#66)
This resolves (1.viii) from
https://github.com/valkey-io/valkey/issues/43
> REDIS_FLAGS will be updated to SERVER_FLAGS. Maybe we should also
allow REDIS_FLAGS -> SERVER_FLAGS as well, for an extra layer of
compatibility.

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
2024-04-06 20:43:52 -07:00
Madelyn OlsonandMadelyn Olson 9c4237c17e Update Server version to valkey version (#232)
This commit updates the following fields:
1. server_version -> valkey_version in server info. Since we would like
to advertise specific compatibility, we are making the version specific
to valkey. servername will remain as an optional indicator, and other
valkey compatible stores might choose to advertise something else.
1. We dropped redis-ver from the API. This isn't related to API
compatibility, but we didn't want to "fake" that valkey was creating an
rdb from a Redis version.
1. Renamed server-ver -> valkey_version in rdb info. Same as point one,
we want to explicitly indicate this was created by a valkey server.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 20:43:42 -07:00
Madelyn OlsonandMadelyn Olson 3a9009d3bd Update versioning fields for compatibility (#47)
New info information to be used to determine the valkey versioning info.

Internally, introduce new define values for "SERVER_VERSION" which is
different from the Redis compatibility version, "REDIS_VERSION".

Add two new info fields:
`server_version`: The Valkey server version
`server_name`: Indicates that the server is valkey.

Add one new RDB field: `server_ver`, which indicates the valkey version
that produced the server.

Add 3 new LUA globals: `SERVER_VERSION_NUM`, `SERVER_VERSION`, and
`SERVER_NAME`. Which reflect the valkey version instead of the Redis
compatibility version.

Also clean up various places where Redis and configuration was being
used that is no longer necessary.

---------

Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 20:36:05 -07:00
Madelyn Olson 20339015f0 Squash changes for creating valkey-server
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
2024-04-06 20:33:09 -07:00
Oran Agra 2c9af0f00d Redis 7.2.4 2024-01-09 13:51:49 +02:00
BinbinandOran Agra 122e375eeb Fix CLUSTER SHARDS crash in 7.0/7.2 mixed clusters where shard ids are not sync (#12832)
Crash reported in #12695. In the process of upgrading the cluster from
7.0 to 7.2, because the 7.0 nodes will not gossip shard id, in 7.2 we
will rely on shard id to build the server.cluster->shards dict.

In some cases, for example, the 7.0 master node and the 7.2 replica node.
From the view of 7.2 replica node, the cluster->shards dictionary does not
have its master node. In this case calling CLUSTER SHARDS on the 7.2 replica
node may crash.

We should fix the underlying assumption of updateShardId, which is that the
shard dict should be always in sync with the node's shard_id. The fix was
suggested by PingXie, see more details in #12695.

(cherry picked from commit 77f48cd894)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra d70a9d1653 Use shard-id of the master if the replica does not support shard-id (#12805)
If there are nodes in the cluster that do not support shard-id, they
will gossip shard-id. From the perspective of nodes that support shard-id,
their shard-id is meaningless (since shard-id is randomly generated when
we create a node.)

Nodes that support shard-id will save the shard-id information in nodes.conf.
If the node is restarted according to nodes.conf, the server will report a
corrupted cluster config file error. Because auxShardIdSetter will reject
configurations with inconsistent master-replica shard-ids.

A cluster-wide consensus for the node's shard_id is not necessary. The key
is maintaining consistency of the shard_id on each individual 7.2 node.
As the cluster progressively upgrades to version 7.2, we can expect the
shard_ids across all nodes to naturally converge and align.

In this PR, when processing the gossip, if sender is a replica and does not
support shard-id, set the shard_id to the shard_id of its master.

(cherry picked from commit e4811e61d6)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra e52f3bfdd1 Un-register notification and server event when RedisModule_OnLoad fails (#12809)
When we register notification or server event in RedisModule_OnLoad, but
RedisModule_OnLoad eventually fails, triggering notification or server
event
will cause the server to crash.

If the loading fails on a later stage of moduleLoad, we do call
moduleUnload
which handles all un-registration, but when it fails on the
RedisModule_OnLoad
call, we only un-register several specific things and these were
missing:

- moduleUnsubscribeNotifications
- moduleUnregisterFilters
- moduleUnsubscribeAllServerEvents

Refactored the code to reuse the code from moduleUnload.

Fixes #12808.

(cherry picked from commit 4b0f42817a)
2024-01-09 13:51:49 +02:00
Meir Shpilraien (Spielrein)andOran Agra 94b5ec459a Before evicted and before expired server events are not executed inside an execution unit. (#12733)
Redis 7.2 (#9406) introduced a new modules event, `RedisModuleEvent_Key`.
This new event allows the module to read the key data just before it is removed
from the database (either deleted, expired, evicted, or overwritten).

When the key is removed from the database, either by active expire or eviction.
The new event was not called as part of an execution unit. This can cause an
issue if the module registers a post notification job inside the event. This job will
not be executed atomically with the expiration/eviction operation and will not
replicated inside a Multi/Exec. Moreover, the post notification job will be executed
right after the event where it is still not safe to perform any write operation, this will
violate the promise that post notification job will be called atomically with the
operation that triggered it and **only when it is safe to write**.

This PR fixes the issue by wrapping each expiration/eviction of a key with an execution
unit. This makes sure the entire operation will run atomically and all the post notification
jobs will be executed at the end where it is safe to write.

Tests were modified to verify the fix.

(cherry picked from commit fad56faead)
2024-01-09 13:51:49 +02:00
SankarandOran Agra 802a70e3ad Clear owner_not_claiming_slot bit for the slot in clusterDelSlot (#12564)
Clear owner_not_claiming_slot bit for the slot in clusterDelSlot to keep it
consistent with slot ownership information.

(cherry picked from commit 11ded735b3)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra 22c9e05439 Use CLZ in _dictNextExp to get the next power of two (#12815)
In the past, we did not call _dictNextExp frequently. It was only
called when the dictionary was expanded.

Later, dictTypeExpandAllowed was introduced in #7954, which is 6.2.
For the data dict and the expire dict, we can check maxmemory before
actually expanding the dict. This is a good optimization to avoid
maxmemory being exceeded due to the dict expansion.

And in #11692, we moved the dictTypeExpandAllowed check before the
threshold check, this caused a bit of performance degradation, every
time a key is added to the dict, dictTypeExpandAllowed is called to
check.

The main reason for degradation is that in a large dict, we need to
call _dictNextExp frequently, that is, every time we add a key, we
need to call _dictNextExp once. Then the threshold is checked to see
if the dict needs to be expanded. We can see that the order of checks
here can be optimized.

So we moved the dictTypeExpandAllowed check back to after the threshold
check in #12789. In this way, before the dict is actually expanded (that
is, before the threshold is reached), we will not do anything extra
compared to before, that is, we will not call _dictNextExp frequently.

But note we'll still hit the degradation when we over the thresholds.
When the threshold is reached, because #7954, we may delay the dict
expansion due to maxmemory limitations. In this case, we will call
_dictNextExp every time we add a key during this period.

This PR use CLZ in _dictNextExp to get the next power of two. CLZ (count
leading zeros) can easily give you the next power of two. It should be
noted that we have actually introduced the use of __builtin_clzl in
#8687,
which is 7.0. So i suppose all the platforms we use have it (even if the
CPU doesn't have an instruction).

We build 67108864 (2**26) keys through DEBUG POPULTE, which will use
approximately 5.49G memory (used_memory:5898522936). If expansion is
triggered, the additional hash table will consume approximately 1G
memory (2 ** 27 * 8). So we set maxmemory to 6871947673 (that is, 6.4G),
which will be less than 5.49G + 1G, so we will delay the dict rehash
while addint the keys.

After that, each time an element is added to the dict, an allow check
will be performed, that is, we can frequently call _dictNextExp to test
the comparison before and after the optimization. Using DEBUG HTSTATS 0
to
check and make sure that our dict expansion is dealyed.

Using `./src/redis-server redis.conf --save "" --maxmemory 6871947673`.
Using `./src/redis-benchmark -P 100 -r 1000000000 -t set -n 5000000`.
After ten rounds of testing:
```
unstable:           this PR:
769585.94           816860.00
771724.00           818196.69
775674.81           822368.44
781983.12           822503.69
783576.25           828088.75
784190.75           828637.75
791389.69           829875.50
794659.94           835660.69
798212.00           830013.25
801153.62           833934.56
```

We can see there is about 4-5% performance improvement in this case.

(cherry picked from commit a0059b3607)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra 44eff3aad4 Optimize dict expand check, move allow check after the thresholds check (#12789)
dictExpandAllowed (for the main db dict and the expire dict) seems to
involve a few function calls and memory accesses, and we can do it only
after the thresholds checks and can get some performance improvements.

A simple benchmark test: there are 11032768 fixed keys in the database,
start a redis-server with `--maxmemory big_number --save ""`,
start a redis-benchmark with `-P 100 -r 1000000000 -t set -n 5000000`,
collect `throughput summary: n requests per second` result.

After five rounds of testing:
```
unstable     this PR
848032.56    897988.56
854408.69    913408.88
858663.94    914076.81
871839.56    916758.31
882612.56    920640.75
```

We can see a 5% performance improvement in general condition.
But note we'll still hit the degradation when we over the thresholds.

(cherry picked from commit f3720f74bd)
2024-01-09 13:51:49 +02:00
Oran Agra 8e5683fb1c Fix possible corruption in sdsResize (CVE-2023-41056)
#11766 introduced a bug in sdsResize where it could forget to update
the sds type in the sds header and then cause an overflow in sdsalloc.
it looks like the only implication of that is a possible assertion in HLL,
but it's hard to rule out possible heap corruption issues with clientsCronResizeQueryBuffer
2024-01-09 13:51:49 +02:00
Oran Agra bb9130d9c6 Redis 7.2.3 2023-11-01 14:38:13 +02:00
Viktor SöderqvistandOran Agra 9d1150494b Don't crash when adding a forgotten node to blacklist twice (#12702)
Add a defensive checks to prevent double freeing a node from the cluster blacklist.

(cherry picked from commit 4453eabb8e)
2023-11-01 14:38:13 +02:00
Oran Agra 19d6702358 Fix fd leak causing deleted files to remain open and eat disk space (#12693)
This was introduced in v7.2 by #11248

(cherry picked from commit 4d2ac760c3)
2023-11-01 14:38:13 +02:00
Oran Agra 629b757cc4 Redis 7.2.2 2023-10-18 10:44:10 +03:00
BinbinandOran Agra 708a7e9ea1 Fix crash when running rebalance command in a mixed cluster of 7.0 and 7.2 (#12604)
In #10536, we introduced the assert, some older versions of servers
(like 7.0) doesn't gossip shard_id, so we will not add the node to
cluster->shards, and node->shard_id is filled in randomly and may not
be found here.

It causes that if we add a 7.2 node to a 7.0 cluster and allocate slots
to the 7.2 node, the 7.2 node will crash when it hits this assert. Somehow
like #12538.

In this PR, we remove the assert and replace it with an unconditional removal.

(cherry picked from commit 195e6a362d)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 67bb61fa5c Fix redis-cli pubsub_mode and connect minor prompt / crash issue (#12571)
When entering pubsub mode and using the redis-cli only
connect command, we need to reset pubsub_mode because
we switch to a different connection.

This will affect the prompt when the connection is successful,
and redis-cli will crash when the connect fails:
```
127.0.0.1:6379> subscribe ch
1) "subscribe"
2) "ch"
3) (integer) 1
127.0.0.1:6379(subscribed mode)> connect 127.0.0.1 6380
127.0.0.1:6380(subscribed mode)> ping
PONG
127.0.0.1:6380(subscribed mode)> connect a b
Could not connect to Redis at a:0: Name or service not known
Segmentation fault
```

(cherry picked from commit 91925792d6)
2023-10-18 10:44:10 +03:00
guybe7andOran Agra 93fee40390 WAITAOF: Update fsynced_reploff_pending just before starting the initial AOFRW fork (#12620)
If we set `fsynced_reploff_pending` in `startAppendOnly`, and the fork doesn't start
immediately (e.g. there's another fork active at the time), any subsequent commands
will increment `server.master_repl_offset`, but will not cause a fsync (given they were
executed before the fork started, they just ended up in the RDB part of it)
Therefore, any WAITAOF will wait on the new master_repl_offset, but it will time out
because no fsync will be executed.

Release notes:
```
WAITAOF could timeout in the absence of write traffic in case a new AOF is created and
an AOFRW can't immediately start.
This can happen by the appendonly config is changed at runtime, but also after FLUSHALL,
and replica full sync.
```

(cherry picked from commit d967ed0e08)
2023-10-18 10:44:10 +03:00
Nir RattnerandOran Agra 64aa4537ba Fix overflow calculation for next timer event (#12474)
The `retval` variable is defined as an `int`, so with 4 bytes, it cannot properly represent
microsecond values greater than the equivalent of about 35 minutes.

This bug shouldn't impact standard Redis behavior because Redis doesn't have timer
events that are scheduled as far as 35 minutes out, but it may affect custom Redis modules
which interact with the event timers via the RM_CreateTimer API.

The impact is that `usUntilEarliestTimer` may return 0 for as long as `retval` is scaled to
an overflowing value. While `usUntilEarliestTimer` continues to return `0`, `aeApiPoll`
will have a zero timeout, and so Redis will use significantly more CPU iterating through
its event loop without pause. For timers scheduled far enough into the future, Redis will
cycle between ~35 minute periods of high CPU usage and ~35 minute periods of standard
CPU usage.

(cherry picked from commit d16dd42764)
2023-10-18 10:44:10 +03:00
Chen TianjieandOran Agra 5bcce7a39d Use server.current_client to decide whether cluster commands should return TLS info. (#12569)
Starting a change in #12233 (released in 7.2), CLUSTER commands use client's
connection to decide whether to return TLS port or non-TLS port, but commands
called by Lua script and module's RM_Call don't have a real client with connection,
and would currently be regarded as non-TLS connections.

We can use server.current_client instead when it is available. When it is not (module calls
commands without a real client), we may see this as an undefined behavior, and return null
or default port (currently in this PR it returns default port, judged by server.tls_cluster).

(cherry picked from commit 115ed5e156)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 20b20d404c Fix that slot return in CLUSTER SHARDS should be integer (#12561)
An unintentional change was introduced in #10536, we used
to use addReplyLongLong and now it is addReplyBulkLonglong,
revert it back the previous behavior.

(cherry picked from commit 2a958dc37b)
2023-10-18 10:44:10 +03:00
JachinandOran Agra 139d93266d Fix compile on macOS 13 (#12611)
Use the __MAC_OS_X_VERSION_MIN_REQUIRED macro to detect the
macOS system version instead of using MAC_OS_X_VERSION_10_6.

From MacOSX14.0.sdk, the default definitions of MAC_OS_X_VERSION_xxx have
been removed in usr/include/AvailabilityMacros.h. It includes AvailabilityVersions.h,
where the following condition must be met:
`#if (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) || defined(_DARWIN_C_SOURCE)`
Only then will MAC_OS_X_VERSION_xxx be defined.
However, in the project, _DARWIN_C_SOURCE is not defined, which leads to the
loss of the definition for MAC_OS_X_VERSION_10_6.

(cherry picked from commit 0267fbb0a3)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 4d78a6061f Support NO ONE block in REPLICAOF command json (#12633)
The current commands.json doesn't mention the special NO ONE arguments.
This change is also applied to SLAVEOF

(cherry picked from commit f6d839fde2)
2023-10-18 10:44:10 +03:00
Yossi GottliebandOran Agra c2ebbd0668 Fix issue of listen before chmod on Unix sockets (CVE-2023-45145)
Before this commit, Unix socket setup performed chmod(2) on the socket
file after calling listen(2). Depending on what umask is used, this
could leave the file with the wrong permissions for a short period of
time. As a result, another process could exploit this race condition and
establish a connection that would otherwise not be possible.

We now make sure the socket permissions are set up prior to calling
listen(2).
2023-10-18 10:44:10 +03:00
Oran Agra 68d6dffe50 Redis 7.2.1 2023-09-06 20:56:15 +03:00
nihohitandOran Agra 22087fc2b4 Update command tips on more admin / configuration commands (#12545)
Updated the command tips for ACL SAVE / SETUSER / DELUSER, CLIENT SETNAME / SETINFO, and LATENCY RESET.
The tips now match CONFIG SET, since there's a similar behavior for all of these commands - the
user expects to update the various configurations & states on all nodes, not only on a single, random node.
For LATENCY RESET the response tip is now agg_sum.

Co-authored-by: Shachar Langbeheim <shachlan@amazon.com>
(cherry picked from commit 1c2c30ef03)
2023-09-06 20:56:15 +03:00
secwallandOran Agra b3d6f4bbf2 Check shard_id pointer validity in updateShardId (#12538)
When connecting between a 7.0 and 7.2 cluster, the 7.0 cluster will not populate the shard_id field, which is expect on the 7.2 cluster. This is not intended behavior, as the 7.2 cluster is supposed to use a temporary shard_id while the node is in the upgrading state, but it wasn't being correctly set in this case.

(cherry picked from commit 0a680287a6)
2023-09-06 20:56:15 +03:00
BinbinandOran Agra d5f5378d47 Update sort_ro reply_schema to mention the null reply (#12534)
Also added a test to cover this case, so this can
cover the reply schemas check.

(cherry picked from commit 2550dcf968)
2023-09-06 20:56:15 +03:00
bodong.ybdandOran Agra 93784e2ee3 Fix sort_ro get-keys function return wrong key number (#12522)
Before:
```
127.0.0.1:6379> command getkeys sort_ro key
(empty array)
127.0.0.1:6379>
```
After:
```
127.0.0.1:6379> command getkeys sort_ro key
1) "key"
127.0.0.1:6379>
```

(cherry picked from commit 7d43d7604e)
2023-09-06 20:56:15 +03:00
nihohitandOran Agra 41275dd3d3 Align CONFIG RESETSTAT/REWRITE tips with SET. (#12530)
Since the three commands have similar behavior (change config, return
OK), the tips that govern how they should behave should be similar.

Co-authored-by: Shachar Langbeheim <shachlan@amazon.com>
(cherry picked from commit 056498406c)
2023-09-06 20:56:15 +03:00
Oran AgraandGitHub 42d7d976ed Release Redis 7.2.0 GA 2023-08-15 12:38:36 +03:00
Oran Agra 606f3502c9 Redis 7.2.0 2023-08-15 10:04:17 +03:00
Oran Agra 146644355e Merge branch 'unstable' into 7.2 to prepare for 7.2.0 GA 2023-08-15 09:43:09 +03:00
Oran AgraandGitHub ff1c49d07b Release Redis 7.2 RC3 2023-07-10 14:55:20 +03:00
Oran Agra cd05870817 Redis 7.2 RC3 2023-07-10 11:52:53 +03:00
Oran Agra 1bc735bf30 Merge remote-tracking branch 'origin/unstable' into 7.2 2023-07-10 10:26:53 +03:00
Oran AgraandGitHub b1fe900842 Release Redis 7.2 RC2 2023-05-15 13:08:15 +03:00
Oran Agra df68e754a7 Redis 7.2 RC2 2023-05-15 09:53:36 +03:00
Oran Agra f4ba22d9b6 Merge remote-tracking branch 'origin/unstable' into 7.2 2023-05-15 09:50:01 +03:00
Oran Agra 45f46a1a5a Redis 7.2 RC1 2023-03-22 15:58:15 +02:00
1335 changed files with 87158 additions and 192052 deletions
-76
View File
@@ -1,76 +0,0 @@
format:
_help_line_width:
- How wide to allow formatted cmake files
line_width: 120
_help_tab_size:
- How many spaces to tab for indent
tab_size: 4
_help_use_tabchars:
- If true, lines are indented using tab characters (utf-8
- 0x09) instead of <tab_size> space characters (utf-8 0x20).
- In cases where the layout would require a fractional tab
- character, the behavior of the fractional indentation is
- governed by <fractional_tab_policy>
use_tabchars: false
_help_separate_ctrl_name_with_space:
- If true, separate flow control names from their parentheses
- with a space
separate_ctrl_name_with_space: true
_help_min_prefix_chars:
- If the statement spelling length (including space and
- parenthesis) is smaller than this amount, then force reject
- nested layouts.
min_prefix_chars: 4
_help_max_prefix_chars:
- If the statement spelling length (including space and
- parenthesis) is larger than the tab width by more than this
- amount, then force reject un-nested layouts.
max_prefix_chars: 10
_help_max_lines_hwrap:
- If a candidate layout is wrapped horizontally but it exceeds
- this many lines, then reject the layout.
max_lines_hwrap: 2
_help_line_ending:
- What style line endings to use in the output.
line_ending: unix
_help_command_case:
- Format command names consistently as 'lower' or 'upper' case
command_case: lower
_help_keyword_case:
- Format keywords consistently as 'lower' or 'upper' case
keyword_case: unchanged
_help_always_wrap:
- A list of command names which should always be wrapped
always_wrap: []
_help_enable_sort:
- If true, the argument lists which are known to be sortable
- will be sorted lexicographicall
enable_sort: true
_help_autosort:
- If true, the parsers may infer whether or not an argument
- list is sortable (without annotation).
autosort: false
_help_require_valid_layout:
- By default, if cmake-format cannot successfully fit
- everything into the desired linewidth it will apply the
- last, most aggressive attempt that it made. If this flag is
- True, however, cmake-format will print error, exit with non-
- zero status code, and write-out nothing
require_valid_layout: false
_help_layout_passes:
- A dictionary mapping layout nodes to a list of wrap
- decisions. See the documentation for more information.
layout_passes: {}
encode:
_help_emit_byteorder_mark:
- If true, emit the unicode byte-order mark (BOM) at the start
- of the file
emit_byteorder_mark: false
_help_input_encoding:
- Specify the encoding of the input file. Defaults to utf-8
input_encoding: utf-8
_help_output_encoding:
- Specify the encoding of the output file. Defaults to utf-8.
- Note that cmake only claims to support utf-8 so be careful
- when using anything else
output_encoding: utf-8
+5
View File
@@ -0,0 +1,5 @@
[codespell]
quiet-level = 2
count =
skip = ./deps,./src/crc16_slottable.h,tmp*,./.git,./lcov-html
ignore-words = ./.codespell/wordlist.txt
+1
View File
@@ -0,0 +1 @@
codespell==2.2.4
+22
View File
@@ -0,0 +1,22 @@
ake
bale
fle
fo
gameboy
mutli
nd
nees
oll
optin
ot
smove
te
tre
cancelability
ist
statics
filetest
ro
exat
clen
parth
-7
View File
@@ -1,7 +0,0 @@
formatter:
type: basic
indent: 2
retain_line_breaks_single: true
exclude:
- "deps/"
-73
View File
@@ -1,73 +0,0 @@
# See https://github.com/crate-ci/typos/blob/master/docs/reference.md to configure typos
[files]
extend-exclude = [
".git/",
"deps/",
# crc16_slottable is primarily pre-generated random strings.
"src/crc16_slottable.h",
]
ignore-hidden = false
[default.extend-words]
exat = "exat"
optin = "optin"
smove = "smove"
Parth = "Parth" # seems like the spellchecker does not like it is similar to "Path"
nd = "nd"
[default]
extend-ignore-re = [
"SELECTed",
"WATCHed",
]
[type.c]
extend-ignore-re = [
"BA3E2571", # sha1.c
"D4C4DAA4", # sha1.c
"Georg Nees",
"\\[l\\]ist", # eval.c
'"LKE"', # test_rax.c
]
[type.tcl]
extend-ignore-re = [
"DUMPed",
]
[type.c.extend-identifiers]
advices = "advices"
clen = "clen"
fle = "fle"
ot = "ot"
[type.tcl.extend-identifiers]
oll = "oll"
stressers = "stressers"
[type.sv.extend-identifiers]
# sv = .h
fo = "fo"
[type.sv.extend-words]
# sv = .h
seeked = "seeked"
[type.c.extend-words]
arange = "arange"
fo = "fo"
frst = "frst"
limite = "limite"
pathc = "pathc"
pn = "pn"
seeked = "seeked"
tre = "tre"
[type.systemd.extend-words]
# systemd = .conf
ake = "ake"
[type.tcl.extend-words]
fo = "fo"
tre = "tre"
-17
View File
@@ -1,17 +0,0 @@
# This is a file that can be used by git-blame to ignore some revisions.
# (git 2.23+, released in August 2019)
#
# Can be configured as follow:
#
# $ git config blame.ignoreRevsFile .git-blame-ignore-revs
#
# For more information you can look at git-blame(1) man page.
# Applied clang-format (#323)
c41dd77a3e93e02be3c4bc75d8c76b7b4169a4ce
# Removed terms `master` and `slave` from the source code (#591)
54c97479356ecf41b4b63733494a1be2ab919e17
# Set ColumnLimit to 0 and reformat (#1045)
af811748e7819a5ac31a6df4b21622aa58c64ae4
+2 -2
View File
@@ -1,8 +1,8 @@
---
name: Bug report
about: Help us improve by reporting a bug
about: Help us improve Redis by reporting a bug
title: '[BUG]'
labels: ['bug']
labels: ''
assignees: ''
---
-17
View File
@@ -1,17 +0,0 @@
blank_issues_enabled: true
contact_links:
- name: Questions?
url: https://github.com/kv-io/kv/discussions
about: Ask and answer questions on GitHub Discussions.
- name: Chat with us on Discord?
url: https://discord.gg/zbcPa5umUB
about: We are on Discord!
- name: Chat with us on Matrix?
url: https://matrix.to/#/#kv:matrix.org
about: We are on Matrix too!
- name: Chat with us on Slack?
url: https://join.slack.com/t/kv-oss-developer/shared_invite/zt-2nxs51chx-EB9hu9Qdch3GMfRcztTSkQ
about: We are on Slack too!
- name: Documentation issue?
url: https://github.com/kv-io/kv-doc/issues
about: Report it on the kv-doc repo.
+25
View File
@@ -0,0 +1,25 @@
---
name: Crash report
about: Submit a crash report
title: '[CRASH] <short description>'
labels: ''
assignees: ''
---
Notice!
- If a Redis module was involved, please open an issue in the module's repo instead!
- If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM!
**Crash report**
Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context.
```
```
**Additional information**
1. OS distribution and version
2. Steps to reproduce (if any)
-35
View File
@@ -1,35 +0,0 @@
name: Crash report
description: Submit a crash report
title: '[CRASH] <short description>'
labels: ['bug']
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to post your crash report!
Please notice:
- If a module was involved, please open an issue in the module's repo instead!
- If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM!
- type: textarea
id: crash-report
attributes:
label: Crash report
description: Paste the complete crash log. Please include a few lines from the log preceding the crash report to provide some context.
render: shell
validations:
required: true
- type: markdown
attributes:
value: |
Please provide the following additional information below:
- OS distribution and version
- Steps to reproduce (if any)
- type: textarea
id: additional-information
attributes:
label: Additional information
description: OS version, steps to reproduce and other useful info.
render: shell
validations:
required: false
+2 -2
View File
@@ -1,8 +1,8 @@
---
name: Feature request
about: Suggest a feature
about: Suggest a feature for Redis
title: '[NEW]'
labels: ['enhancement']
labels: ''
assignees: ''
---
+8
View File
@@ -0,0 +1,8 @@
---
name: Other
about: Can't find the right issue type? Use this one!
title: ''
labels: ''
assignees: ''
---
+21
View File
@@ -0,0 +1,21 @@
---
name: Question
about: Ask the Redis developers
title: '[QUESTION]'
labels: ''
assignees: ''
---
Please keep in mind that this issue tracker should be used for reporting bugs or proposing improvements to the Redis server.
Generally, questions about using Redis should be directed to the [community](https://redis.io/community):
* [the mailing list](https://groups.google.com/forum/#!forum/redis-db)
* [the `redis` tag at StackOverflow](http://stackoverflow.com/questions/tagged/redis)
* [/r/redis subreddit](http://www.reddit.com/r/redis)
* [github discussions](https://github.com/redis/redis/discussions)
It is also possible that your question was already asked here, so please do a quick issues search before submitting. Lastly, if your question is about one of Redis' [clients](https://redis.io/clients), you may to contact your client's developers for help.
That said, please feel free to replace all this with your question :)
-24
View File
@@ -1,24 +0,0 @@
---
name: Test failure
about: Report a failing or a flaky test
title: '[TEST-FAILURE] <short description>'
labels: ['test-failure']
assignees: ''
---
<!--Before submitting a test failure report, please check open issues to ensure this failure has not already been reported. -->
**Summary**
A short description of the failure.
**Failing test(s)**
- Test name:
- CI link(s):
**Error stack trace**
A relevant stack trace for the test failure.
> **Tip:** Copy and paste the full stack trace from the CI output into your issue, as CI links may expire over time.
@@ -1,20 +0,0 @@
[
{
"duration": 180,
"keyspacelen": [3000000],
"data_sizes": [16,96],
"pipelines": [1, 10],
"clients": [1600],
"commands": [
"SET",
"GET"
],
"cluster_mode": false,
"tls_mode": false,
"warmup": 30,
"io-threads": [1,9],
"benchmark-threads": 90,
"server_cpu_range": "0-8",
"client_cpu_range": "96-191"
}
]
@@ -1,20 +0,0 @@
[
{
"duration": 180,
"keyspacelen": [3000000],
"data_sizes": [16,96],
"pipelines": [1, 10],
"clients": [1600],
"commands": [
"SET",
"GET"
],
"cluster_mode": false,
"tls_mode": false,
"warmup": 30,
"io-threads": [1,9],
"benchmark-threads": 90,
"server_cpu_range": "0-8",
"client_cpu_range": "144-191,48-95"
}
]
-30
View File
@@ -1,30 +0,0 @@
# KV Project Instructions
You are an expert code reviewer for the KV project. Provide helpful, constructive feedback on code quality, safety, and adherence to project standards.
## 1. Review Tone & Focus
- **Tone:** Be professional, direct, constructive, and empathetic.
- **Focus:** Critique the *code*, never the *person*.
- **Constructive:** Suggest improvements, explain *why*, provide examples.
## 2. Critical Checks
- **DCO:** **Flag missing** `Signed-off-by: Name <email>` in commits. Every commit needs it.
- **Security:** If PR fixes a security vulnerability, flag it: "Security fixes should be reported privately to security@hanzo.ai, not via public PRs."
## 3. Major Decision Detection
Flag PRs that appear to be "Technical Major Decisions" requiring TSC consensus:
- Fundamental changes to core datastructures
- New data structures or APIs
- Backward compatibility breaks
- New user-visible fields requiring long-term maintenance
- New external libraries affecting runtime behavior
**Action:** Comment mentioning **@core-team** that this appears to require TSC review and ask if consensus was reached in a linked Issue.
## 4. Documentation Reminder
If PR changes user-facing behavior (new commands, changed semantics, new config):
- **Remind** author that docs at [kv-doc](https://github.com/hanzoai/kv-doc) may need updating.
- **Suggest** linking PR to related Issue with "Fixes #xyz" pattern if applicable.
## 5. Governance Changes
**ANY change to `GOVERNANCE.md`** requires special attention - comment mentioning **@core-team** for review.
+5
View File
@@ -2,9 +2,14 @@
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
- package-ecosystem: pip
directory: /.codespell
schedule:
interval: weekly
@@ -1,48 +0,0 @@
---
applyTo:
- "src/**/*.{c,h}"
- "kv.conf"
---
# KV Core Engine Review Standards
Apply these standards to core engine C code. Do NOT apply to `deps/` (vendored dependencies).
## 1. Code Style (from DEVELOPMENT_GUIDE.md)
- **Formatting:** Follow clang-format (4-space indent, no tabs, braces attached).
- **Comments:**
- C-style `/* ... */` for single or multi-line.
- C++ `//` only for single-line.
- Multi-line: align leading `*`, final `*/` on last text line.
- Document *why* code exists, not just *what*. Document all functions.
- **Line Length:** Keep below 90 characters when reasonable.
- **Types:** Use the boolean type for true/false values.
- **Static:** Use `static` for file-local functions.
## 2. Naming Conventions
- **Variables:** `snake_case` or lowercase (e.g. `cached_reply`, `keylen`).
- **Functions:** `camelCase` or `namespace_camelCase` (e.g. `createStringObject`, `IOJobQueue_isFull`).
- **Macros:** `UPPER_CASE` (e.g. `MAKE_CMD`).
- **Structures:** `camelCase` (e.g. `user`).
## 3. Safety & Correctness
- **Memory:** Strict check for buffer overflows and leaks.
- **Strings:** Validate `sds` string handling.
- **Concurrency:** Verify thread safety in threaded I/O paths.
## 4. Design Guidelines
- **Configuration:** Avoid new configs if heuristics suffice. Only add for explicit trade-offs (CPU vs memory).
- **Metrics:** No new metrics on hot paths without zero-overhead proof.
- **PR Scope:** Separate refactoring from functional changes for easier backporting.
## 5. Testing & Documentation
- **Unit Tests:** Required for data structures in `src/unit/`. Test files should follow `test_*.c` naming.
- **Integration Tests:** Required for commands in `tests/`.
- **Command Changes:** New/modified commands need corresponding updates in `src/commands/*.json`.
- **New C Files:** Remind to update `CMakeLists.txt` when adding new `.c` source files.
- **License:** New files need BSD-3-Clause header. Material changes (>100 lines) also require it.
- **Documentation:** User-facing changes need docs at [kv-doc](https://github.com/hanzoai/kv-doc).
## 6. Critical Escalation
- **Trigger:** Changes to `cluster*.c`, `replication.c`, `rdb.c`, `aof.c`.
- **Action:** Comment mentioning **@core-team** to request architectural review.
@@ -1,23 +0,0 @@
---
applyTo:
- "tests/**/*.tcl"
---
# KV Integration Test Review Standards
Apply these standards to Tcl-based integration tests (from DEVELOPMENT_GUIDE.md).
## 1. Test Organization
- **Cluster:** Use `tests/unit/cluster/` (NOT legacy `tests/cluster/` which is deprecated).
- **Coverage:** All contributions should include tests. New commands require integration tests.
- **Naming:** Use descriptive test names that explain what is being tested.
## 2. Test Quality
- **Isolation:** Tests should not depend on execution order.
- **Cleanup:** Ensure proper cleanup of resources and temporary files.
- **Assertions:** Use clear assertions with meaningful error messages.
## 3. Best Practices
- **Readability:** Keep tests simple and focused on one scenario.
- **Reliability:** Avoid timing-dependent tests; use proper synchronization.
- **Documentation:** Comment complex test scenarios to explain intent.
@@ -1,24 +0,0 @@
---
applyTo:
- "utils/**/*"
---
# KV Utilities Review Standards
Apply these standards to utility scripts and tools.
## 1. Script Quality
- **Portability:** Scripts should work across common platforms (Linux, macOS).
- **Error Handling:** Check for errors and provide clear error messages.
- **Documentation:** Include usage instructions in comments or help text.
## 2. Code Standards
- **Python:** Follow PEP 8 style guidelines.
- **Ruby:** Follow standard Ruby conventions.
- **Shell:** Use shellcheck-compatible patterns.
- **C Tools:** Follow LLVM style (4-space indent, no tabs).
## 3. Best Practices
- **Dependencies:** Minimize external dependencies.
- **Safety:** Validate inputs and avoid destructive operations without confirmation.
- **Maintainability:** Keep utilities simple and well-commented.
-14
View File
@@ -1,14 +0,0 @@
name: Auto Author Assign
on:
pull_request_target:
types: [opened, reopened]
permissions:
pull-requests: write
jobs:
assign-author:
runs-on: ubuntu-latest
steps:
- uses: toshimaru/auto-author-assign@4d585cc37690897bd9015942ed6e766aa7cdb97f # v3.0.1
-173
View File
@@ -1,173 +0,0 @@
name: On-Demand Labeled Benchmark
on:
pull_request_target:
types: [labeled]
concurrency:
group: ec2-al-2023-pr-benchmarking-arm64
cancel-in-progress: false
defaults:
run:
shell: "bash -Eeuo pipefail -x {0}"
permissions:
contents: read
pull-requests: write
issues: write
jobs:
benchmark:
if: |
github.event.action == 'labeled' && github.event.label.name == 'run-benchmark' &&
github.repository == 'kv-io/kv'
runs-on: ["self-hosted", "ec2-al-2023-pr-benchmarking-arm64"]
timeout-minutes: 7200
steps:
- name: Checkout kv
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: kv
fetch-depth: 0
ref: ${{ github.event.pull_request.merge_commit_sha }}
persist-credentials: false
- name: Checkout kv-perf-benchmark
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ${{ github.repository_owner }}/kv-perf-benchmark
path: kv-perf-benchmark
fetch-depth: 1
persist-credentials: false
- name: Checkout kv for latest benchmark.
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: kv-io/kv
ref: "unstable"
path: kv_latest
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: kishaningithub/setup-python-amazon-linux@a326cdc792983fe0fbd04c81d3d62b59b6123a6c # v1.1.0
with:
python-version: "3.10"
cache: "pip"
- name: Install dependencies
working-directory: kv-perf-benchmark
run: |
sudo dnf groupinstall "Development Tools" -y
sudo dnf install -y gcc gcc-c++ make \
python3-devel \
openssl-devel \
bzip2-devel \
libffi-devel
pip install -r requirements.txt
- name: Build latest kv_latest
working-directory: kv_latest
run: |
echo "Building latest kv-benchmark for latest benchmark executable..."
make distclean || true
make -j
if [[ -f "src/kv-benchmark" ]]; then
echo "Successfully built latest kv-benchmark"
ls -la src/kv-benchmark
./src/kv-benchmark --version || echo "Version check completed"
else
echo "Failed to build kv-benchmark"
exit 1
fi
KV_BENCHMARK_PATH="$(pwd)/src/kv-benchmark"
echo "KV_BENCHMARK_PATH=$KV_BENCHMARK_PATH" >> $GITHUB_ENV
echo "Latest kv-benchmark path: $KV_BENCHMARK_PATH"
- name: Run benchmarks
working-directory: kv-perf-benchmark
run: |
CONFIG_FILE="../kv/.github/benchmark_configs/benchmark-config-arm.json"
# Base benchmark arguments
BENCHMARK_ARGS=(
--config "$CONFIG_FILE"
--commits "${{ github.event.pull_request.merge_commit_sha }}"
--baseline "${{ github.event.pull_request.base.ref }}"
--kv-benchmark-path "$KV_BENCHMARK_PATH"
--target-ip ${{ secrets.EC2_ARM64_IP }}
--kv-path "../kv"
--results-dir "results"
--runs 5
)
# Run benchmark
python ./benchmark.py "${BENCHMARK_ARGS[@]}"
- name: Compare results
working-directory: kv-perf-benchmark
run: |
python ./utils/compare_benchmark_results.py \
--baseline ./results/${{ github.event.pull_request.base.ref }}/metrics.json \
--new ./results/${{ github.event.pull_request.merge_commit_sha }}/metrics.json \
--output ../comparison.md \
--metrics rps
- name: Upload artifacts
if: always()
continue-on-error: true
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: benchmark-results
path: |
./kv-perf-benchmark/results/${{ github.event.pull_request.merge_commit_sha }}/metrics.json
./kv-perf-benchmark/results/${{ github.event.pull_request.base.ref }}/metrics.json
comparison.md
- name: Comment PR with results
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const body = fs.readFileSync('comparison.md', 'utf8');
const {owner, repo} = context.repo;
const sha = '${{ github.event.pull_request.head.sha }}';
const short = sha.slice(0,7);
const link = `[\`${short}\`](https://github.com/${owner}/${repo}/commit/${sha})`
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner,
repo,
body: `**Benchmark ran on this commit:** ${link}\n\n${body}`
});
- name: Cleanup any running kv processes
if: always()
continue-on-error: true
run: |
rm -rf comparison.md kv*
pkill -f kv
exit_code=$?
if [ $exit_code -eq 0 ]; then
echo "Killed running kv processes"
elif [ $exit_code -eq 1 ]; then
echo "No kv processes found to kill"
else
echo "Warning: pkill failed with exit code $exit_code"
fi
- name: Remove ${{ github.event.label.name }} label
if: always() && github.event.label.name == 'run-benchmark'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.removeLabel({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
name: '${{ github.event.label.name }}'
}).catch(err => console.log('label not present', err.message));
-288
View File
@@ -1,288 +0,0 @@
name: Compare KV Versions
on:
workflow_dispatch:
inputs:
version1:
description: "First version to compare (commit SHA, branch, or tag)"
required: true
type: string
version2:
description: "Second version to compare (commit SHA, branch, or tag)"
required: true
type: string
issue_id:
description: "Issue ID to comment results on"
required: true
type: string
runs:
description: "Number of benchmark runs per configuration"
required: false
type: number
default: 1
defaults:
run:
shell: "bash -Eeuo pipefail -x {0}"
permissions:
contents: read
pull-requests: write
issues: write
jobs:
benchmark:
if: github.repository == 'kv-io/kv'
strategy:
matrix:
include:
- arch: x86
machine: ec2-al-2023-pr-benchmarking-x86
config: benchmark-config-x86.json
- arch: arm64
machine: ec2-al-2023-pr-benchmarking-arm64
config: benchmark-config-arm.json
concurrency:
group: ${{ matrix.machine }}
cancel-in-progress: false
runs-on: ["self-hosted", "${{ matrix.machine }}"]
timeout-minutes: 7200
steps:
- name: Validate inputs
run: |
echo "Version 1: ${{ github.event.inputs.version1 }}"
echo "Version 2: ${{ github.event.inputs.version2 }}"
echo "Issue ID: ${{ github.event.inputs.issue_id }}"
echo "Architecture: ${{ matrix.arch }}"
echo "Config: ${{ matrix.config }}"
echo "Runs: ${{ github.event.inputs.runs }}"
# Validate issue ID is numeric
if ! [[ "${{ github.event.inputs.issue_id }}" =~ ^[0-9]+$ ]]; then
echo "Error: Issue ID must be a number"
exit 1
fi
# Validate runs is a positive number
if ! [[ "${{ github.event.inputs.runs }}" =~ ^[1-9][0-9]*$ ]]; then
echo "Error: Runs must be a positive number"
exit 1
fi
- name: Checkout kv for latest benchmark.
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: kv-io/kv
ref: "unstable"
path: kv_latest
fetch-depth: 0
persist-credentials: false
- name: Checkout kv
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: kv
- name: Checkout kv-perf-benchmark
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ${{ github.repository_owner }}/kv-perf-benchmark
path: kv-perf-benchmark
fetch-depth: 1
persist-credentials: false
- name: Set up Python
uses: kishaningithub/setup-python-amazon-linux@a326cdc792983fe0fbd04c81d3d62b59b6123a6c # v1.1.0
with:
python-version: "3.10"
cache: "pip"
- name: Install dependencies
working-directory: kv-perf-benchmark
run: |
sudo dnf groupinstall "Development Tools" -y
sudo dnf install -y gcc gcc-c++ make \
python3-devel \
openssl-devel \
bzip2-devel \
libffi-devel
pip install -r requirements.txt
- name: Build latest kv_latest
working-directory: kv_latest
run: |
echo "Building latest kv-benchmark for latest benchmark executable..."
# Clean any previous builds
make distclean || true
# Build kv-benchmark with latest code
make -j$(nproc)
# Verify the binary was created
if [[ -f "src/kv-benchmark" ]]; then
echo "✓ Successfully built latest kv-benchmark"
ls -la src/kv-benchmark
# Test the binary
echo "Testing kv-benchmark binary..."
./src/kv-benchmark --version || echo "Version check completed"
else
echo "Failed to build kv-benchmark"
exit 1
fi
# Store the absolute path for later use
KV_BENCHMARK_PATH="$(pwd)/src/kv-benchmark"
echo "KV_BENCHMARK_PATH=$KV_BENCHMARK_PATH" >> $GITHUB_ENV
echo "Latest kv-benchmark path: $KV_BENCHMARK_PATH"
- name: Run benchmarks
working-directory: kv-perf-benchmark
env:
EC2_X86_IP: ${{ secrets.EC2_X86_IP }}
EC2_ARM64_IP: ${{ secrets.EC2_ARM64_IP }}
run: |
# Set the target IP based on the matrix architecture
if [[ "${{ matrix.arch }}" == "x86" ]]; then
TARGET_IP=$EC2_X86_IP
echo "Using x86 machine IP"
elif [[ "${{ matrix.arch }}" == "arm64" ]]; then
TARGET_IP=$EC2_ARM64_IP
echo "Using ARM64 machine IP"
else
echo "Error: Unknown architecture: ${{ matrix.arch }}"
exit 1
fi
CONFIG_FILE="../kv/.github/benchmark_configs/${{ matrix.config }}"
# Verify our custom kv-benchmark exists
if [[ ! -f "$KV_BENCHMARK_PATH" ]]; then
echo "Custom kv-benchmark not found at: $KV_BENCHMARK_PATH"
exit 1
fi
echo "Using custom kv-benchmark from: $KV_BENCHMARK_PATH"
# Base benchmark arguments with custom kv-benchmark path
BENCHMARK_ARGS=(
--config "$CONFIG_FILE"
--commits ${{ github.event.inputs.version1 }} ${{ github.event.inputs.version2 }}
--kv-benchmark-path "$KV_BENCHMARK_PATH"
--target-ip $TARGET_IP
--results-dir "results"
--runs ${{ github.event.inputs.runs }}
)
echo "Running benchmarks with the following setup:"
echo "- Version 1: ${{ github.event.inputs.version1 }}"
echo "- Version 2: ${{ github.event.inputs.version2 }}"
echo "- Architecture: ${{ matrix.arch }}"
echo "- Config: ${{ matrix.config }}"
echo "- Using latest kv-benchmark: $KV_BENCHMARK_PATH"
echo "- This ensures both versions use the same (latest) benchmark tool for consistent results"
# Run benchmark with custom kv-benchmark executable
python ./benchmark.py "${BENCHMARK_ARGS[@]}"
- name: Compare results
working-directory: kv-perf-benchmark
run: |
# Find the actual result directories (they might have different names than input)
VERSION1_DIR=$(find ./results -maxdepth 1 -type d -name "*${{ github.event.inputs.version1 }}*" | head -1)
VERSION2_DIR=$(find ./results -maxdepth 1 -type d -name "*${{ github.event.inputs.version2 }}*" | head -1)
if [[ -z "$VERSION1_DIR" ]]; then
echo "Could not find results for version1: ${{ github.event.inputs.version1 }}"
echo "Available result directories:"
ls -la ./results/
exit 1
fi
if [[ -z "$VERSION2_DIR" ]]; then
echo "Could not find results for version2: ${{ github.event.inputs.version2 }}"
echo "Available result directories:"
ls -la ./results/
exit 1
fi
echo "Comparing results:"
echo "Version 1 (${{ github.event.inputs.version1 }}): $VERSION1_DIR"
echo "Version 2 (${{ github.event.inputs.version2 }}): $VERSION2_DIR"
# Generate RPS-focused comparison and graphs for GitHub comments
python utils/compare_benchmark_results.py \
--baseline "$VERSION1_DIR/metrics.json" \
--new "$VERSION2_DIR/metrics.json" \
--output ../comparison.md \
--metrics rps
- name: Upload artifacts
if: always()
continue-on-error: true
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: benchmark-results-${{ matrix.arch }}-${{ github.event.inputs.issue_id }}
path: |
./kv-perf-benchmark/results/*
comparison.md
- name: Cleanup any running kv processes and files
if: always()
continue-on-error: true
run: |
pkill -f kv || echo "No kv processes found to kill"
rm -rf *
combine-results:
needs: benchmark
runs-on: ubuntu-latest
steps:
- name: Download all artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
path: artifacts
- name: Combine results and create comprehensive report
run: |
echo "# Multi-Architecture Benchmark Comparison: ${{ github.event.inputs.version1 }} vs ${{ github.event.inputs.version2 }}" > combined_report.md
echo "" >> combined_report.md
echo "**Versions Compared:**" >> combined_report.md
echo "- Version 1: \`${{ github.event.inputs.version1 }}\`" >> combined_report.md
echo "- Version 2: \`${{ github.event.inputs.version2 }}\`" >> combined_report.md
echo "" >> combined_report.md
echo "**Runs:** ${{ github.event.inputs.runs }} per configuration" >> combined_report.md
echo "**Workflow Run:** [View Details](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})" >> combined_report.md
echo "" >> combined_report.md
echo "---" >> combined_report.md
echo "" >> combined_report.md
# Process each architecture's results
for arch in x86 arm64; do
artifact_dir="artifacts/benchmark-results-${arch}-${{ github.event.inputs.issue_id }}"
if [[ -d "$artifact_dir" && -f "$artifact_dir/comparison.md" ]]; then
echo "## ${arch^^} Architecture Results" >> combined_report.md
echo "" >> combined_report.md
cat "$artifact_dir/comparison.md" >> combined_report.md
echo "" >> combined_report.md
echo "---" >> combined_report.md
echo "" >> combined_report.md
fi
done
- name: Comment on issue with combined results
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
// Read the combined report
const body = fs.readFileSync('combined_report.md', 'utf8');
await github.rest.issues.createComment({
issue_number: ${{ github.event.inputs.issue_id }},
owner: context.repo.owner,
repo: context.repo.repo,
body: body
});
+46 -285
View File
@@ -1,323 +1,84 @@
name: CI
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
concurrency:
group: ci-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
on: [push, pull_request]
jobs:
test-ubuntu-latest:
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make -j4 all-with-unit-tests SERVER_CFLAGS='-Werror' BUILD_TLS=yes USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes
- name: test
run: |
sudo apt-get install tcl8.6 tclx
./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs
- name: validate commands.def up to date
run: |
touch src/commands/ping.json
make commands.def
dirty="$(git diff)"
if [[ ! -z "$dirty" ]]; then echo "$dirty"; exit 1; fi
- name: unit tests
run: |
./src/kv-unit-tests
test-ubuntu-latest-compatibility:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
server:
- {version: "7.2.11", file: "kv-7.2.11-noble-x86_64.tar.gz"}
- {version: "8.0.6", file: "kv-8.0.6-noble-x86_64.tar.gz"}
- {version: "8.1.4", file: "kv-8.1.4-noble-x86_64.tar.gz"}
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make -j4 all-with-unit-tests SERVER_CFLAGS='-Werror' BUILD_TLS=yes USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes
- name: Install old server (${{ matrix.server.version }}) for compatibility testing
run: |
mkdir -p tests/tmp
cd tests/tmp
wget https://download.kv.io/releases/${{ matrix.server.file }}
tar -xvf ${{ matrix.server.file }}
- name: Run compatibility tests against ${{ matrix.server.version }}
run: |
sudo apt-get install -y tcl8.6 tclx
./runtest --verbose --tags "-slow needs:other-server" --dump-logs \
--other-server-path tests/tmp/kv-${{ matrix.server.version }}-noble-x86_64/bin/kv-server
- name: Module API tests against ${{ matrix.server.version }}
run: |
CFLAGS='-Werror' ./runtest-moduleapi --tags needs:other-server --verbose --dump-logs \
--other-server-path tests/tmp/kv-${{ matrix.server.version }}-noble-x86_64/bin/kv-server
test-ubuntu-latest-cmake-tls:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: cmake and make
run: |
sudo apt-get install -y cmake libssl-dev
mkdir -p build-release
cd build-release
cmake -DCMAKE_BUILD_TYPE=Release .. -DBUILD_TLS=yes -DBUILD_UNIT_TESTS=yes
make -j$(nproc)
- name: test
run: |
sudo apt-get install -y tcl8.6 tclx tcl-tls
./utils/gen-test-certs.sh
./build-release/runtest --verbose --tags -slow --dump-logs --tls
- name: unit tests
run: |
./build-release/bin/kv-unit-tests
- uses: actions/checkout@v3
- name: make
# Fail build if there are warnings
# build with TLS just for compilation coverage
run: make SERVER_CFLAGS='-Werror' BUILD_TLS=yes
- name: test
run: |
sudo apt-get install tcl8.6 tclx
./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: ./runtest-moduleapi --verbose --dump-logs
- name: validate commands.def up to date
run: |
touch src/commands/ping.json
make commands.def
dirty=$(git diff)
if [[ ! -z $dirty ]]; then echo $dirty; exit 1; fi
test-sanitizer-address:
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@v3
- name: make
# build with TLS module just for compilation coverage
run: make -j4 all-with-unit-tests SANITIZER=address SERVER_CFLAGS='-Werror' BUILD_TLS=module USE_LIBBACKTRACE=yes
run: make SANITIZER=address SERVER_CFLAGS='-Werror' BUILD_TLS=module
- name: testprep
run: sudo apt-get install tcl8.6 tclx -y
- name: test
run: ./runtest --verbose --tags -slow --dump-logs
- name: module api test
run: CFLAGS='-Werror' ./runtest-moduleapi --verbose --dump-logs
- name: unit tests
run: ./src/kv-unit-tests
test-rdma:
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: prepare-development-libraries
run: sudo apt-get install librdmacm-dev libibverbs-dev
- name: make-rdma-module
run: make -j4 BUILD_RDMA=module USE_LIBBACKTRACE=yes
- name: make-rdma-builtin
run: |
make distclean
make -j4 BUILD_RDMA=yes USE_LIBBACKTRACE=yes
- name: clone-rxe-kmod
run: |
mkdir -p tests/rdma/rxe
git clone https://github.com/pizhenwei/rxe.git tests/rdma/rxe
make -C tests/rdma/rxe
- name: clear-kernel-log
run: sudo dmesg -c > /dev/null
- name: test
run: sudo ./runtest-rdma --install-rxe
- name: show-kernel-log
run: sudo dmesg -c
test-tls-only:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: generate-test-certificates
run: ./utils/gen-test-certs.sh
- name: install-test-dependencies
run: sudo apt-get install -y tcl8.6 tclx tcl-tls
- name: make-tls-module
run: make -j4 BUILD_TLS=module SERVER_CFLAGS='-Werror'
- name: test-tls-module
run: ./runtest --verbose --single unit/tls --dump-logs --tls-module
- name: make-tls-builtin
run: |
make distclean
make -j4 BUILD_TLS=yes SERVER_CFLAGS='-Werror'
- name: test-tls-builtin
run: ./runtest --verbose --single unit/tls --dump-logs --tls
run: ./runtest-moduleapi --verbose --dump-logs
build-debian-old:
runs-on: ubuntu-latest
container: debian:bullseye
container: debian:buster
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- name: Build libbacktrace
run: |
apt-get update && apt-get install -y build-essential
cd libbacktrace && ./configure && make && make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
run: make -j4 SERVER_CFLAGS='-Werror' USE_LIBBACKTRACE=yes
- uses: actions/checkout@v3
- name: make
run: |
apt-get update && apt-get install -y build-essential
make SERVER_CFLAGS='-Werror'
build-macos-latest:
runs-on: macos-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
# Build with additional upcoming features
run: make -j3 all-with-unit-tests SERVER_CFLAGS='-Werror' USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes
- uses: actions/checkout@v3
- name: make
run: make SERVER_CFLAGS='-Werror'
build-32bit:
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- name: Build libbacktrace (32-bit)
run: |
sudo apt-get update
sudo apt-get install libc6-dev-i386 libstdc++-11-dev-i386-cross gcc-multilib g++-multilib
cd libbacktrace && ./configure CFLAGS="-m32" --prefix=/usr/local/libbacktrace32 && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
# Fast float requires C++ 32-bit libraries to compile on 64-bit ubuntu
# machine i.e. "-cross" suffixed version. Cross-compiling c++ to 32-bit
# also requires multilib support for g++ compiler i.e. "-multilib"
# suffixed version of g++. g++-multilib generally includes libstdc++.
# *cross version as well, but it is also added explicitly just in case.
run: make -j4 SERVER_CFLAGS='-Werror' 32bit USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes LIBBACKTRACE_PREFIX=/usr/local/libbacktrace32
- name: unit tests
run: |
./src/kv-unit-tests
- uses: actions/checkout@v3
- name: make
run: |
sudo apt-get update && sudo apt-get install libc6-dev-i386
make SERVER_CFLAGS='-Werror' 32bit
build-libc-malloc:
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
run: make -j4 SERVER_CFLAGS='-Werror' MALLOC=libc USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes
- uses: actions/checkout@v3
- name: make
run: make SERVER_CFLAGS='-Werror' MALLOC=libc
build-almalinux8-jemalloc:
build-centos7-jemalloc:
runs-on: ubuntu-latest
container: almalinux:8
container: centos:7
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- name: Build libbacktrace
run: |
dnf -y install epel-release gcc gcc-c++ make procps-ng which
cd libbacktrace && ./configure && make && make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: make
run: make -j4 SERVER_CFLAGS='-Werror' USE_FAST_FLOAT=yes USE_LIBBACKTRACE=yes
- uses: actions/checkout@v3
- name: make
run: |
yum -y install gcc make
make SERVER_CFLAGS='-Werror'
format-yaml:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Go
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: "1.22.4"
- name: Setup YAML formatter
run: |
go install github.com/google/yamlfmt/cmd/yamlfmt@latest
- name: Run yamlfmt
id: yamlfmt
run: |
yamlfmt -lint -conf .config/format.yml .
# Capture the diff output
DIFF=$(git diff)
if [ ! -z "$DIFF" ]; then
# Encode the diff in Base64 to ensure it's handled as a single line
ENCODED_DIFF=$(echo "$DIFF" | base64 -w 0)
echo "diff=$ENCODED_DIFF" >> $GITHUB_OUTPUT
fi
shell: bash
- name: Check for formatting changes
if: ${{ steps.yamlfmt.outputs.diff }}
run: |
echo "ERROR: YAML file is not formatted properly. Here is the diff: "
# Decode the Base64 diff to display it
echo "${{ steps.clang-format.outputs.diff }}" | base64 --decode
exit 1
shell: bash
-59
View File
@@ -1,59 +0,0 @@
name: Clang Format Check
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
concurrency:
group: clang-${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
clang-format-check:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Clang
run: |
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install software-properties-common -y
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/llvm-toolchain.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/llvm-toolchain.gpg] http://apt.llvm.org/$(lsb_release -cs)/ llvm-toolchain-$(lsb_release -cs)-18 main" | sudo tee /etc/apt/sources.list.d/llvm.list
sudo apt-get update -y
sudo apt-get install clang-format-18 -y
- name: Run clang-format
id: clang-format
run: |
# Run clang-format and capture the diff
cd src
shopt -s globstar
clang-format-18 -i **/*.c **/*.h
# Capture the diff output
DIFF=$(git diff)
if [ ! -z "$DIFF" ]; then
# Encode the diff in Base64 to ensure it's handled as a single line
ENCODED_DIFF=$(echo "$DIFF" | base64 -w 0)
echo "diff=$ENCODED_DIFF" >> $GITHUB_OUTPUT
fi
shell: bash
- name: Check for formatting changes
if: ${{ steps.clang-format.outputs.diff }}
run: |
echo "ERROR: Code is not formatted correctly. Here is the diff:"
# Decode the Base64 diff to display it
echo "${{ steps.clang-format.outputs.diff }}" | base64 --decode
exit 1
shell: bash
-44
View File
@@ -1,44 +0,0 @@
name: "Codecov"
# Enabling on each push is to display the coverage changes in every PR,
# where each PR needs to be compared against the coverage of the head commit
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
concurrency:
group: codecov-${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
code-coverage:
runs-on: ubuntu-22.04
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install lcov and run test
run: |
sudo apt-get install lcov tclx
make lcov USE_LIBBACKTRACE=yes
- name: Upload code coverage
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./src/kv.info
+13 -31
View File
@@ -1,51 +1,33 @@
name: "CodeQL"
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
schedule:
# run weekly new vulnerability was added to the database
- cron: '0 3 * * 0'
concurrency:
group: codeql-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
- cron: '0 0 * * 0'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'kv-io/kv'
permissions:
security-events: write
if: github.event_name != 'schedule' || github.repository == 'redis/redis'
strategy:
fail-fast: false
matrix:
language: ['cpp']
language: [ 'cpp' ]
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Checkout repository
uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@b5ebac6f4c00c8ccddb7cdcd45fdb248329f808a # v3.32.2
with:
languages: ${{ matrix.language }}
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
- name: Autobuild
uses: github/codeql-action/autobuild@b5ebac6f4c00c8ccddb7cdcd45fdb248329f808a # v3.32.2
- name: Autobuild
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@b5ebac6f4c00c8ccddb7cdcd45fdb248329f808a # v3.32.2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
-47
View File
@@ -1,47 +0,0 @@
# Creates and uploads a Coverity build on a schedule
name: Coverity Scan
on:
schedule:
# Run once daily, since below 500k LOC can have 21 builds per week, per https://scan.coverity.com/faq#frequency
- cron: '0 1 * * *'
# Support manual execution
workflow_dispatch:
concurrency:
group: coverity-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
coverity:
if: github.repository == 'kv-io/kv'
runs-on: ubuntu-latest
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download and extract the Coverity Build Tool
run: |
wget -q https://scan.coverity.com/download/cxx/linux64 --post-data "token=${{ secrets.COVERITY_SCAN_TOKEN }}&project=kv-io%2Fkv" -O cov-analysis-linux64.tar.gz
mkdir cov-analysis-linux64
tar xzf cov-analysis-linux64.tar.gz --strip 1 -C cov-analysis-linux64
- name: Install KV dependencies
run: sudo apt install -y gcc procps libssl-dev
- name: Build with cov-build
run: cov-analysis-linux64/bin/cov-build --dir cov-int make USE_LIBBACKTRACE=yes
- name: Upload the result
run: |
tar czvf cov-int.tgz cov-int
curl \
--form email=${{ secrets.COVERITY_SCAN_EMAIL }} \
--form token=${{ secrets.COVERITY_SCAN_TOKEN }} \
--form file=@cov-int.tgz \
https://scan.coverity.com/builds?project=kv-io%2Fkv
+896 -1650
View File
File diff suppressed because it is too large Load Diff
-58
View File
@@ -1,58 +0,0 @@
name: Build and Deploy Hanzo KV
on:
push:
branches: [main]
tags: ['v*']
paths:
- 'src/**'
- 'deps/**'
- 'Dockerfile'
- '.github/workflows/deploy.yml'
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Image metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/hanzoai/kv
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=9,enable={{is_default_branch}}
type=sha,prefix=
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=kv
cache-to: type=gha,mode=max,scope=kv
+47 -87
View File
@@ -1,116 +1,76 @@
name: External Server Tests
on:
push:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
pull_request:
paths-ignore:
- '**/*.md'
- '**/00-RELEASENOTES'
- '**/COPYING'
push:
schedule:
- cron: '0 2 * * *'
concurrency:
group: external-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
- cron: '0 0 * * *'
jobs:
test-external-standalone:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'kv-io/kv'
timeout-minutes: 1440
if: github.event_name != 'schedule' || github.repository == 'valkey-io/valkey'
timeout-minutes: 14400
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Build
run: make SERVER_CFLAGS=-Werror USE_LIBBACKTRACE=yes
- name: Start kv-server
run: |
./src/kv-server --daemonize yes --save "" --logfile external-server.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Run external test
run: |
- uses: actions/checkout@v4
- name: Build
run: make SERVER_CFLAGS=-Werror
- name: Start valkey-server
run: |
./src/valkey-server --daemonize yes --save "" --logfile external-server.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Run external test
run: |
./runtest \
--host 127.0.0.1 --port 6379 \
--verbose \
--tags -slow
- name: Archive server log
if: ${{ failure() }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: test-external-standalone-log
path: external-server.log
- name: Archive server log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-external-server-log
path: external-server.log
test-external-cluster:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'kv-io/kv'
timeout-minutes: 1440
if: github.event_name != 'schedule' || github.repository == 'valkey-io/valkey'
timeout-minutes: 14400
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Build
run: make SERVER_CFLAGS=-Werror USE_LIBBACKTRACE=yes
- name: Start kv-server
run: |
./src/kv-server --cluster-enabled yes --cluster-databases 16 --daemonize yes \
--save "" --logfile external-server.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Create a single node cluster
run: ./src/kv-cli cluster addslots $(for slot in {0..16383}; do echo $slot; done); sleep 5
- name: Run external test
run: |
- uses: actions/checkout@v4
- name: Build
run: make SERVER_CFLAGS=-Werror
- name: Start valkey-server
run: |
./src/valkey-server --cluster-enabled yes --daemonize yes --save "" --logfile external-server.log \
--enable-protected-configs yes --enable-debug-command yes --enable-module-command yes
- name: Create a single node cluster
run: ./src/valkey-cli cluster addslots $(for slot in {0..16383}; do echo $slot; done); sleep 5
- name: Run external test
run: |
./runtest \
--host 127.0.0.1 --port 6379 \
--verbose \
--cluster-mode \
--tags -slow
- name: Archive server log
if: ${{ failure() }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: test-external-cluster-log
path: external-server.log
- name: Archive server log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-external-cluster-log
path: external-server.log
test-external-nodebug:
runs-on: ubuntu-latest
if: github.event_name != 'schedule' || github.repository == 'kv-io/kv'
timeout-minutes: 1440
if: github.event_name != 'schedule' || github.repository == 'valkey-io/valkey'
timeout-minutes: 14400
steps:
- name: Install libbacktrace
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: ianlancetaylor/libbacktrace
ref: b9e40069c0b47a722286b94eb5231f7f05c08713
path: libbacktrace
- run: cd libbacktrace && ./configure && make && sudo make install
- name: Checkout KV
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@v4
- name: Build
run: make SERVER_CFLAGS=-Werror USE_LIBBACKTRACE=yes
- name: Start kv-server
run: make SERVER_CFLAGS=-Werror
- name: Start valkey-server
run: |
./src/kv-server --daemonize yes --save "" --logfile external-server.log
./src/valkey-server --daemonize yes --save "" --logfile external-server.log
- name: Run external test
run: |
./runtest \
@@ -119,7 +79,7 @@ jobs:
--tags "-slow -needs:debug"
- name: Archive server log
if: ${{ failure() }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
uses: actions/upload-artifact@v3
with:
name: test-external-nodebug-log
path: external-server.log
name: test-external-server-log
path: external-server.log
+2 -9
View File
@@ -8,20 +8,13 @@ on:
paths:
- 'src/commands/*.json'
concurrency:
group: reply-schemas-linter-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
reply-schemas-linter:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@v3
- name: Setup nodejs
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
uses: actions/setup-node@v3
- name: Install packages
run: npm install ajv
- name: linter
+10 -12
View File
@@ -9,13 +9,6 @@ on:
push:
pull_request:
concurrency:
group: spellcheck-${{ github.head_ref || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Spellcheck
@@ -23,12 +16,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@v3
- name: Install typos
uses: taiki-e/install-action@d4422f254e595ee762a758628fe4f16ce050fa2e # v2.67.28
- name: pip cache
uses: actions/cache@v3
with:
tool: typos
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: ${{ runner.os }}-pip-
- name: Install prerequisites
run: sudo pip install -r ./.codespell/requirements.txt
- name: Spell check
run: typos --config=./.config/typos.toml
run: codespell --config=./.codespell/.codespellrc
@@ -1,50 +0,0 @@
name: Trigger Build Release
on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: Version of KV to build
required: true
environment:
description: Environment to build
required: true
type: choice
options:
- dev
- prod
jobs:
trigger:
runs-on: ubuntu-latest
steps:
- name: Determine version and environment
id: determine-vars
run: |
if [[ "${{ github.event_name }}" == "release" ]]; then
echo "Triggered by a release event."
VERSION=${{ github.event.release.tag_name }}
ENVIRONMENT="prod"
else
echo "Triggered manually (workflow_dispatch)."
VERSION=${{ inputs.version }}
ENVIRONMENT=${{ inputs.environment }}
fi
# Set the outputs for version and environment
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "environment=$ENVIRONMENT" >> $GITHUB_OUTPUT
- name: Trigger build
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.AUTOMATION_PAT }}
repository: ${{ github.repository_owner }}/kv-release-automation
event-type: build-release
client-payload: >
{
"version": "${{ steps.determine-vars.outputs.version }}",
"environment": "${{ steps.determine-vars.outputs.environment }}"
}
-68
View File
@@ -1,68 +0,0 @@
name: Weekly Test Workflow for Released Branches
on:
schedule:
- cron: '0 6 * * 0'
workflow_dispatch: {}
permissions:
actions: read
contents: read
pull-requests: read
concurrency:
group: weekly-release-tests
cancel-in-progress: false
jobs:
determine-release-branches:
if: github.repository == 'kv-io/kv'
runs-on: ubuntu-latest
outputs:
branches: ${{ steps.release-branches.outputs.branches }}
has-branches: ${{ steps.release-branches.outputs.has-branches }}
steps:
- name: Collect release branches
id: release-branches
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { owner, repo } = context.repo;
const MIN_MAJOR = 7;
const MIN_MINOR = 2;
const branches = await github.paginate(github.rest.repos.listBranches, {
owner,
repo,
per_page: 100,
});
const releaseBranches = branches
.map(({ name }) => name)
.filter(name => /^\d+\.\d+$/.test(name))
.filter(name => {
const [major, minor] = name.split('.').map(Number);
return Number.isInteger(major) &&
Number.isInteger(minor) &&
(major > MIN_MAJOR || (major === MIN_MAJOR && minor >= MIN_MINOR));
})
.sort((a, b) => a.localeCompare(b, undefined, { numeric: true }));
core.info(`Weekly release branches: ${releaseBranches.join(', ') || '(none)'}`);
core.setOutput('branches', JSON.stringify(releaseBranches));
core.setOutput('has-branches', releaseBranches.length > 0 ? 'true' : 'false');
run-daily-for-release-branches:
needs: determine-release-branches
if: needs.determine-release-branches.outputs.has-branches == 'true' && github.repository == 'kv-io/kv'
permissions:
actions: read
contents: read
pull-requests: read
strategy:
fail-fast: false
matrix:
release-branch: ${{ fromJson(needs.determine-release-branches.outputs.branches) }}
max-parallel: 1
uses: ./.github/workflows/daily.yml
with:
use_repo: kv-io/kv
use_git_ref: ${{ matrix.release-branch }}
skipjobs: ""
skiptests: ""
secrets: inherit
+2 -18
View File
@@ -1,13 +1,10 @@
.*.swp
*.o
*.a
*.xo
*.so
*.d
*.lo
*.la
*.log
dump*.rdb
dump.rdb
*-benchmark
*-check-aof
*-check-rdb
@@ -15,13 +12,12 @@ dump*.rdb
*-cli
*-sentinel
*-server
*-unit-tests
doc-tools
release
misc/*
src/release.h
appendonly.aof*
appendonlydir*
appendonlydir
SHORT_TERM_TODO
release.h
src/transfer.sh
@@ -44,15 +40,3 @@ Makefile.dep
.ccls-cache/*
compile_commands.json
redis.code-workspace
.cache
.cscope*
.swp
nodes*.conf
tests/cluster/tmp/*
tests/rdma/rdma-test
tags
build/
build-debug/
build-release/
cmake-build-debug/
cmake-build-release/
+102 -11
View File
@@ -1,16 +1,107 @@
Hello! This file is just a placeholder, since this is the "unstable" branch
of KV, the place where all the development happens.
Valkey 7.2 release notes
========================
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
--------------------------------------------------------------------------------
Upgrade urgency levels:
Usually "unstable" is stable enough for you to use it in development environments
however you should never use it in production environments. It is possible
to download the latest stable release here:
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
https://kv.io/download/
================================================================================
Valkey 7.2.5 GA - Released Tue 16 Apr 2024
================================================================================
More information is available at https://kv.io
Upgrade urgency LOW: This is the first stable release of Valkey 7.2.5.
Happy hacking!
Changes to support Valkey branding
=========
* Update template config files to remove references of redis.io and replace them
with valkey.io. (#320)
================================================================================
Valkey 7.2.5 RC1 - Released Fri 12 Apr 2024
================================================================================
Upgrade urgency LOW: Second release candidate for Valkey with API compatibility
for OSS Redis 7.2.4. Moving to a release candidate on 7.2.5, to make it clearer
this is a patch iteration as opposed to an exact copy of OSS Redis. Also includes
fixes to minor bugs present in 7.2.4 RC1 and more compatibility changes.
Changes to support Valkey branding
=========
* Update README to remove Redis references.
* Update valkey-server and valkey-cli help info to show only Valkey (#222)
* Add compatibility for lua debugger to use 'server' instead of redis. (#303)
Bug fixes
=========
* Fix module event name to maintain Redis compatibility. (#289)
* Fix issue where Redis symlinks were created with the wrong name. (#282)
================================================================================
Valkey 7.2.4 RC1 - Released Tue 09 Apr 2024
================================================================================
Upgrade urgency LOW: Initial release of Valkey with API compatibility for OSS
Redis 7.2.4. This release is based off of OSS Redis 7.2.4, with additional
functionality to better brand the server as Valkey instead of Redis. All APIs
are fully backwards compatible and care was taken to minimize the number of
log lines that were changed to best support existing tooling around log parsing.
Changes to support Valkey branding
=========
* Rename the 6 Redis binaries to ones with `valkey` prefixes: `valkey-server`,
`valkey-cli`, `valkey-benchmark`, `valkey-check-aof`, `valkey-check-rdb` and
`valkey-sentinel`. (#62)
* During install, create symlinks mapping the corresponding Redis binary names
to the new Valkey binaries. (#193)
* INFO fields: Introduce `valkey_version` and `server_name` fields. The
`redis_version` will continue to be present and will indicate the Redis OSS
version that the server is compatible with. (#47, #232)
* RDB file format: Introduce a `valkey-ver` RDB field to indicate an RDB file
was produced by a Valkey server. This field is ignored when the file is
loaded by Redis OSS. (#47)
* Module API changes: Introduce Valkey Module API in a new file, valkeymodule.h,
with functions and types prefixed by ValkeyModule. Valkey is still fully ABI
compatible with the Redis Module API, meaning Valkey supports running modules
compiled with either the Redis or Valkey Module APIs. For full source
compatibility, the Redis Module API (redismodule.h) is also kept with the
RedisModule prefixed functions and types. (#194, #243, #262)
* Scripting: Introduce a new top level `server` object for functions and scripts
that can be instead of the existing `redis` object. This allows users to
replace calls like `redis.call(...)` with `server.call(...)`. (#213)
* Makefile: Introduce `SERVER_CFLAGS` and `SERVER_LDFLAGS` as an alternative to
`REDIS_CFLAGS` and `REDIS_LDFLAGS`. (#46)
* Update template config files to replace references to Redis with Valkey. The
file valkey.conf replaces redis.conf. (#29, #171)
* Logging: Update startup, shutdown, and help logs to reference Valkey instead of
Redis. Valkey server will also show the Valkey logo at startup and show the
Valkey repositories when crashing. (#251, #252, #263, #113)
Bug fixes
=========
* Cluster: Fix an issue where cluster nodes running on Redis versions earlier
than 7.0 would be unable to communicate with nodes running on Valkey 7.2. This
change also introduces a slight delay when hostnames are not displayed when a
node is first added to the cluster. (#52)
A special thank you for the amount of work put into this release by:
- Viktor Söderqvist
- Madelyn Olson
- Wen Hui
- Ping Xie
- Parth Patel
- Roshan Katri
- Zhu Binbin
- Zhao Zhao
- Bany
- Harkrishn Patro
- Vitah Lin
- Ziyang Zeng (For creating the logo!)
- The many other community members who provided their support!
-1
View File
@@ -1 +0,0 @@
LLM.md
+1
View File
@@ -0,0 +1 @@
Please check https://github.com/redis/redis/issues
-1
View File
@@ -1 +0,0 @@
LLM.md
-78
View File
@@ -1,78 +0,0 @@
cmake_minimum_required(VERSION 3.10)
# Must be done first
if (APPLE)
# Force clang compiler on macOS
find_program(CLANGPP "clang++")
find_program(CLANG "clang")
if (CLANG AND CLANGPP)
message(STATUS "Found ${CLANGPP}, ${CLANG}")
set(CMAKE_CXX_COMPILER ${CLANGPP})
set(CMAKE_C_COMPILER ${CLANG})
endif ()
endif ()
# Options
option(BUILD_LUA "Build KV Lua scripting engine" ON)
option(BUILD_UNIT_TESTS "Build kv-unit-tests" OFF)
option(BUILD_TEST_MODULES "Build all test modules" OFF)
option(BUILD_EXAMPLE_MODULES "Build example modules" OFF)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")
project("kv")
add_compile_options(-Wundef)
set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_EXTENSIONS ON)
include(KVSetup)
add_subdirectory(src)
add_subdirectory(tests)
# Include the packaging module
include(Packaging)
# Clear cached variables from the cache
unset(BUILD_LUA CACHE)
unset(BUILD_TESTS CACHE)
unset(CLANGPP CACHE)
unset(CLANG CACHE)
unset(BUILD_RDMA_MODULE CACHE)
unset(BUILD_TLS_MODULE CACHE)
unset(BUILD_UNIT_TESTS CACHE)
unset(BUILD_TEST_MODULES CACHE)
unset(BUILD_EXAMPLE_MODULES CACHE)
unset(USE_TLS CACHE)
unset(DEBUG_FORCE_DEFRAG CACHE)
# Helper to copy runtest scripts to allow running tests with CMake built binaries
function(copy_runtest_script script_name)
set(src "${CMAKE_SOURCE_DIR}/${script_name}")
set(dst "${CMAKE_BINARY_DIR}/${script_name}")
file(READ "${src}" contents)
# Split at the first newline (after shebang #!/bin/sh)
string(FIND "${contents}" "\n" index_of_first_newline_char)
math(EXPR first_index_script_body "${index_of_first_newline_char} + 1")
string(SUBSTRING "${contents}" 0 ${first_index_script_body} shebang_line)
string(SUBSTRING "${contents}" ${first_index_script_body} -1 script_body)
# Insert our environment variable lines
set(insert_content
"# Most tests assume running from the project root
cd ${CMAKE_SOURCE_DIR}
export KV_BIN_DIR=\"${CMAKE_BINARY_DIR}/bin\"
")
# Reconstruct the full script
set(new_contents "${shebang_line}${insert_content}${script_body}")
file(WRITE "${dst}" "${new_contents}")
file(CHMOD "${dst}" PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE)
endfunction()
copy_runtest_script("runtest")
copy_runtest_script("runtest-cluster")
copy_runtest_script("runtest-moduleapi")
copy_runtest_script("runtest-rdma")
copy_runtest_script("runtest-sentinel")
+3 -3
View File
@@ -26,7 +26,7 @@ Examples of unacceptable behavior include:
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
* Publishing others private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
@@ -49,7 +49,7 @@ representative at an online or offline event.
Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
this email address: maintainers@hanzo.ai.
this email address: redis@redis.io.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
@@ -89,7 +89,7 @@ Attribution
This Code of Conduct is adapted from the Contributor Covenant,
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by Mozilla's code of conduct
Community Impact Guidelines were inspired by Mozillas code of conduct
enforcement ladder.
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
+35 -110
View File
@@ -1,131 +1,56 @@
Contributing to KV
======================
Note: by contributing code to the Redis project in any form, including sending
a pull request via Github, a code fragment or patch via private email or
public discussion groups, you agree to release your code under the terms
of the BSD license that you can find in the COPYING file included in the Redis
source distribution. You will include BSD license in the COPYING file within
each source file that you contribute.
Welcome and thank you for wanting to contribute!
# IMPORTANT: HOW TO USE REDIS GITHUB ISSUES
# Project governance
Github issues SHOULD ONLY BE USED to report bugs, and for DETAILED feature
requests. Everything else belongs to the Redis Google Group:
https://groups.google.com/forum/m/#!forum/Redis-db
The KV project is led by a Technical Steering Committee, whose responsibilities are laid out in [GOVERNANCE.md](GOVERNANCE.md).
PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
bugs in the Github issues system. We'll be very happy to help you and provide
all the support in the mailing list.
## Get started
There is also an active community of Redis users at Stack Overflow:
* Have a question? Ask it on
[GitHub Discussions](https://github.com/hanzoai/kv/discussions)
or [KV's Discord](https://discord.gg/zbcPa5umUB)
or [KV's Matrix](https://matrix.to/#/#kv:matrix.org)
* Found a bug? [Report it here](https://github.com/hanzoai/kv/issues/new?template=bug_report.md&title=%5BBUG%5D)
* KV crashed? [Submit a crash report here](https://github.com/hanzoai/kv/issues/new?template=crash_report.md&title=%5BCRASH%5D+%3Cshort+description%3E)
* Suggest a new feature? [Post your detailed feature request here](https://github.com/hanzoai/kv/issues/new?template=feature_request.md&title=%5BNEW%5D)
* Report a test failure? [Report it here](https://github.com/hanzoai/kv/issues/new?template=test-failure.md)
* Want to help with documentation? [Move on to kv-doc](https://github.com/hanzoai/kv-doc)
* Report a vulnerability? See [SECURITY.md](SECURITY.md)
https://stackoverflow.com/questions/tagged/redis
## Developer Certificate of Origin
Issues and pull requests for documentation belong on the redis-doc repo:
We respect the intellectual property rights of others and we want to make sure
all incoming contributions are correctly attributed and licensed. A Developer
Certificate of Origin (DCO) is a lightweight mechanism to do that. The DCO is
a declaration attached to every commit. In the commit message of the contribution,
the developer simply adds a `Signed-off-by` statement and thereby agrees to the DCO,
which you can find below or at [DeveloperCertificate.org](http://developercertificate.org/).
https://github.com/redis/redis-doc
```text
Developer's Certificate of Origin 1.1
If you are reporting a security bug or vulnerability, see SECURITY.md.
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the
best of my knowledge, is covered under an appropriate open
source license and I have the right under that license to
submit that work with modifications, whether created in whole
or in part by me, under the same open source license (unless
I am permitted to submit under a different license), as
Indicated in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including
all personal information I submit with it, including my
sign-off) is maintained indefinitely and may be redistributed
consistent with this project or the open source license(s)
involved.
```
We require that every contribution to KV to be signed with a DCO. We require the
usage of known identity (such as a real or preferred name). We do not accept anonymous
contributors nor those utilizing pseudonyms. A DCO signed commit will contain a line like:
```text
Signed-off-by: Jane Smith <jane.smith@email.com>
```
You may type this line on your own when writing your commit messages. However, if your
user.name and user.email are set in your git configs, you can use `git commit` with `-s`
or `--signoff` to add the `Signed-off-by` line to the end of the commit message. We also
require revert commits to include a DCO.
If you're contributing code to the KV project in any other form, including
sending a code fragment or patch via private email or public discussion groups,
you need to ensure that the contribution is in accordance with the DCO.
## How to provide a patch or a new feature
# How to provide a patch for a new feature
1. If it is a major feature or a semantical change, please don't start coding
straight away: if your feature is not a conceptual fit you'll lose a lot of
time writing the code without any reason. Start by creating an issue at Github with the
description of, exactly, what you want to accomplish and why. Use cases are important for
features to be accepted. Here you can see if there is consensus about your idea.
time writing the code without any reason. Start by posting in the mailing list
and creating an issue at Github with the description of, exactly, what you want
to accomplish and why. Use cases are important for features to be accepted.
Here you'll see if there is consensus about your idea.
2. If in step 1 you get an acknowledgment from the project leaders, use the following
procedure to submit a patch:
1. Fork KV on GitHub ([HOWTO](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo))
1. Create a topic branch (`git checkout -b my_branch`)
1. Make the needed changes and commit with a DCO. (`git commit -s`)
1. Push to your branch (`git push origin my_branch`)
1. Initiate a pull request on GitHub ([HOWTO](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request))
1. Done :)
2. If in step 1 you get an acknowledgment from the project leaders, use the
following procedure to submit a patch:
a. Fork Redis on github ( https://docs.github.com/en/github/getting-started-with-github/fork-a-repo )
b. Create a topic branch (git checkout -b my_branch)
c. Push to your branch (git push origin my_branch)
d. Initiate a pull request on github ( https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request )
e. Done :)
3. Keep in mind that we are very overloaded, so issues and PRs sometimes wait
for a *very* long time. However this is not a lack of interest, as the project
for a *very* long time. However this is not lack of interest, as the project
gets more and more users, we find ourselves in a constant need to prioritize
certain issues/PRs over others. If you think your issue/PR is very important
try to popularize it, have other users commenting and sharing their point of
view, and so forth. This helps.
view and so forth. This helps.
4. While developing code, make sure to refer to our [DEVELOPMENT_GUIDE.md](DEVELOPMENT_GUIDE.md),
which includes documentation about various best practices for writing KV code.
5. For minor fixes, open a pull request on GitHub.
To link a pull request to an existing issue, please write "Fixes #xyz" somewhere
in the pull request description, where xyz is the issue number.
## Running the daily workflow on demand for your branch
Use [`.github/workflows/daily.yml`](.github/workflows/daily.yml) with
`workflow_dispatch` to run daily tests manually on any branch in your fork.
1. Open your fork on GitHub and go to **Actions** -> **Daily**.
2. Click **Run workflow**.
3. In the **Branch** dropdown, select the branch that contains the workflow file you want to use.
4. In the input fields, set:
* `use_repo` to your fork (for example, `your-user/kv`)
* `use_git_ref` to your branch name (or a specific commit SHA)
5. Optionally set `skipjobs`, `skiptests`, `test_args`, and `cluster_test_args`.
6. Click **Run workflow**.
Notes:
* To run the full matrix, set `skipjobs` and `skiptests` to `none`.
Do not leave them empty, since the workflow input defaults may be applied.
* The scheduled part of this workflow is gated to `hanzoai/kv`, but manual
`workflow_dispatch` runs work for forks.
4. For minor fixes just open a pull request on Github.
Thanks!
+1 -22
View File
@@ -1,25 +1,4 @@
# License 1
SPDX-License-Identifier: BSD-3-Clause
BSD 3-Clause License
Copyright (c) 2024-present, KV contributors
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# License 2
BSD 3-Clause License
Copyright (c) 2006-2020, Redis Ltd.
Copyright (c) 2006-2020, Salvatore Sanfilippo
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
-75
View File
@@ -1,75 +0,0 @@
# KV development guidelines
This document provides a general overview for writing and designing code for KV.
During our long development history, we've made a lot of inconsistent decisions, but we strive to get incrementally better.
## General Best practices
1. Try to limit the number of lines changed in a PR when possible.
We do a lot of backporting as a project, and the more lines changed, the higher the chance of having to resolve merge conflicts.
Please separate refactoring and functional changes into separate PRs, to make it easier to handle backporting.
1. Avoid adding configuration when a feature can be fully controlled by heuristics.
We want KV to work correctly out of the box without much tuning.
Configurations can be added to provide additional tuning of features.
When the workload characteristics can't be inferred or imply a tradeoff (CPU vs memory), then provide a configuration.
## General style guidelines
Most of the style guidelines are enforced by clang format, but some additional comments are included here.
1. C style comments `/* comment */` can be used for both single and multi-line comments.
C++ comments `//` can only be used for single line comments.
Multi line comments should have the leading `*` align and the final `*/` should be on the same line as the last line of text.
e.g.
```c
/* Blah Blah
* Blah Blah. */
```
2. Comments should generally be used to describe behavior that is not obvious from reading the code itself.
This includes complex behavior, why code was written the way it was and describing non-obvious behavior.
Additionally, functions should be documented to explain all of the function's behavior without having to read the code.
1. Generally keep line lengths below 90 characters when reasonable, however there is no explicit line length enforcement.
Use your best judgement for readability.
1. Use static functions when a function is only intended to be accessed from the same file.
For historical reasons, some private functions are prefixed by `_`, and they are kept as is to make it easier to backport changes.
1. Use the boolean type for true/false values.
For historical reasons, some functions used the integer type, and they are kept as is to make it easier to backport changes.
## Naming conventions
KV has a long history of inconsistent naming conventions.
Generally follow the style of the surrounding code, but you can also always use the following conventions for variable and structure names:
- Variable names: `snake_case` or all lower case for short names (e.g. `cached_reply` or `keylen`).
- Function names: `camelCase` or `namespace_camelCase` (e.g. `createStringObject` or `IOJobQueue_isFull`).
- Macros: `UPPER_CASE` (e.g. `MAKE_CMD`).
- Structures: `camelCase` (e.g. `user`).
## Licensing information
When creating new source code files, use the following snippet to indicate the license:
```
/*
* Copyright (c) KV Contributors
* All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
```
If you are making material changes to a file that has a different license at the top, also add the above license snippet.
There isn't a well defined test for what is considered a material change, but a good rule of thumb is that material changes are more than 100 lines of code.
## Test coverage
KV uses two types of tests: unit and integration tests.
All contributions should include a test of some form.
Unit tests are present in the `src/unit` directory, and are intended to test individual structures or files.
For example, most changes to data structures should include corresponding unit tests.
Integration tests are located in the `tests/` directory, and are intended to test end-to-end functionality.
Adding new commands should come with corresponding integration tests.
When writing cluster mode tests, do not use the legacy `tests/cluster` framework, which has been deprecated, and instead write tests in `unit/cluster`.
## Documentation
KV keeps most of the user documentation in the [kv-doc](https://github.com/hanzoai/kv-doc) repository in a few areas:
1. Major functionality is documented in the [topics](https://github.com/hanzoai/kv-doc/tree/main/topics) section.
1. Specific command behavior is documented in the [commands](https://github.com/hanzoai/kv-doc/tree/main/commands) section.
Command history is also documented in the [command json file](https://github.com/hanzoai/kv/tree/unstable/src/commands).
1. Server info fields are documented in the [INFO](https://github.com/hanzoai/kv-doc/blob/main/commands/info.md) command.
When a PR is opened that requires documentation to be updated, the `needs-doc-pr` should be added until the corresponding documentation PR is open.
-31
View File
@@ -1,31 +0,0 @@
ARG KV_VERSION=9
# Hanzo KV: High-performance key-value store
FROM kv/kv:${KV_VERSION}-alpine AS base
FROM base
LABEL maintainer="dev@hanzo.ai"
LABEL org.opencontainers.image.source="https://github.com/hanzoai/kv"
LABEL org.opencontainers.image.description="Hanzo KV - High-performance key-value store"
LABEL org.opencontainers.image.vendor="Hanzo AI"
# Install Hanzo KV CLI tools
# Primary names are kv-* ; legacy kv-* names remain as symlinks
RUN cp /usr/local/bin/kv-server /usr/local/bin/kv-server \
&& cp /usr/local/bin/kv-cli /usr/local/bin/kv-cli \
&& ln -sf /usr/local/bin/kv-cli /usr/local/bin/kv \
&& cp /usr/local/bin/kv-sentinel /usr/local/bin/kv-sentinel 2>/dev/null; \
cp /usr/local/bin/kv-benchmark /usr/local/bin/kv-benchmark 2>/dev/null; \
cp /usr/local/bin/kv-check-aof /usr/local/bin/kv-check-aof 2>/dev/null; \
cp /usr/local/bin/kv-check-rdb /usr/local/bin/kv-check-rdb 2>/dev/null; \
true
EXPOSE 6379
HEALTHCHECK --interval=15s --timeout=3s --start-period=10s --retries=3 \
CMD kv ping | grep -q PONG || exit 1
ENTRYPOINT ["kv-server"]
CMD ["--bind", "0.0.0.0", "--dir", "/data", "--maxmemory-policy", "allkeys-lru", "--protected-mode", "no"]
-93
View File
@@ -1,93 +0,0 @@
# Project Governance
The KV project is managed by a Technical Steering Committee (TSC) composed of the maintainers of the KV repository.
The KV project includes all of the current and future repositories under the hanzoai organization.
Committers are defined as individuals with write access to the code within a repository.
Maintainers are defined as individuals with full access to a repository and own its governance.
Both maintainers and committers shall be clearly listed in the MAINTAINERS.md file in a given project's repository.
Maintainers of other repositories within the KV project are not members of the TSC unless explicitly added.
## Technical Steering Committee
The TSC is responsible for oversight of all technical, project, approval, and policy matters for KV.
The TSC members are listed in the [MAINTAINERS.md](MAINTAINERS.md) file in the KV repository.
At any time, no more than one third (1/3) of the TSC members may be employees, contractors, or representatives of the same organization or affiliated organizations.
For the purposes of this document, “organization” includes companies, corporations, universities, research institutes, non-profits, governmental institutions, and any of their subsidiaries or affiliates.
If, at any time, the 1/3 organization limit is exceeded (for example, due to changes in employment, company acquisitions, or organizational affiliations), the TSC shall be notified as soon as possible.
The TSC must promptly take action to restore compliance, which may include removing or reassigning members in accordance with the procedures outlined in the [Termination of Membership](#termination-of-membership) section.
The TSC shall strive to resolve the situation within 30 days of notification, and document the steps taken to restore compliance.
The TSC shall appoint a Chair responsible for organizing TSC meetings.
If the TSC Chair is removed from the TSC (or the Chair steps down from that role), it is the responsibility of the TSC to appoint a new Chair.
The TSC may, at its discretion, add or remove members who are not maintainers of the main KV repository.
The TSC may, at its discretion, add or remove maintainers from other repositories within the KV project.
## Voting
The TSC shall strive for all decisions to be made by consensus.
While explicit agreement of the entire TSC is preferred, it is not required for consensus.
Rather, the TSC shall determine consensus based on their good faith consideration of a number of factors, including the dominant view of the TSC and nature of support and objections.
The TSC shall document evidence of consensus in accordance with these requirements.
If consensus cannot be reached, the TSC shall make the decision by a vote.
A vote shall also be called when an issue or pull request is marked as a major decision, which are decisions that have a significant impact on the KV architecture or design.
### Technical Major Decisions
Technical major decisions include:
* Fundamental changes to the KV core datastructures
* Adding a new data structure or API
* Changes that affect backward compatibility
* New user visible fields that need to be maintained
* Adding or removing a new external library such as a client or module to the project when it affects runtime behavior
Technical major decisions shall be approved by a simple majority vote whenever one can be obtained.
If a simple majority cannot be reached within a two-week voting period, and no TSC member has voted against, the decision may instead be approved through explicit “+2” support from at least two TSC members, recorded on the relevant issue or pull request.
If the pull request author or issue proposer is a TSC member, their +1 counts toward the +2.
If any TSC member casts a negative vote, the decision must follow the simple majority voting process and cannot be approved through +2.
Once a technical major decision has been approved through the +2 mechanism, any subsequent concerns shall be raised through a new major decision process; +2 approvals are not retracted directly.
### Governance Major Decisions
Governance major decisions include:
* Adding TSC members or involuntary removal of TSC members
* Modifying this governance document
* Delegation of maintainership for projects or governance authority
* Creating, modifying, or removing roles within the KV project
* Any change that alters voting rules, TSC responsibilities, or project oversight
* Structural changes to the TSC, including composition limits
Governance major decisions shall require approval by a super-majority vote of at least two thirds (2/3) of the entire TSC.
Any member of the TSC can call a vote with reasonable notice to the TSC, setting out a discussion period and a separate voting period.
Any discussion may be conducted in person or electronically by text, voice, or video.
The discussion shall be open to the public, with the notable exception of discussions involving embargoed security issues or the addition or removal of maintainers, which will be private.
In any vote, each voting TSC member will have one vote.
The TSC shall give at least two weeks for all members to submit their vote.
Except as specifically noted elsewhere in this document, decisions by vote require a simple majority vote of all voting members.
If a vote results in a tie, the status quo is preserved.
It is the responsibility of the TSC Chair to help facilitate the voting process as needed to make sure it completes within the voting period.
## Termination of Membership
A maintainer's access (and accordingly, their position on the TSC) will be removed if any of the following occur:
* Involuntary Removal: Removal via the [Governance Major Decision](#governance-major-decisions) voting process.
* Resignation: Written notice of resignation to the TSC.
* Unreachable Member: If a member is unresponsive for more than six months, the remaining active members of the TSC may vote to remove the unreachable member by a simple majority.
## Technical direction for other KV projects
The TSC may delegate decision making for other projects within the KV organization to the maintainers responsible for those projects.
Delegation of decision making for a project is considered a [Governance Major Decision](#governance-major-decisions).
Projects within the KV organization must indicate the individuals with commit permissions by updating the MAINTAINERS.md within their repositories.
The TSC may, at its discretion, overrule the decisions made by other projects within the KV organization, although they shall show restraint in doing so.
## License of this document
This document may be used, modified, and/or distributed under the terms of the
[Creative Commons Attribution 4.0 International (CC-BY) license](https://creativecommons.org/licenses/by/4.0/legalcode).
+1
View File
@@ -0,0 +1 @@
See README
-7
View File
@@ -1,7 +0,0 @@
# kv — AI Assistant Context
<p align="center">
<strong>Hanzo KV</strong>
</p>
<p align="center">
-38
View File
@@ -1,38 +0,0 @@
## Overview
This document contains a list of maintainers in this repo.
See [GOVERNANCE.md](GOVERNANCE.md) that explains the function of this file.
## Committee Chair
- **Madelyn Olson**
Term: March 28, 2024 Present
## Current Maintainers
Maintainers listed in alphabetical order by their github ID.
| Maintainer | GitHub ID | Affiliation |
| ------------------- | ------------- | ----------- |
| Binbin Zhu | @enjoy-binbin | Tencent |
| Harkrishn Patro | @hpatro | Amazon |
| Lucas Yang | @lucasyonge | - |
| Madelyn Olson | @madolson | Amazon |
| Jacob Murphy | @murphyjacob4 | Google |
| Ping Xie | @pingxie | Oracle |
| Ran Shidlansik | @ranshid | Amazon |
| Zhao Zhao | @soloestoy | Alibaba |
| Viktor Söderqvist | @zuiderkwast | Ericsson |
## Current Committers
Committers listed in alphabetical order by their github ID.
| Committer | GitHub ID | Affiliation |
| ------------------- | ------------- | ----------- |
| Ricardo Dias | @rjd15372 | Percona |
## Former Maintainers and Committers
| Name | GitHub ID | Role | Term | Affiliation |
| ------------------- | ------------- | ---- | ---- | ----------- |
+106
View File
@@ -0,0 +1,106 @@
[Note: this is the Redis manifesto, for general information about
installing and running Redis read the README file instead.]
Redis Manifesto
===============
1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language)
that manipulates abstract data types and implemented as a TCP daemon.
Commands manipulate a key space where keys are binary-safe strings and
values are different kinds of abstract data types. Every data type
represents an abstract version of a fundamental data structure. For instance
Redis Lists are an abstract representation of linked lists. In Redis, the
essence of a data type isn't just the kind of operations that the data types
support, but also the space and time complexity of the data type and the
operations performed upon it.
2 - Memory storage is #1. The Redis data set, composed of defined key-value
pairs, is primarily stored in the computer's memory. The amount of memory in
all kinds of computers, including entry-level servers, is increasing
significantly each year. Memory is fast, and allows Redis to have very
predictable performance. Datasets composed of 10k or 40 millions keys will
perform similarly. Complex data types like Redis Sorted Sets are easy to
implement and manipulate in memory with good performance, making Redis very
simple. Redis will continue to explore alternative options (where data can
be optionally stored on disk, say) but the main goal of the project remains
the development of an in-memory database.
3 - Fundamental data structures for a fundamental API. The Redis API is a direct
consequence of fundamental data structures. APIs can often be arbitrary but
not an API that resembles the nature of fundamental data structures. If we
ever meet intelligent life forms from another part of the universe, they'll
likely know, understand and recognize the same basic data structures we have
in our computer science books. Redis will avoid intermediate layers in API,
so that the complexity is obvious and more complex operations can be
performed as the sum of the basic operations.
4 - We believe in code efficiency. Computers get faster and faster, yet we
believe that abusing computing capabilities is not wise: the amount of
operations you can do for a given amount of energy remains anyway a
significant parameter: it allows to do more with less computers and, at
the same time, having a smaller environmental impact. Similarly Redis is
able to "scale down" to smaller devices. It is perfectly usable in a
Raspberry Pi and other small ARM based computers. Faster code having
just the layers of abstractions that are really needed will also result,
often, in more predictable performances. We think likewise about memory
usage, one of the fundamental goals of the Redis project is to
incrementally build more and more memory efficient data structures, so that
problems that were not approachable in RAM in the past will be perfectly
fine to handle in the future.
5 - Code is like a poem; it's not just something we write to reach some
practical result. Sometimes people that are far from the Redis philosophy
suggest using other code written by other authors (frequently in other
languages) in order to implement something Redis currently lacks. But to us
this is like if Shakespeare decided to end Enrico IV using the Paradiso from
the Divina Commedia. Is using any external code a bad idea? Not at all. Like
in "One Thousand and One Nights" smaller self contained stories are embedded
in a bigger story, we'll be happy to use beautiful self contained libraries
when needed. At the same time, when writing the Redis story we're trying to
write smaller stories that will fit in to other code.
6 - We're against complexity. We believe designing systems is a fight against
complexity. We'll accept to fight the complexity when it's worthwhile but
we'll try hard to recognize when a small feature is not worth 1000s of lines
of code. Most of the time the best way to fight complexity is by not
creating it at all. Complexity is also a form of lock-in: code that is
very hard to understand cannot be modified by users in an independent way
regardless of the license. One of the main Redis goals is to remain
understandable, enough for a single programmer to have a clear idea of how
it works in detail just reading the source code for a couple of weeks.
7 - Threading is not a silver bullet. Instead of making Redis threaded we
believe on the idea of an efficient (mostly) single threaded Redis core.
Multiple of such cores, that may run in the same computer or may run
in multiple computers, are abstracted away as a single big system by
higher order protocols and features: Redis Cluster and the upcoming
Redis Proxy are our main goals. A shared nothing approach is not just
much simpler (see the previous point in this document), is also optimal
in NUMA systems. In the specific case of Redis it allows for each instance
to have a more limited amount of data, making the Redis persist-by-fork
approach more sounding. In the future we may explore parallelism only for
I/O, which is the low hanging fruit: minimal complexity could provide an
improved single process experience.
8 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
naturally into a distributed version of Redis and 2) a more complex API that
supports multi-key operations. Both are useful if used judiciously but
there's no way to make the more complex multi-keys API distributed in an
opaque way without violating our other principles. We don't want to provide
the illusion of something that will work magically when actually it can't in
all cases. Instead we'll provide commands to quickly migrate keys from one
instance to another to perform multi-key operations and expose the
trade-offs to the user.
9 - We optimize for joy. We believe writing code is a lot of hard work, and the
only way it can be worth is by enjoying it. When there is no longer joy in
writing code, the best thing to do is stop. To prevent this, we'll avoid
taking paths that will make Redis less of a joy to develop.
10 - All the above points are put together in what we call opportunistic
programming: trying to get the most for the user with minimal increases
in complexity (hanging fruits). Solve 95% of the problem with 5% of the
code when it is acceptable. Avoid a fixed schedule but follow the flow of
user requests, inspiration, Redis internal readiness for certain features
(sometimes many past changes reach a critical point making a previously
complex feature very easy to obtain).
+1 -1
View File
@@ -1,4 +1,4 @@
# Top level makefile, the real magic is at src/Makefile
# Top level makefile, the real shit is at src/Makefile
default: all
+174 -104
View File
@@ -1,145 +1,215 @@
<p align="center">
<strong>Hanzo KV</strong>
</p>
This README is under construction as we work to build a new community driven high performance key-value store.
<p align="center">
High-performance key-value store for the Hanzo ecosystem.<br/>
In-memory data store used as database, cache, streaming engine, and message broker.
</p>
This project was forked from the open source Redis project right before the transition to their new source available licenses.
<p align="center">
<a href="https://github.com/hanzoai/kv/actions"><img src="https://github.com/hanzoai/kv/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
<a href="https://github.com/hanzoai/kv/releases"><img src="https://img.shields.io/github/v/release/hanzoai/kv" alt="Release"></a>
<a href="https://github.com/hanzoai/kv/blob/main/LICENSE"><img src="https://img.shields.io/github/license/hanzoai/kv" alt="License"></a>
</p>
This README is just a fast *quick start* document. We are currently working on a more permanent documentation page.
---
What is Valkey?
--------------
Valkey is a high-performance data structure server that primarily serves key/value workloads.
It supports a wide range of native structures and an extensible plugin system for adding new data structures and access patterns.
## Features
Building Valkey
--------------
- **In-memory key-value store** -- sub-millisecond reads and writes
- **Redis-compatible protocol** -- drop-in replacement for existing Redis clients
- **Persistence** -- RDB snapshots and AOF (append-only file) for durability
- **Replication** -- primary-replica with automatic failover via Sentinel
- **Lua scripting** -- server-side scripting for atomic operations
- **Pub/Sub** -- publish and subscribe messaging
- **Streams** -- append-only log data structure for event sourcing
- **Cluster mode** -- horizontal scaling with automatic sharding
- **Modules** -- extensible plugin system for custom data structures
- **ZAP native** -- built-in [ZAP binary protocol](https://github.com/luxfi/zap) on port 9653 (17x faster than JSON-RPC)
- **Multi-arch** -- linux/amd64 and linux/arm64
Valkey can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
We support big endian and little endian architectures, and both 32 bit
and 64 bit systems.
## Quick Start
It may compile on Solaris derived systems (for instance SmartOS) but our
support for this platform is *best effort* and Valkey is not guaranteed to
work as well as in Linux, OSX, and \*BSD.
### Docker
It is as simple as:
```bash
docker run -d --name hanzo-kv -p 6379:6379 ghcr.io/hanzoai/kv
```
% make
### Connect
To build with TLS support, you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu) and run:
```bash
docker exec -it hanzo-kv kv
% make BUILD_TLS=yes
127.0.0.1:6379> SET hello world
OK
127.0.0.1:6379> GET hello
"world"
```
To build with systemd support, you'll need systemd development libraries (such
as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS) and run:
Any Redis-compatible CLI also works out of the box.
% make USE_SYSTEMD=yes
### Build from Source
To append a suffix to Valkey program names, use:
```bash
make
make test
make install
```
% make PROG_SUFFIX="-alt"
## CLI Tools
You can build a 32 bit Valkey binary using:
| Command | Description |
|---------|-------------|
| `kv` | Interactive CLI (default) |
| `kv-server` | Start KV server |
| `kv-cli` | Command-line client |
| `kv-sentinel` | High-availability sentinel |
| `kv-benchmark` | Performance benchmarking |
| `kv-check-aof` | AOF file integrity check |
| `kv-check-rdb` | RDB file integrity check |
% make 32bit
## Configuration
After building Valkey, it is a good idea to test it using:
Pass a config file at startup:
% make test
```bash
kv-server /etc/kv/kv.conf
```
If TLS is built, running the tests with TLS enabled (you will need `tcl-tls`
installed):
Or set options via command line:
% ./utils/gen-test-certs.sh
% ./runtest --tls
```bash
kv-server --port 6379 --maxmemory 256mb --appendonly yes
```
## ZAP Binary Protocol
Fixing build problems with dependencies or cached build options
---------
Hanzo KV speaks [ZAP](https://github.com/luxfi/zap) natively on port **9653** — no sidecar needed.
Valkey has some dependencies which are included in the `deps` directory.
`make` does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
ZAP is a zero-copy binary protocol that's 17x faster than JSON-RPC with 11x less memory usage.
When you update the source code with `git pull` or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
### Enable ZAP
% make distclean
ZAP is enabled by default. Load the module:
This will clean: jemalloc, lua, hiredis, linenoise and other dependencies.
```bash
kv-server --loadmodule /path/to/zap.so
# or with custom port:
kv-server --loadmodule /path/to/zap.so PORT 9653
```
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
those options are cached indefinitely until you issue a `make distclean`
command.
### ZAP Operations
Fixing problems building 32 bit binaries
---------
| Path | Body | Description |
|------|------|-------------|
| `/get` | `{"key":"mykey"}` | GET a key |
| `/set` | `{"key":"mykey","value":"myval"}` | SET a key |
| `/del` | `{"key":"mykey"}` | DEL a key |
| `/cmd` | `{"cmd":"PING","args":[]}` | Execute any command |
If after building Valkey with a 32 bit target you need to rebuild it
with a 64 bit target, or the other way around, you need to perform a
`make distclean` in the root directory of the Valkey distribution.
### Module API
In case of build errors when trying to build a 32 bit binary of Valkey, try
the following steps:
Develop custom modules using the KV Module API:
* Install the package libc6-dev-i386 (also try g++-multilib).
* Try using the following command line instead of `make 32bit`:
`make CFLAGS="-m32 -march=native" LDFLAGS="-m32"`
```c
#include "kvmodule.h"
Allocator
---------
int KVModule_OnLoad(KVModuleCtx *ctx, KVModuleString **argv, int argc) {
if (KVModule_Init(ctx, "mymod", 1, KVMODULE_APIVER_1) == KVMODULE_ERR)
return KVMODULE_ERR;
// register commands...
return KVMODULE_OK;
}
```
Selecting a non-default memory allocator when building Valkey is done by setting
the `MALLOC` environment variable. Valkey is compiled and linked against libc
malloc by default, with the exception of jemalloc being the default on Linux
systems. This default was picked because jemalloc has proven to have fewer
fragmentation problems than libc malloc.
## Client SDKs
To force compiling against libc malloc, use:
| Language | Package | Install |
|----------|---------|---------|
| Python | [hanzo-kv](https://pypi.org/project/hanzo-kv) | `pip install hanzo-kv` |
| Go | [hanzo/kv-go](https://github.com/hanzoai/kv-go) | `go get github.com/hanzoai/kv-go` |
| Node.js | [@hanzo/kv](https://github.com/hanzoai/kv-client) | `npm install @hanzo/kv` |
% make MALLOC=libc
Any Redis-compatible client library will also work.
To compile against jemalloc on Mac OS X systems, use:
## Documentation
% make MALLOC=jemalloc
Full documentation is available at [docs.hanzo.ai](https://docs.hanzo.ai).
Monotonic clock
---------------
## License
By default, Valkey will build using the POSIX clock_gettime function as the
monotonic clock source. On most modern systems, the internal processor clock
can be used to improve performance. Cautions can be found here:
http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/
BSD-3-Clause
To build with support for the processor's internal instruction clock, use:
Copyright (c) 2024-2026 Hanzo AI Inc. All rights reserved.
% make CFLAGS="-DUSE_PROCESSOR_CLOCK"
Verbose build
-------------
Valkey will build with a user-friendly colorized output by default.
If you want to see a more verbose output, use the following:
% make V=1
Running Valkey
-------------
To run Valkey with the default configuration, just type:
% cd src
% ./valkey-server
If you want to provide your valkey.conf, you have to run it using an additional
parameter (the path of the configuration file):
% cd src
% ./valkey-server /path/to/valkey.conf
It is possible to alter the Valkey configuration by passing parameters directly
as options using the command line. Examples:
% ./valkey-server --port 9999 --replicaof 127.0.0.1 6379
% ./valkey-server /etc/valkey/6379.conf --loglevel debug
All the options in valkey.conf are also supported as options using the command
line, with exactly the same name.
Running Valkey with TLS:
------------------
Please consult the [TLS.md](TLS.md) file for more information on
how to use Valkey with TLS.
Playing with Valkey
------------------
You can use valkey-cli to play with Valkey. Start a valkey-server instance,
then in another terminal try the following:
% cd src
% ./valkey-cli
valkey> ping
PONG
valkey> set foo bar
OK
valkey> get foo
"bar"
valkey> incr mycounter
(integer) 1
valkey> incr mycounter
(integer) 2
valkey>
Installing Valkey
-----------------
In order to install Valkey binaries into /usr/local/bin, just use:
% make install
You can use `make PREFIX=/some/other/directory install` if you wish to use a
different destination.
_Note_: For compatibility with Redis, we create symlinks from the Redis names (`redis-server`, `redis-cli`, etc.) to the Valkey binaries installed by `make install`.
The symlinks are created in same directory as the Valkey binaries.
The symlinks are removed when using `make uninstall`.
The creation of the symlinks can be skipped by setting the makefile variable `USE_REDIS_SYMLINKS=no`.
`make install` will just install binaries in your system, but will not configure
init scripts and configuration files in the appropriate place. This is not
needed if you just want to play a bit with Valkey, but if you are installing
it the proper way for a production system, we have a script that does this
for Ubuntu and Debian systems:
% cd utils
% ./install_server.sh
_Note_: `install_server.sh` will not work on Mac OSX; it is built for Linux only.
The script will ask you a few questions and will setup everything you need
to run Valkey properly as a background daemon that will start again on
system reboots.
You'll be able to stop and start Valkey using the script named
`/etc/init.d/valkey_<portnumber>`, for instance `/etc/init.d/valkey_6379`.
Code contributions
-----------------
Please see the [CONTRIBUTING.md][2]. For security bugs and vulnerabilities, please see [SECURITY.md][3].
[1]: https://github.com/valkey-io/valkey/blob/unstable/COPYING
[2]: https://github.com/valkey-io/valkey/blob/unstable/CONTRIBUTING.md
[3]: https://github.com/valkey-io/valkey/blob/unstable/SECURITY.md
+3 -3
View File
@@ -1,6 +1,6 @@
## Reporting a Vulnerability
If you believe you've discovered a security vulnerability, please contact the KV team at security@hanzo.ai.
If you believe you've discovered a security vulnerability, please contact the Valkey team at security@lists.valkey.io.
Please *DO NOT* create an issue.
We follow a responsible disclosure procedure, so depending on the severity of the issue we may notify KV vendors about the issue before releasing it publicly.
If you would like to be added to our list of vendors, please reach out to the KV team at maintainers@hanzo.ai.
We follow a responsible disclore procedure, so depending on the severity of the issue we may notify Valkey vendors about the issue before releasing it publicly.
If you would like to be added to our list of vendors, please reach out to the Valkey team at maintainers@lists.valkey.io.
+104
View File
@@ -0,0 +1,104 @@
TLS Support
===========
Getting Started
---------------
### Building
To build with TLS support you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu).
To build TLS support as Redis built-in:
Run `make BUILD_TLS=yes`.
Or to build TLS as Redis module:
Run `make BUILD_TLS=module`.
Note that sentinel mode does not support TLS module.
### Tests
To run Redis test suite with TLS, you'll need TLS support for TCL (i.e.
`tcl-tls` package on Debian/Ubuntu).
1. Run `./utils/gen-test-certs.sh` to generate a root CA and a server
certificate.
2. Run `./runtest --tls` or `./runtest-cluster --tls` to run Redis and Redis
Cluster tests in TLS mode.
3. Run `./runtest --tls-module` or `./runtest-cluster --tls-module` to
run Redis and Redis cluster tests in TLS mode with Redis module.
### Running manually
To manually run a Redis server with TLS mode (assuming `gen-test-certs.sh` was
invoked so sample certificates/keys are available):
For TLS built-in mode:
./src/redis-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/redis.crt \
--tls-key-file ./tests/tls/redis.key \
--tls-ca-cert-file ./tests/tls/ca.crt
For TLS module mode:
./src/redis-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/redis.crt \
--tls-key-file ./tests/tls/redis.key \
--tls-ca-cert-file ./tests/tls/ca.crt \
--loadmodule src/redis-tls.so
To connect to this Redis server with `redis-cli`:
./src/redis-cli --tls \
--cert ./tests/tls/redis.crt \
--key ./tests/tls/redis.key \
--cacert ./tests/tls/ca.crt
This will disable TCP and enable TLS on port 6379. It's also possible to have
both TCP and TLS available, but you'll need to assign different ports.
To make a Replica connect to the master using TLS, use `--tls-replication yes`,
and to make Redis Cluster use TLS across nodes use `--tls-cluster yes`.
Connections
-----------
All socket operations now go through a connection abstraction layer that hides
I/O and read/write event handling from the caller.
**Multi-threading I/O is not currently supported for TLS**, as a TLS connection
needs to do its own manipulation of AE events which is not thread safe. The
solution is probably to manage independent AE loops for I/O threads and longer
term association of connections with threads. This may potentially improve
overall performance as well.
Sync IO for TLS is currently implemented in a hackish way, i.e. making the
socket blocking and configuring socket-level timeout. This means the timeout
value may not be so accurate, and there would be a lot of syscall overhead.
However I believe that getting rid of syncio completely in favor of pure async
work is probably a better move than trying to fix that. For replication it would
probably not be so hard. For cluster keys migration it might be more difficult,
but there are probably other good reasons to improve that part anyway.
To-Do List
----------
- [ ] redis-benchmark support. The current implementation is a mix of using
hiredis for parsing and basic networking (establishing connections), but
directly manipulating sockets for most actions. This will need to be cleaned
up for proper TLS support. The best approach is probably to migrate to hiredis
async mode.
- [ ] redis-cli `--slave` and `--rdb` support.
Multi-port
----------
Consider the implications of allowing TLS to be configured on a separate port,
making Redis listening on multiple ports:
1. Startup banner port notification
2. Proctitle
3. How slaves announce themselves
4. Cluster bus port calculation
-379
View File
@@ -1,379 +0,0 @@
include(CheckIncludeFiles)
include(ProcessorCount)
include(Utils)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin")
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
# Generate compile_commands.json file for IDEs code completion support
set(CMAKE_EXPORT_COMPILE_COMMANDS 1)
processorcount(KV_PROCESSOR_COUNT)
message(STATUS "Processor count: ${KV_PROCESSOR_COUNT}")
# Installed executables will have this permissions
set(KV_EXE_PERMISSIONS
OWNER_EXECUTE
OWNER_WRITE
OWNER_READ
GROUP_EXECUTE
GROUP_READ
WORLD_EXECUTE
WORLD_READ)
set(KV_SERVER_CFLAGS "")
set(KV_SERVER_LDFLAGS "")
# ----------------------------------------------------
# Helper functions & macros
# ----------------------------------------------------
macro (add_kv_server_compiler_options value)
set(KV_SERVER_CFLAGS "${KV_SERVER_CFLAGS} ${value}")
endmacro ()
macro (add_kv_server_linker_option value)
list(APPEND KV_SERVER_LDFLAGS ${value})
endmacro ()
macro (get_kv_server_linker_option return_value)
list(JOIN KV_SERVER_LDFLAGS " " ${value} ${return_value})
endmacro ()
set(IS_FREEBSD 0)
if (CMAKE_SYSTEM_NAME MATCHES "^.*BSD$|DragonFly")
message(STATUS "Building for FreeBSD compatible system")
set(IS_FREEBSD 1)
include_directories("/usr/local/include")
add_kv_server_compiler_options("-DUSE_BACKTRACE")
endif ()
# Helper function for creating symbolic link so that: link -> source
macro (kv_create_symlink source link)
add_custom_command(
TARGET ${source} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E create_symlink
"$<TARGET_FILE_NAME:${source}>"
"$<TARGET_FILE_DIR:${source}>/${link}"
VERBATIM
)
endmacro ()
# Install a binary
macro (kv_install_bin target)
# Install cli tool and create a redis symbolic link
install(
TARGETS ${target}
DESTINATION ${CMAKE_INSTALL_BINDIR}
PERMISSIONS ${KV_EXE_PERMISSIONS}
COMPONENT "kv")
endmacro ()
# Helper function that defines, builds and installs `target` In addition, it creates a symbolic link between the target
# and `link_name`
macro (kv_build_and_install_bin target sources ld_flags libs link_name)
add_executable(${target} ${sources})
if (USE_JEMALLOC
OR USE_TCMALLOC
OR USE_TCMALLOC_MINIMAL)
# Using custom allocator
target_link_libraries(${target} ${ALLOCATOR_LIB})
endif ()
# Place this line last to ensure that ${ld_flags} is placed last on the linker line
target_link_libraries(${target} ${libs} ${ld_flags})
target_link_libraries(${target} kv::kv)
if (USE_TLS)
# Add required libraries needed for TLS
target_link_libraries(${target} OpenSSL::SSL kv::kv_tls)
endif ()
if (USE_RDMA)
# Add required libraries needed for RDMA
target_link_libraries(${target} kv::kv_rdma)
endif ()
if (IS_FREEBSD)
target_link_libraries(${target} execinfo)
endif ()
# Enable all warnings + fail on warning
target_compile_options(${target} PRIVATE -Werror -Wall)
# Install cli tool and create a redis symbolic link
kv_install_bin(${target})
kv_create_symlink(${target} ${link_name})
endmacro ()
# Determine if we are building in Release or Debug mode
if (CMAKE_BUILD_TYPE MATCHES Debug OR CMAKE_BUILD_TYPE MATCHES DebugFull)
set(KV_DEBUG_BUILD 1)
set(KV_RELEASE_BUILD 0)
message(STATUS "Building in debug mode")
else ()
set(KV_DEBUG_BUILD 0)
set(KV_RELEASE_BUILD 1)
message(STATUS "Building in release mode")
endif ()
# ----------------------------------------------------
# Helper functions - end
# ----------------------------------------------------
# ----------------------------------------------------
# Build options (allocator, tls, rdma et al)
# ----------------------------------------------------
if (NOT BUILD_MALLOC)
if (APPLE)
set(BUILD_MALLOC "libc")
elseif (UNIX)
set(BUILD_MALLOC "jemalloc")
endif ()
endif ()
# User may pass different allocator library. Using -DBUILD_MALLOC=<libname>, make sure it is a valid value
if (BUILD_MALLOC)
if ("${BUILD_MALLOC}" STREQUAL "jemalloc")
set(MALLOC_LIB "jemalloc")
set(ALLOCATOR_LIB "jemalloc")
add_kv_server_compiler_options("-DUSE_JEMALLOC")
set(USE_JEMALLOC 1)
elseif ("${BUILD_MALLOC}" STREQUAL "libc")
set(MALLOC_LIB "libc")
elseif ("${BUILD_MALLOC}" STREQUAL "tcmalloc")
set(MALLOC_LIB "tcmalloc")
kv_pkg_config(libtcmalloc ALLOCATOR_LIB)
add_kv_server_compiler_options("-DUSE_TCMALLOC")
set(USE_TCMALLOC 1)
elseif ("${BUILD_MALLOC}" STREQUAL "tcmalloc_minimal")
set(MALLOC_LIB "tcmalloc_minimal")
kv_pkg_config(libtcmalloc_minimal ALLOCATOR_LIB)
add_kv_server_compiler_options("-DUSE_TCMALLOC")
set(USE_TCMALLOC_MINIMAL 1)
else ()
message(FATAL_ERROR "BUILD_MALLOC can be one of: jemalloc, libc, tcmalloc or tcmalloc_minimal")
endif ()
endif ()
message(STATUS "Using ${MALLOC_LIB}")
# TLS support
if (BUILD_TLS)
kv_parse_build_option(${BUILD_TLS} USE_TLS)
if (USE_TLS EQUAL 1)
# Only search for OpenSSL if needed
find_package(OpenSSL REQUIRED)
message(STATUS "OpenSSL include dir: ${OPENSSL_INCLUDE_DIR}")
message(STATUS "OpenSSL libraries: ${OPENSSL_LIBRARIES}")
include_directories(${OPENSSL_INCLUDE_DIR})
endif ()
if (USE_TLS EQUAL 1)
add_kv_server_compiler_options("-DUSE_OPENSSL=1")
add_kv_server_compiler_options("-DBUILD_TLS_MODULE=0")
else ()
# Build TLS as a module RDMA can only be built as a module. So disable it
message(WARNING "BUILD_TLS can be one of: [ON | OFF | 1 | 0], but '${BUILD_TLS}' was provided")
message(STATUS "TLS support is disabled")
set(USE_TLS 0)
endif ()
else ()
# By default, TLS is disabled
message(STATUS "TLS is disabled")
set(USE_TLS 0)
endif ()
if (BUILD_RDMA)
set(BUILD_RDMA_MODULE 0)
# RDMA support (Linux only)
if (LINUX AND NOT APPLE)
kv_parse_build_option(${BUILD_RDMA} USE_RDMA)
find_package(PkgConfig REQUIRED)
# Locate librdmacm & libibverbs, fail if we can't find them
kv_pkg_config(librdmacm RDMACM_LIBS)
kv_pkg_config(libibverbs IBVERBS_LIBS)
message(STATUS "${RDMACM_LIBS};${IBVERBS_LIBS}")
list(APPEND RDMA_LIBS "${RDMACM_LIBS};${IBVERBS_LIBS}")
if (USE_RDMA EQUAL 2) # Module
message(STATUS "Building RDMA as module")
add_kv_server_compiler_options("-DUSE_RDMA=2")
set(BUILD_RDMA_MODULE 2)
elseif (USE_RDMA EQUAL 1) # Builtin
message(STATUS "Building RDMA as builtin")
add_kv_server_compiler_options("-DUSE_RDMA=1")
add_kv_server_compiler_options("-DBUILD_RDMA_MODULE=0")
list(APPEND SERVER_LIBS "${RDMA_LIBS}")
endif ()
else ()
message(WARNING "RDMA is only supported on Linux platforms")
endif ()
else ()
# By default, RDMA is disabled
message(STATUS "RDMA is disabled")
set(USE_RDMA 0)
endif ()
set(BUILDING_ARM64 0)
set(BUILDING_ARM32 0)
if ("${CMAKE_SYSTEM_PROCESSOR}" STREQUAL "arm64" OR "${CMAKE_SYSTEM_PROCESSOR}" STREQUAL "aarch64")
set(BUILDING_ARM64 1)
endif ()
if ("${CMAKE_SYSTEM_PROCESSOR}" STREQUAL "arm")
set(BUILDING_ARM32 1)
endif ()
message(STATUS "Building on ${CMAKE_HOST_SYSTEM_NAME}")
if (BUILDING_ARM64)
message(STATUS "Compiling kv for ARM64")
add_kv_server_linker_option("-funwind-tables")
endif ()
if (APPLE)
add_kv_server_linker_option("-rdynamic")
add_kv_server_linker_option("-ldl")
elseif (UNIX)
add_kv_server_linker_option("-rdynamic")
add_kv_server_linker_option("-pthread")
add_kv_server_linker_option("-ldl")
add_kv_server_linker_option("-lm")
endif ()
if (KV_DEBUG_BUILD)
# Debug build, use enable "-fno-omit-frame-pointer"
add_kv_server_compiler_options("-fno-omit-frame-pointer")
endif ()
# Check for Atomic
check_include_files(stdatomic.h HAVE_C11_ATOMIC)
if (HAVE_C11_ATOMIC)
add_kv_server_compiler_options("-std=gnu11")
else ()
add_kv_server_compiler_options("-std=c99")
endif ()
# Sanitizer
if (BUILD_SANITIZER)
# Common CFLAGS
list(APPEND KV_SANITAIZER_CFLAGS "-fno-sanitize-recover=all")
list(APPEND KV_SANITAIZER_CFLAGS "-fno-omit-frame-pointer")
if ("${BUILD_SANITIZER}" STREQUAL "address")
list(APPEND KV_SANITAIZER_CFLAGS "-fsanitize=address")
list(APPEND KV_SANITAIZER_LDFLAGS "-fsanitize=address")
elseif ("${BUILD_SANITIZER}" STREQUAL "thread")
list(APPEND KV_SANITAIZER_CFLAGS "-fsanitize=thread")
list(APPEND KV_SANITAIZER_LDFLAGS "-fsanitize=thread")
elseif ("${BUILD_SANITIZER}" STREQUAL "undefined")
list(APPEND KV_SANITAIZER_CFLAGS "-fsanitize=undefined")
list(APPEND KV_SANITAIZER_LDFLAGS "-fsanitize=undefined")
else ()
message(FATAL_ERROR "Unknown sanitizer: ${BUILD_SANITIZER}")
endif ()
endif ()
include_directories("${CMAKE_SOURCE_DIR}/deps/libkv/include")
include_directories("${CMAKE_SOURCE_DIR}/src/modules/lua")
include_directories("${CMAKE_SOURCE_DIR}/deps/linenoise")
include_directories("${CMAKE_SOURCE_DIR}/deps/hdr_histogram")
include_directories("${CMAKE_SOURCE_DIR}/deps/fpconv")
add_subdirectory("${CMAKE_SOURCE_DIR}/deps")
# Update linker flags for the allocator
if (USE_JEMALLOC)
include_directories("${CMAKE_SOURCE_DIR}/deps/jemalloc/include")
endif ()
# Common compiler flags
add_kv_server_compiler_options("-pedantic")
if (NOT BUILD_LUA)
message(STATUS "Lua scripting engine is disabled")
endif()
# ----------------------------------------------------
# Build options (allocator, tls, rdma et al) - end
# ----------------------------------------------------
# -------------------------------------------------
# Code Generation section
# -------------------------------------------------
find_program(PYTHON_EXE python3)
if (PYTHON_EXE)
# Python based code generation
message(STATUS "Found python3: ${PYTHON_EXE}")
# Rule for generating commands.def file from json files
message(STATUS "Adding target generate_commands_def")
file(GLOB COMMAND_FILES_JSON "${CMAKE_SOURCE_DIR}/src/commands/*.json")
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/commands_def_generated
DEPENDS ${COMMAND_FILES_JSON}
COMMAND ${PYTHON_EXE} ${CMAKE_SOURCE_DIR}/utils/generate-command-code.py
COMMAND touch ${CMAKE_BINARY_DIR}/commands_def_generated
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/src")
add_custom_target(generate_commands_def DEPENDS ${CMAKE_BINARY_DIR}/commands_def_generated)
# Rule for generating fmtargs.h
message(STATUS "Adding target generate_fmtargs_h")
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/fmtargs_generated
DEPENDS ${CMAKE_SOURCE_DIR}/utils/generate-fmtargs.py
COMMAND sed '/Everything/,$$d' fmtargs.h > fmtargs.h.tmp
COMMAND ${PYTHON_EXE} ${CMAKE_SOURCE_DIR}/utils/generate-fmtargs.py >> fmtargs.h.tmp
COMMAND mv fmtargs.h.tmp fmtargs.h
COMMAND touch ${CMAKE_BINARY_DIR}/fmtargs_generated
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/src")
add_custom_target(generate_fmtargs_h DEPENDS ${CMAKE_BINARY_DIR}/fmtargs_generated)
# Rule for generating test_files.h
message(STATUS "Adding target generate_test_files_h")
file(GLOB UNIT_TEST_SRCS "${CMAKE_SOURCE_DIR}/src/unit/*.c")
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/test_files_generated
DEPENDS "${UNIT_TEST_SRCS};${CMAKE_SOURCE_DIR}/utils/generate-unit-test-header.py"
COMMAND ${PYTHON_EXE} ${CMAKE_SOURCE_DIR}/utils/generate-unit-test-header.py
COMMAND touch ${CMAKE_BINARY_DIR}/test_files_generated
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/src")
add_custom_target(generate_test_files_h DEPENDS ${CMAKE_BINARY_DIR}/test_files_generated)
else ()
# Fake targets
add_custom_target(generate_commands_def)
add_custom_target(generate_fmtargs_h)
add_custom_target(generate_test_files_h)
endif ()
# Generate release.h file (always)
add_custom_target(
release_header
COMMAND sh -c '${CMAKE_SOURCE_DIR}/src/mkreleasehdr.sh'
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/src")
# -------------------------------------------------
# Code Generation section - end
# -------------------------------------------------
# ----------------------------------------------------------
# All our source files are defined in SourceFiles.cmake file
# ----------------------------------------------------------
include(SourceFiles)
# Clear the below variables from the cache
unset(CMAKE_C_FLAGS CACHE)
unset(KV_SERVER_LDFLAGS CACHE)
unset(KV_SERVER_CFLAGS CACHE)
unset(PYTHON_EXE CACHE)
unset(HAVE_C11_ATOMIC CACHE)
unset(USE_TLS CACHE)
unset(USE_RDMA CACHE)
unset(BUILD_TLS CACHE)
unset(BUILD_RDMA CACHE)
unset(BUILD_MALLOC CACHE)
unset(USE_JEMALLOC CACHE)
unset(BUILD_TLS_MODULE CACHE)
unset(BUILD_TLS_BUILTIN CACHE)
-44
View File
@@ -1,44 +0,0 @@
set(CPACK_PACKAGE_NAME "kv")
kv_parse_version(CPACK_PACKAGE_VERSION_MAJOR CPACK_PACKAGE_VERSION_MINOR CPACK_PACKAGE_VERSION_PATCH)
set(CPACK_PACKAGE_CONTACT "maintainers@lists.kv.io")
set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "KV is an open source (BSD) high-performance key/value datastore")
set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_SOURCE_DIR}/COPYING")
set(CPACK_RESOURCE_FILE_README "${CMAKE_SOURCE_DIR}/README.md")
set(CPACK_STRIP_FILES TRUE)
kv_get_distro_name(DISTRO_NAME)
message(STATUS "Current host distro: ${DISTRO_NAME}")
if (DISTRO_NAME MATCHES ubuntu
OR DISTRO_NAME MATCHES debian
OR DISTRO_NAME MATCHES mint)
message(STATUS "Adding target package for ${DISTRO_NAME}")
set(CPACK_PACKAGING_INSTALL_PREFIX "/opt/kv")
# Debian related parameters
set(CPACK_DEBIAN_PACKAGE_MAINTAINER "KV contributors")
set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
set(CPACK_DEBIAN_FILE_NAME DEB-DEFAULT)
set(CPACK_GENERATOR "DEB")
endif ()
include(CPack)
unset(DISTRO_NAME CACHE)
# ---------------------------------------------------
# Create a helper script for creating symbolic links
# ---------------------------------------------------
write_file(
${CMAKE_BINARY_DIR}/CreateSymlink.sh
"\
#!/bin/bash \n\
if [ -z \${DESTDIR} ]; then \n\
# Script is called during 'make install' \n\
PREFIX=${CMAKE_INSTALL_PREFIX}/bin \n\
else \n\
# Script is called during 'make package' \n\
PREFIX=\${DESTDIR}${CPACK_PACKAGING_INSTALL_PREFIX}/bin \n\
fi \n\
cd \$PREFIX \n\
ln -sf \$1 \$2")
-178
View File
@@ -1,178 +0,0 @@
# -------------------------------------------------
# Define the sources to be built
# -------------------------------------------------
# kv-server source files
set(KV_SERVER_SRCS
${CMAKE_SOURCE_DIR}/src/threads_mngr.c
${CMAKE_SOURCE_DIR}/src/adlist.c
${CMAKE_SOURCE_DIR}/src/vector.c
${CMAKE_SOURCE_DIR}/src/quicklist.c
${CMAKE_SOURCE_DIR}/src/ae.c
${CMAKE_SOURCE_DIR}/src/anet.c
${CMAKE_SOURCE_DIR}/src/dict.c
${CMAKE_SOURCE_DIR}/src/hashtable.c
${CMAKE_SOURCE_DIR}/src/kvstore.c
${CMAKE_SOURCE_DIR}/src/sds.c
${CMAKE_SOURCE_DIR}/src/zmalloc.c
${CMAKE_SOURCE_DIR}/src/lzf_c.c
${CMAKE_SOURCE_DIR}/src/lzf_d.c
${CMAKE_SOURCE_DIR}/src/pqsort.c
${CMAKE_SOURCE_DIR}/src/zipmap.c
${CMAKE_SOURCE_DIR}/src/sha1.c
${CMAKE_SOURCE_DIR}/src/ziplist.c
${CMAKE_SOURCE_DIR}/src/release.c
${CMAKE_SOURCE_DIR}/src/memory_prefetch.c
${CMAKE_SOURCE_DIR}/src/io_threads.c
${CMAKE_SOURCE_DIR}/src/networking.c
${CMAKE_SOURCE_DIR}/src/util.c
${CMAKE_SOURCE_DIR}/src/object.c
${CMAKE_SOURCE_DIR}/src/db.c
${CMAKE_SOURCE_DIR}/src/replication.c
${CMAKE_SOURCE_DIR}/src/rdb.c
${CMAKE_SOURCE_DIR}/src/t_string.c
${CMAKE_SOURCE_DIR}/src/t_list.c
${CMAKE_SOURCE_DIR}/src/t_set.c
${CMAKE_SOURCE_DIR}/src/t_zset.c
${CMAKE_SOURCE_DIR}/src/t_hash.c
${CMAKE_SOURCE_DIR}/src/config.c
${CMAKE_SOURCE_DIR}/src/aof.c
${CMAKE_SOURCE_DIR}/src/pubsub.c
${CMAKE_SOURCE_DIR}/src/multi.c
${CMAKE_SOURCE_DIR}/src/debug.c
${CMAKE_SOURCE_DIR}/src/sort.c
${CMAKE_SOURCE_DIR}/src/intset.c
${CMAKE_SOURCE_DIR}/src/syncio.c
${CMAKE_SOURCE_DIR}/src/cluster.c
${CMAKE_SOURCE_DIR}/src/cluster_migrateslots.c
${CMAKE_SOURCE_DIR}/src/cluster_legacy.c
${CMAKE_SOURCE_DIR}/src/cluster_slot_stats.c
${CMAKE_SOURCE_DIR}/src/crc16.c
${CMAKE_SOURCE_DIR}/src/commandlog.c
${CMAKE_SOURCE_DIR}/src/eval.c
${CMAKE_SOURCE_DIR}/src/bio.c
${CMAKE_SOURCE_DIR}/src/rio.c
${CMAKE_SOURCE_DIR}/src/rand.c
${CMAKE_SOURCE_DIR}/src/memtest.c
${CMAKE_SOURCE_DIR}/src/syscheck.c
${CMAKE_SOURCE_DIR}/src/crcspeed.c
${CMAKE_SOURCE_DIR}/src/crccombine.c
${CMAKE_SOURCE_DIR}/src/crc64.c
${CMAKE_SOURCE_DIR}/src/bitops.c
${CMAKE_SOURCE_DIR}/src/sentinel.c
${CMAKE_SOURCE_DIR}/src/notify.c
${CMAKE_SOURCE_DIR}/src/setproctitle.c
${CMAKE_SOURCE_DIR}/src/blocked.c
${CMAKE_SOURCE_DIR}/src/hyperloglog.c
${CMAKE_SOURCE_DIR}/src/latency.c
${CMAKE_SOURCE_DIR}/src/sparkline.c
${CMAKE_SOURCE_DIR}/src/kv-check-rdb.c
${CMAKE_SOURCE_DIR}/src/kv-check-aof.c
${CMAKE_SOURCE_DIR}/src/geo.c
${CMAKE_SOURCE_DIR}/src/lazyfree.c
${CMAKE_SOURCE_DIR}/src/module.c
${CMAKE_SOURCE_DIR}/src/lrulfu.c
${CMAKE_SOURCE_DIR}/src/evict.c
${CMAKE_SOURCE_DIR}/src/expire.c
${CMAKE_SOURCE_DIR}/src/geohash.c
${CMAKE_SOURCE_DIR}/src/geohash_helper.c
${CMAKE_SOURCE_DIR}/src/childinfo.c
${CMAKE_SOURCE_DIR}/src/allocator_defrag.c
${CMAKE_SOURCE_DIR}/src/defrag.c
${CMAKE_SOURCE_DIR}/src/siphash.c
${CMAKE_SOURCE_DIR}/src/rax.c
${CMAKE_SOURCE_DIR}/src/t_stream.c
${CMAKE_SOURCE_DIR}/src/listpack.c
${CMAKE_SOURCE_DIR}/src/localtime.c
${CMAKE_SOURCE_DIR}/src/lolwut.c
${CMAKE_SOURCE_DIR}/src/lolwut5.c
${CMAKE_SOURCE_DIR}/src/lolwut6.c
${CMAKE_SOURCE_DIR}/src/lolwut9.c
${CMAKE_SOURCE_DIR}/src/acl.c
${CMAKE_SOURCE_DIR}/src/tracking.c
${CMAKE_SOURCE_DIR}/src/socket.c
${CMAKE_SOURCE_DIR}/src/tls.c
${CMAKE_SOURCE_DIR}/src/rdma.c
${CMAKE_SOURCE_DIR}/src/sha256.c
${CMAKE_SOURCE_DIR}/src/timeout.c
${CMAKE_SOURCE_DIR}/src/setcpuaffinity.c
${CMAKE_SOURCE_DIR}/src/monotonic.c
${CMAKE_SOURCE_DIR}/src/mt19937-64.c
${CMAKE_SOURCE_DIR}/src/resp_parser.c
${CMAKE_SOURCE_DIR}/src/call_reply.c
${CMAKE_SOURCE_DIR}/src/script.c
${CMAKE_SOURCE_DIR}/src/functions.c
${CMAKE_SOURCE_DIR}/src/scripting_engine.c
${CMAKE_SOURCE_DIR}/src/trace/trace.c
${CMAKE_SOURCE_DIR}/src/trace/trace_rdb.c
${CMAKE_SOURCE_DIR}/src/trace/trace_aof.c
${CMAKE_SOURCE_DIR}/src/trace/trace_commands.c
${CMAKE_SOURCE_DIR}/src/trace/trace_db.c
${CMAKE_SOURCE_DIR}/src/trace/trace_cluster.c
${CMAKE_SOURCE_DIR}/src/trace/trace_server.c
${CMAKE_SOURCE_DIR}/src/commands.c
${CMAKE_SOURCE_DIR}/src/strl.c
${CMAKE_SOURCE_DIR}/src/connection.c
${CMAKE_SOURCE_DIR}/src/unix.c
${CMAKE_SOURCE_DIR}/src/server.c
${CMAKE_SOURCE_DIR}/src/logreqres.c
${CMAKE_SOURCE_DIR}/src/entry.c
${CMAKE_SOURCE_DIR}/src/vset.c
${CMAKE_SOURCE_DIR}/src/fifo.c
${CMAKE_SOURCE_DIR}/src/mutexqueue.c)
# kv-cli
set(KV_CLI_SRCS
${CMAKE_SOURCE_DIR}/src/anet.c
${CMAKE_SOURCE_DIR}/src/adlist.c
${CMAKE_SOURCE_DIR}/src/dict.c
${CMAKE_SOURCE_DIR}/src/sds.c
${CMAKE_SOURCE_DIR}/src/sha256.c
${CMAKE_SOURCE_DIR}/src/util.c
${CMAKE_SOURCE_DIR}/src/kv-cli.c
${CMAKE_SOURCE_DIR}/src/zmalloc.c
${CMAKE_SOURCE_DIR}/src/release.c
${CMAKE_SOURCE_DIR}/src/ae.c
${CMAKE_SOURCE_DIR}/src/serverassert.c
${CMAKE_SOURCE_DIR}/src/crcspeed.c
${CMAKE_SOURCE_DIR}/src/crccombine.c
${CMAKE_SOURCE_DIR}/src/crc64.c
${CMAKE_SOURCE_DIR}/src/siphash.c
${CMAKE_SOURCE_DIR}/src/crc16.c
${CMAKE_SOURCE_DIR}/src/monotonic.c
${CMAKE_SOURCE_DIR}/src/cli_common.c
${CMAKE_SOURCE_DIR}/src/mt19937-64.c
${CMAKE_SOURCE_DIR}/src/strl.c
${CMAKE_SOURCE_DIR}/src/cli_commands.c)
# kv-benchmark
set(KV_BENCHMARK_SRCS
${CMAKE_SOURCE_DIR}/src/ae.c
${CMAKE_SOURCE_DIR}/src/anet.c
${CMAKE_SOURCE_DIR}/src/sds.c
${CMAKE_SOURCE_DIR}/src/sha256.c
${CMAKE_SOURCE_DIR}/src/util.c
${CMAKE_SOURCE_DIR}/src/kv-benchmark.c
${CMAKE_SOURCE_DIR}/src/adlist.c
${CMAKE_SOURCE_DIR}/src/dict.c
${CMAKE_SOURCE_DIR}/src/zmalloc.c
${CMAKE_SOURCE_DIR}/src/serverassert.c
${CMAKE_SOURCE_DIR}/src/release.c
${CMAKE_SOURCE_DIR}/src/crcspeed.c
${CMAKE_SOURCE_DIR}/src/crccombine.c
${CMAKE_SOURCE_DIR}/src/crc64.c
${CMAKE_SOURCE_DIR}/src/siphash.c
${CMAKE_SOURCE_DIR}/src/crc16.c
${CMAKE_SOURCE_DIR}/src/monotonic.c
${CMAKE_SOURCE_DIR}/src/cli_common.c
${CMAKE_SOURCE_DIR}/src/mt19937-64.c
${CMAKE_SOURCE_DIR}/src/strl.c
${CMAKE_SOURCE_DIR}/src/fuzzer_client.c
${CMAKE_SOURCE_DIR}/src/fuzzer_command_generator.c)
# kv-rdma module
set(KV_RDMA_MODULE_SRCS ${CMAKE_SOURCE_DIR}/src/rdma.c)
# kv-tls module
set(KV_TLS_MODULE_SRCS ${CMAKE_SOURCE_DIR}/src/tls.c)
-115
View File
@@ -1,115 +0,0 @@
# Return the current host distro name. For example: ubuntu, debian, amzn etc
function (kv_get_distro_name DISTRO_NAME)
if (LINUX AND NOT APPLE)
execute_process(
COMMAND /bin/bash "-c" "cat /etc/os-release |grep ^ID=|cut -d = -f 2"
OUTPUT_VARIABLE _OUT_VAR
OUTPUT_STRIP_TRAILING_WHITESPACE)
# clean the output
string(REPLACE "\"" "" _OUT_VAR "${_OUT_VAR}")
string(REPLACE "." "" _OUT_VAR "${_OUT_VAR}")
set(${DISTRO_NAME}
"${_OUT_VAR}"
PARENT_SCOPE)
elseif (APPLE)
set(${DISTRO_NAME}
"darwin"
PARENT_SCOPE)
elseif (IS_FREEBSD)
set(${DISTRO_NAME}
"freebsd"
PARENT_SCOPE)
else ()
set(${DISTRO_NAME}
"unknown"
PARENT_SCOPE)
endif ()
endfunction ()
function (kv_parse_version OUT_MAJOR OUT_MINOR OUT_PATCH)
# Read and parse package version from version.h file
file(STRINGS ${CMAKE_SOURCE_DIR}/src/version.h VERSION_LINES)
foreach (LINE ${VERSION_LINES})
string(FIND "${LINE}" "#define KV_VERSION " VERSION_STR_POS)
if (VERSION_STR_POS GREATER -1)
string(REPLACE "#define KV_VERSION " "" LINE "${LINE}")
string(REPLACE "\"" "" LINE "${LINE}")
# Change "." to ";" to make it a list
string(REPLACE "." ";" LINE "${LINE}")
list(GET LINE 0 _MAJOR)
list(GET LINE 1 _MINOR)
list(GET LINE 2 _PATCH)
message(STATUS "KV version: ${_MAJOR}.${_MINOR}.${_PATCH}")
# Set the output variables
set(${OUT_MAJOR}
${_MAJOR}
PARENT_SCOPE)
set(${OUT_MINOR}
${_MINOR}
PARENT_SCOPE)
set(${OUT_PATCH}
${_PATCH}
PARENT_SCOPE)
endif ()
endforeach ()
endfunction ()
# Given input argument `OPTION_VALUE`, check that the `OPTION_VALUE` is from the allowed values (one of:
# module/yes/no/1/0/true/false)
#
# Return value:
#
# If ARG is valid, return its number where:
#
# ~~~
# - `no` | `0` | `off` => return `0`
# - `yes` | `1` | `on` => return `1`
# - `module` => return `2`
# ~~~
function (kv_parse_build_option OPTION_VALUE OUT_ARG_ENUM)
list(APPEND VALID_OPTIONS "yes")
list(APPEND VALID_OPTIONS "1")
list(APPEND VALID_OPTIONS "on")
list(APPEND VALID_OPTIONS "no")
list(APPEND VALID_OPTIONS "0")
list(APPEND VALID_OPTIONS "off")
list(APPEND VALID_OPTIONS "module")
string(TOLOWER "${OPTION_VALUE}" OPTION_VALUE)
list(FIND VALID_OPTIONS "${ARG}" OPT_INDEX)
if (VERSION_STR_POS GREATER -1)
message(FATAL_ERROR "Invalid value passed ''${OPTION_VALUE}'")
endif ()
if ("${OPTION_VALUE}" STREQUAL "yes"
OR "${OPTION_VALUE}" STREQUAL "1"
OR "${OPTION_VALUE}" STREQUAL "on")
set(${OUT_ARG_ENUM}
1
PARENT_SCOPE)
elseif (
"${OPTION_VALUE}" STREQUAL "no"
OR "${OPTION_VALUE}" STREQUAL "0"
OR "${OPTION_VALUE}" STREQUAL "off")
set(${OUT_ARG_ENUM}
0
PARENT_SCOPE)
else ()
set(${OUT_ARG_ENUM}
2
PARENT_SCOPE)
endif ()
endfunction ()
function (kv_pkg_config PKGNAME OUT_VARIABLE)
if (NOT FOUND_PKGCONFIG)
# Locate pkg-config once
find_package(PkgConfig REQUIRED)
set(FOUND_PKGCONFIG 1)
endif ()
pkg_check_modules(__PREFIX REQUIRED ${PKGNAME})
message(STATUS "Found library for '${PKGNAME}': ${__PREFIX_LIBRARIES}")
set(${OUT_VARIABLE}
"${__PREFIX_LIBRARIES}"
PARENT_SCOPE)
endfunction ()
-19
View File
@@ -1,19 +0,0 @@
coverage:
status:
patch:
default:
informational: true
project:
default:
informational: true
comment:
require_changes: false
require_head: false
require_base: false
layout: "condensed_header, diff, files"
hide_project_coverage: false
behavior: default
github_checks:
annotations: false
-38
View File
@@ -1,38 +0,0 @@
if (USE_JEMALLOC)
add_subdirectory(jemalloc)
endif ()
add_subdirectory(lua)
# Set libkv options. We need to disable the defaults set in the OPTION(..) we do this by setting them in the CACHE
set(BUILD_SHARED_LIBS
OFF
CACHE BOOL "Build shared libraries")
set(DISABLE_TESTS
ON
CACHE BOOL "If tests should be compiled or not")
if (USE_TLS) # Module or no module
message(STATUS "Building kv_tls")
set(ENABLE_TLS
ON
CACHE BOOL "If TLS support should be compiled or not")
endif ()
if (USE_RDMA) # Module or no module
message(STATUS "Building kv_rdma")
set(ENABLE_RDMA
ON
CACHE BOOL "If RDMA support should be compiled or not")
endif ()
# Let libkv use sds and dict provided by kv.
set(DICT_INCLUDE_DIR ${CMAKE_SOURCE_DIR}/src)
set(SDS_INCLUDE_DIR ${CMAKE_SOURCE_DIR}/src)
add_subdirectory(libkv)
add_subdirectory(linenoise)
add_subdirectory(fpconv)
add_subdirectory(hdr_histogram)
# Clear any cached variables passed to libkv from the cache
unset(BUILD_SHARED_LIBS CACHE)
unset(DISABLE_TESTS CACHE)
unset(ENABLE_TLS CACHE)
unset(ENABLE_RDMA CACHE)
+9 -21
View File
@@ -1,4 +1,4 @@
# Dependency Makefile
# Redis dependency Makefile
uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
@@ -36,31 +36,25 @@ ifneq ($(shell sh -c '[ -f .make-ldflags ] && cat .make-ldflags || echo none'),
endif
distclean:
-(cd libkv && $(MAKE) clean) > /dev/null || true
-(cd hiredis && $(MAKE) clean) > /dev/null || true
-(cd linenoise && $(MAKE) clean) > /dev/null || true
-(cd lua && $(MAKE) clean) > /dev/null || true
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
-(cd hdr_histogram && $(MAKE) clean) > /dev/null || true
-(cd fpconv && $(MAKE) clean) > /dev/null || true
-(cd fast_float_c_interface && $(MAKE) clean) > /dev/null || true
-(rm -f .make-*)
.PHONY: distclean
LIBKV_MAKE_FLAGS = SDS_INCLUDE_DIR=../../src/ DICT_INCLUDE_DIR=../../src/
ifneq (,$(filter $(BUILD_TLS),yes module))
LIBKV_MAKE_FLAGS += USE_TLS=1
HIREDIS_MAKE_FLAGS = USE_SSL=1
endif
ifneq (,$(filter $(BUILD_RDMA),yes module))
LIBKV_MAKE_FLAGS += USE_RDMA=1
endif
libkv: .make-prerequisites
hiredis: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd libkv && $(MAKE) static $(LIBKV_MAKE_FLAGS)
cd hiredis && $(MAKE) static $(HIREDIS_MAKE_FLAGS)
.PHONY: libkv
.PHONY: hiredis
linenoise: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
@@ -76,7 +70,7 @@ hdr_histogram: .make-prerequisites
fpconv: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fpconv && $(MAKE) CFLAGS="-fPIC $(CFLAGS)"
cd fpconv && $(MAKE)
.PHONY: fpconv
@@ -85,12 +79,12 @@ ifeq ($(uname_S),SunOS)
LUA_CFLAGS= -D__C99FEATURES__=1
endif
LUA_CFLAGS+= -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DLUA_USE_MKSTEMP $(CFLAGS) -fPIC
LUA_CFLAGS+= -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DREDIS_STATIC='' -DLUA_USE_MKSTEMP $(CFLAGS)
LUA_LDFLAGS+= $(LDFLAGS)
ifeq ($(LUA_DEBUG),yes)
LUA_CFLAGS+= -O0 -g -DLUA_USE_APICHECK
else
LUA_CFLAGS+= -O2
LUA_CFLAGS+= -O2
endif
ifeq ($(LUA_COVERAGE),yes)
LUA_CFLAGS += -fprofile-arcs -ftest-coverage
@@ -122,9 +116,3 @@ jemalloc: .make-prerequisites
cd jemalloc && $(MAKE) lib/libjemalloc.a
.PHONY: jemalloc
fast_float_c_interface: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fast_float_c_interface && $(MAKE)
.PHONY: fast_float_c_interface
+16 -33
View File
@@ -1,12 +1,11 @@
This directory contains all KV dependencies, except for the libc that
This directory contains all Redis dependencies, except for the libc that
should be provided by the operating system.
* **Jemalloc** is our memory allocator, used as replacement for libc malloc on Linux by default. It has good performances and excellent fragmentation behavior. This component is upgraded from time to time.
* **libkv** is the official C client library for KV. It is used by kv-cli, kv-benchmark and KV Sentinel. It is managed in a separate project and updated as needed.
* **linenoise** is a readline replacement. It is developed by the same authors of KV but is managed as a separated project and updated as needed.
* **hiredis** is the official C client library for Redis. It is used by redis-cli, redis-benchmark and Redis Sentinel. It is part of the Redis official ecosystem but is developed externally from the Redis repository, so we just upgrade it as needed.
* **linenoise** is a readline replacement. It is developed by the same authors of Redis but is managed as a separated project and updated as needed.
* **lua** is Lua 5.1 with minor changes for security and additional libraries.
* **hdr_histogram** Used for per-command latency tracking histograms.
* **fast_float** is a replacement for strtod to convert strings to floats efficiently.
How to upgrade the above dependencies
===
@@ -14,10 +13,10 @@ How to upgrade the above dependencies
Jemalloc
---
Jemalloc is modified with changes that allow us to implement the KV
active defragmentation logic. However this feature of KV is not mandatory
and KV is able to understand if the Jemalloc version it is compiled
against supports such KV-specific modifications. So in theory, if you
Jemalloc is modified with changes that allow us to implement the Redis
active defragmentation logic. However this feature of Redis is not mandatory
and Redis is able to understand if the Jemalloc version it is compiled
against supports such Redis-specific modifications. So in theory, if you
are not interested in the active defragmentation, you can replace Jemalloc
just following these steps:
@@ -29,7 +28,7 @@ just following these steps:
Jemalloc configuration script is broken and will not work nested in another
git repository.
However note that we change Jemalloc settings via the `configure` script of Jemalloc using the `--with-lg-quantum` option, setting it to the value of 3 instead of 4. This provides us with more size classes that better suit the KV data structures, in order to gain memory efficiency.
However note that we change Jemalloc settings via the `configure` script of Jemalloc using the `--with-lg-quantum` option, setting it to the value of 3 instead of 4. This provides us with more size classes that better suit the Redis data structures, in order to gain memory efficiency.
If you want to upgrade Jemalloc while also providing support for
active defragmentation, in addition to the above steps you need to perform
@@ -39,7 +38,7 @@ the following additional steps:
to add `#define JEMALLOC_FRAG_HINT`.
6. Implement the function `je_get_defrag_hint()` inside `src/jemalloc.c`. You
can see how it is implemented in the current Jemalloc source tree shipped
with KV, and rewrite it according to the new Jemalloc internals, if they
with Redis, and rewrite it according to the new Jemalloc internals, if they
changed, otherwise you could just copy the old implementation if you are
upgrading just to a similar version of Jemalloc.
@@ -59,21 +58,20 @@ cd deps/jemalloc
4. Update jemalloc's version in `deps/Makefile`: search for "`--with-version=<old-version-tag>-0-g0`" and update it accordingly.
5. Commit the changes (VERSION,configure,Makefile).
Libkv
Hiredis
---
Libkv is used by Sentinel, `kv-cli` and `kv-benchmark`.
The library is built without its own version of the sds and dict type and uses the KV provided variant instead.
Hiredis is used by Sentinel, `redis-cli` and `redis-benchmark`. Like Redis, uses the SDS string library, but not necessarily the same version. In order to avoid conflicts, this version has all SDS identifiers prefixed by `hi`.
1. `git subtree pull --prefix deps/libkv https://github.com/hanzoai/kv.git <version-tag> --squash`<br>
1. `git subtree pull --prefix deps/hiredis https://github.com/redis/hiredis.git <version-tag> --squash`<br>
This should hopefully merge the local changes into the new version.
2. Commit the changes.
2. Conflicts will arise (due to our changes) you'll need to resolve them and commit.
Linenoise
---
Linenoise is rarely upgraded as needed. The upgrade process is trivial since
KV uses a non modified version of linenoise, so to upgrade just do the
Redis uses a non modified version of linenoise, so to upgrade just do the
following:
1. Remove the linenoise directory.
@@ -83,11 +81,11 @@ Lua
---
We use Lua 5.1 and no upgrade is planned currently, since we don't want to break
Lua scripts for new Lua features: in the context of KV Lua scripts the
Lua scripts for new Lua features: in the context of Redis Lua scripts the
capabilities of 5.1 are usually more than enough, the release is rock solid,
and we definitely don't want to break old scripts.
So upgrading of Lua is up to the KV project maintainers and should be a
So upgrading of Lua is up to the Redis project maintainers and should be a
manual procedure performed by taking a diff between the different versions.
Currently we have at least the following differences between official Lua 5.1
@@ -96,7 +94,6 @@ and our version:
1. Makefile is modified to allow a different compiler than GCC.
2. We have the implementation source code, and directly link to the following external libraries: `lua_cjson.o`, `lua_struct.o`, `lua_cmsgpack.o` and `lua_bit.o`.
3. There is a security fix in `ldo.c`, line 498: The check for `LUA_SIGNATURE[0]` is removed in order to avoid direct bytecode execution.
4. In `lstring.c`, the luaS_newlstr function's hash calculation has been upgraded from a simple hash function to MurmurHash3, implemented within the same file, to enhance performance, particularly for operations involving large strings.
Hdr_Histogram
---
@@ -107,17 +104,3 @@ We use a customized version based on master branch commit e4448cf6d1cd08fff51981
2. Copy updated files from newer version onto files in /hdr_histogram.
3. Apply the changes from 1 above to the updated files.
fast_float
---
The fast_float library provides fast header-only implementations for the C++ from_chars functions for `float` and `double` types as well as integer types. These functions convert ASCII strings representing decimal values (e.g., `1.3e10`) into binary types. The functions are much faster than comparable number-parsing functions from existing C++ standard libraries.
Specifically, `fast_float` provides the following function to parse floating-point numbers with a C++17-like syntax (the library itself only requires C++11):
template <typename T, typename UC = char, typename = FASTFLOAT_ENABLE_IF(is_supported_float_type<T>())>
from_chars_result_t<UC> from_chars(UC const *first, UC const *last, T &value, chars_format fmt = chars_format::general);
To upgrade the library,
1. Check out https://github.com/fastfloat/fast_float/tree/main
2. cd fast_float
3. Invoke "python3 ./script/amalgamate.py --output fast_float.h"
4. Copy fast_float.h file to "deps/fast_float/".
-3912
View File
File diff suppressed because it is too large Load Diff
-37
View File
@@ -1,37 +0,0 @@
CCCOLOR:="\033[34m"
SRCCOLOR:="\033[33m"
ENDCOLOR:="\033[0m"
CXX?=c++
# we need = instead of := so that $@ in QUIET_CXX gets evaluated in the rule and is assigned appropriate value.
TEMP:=$(CXX)
QUIET_CXX=@printf ' %b %b\n' $(CCCOLOR)C++$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR) 1>&2;
CXX=$(QUIET_CXX)$(TEMP)
WARN=-Wall -W -Wno-missing-field-initializers
STD=-pedantic -std=c++11
OPT?=-O3
CLANG := $(findstring clang,$(shell sh -c '$(CC) --version | head -1'))
ifeq ($(OPT),-O3)
ifeq (clang,$(CLANG))
OPT+=-flto
else
OPT+=-flto=auto -ffat-lto-objects
endif
endif
# 1) Today src/Makefile passes -m32 flag for explicit 32-bit build on 64-bit machine, via CFLAGS. For 32-bit build on
# 32-bit machine and 64-bit on 64-bit machine, CFLAGS are empty. No other flags are set that can conflict with C++,
# therefore let's use CFLAGS without changes for now.
# 2) FASTFLOAT_ALLOWS_LEADING_PLUS allows +inf to be parsed as inf, instead of error.
CXXFLAGS=$(STD) $(OPT) $(WARN) -static -fPIC -fno-exceptions $(CFLAGS) -D FASTFLOAT_ALLOWS_LEADING_PLUS
.PHONY: all clean
all: fast_float_strtod.o
clean:
rm -f *.o || true;
-24
View File
@@ -1,24 +0,0 @@
/*
* Copyright (c) Valkey Contributors
* All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#include "../fast_float/fast_float.h"
#include <cerrno>
extern "C"
{
double fast_float_strtod(const char *str, const char** endptr)
{
double temp = 0;
auto answer = fast_float::from_chars(str, str + strlen(str), temp);
if (answer.ec != std::errc()) {
errno = (answer.ec == std::errc::result_out_of_range) ? ERANGE : EINVAL;
}
if (endptr) {
*endptr = answer.ptr;
}
return temp;
}
}
-6
View File
@@ -1,6 +0,0 @@
project(fpconv)
set(SRCS "${CMAKE_CURRENT_LIST_DIR}/fpconv_dtoa.c" "${CMAKE_CURRENT_LIST_DIR}/fpconv_dtoa.h")
add_library(fpconv STATIC ${SRCS})
# Add -fPIC
set_target_properties(fpconv PROPERTIES POSITION_INDEPENDENT_CODE ON)
+1 -1
View File
@@ -6,7 +6,7 @@
* [1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
* ----------------------------------------------------------------------------
*
* Copyright (c) 2021, Redis Ltd.
* Copyright (c) 2021, Redis Labs
* Copyright (c) 2013-2019, night-shift <as.smljk at gmail dot com>
* Copyright (c) 2009, Florian Loitsch < florian.loitsch at inria dot fr >
* All rights reserved.
-7
View File
@@ -1,7 +0,0 @@
project(hdr_histogram)
set(SRCS "${CMAKE_CURRENT_LIST_DIR}/hdr_histogram.c" "${CMAKE_CURRENT_LIST_DIR}/hdr_histogram.h"
"${CMAKE_CURRENT_LIST_DIR}/hdr_atomic.h" "${CMAKE_CURRENT_LIST_DIR}/hdr_redis_malloc.h")
add_library(hdr_histogram STATIC ${SRCS})
target_compile_definitions(hdr_histogram PRIVATE HDR_MALLOC_INCLUDE=\"hdr_redis_malloc.h\")
+6 -6
View File
@@ -1,13 +1,13 @@
#ifndef HDR_MALLOC_H__
#define HDR_MALLOC_H__
void *kv_malloc(size_t size);
void *zmalloc(size_t size);
void *zcalloc_num(size_t num, size_t size);
void *kv_realloc(void *ptr, size_t size);
void kv_free(void *ptr);
void *zrealloc(void *ptr, size_t size);
void zfree(void *ptr);
#define hdr_malloc kv_malloc
#define hdr_malloc zmalloc
#define hdr_calloc zcalloc_num
#define hdr_realloc kv_realloc
#define hdr_free kv_free
#define hdr_realloc zrealloc
#define hdr_free zfree
#endif
@@ -18,6 +18,7 @@ categories:
- title: 'Breaking Changes'
labels:
- 'breakingchange'
- title: '🧪 Experimental Features'
labels:
- 'experimental'
+177
View File
@@ -0,0 +1,177 @@
name: Build and test
on: [push, pull_request]
jobs:
ubuntu:
name: Ubuntu
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update
sudo apt-get install -y redis-server valgrind libevent-dev
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake .. && make
- name: Build using makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
# - name: Run tests under valgrind
# env:
# SKIPS_AS_FAILS: 1
# TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
# run: $GITHUB_WORKSPACE/test.sh
centos7:
name: CentOS 7
runs-on: ubuntu-latest
container: centos:7
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
yum -y --enablerepo=remi install redis
yum -y install gcc gcc-c++ make openssl openssl-devel cmake3 valgrind libevent-devel
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake3 .. && make
- name: Build using Makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
- name: Run tests under valgrind
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
run: $GITHUB_WORKSPACE/test.sh
centos8:
name: RockyLinux 8
runs-on: ubuntu-latest
container: rockylinux:8
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
dnf -y upgrade --refresh
dnf -y install https://rpms.remirepo.net/enterprise/remi-release-8.rpm
dnf -y module install redis:remi-6.0
dnf -y group install "Development Tools"
dnf -y install openssl-devel cmake valgrind libevent-devel
- name: Build using cmake
env:
EXTRA_CMAKE_OPTS: -DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON -DENABLE_ASYNC_TESTS:BOOL=ON
CFLAGS: -Werror
CXXFLAGS: -Werror
run: mkdir build && cd build && cmake .. && make
- name: Build using Makefile
run: USE_SSL=1 TEST_ASYNC=1 make
- name: Run tests
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
- name: Run tests under valgrind
env:
SKIPS_AS_FAILS: 1
TEST_SSL: 1
TEST_PREFIX: valgrind --error-exitcode=99 --track-origins=yes --leak-check=full
run: $GITHUB_WORKSPACE/test.sh
freebsd:
runs-on: macos-12
name: FreeBSD
steps:
- uses: actions/checkout@v3
- name: Build in FreeBSD
uses: vmactions/freebsd-vm@v0
with:
prepare: pkg install -y gmake cmake
run: |
mkdir build && cd build && cmake .. && make && cd ..
gmake
macos:
name: macOS
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
brew install openssl redis@7.0
brew link redis@7.0 --force
- name: Build hiredis
run: USE_SSL=1 make
- name: Run tests
env:
TEST_SSL: 1
run: $GITHUB_WORKSPACE/test.sh
windows:
name: Windows
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
choco install -y ninja memurai-developer
- uses: ilammy/msvc-dev-cmd@v1
- name: Build hiredis
run: |
mkdir build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_EXAMPLES=ON
ninja -v
- name: Run tests
run: |
./build/hiredis-test.exe
- name: Install Cygwin Action
uses: cygwin/cygwin-install-action@v2
with:
packages: make git gcc-core
- name: Build in cygwin
env:
HIREDIS_PATH: ${{ github.workspace }}
run: |
make clean && make
@@ -4,14 +4,14 @@ on:
push:
# branches to consider in the event; optional, defaults to all
branches:
- main
- master
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
# Drafts your next Release notes as Pull Requests are merged into "master"
- uses: release-drafter/release-drafter@b1476f6e6eb133afa41ed8589daba6dc69b4d3f5 # v6.1.0
- uses: release-drafter/release-drafter@v5
with:
# (Optional) specify config name to use, relative to .github/. Default: release-drafter.yml
config-name: release-drafter-config.yml
+100
View File
@@ -0,0 +1,100 @@
name: C/C++ CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
full-build:
name: Build all, plus default examples, run tests against redis
runs-on: ubuntu-latest
env:
# the docker image used by the test.sh
REDIS_DOCKER: redis:alpine
steps:
- name: Install prerequisites
run: sudo apt-get update && sudo apt-get install -y libev-dev libevent-dev libglib2.0-dev libssl-dev valgrind
- uses: actions/checkout@v3
- name: Run make
run: make all examples
- name: Run unittests
run: make check
- name: Run tests with valgrind
env:
TEST_PREFIX: valgrind --error-exitcode=100
SKIPS_ARG: --skip-throughput
run: make check
build-32-bit:
name: Build and test minimal 32 bit linux
runs-on: ubuntu-latest
steps:
- name: Install prerequisites
run: sudo apt-get update && sudo apt-get install gcc-multilib
- uses: actions/checkout@v3
- name: Run make
run: make all
env:
PLATFORM_FLAGS: -m32
- name: Run unittests
env:
REDIS_DOCKER: redis:alpine
run: make check
build-arm:
name: Cross-compile and test arm linux with Qemu
runs-on: ubuntu-latest
strategy:
matrix:
include:
- name: arm
toolset: arm-linux-gnueabi
emulator: qemu-arm
- name: aarch64
toolset: aarch64-linux-gnu
emulator: qemu-aarch64
steps:
- name: Install qemu
if: matrix.emulator
run: sudo apt-get update && sudo apt-get install -y qemu-user
- name: Install platform toolset
if: matrix.toolset
run: sudo apt-get install -y gcc-${{matrix.toolset}}
- uses: actions/checkout@v3
- name: Run make
run: make all
env:
CC: ${{matrix.toolset}}-gcc
AR: ${{matrix.toolset}}-ar
- name: Run unittests
env:
REDIS_DOCKER: redis:alpine
TEST_PREFIX: ${{matrix.emulator}} -L /usr/${{matrix.toolset}}/
run: make check
build-windows:
name: Build and test on windows 64 bit Intel
runs-on: windows-latest
steps:
- uses: microsoft/setup-msbuild@v1.0.2
- uses: actions/checkout@v3
- name: Run CMake (shared lib)
run: cmake -Wno-dev CMakeLists.txt
- name: Build hiredis (shared lib)
run: MSBuild hiredis.vcxproj /p:Configuration=Debug
- name: Run CMake (static lib)
run: cmake -Wno-dev CMakeLists.txt -DBUILD_SHARED_LIBS=OFF
- name: Build hiredis (static lib)
run: MSBuild hiredis.vcxproj /p:Configuration=Debug
- name: Build hiredis-test
run: MSBuild hiredis-test.vcxproj /p:Configuration=Debug
# use memurai, redis compatible server, since it is easy to install. Can't
# install official redis containers on the windows runner
- name: Install Memurai redis server
run: choco install -y memurai-developer.install
- name: Run tests
run: Debug\hiredis-test.exe
+9
View File
@@ -0,0 +1,9 @@
/hiredis-test
/examples/hiredis-example*
/*.o
/*.so
/*.dylib
/*.a
/*.pc
*.dSYM
tags
+125
View File
@@ -0,0 +1,125 @@
language: c
compiler:
- gcc
- clang
os:
- linux
- osx
dist: bionic
branches:
only:
- staging
- trying
- master
- /^release\/.*$/
install:
- if [ "$TRAVIS_COMPILER" != "mingw" ]; then
wget https://github.com/redis/redis/archive/6.0.6.tar.gz;
tar -xzvf 6.0.6.tar.gz;
pushd redis-6.0.6 && BUILD_TLS=yes make && export PATH=$PWD/src:$PATH && popd;
fi;
before_script:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then
curl -O https://distfiles.macports.org/MacPorts/MacPorts-2.6.2-10.13-HighSierra.pkg;
sudo installer -pkg MacPorts-2.6.2-10.13-HighSierra.pkg -target /;
export PATH=$PATH:/opt/local/bin && sudo port -v selfupdate;
sudo port -N install openssl redis;
fi;
addons:
apt:
packages:
- libc6-dbg
- libc6-dev
- libc6:i386
- libc6-dev-i386
- libc6-dbg:i386
- gcc-multilib
- g++-multilib
- libssl-dev
- libssl-dev:i386
- valgrind
env:
- BITS="32"
- BITS="64"
script:
- EXTRA_CMAKE_OPTS="-DENABLE_EXAMPLES:BOOL=ON -DENABLE_SSL:BOOL=ON -DENABLE_SSL_TESTS:BOOL=ON";
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
else
TEST_PREFIX="valgrind --track-origins=yes --leak-check=full";
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
fi;
export CFLAGS CXXFLAGS LDFLAGS TEST_PREFIX EXTRA_CMAKE_OPTS
- make && make clean;
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
if [ "$BITS" == "64" ]; then
OPENSSL_PREFIX="$(ls -d /usr/local/Cellar/openssl@1.1/*)" USE_SSL=1 make;
fi;
else
USE_SSL=1 make;
fi;
- mkdir build/ && cd build/
- cmake .. ${EXTRA_CMAKE_OPTS}
- make VERBOSE=1
- if [ "$BITS" == "64" ]; then
TEST_SSL=1 SKIPS_AS_FAILS=1 ctest -V;
else
SKIPS_AS_FAILS=1 ctest -V;
fi;
jobs:
include:
# Windows MinGW cross compile on Linux
- os: linux
dist: xenial
compiler: mingw
addons:
apt:
packages:
- ninja-build
- gcc-mingw-w64-x86-64
- g++-mingw-w64-x86-64
script:
- mkdir build && cd build
- CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_WITH_INSTALL_RPATH=on
- ninja -v
# Windows MSVC 2017
- os: windows
compiler: msvc
env:
- MATRIX_EVAL="CC=cl.exe && CXX=cl.exe"
before_install:
- eval "${MATRIX_EVAL}"
install:
- choco install ninja
- choco install -y memurai-developer
script:
- mkdir build && cd build
- cmd.exe //C 'C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat' amd64 '&&'
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_EXAMPLES=ON '&&' ninja -v
- ./hiredis-test.exe
+580
View File
@@ -0,0 +1,580 @@
## [1.2.0](https://github.com/redis/hiredis/tree/v1.2.0) - (2023-06-04)
Announcing Hiredis v1.2.0 with with new adapters, and a great many bug fixes.
## 🚀 New Features
- Add sdevent adapter @Oipo (#1144)
- Allow specifying the keepalive interval @michael-grunder (#1168)
- Add RedisModule adapter @tezc (#1182)
- Helper for setting TCP_USER_TIMEOUT socket option @zuiderkwast (#1188)
## 🐛 Bug Fixes
- Fix a typo in b6a052f. @yossigo (#1190)
- Fix wincrypt symbols conflict @hudayou (#1151)
- Don't attempt to set a timeout if we are in an error state. @michael-grunder (#1180)
- Accept -nan per the RESP3 spec recommendation. @michael-grunder (#1178)
- Fix colliding option values @zuiderkwast (#1172)
- Ensure functionality without `_MSC_VER` definition @windyakin (#1194)
## 🧰 Maintenance
- Add a test for the TCP_USER_TIMEOUT option. @michael-grunder (#1192)
- Add -Werror as a default. @yossigo (#1193)
- CI: Update homebrew Redis version. @yossigo (#1191)
- Fix typo in makefile. @michael-grunder (#1179)
- Write a version file for the CMake package @Neverlord (#1165)
- CMakeLists.txt: respect BUILD_SHARED_LIBS @ffontaine (#1147)
- Cmake static or shared @autoantwort (#1160)
- fix typo @tillkruss (#1153)
- Add a test ensuring we don't clobber connection error. @michael-grunder (#1181)
- Search for openssl on macOS @michael-grunder (#1169)
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/neverlord"><img src="https://github.com/neverlord.png" width="32" height="32"></a>
<a href="https://github.com/Oipo"><img src="https://github.com/Oipo.png" width="32" height="32"></a>
<a href="https://github.com/autoantwort"><img src="https://github.com/autoantwort.png" width="32" height="32"></a>
<a href="https://github.com/ffontaine"><img src="https://github.com/ffontaine.png" width="32" height="32"></a>
<a href="https://github.com/hudayou"><img src="https://github.com/hudayou.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/postgraph"><img src="https://github.com/postgraph.png" width="32" height="32"></a>
<a href="https://github.com/tezc"><img src="https://github.com/tezc.png" width="32" height="32"></a>
<a href="https://github.com/tillkruss"><img src="https://github.com/tillkruss.png" width="32" height="32"></a>
<a href="https://github.com/vityafx"><img src="https://github.com/vityafx.png" width="32" height="32"></a>
<a href="https://github.com/windyakin"><img src="https://github.com/windyakin.png" width="32" height="32"></a>
<a href="https://github.com/yossigo"><img src="https://github.com/yossigo.png" width="32" height="32"></a>
<a href="https://github.com/zuiderkwast"><img src="https://github.com/zuiderkwast.png" width="32" height="32"></a>
## [1.1.0](https://github.com/redis/hiredis/tree/v1.1.0) - (2022-11-15)
Announcing Hiredis v1.1.0 GA with better SSL convenience, new async adapters and a great many bug fixes.
**NOTE**: Hiredis can now return `nan` in addition to `-inf` and `inf` when returning a `REDIS_REPLY_DOUBLE`.
## 🐛 Bug Fixes
- Add support for nan in RESP3 double [@filipecosta90](https://github.com/filipecosta90)
([\#1133](https://github.com/redis/hiredis/pull/1133))
## 🧰 Maintenance
- Add an example that calls redisCommandArgv [@michael-grunder](https://github.com/michael-grunder)
([\#1140](https://github.com/redis/hiredis/pull/1140))
- fix flag reference [@pata00](https://github.com/pata00) ([\#1136](https://github.com/redis/hiredis/pull/1136))
- Make freeing a NULL redisAsyncContext a no op. [@michael-grunder](https://github.com/michael-grunder)
([\#1135](https://github.com/redis/hiredis/pull/1135))
- CI updates ([@bjosv](https://github.com/redis/bjosv) ([\#1139](https://github.com/redis/hiredis/pull/1139))
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/bjsov"><img src="https://github.com/bjosv.png" width="32" height="32"></a>
<a href="https://github.com/filipecosta90"><img src="https://github.com/filipecosta90.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/pata00"><img src="https://github.com/pata00.png" width="32" height="32"></a>
## [1.1.0-rc1](https://github.com/redis/hiredis/tree/v1.1.0-rc1) - (2022-11-06)
Announcing Hiredis v1.1.0-rc1, with better SSL convenience, new async adapters, and a great many bug fixes.
## 🚀 New Features
- Add possibility to prefer IPv6, IPv4 or unspecified [@zuiderkwast](https://github.com/zuiderkwast)
([\#1096](https://github.com/redis/hiredis/pull/1096))
- Add adapters/libhv [@ithewei](https://github.com/ithewei) ([\#904](https://github.com/redis/hiredis/pull/904))
- Add timeout support to libhv adapter. [@michael-grunder](https://github.com/michael-grunder) ([\#1109](https://github.com/redis/hiredis/pull/1109))
- set default SSL verification path [@adobeturchenko](https://github.com/adobeturchenko) ([\#928](https://github.com/redis/hiredis/pull/928))
- Introduce .close method for redisContextFuncs [@pizhenwei](https://github.com/pizhenwei) ([\#1094](https://github.com/redis/hiredis/pull/1094))
- Make it possible to set SSL verify mode [@stanhu](https://github.com/stanhu) ([\#1085](https://github.com/redis/hiredis/pull/1085))
- Polling adapter and example [@kristjanvalur](https://github.com/kristjanvalur) ([\#932](https://github.com/redis/hiredis/pull/932))
- Unsubscribe handling in async [@bjosv](https://github.com/bjosv) ([\#1047](https://github.com/redis/hiredis/pull/1047))
- Add timeout support for libuv adapter [@MichaelSuen-thePointer](https://github.com/@MichaelSuenthePointer) ([\#1016](https://github.com/redis/hiredis/pull/1016))
## 🐛 Bug Fixes
- Update for MinGW cross compile [@bit0fun](https://github.com/bit0fun) ([\#1127](https://github.com/redis/hiredis/pull/1127))
- fixed CPP build error with adapters/libhv.h [@mtdxc](https://github.com/mtdxc) ([\#1125](https://github.com/redis/hiredis/pull/1125))
- Fix protocol error
[@michael-grunder](https://github.com/michael-grunder),
[@mtuleika-appcast](https://github.com/mtuleika-appcast) ([\#1106](https://github.com/redis/hiredis/pull/1106))
- Use a windows specific keepalive function. [@michael-grunder](https://github.com/michael-grunder) ([\#1104](https://github.com/redis/hiredis/pull/1104))
- Fix CMake config path on Linux. [@xkszltl](https://github.com/xkszltl) ([\#989](https://github.com/redis/hiredis/pull/989))
- Fix potential fault at createDoubleObject [@afcidk](https://github.com/afcidk) ([\#964](https://github.com/redis/hiredis/pull/964))
- Fix some undefined behavior [@jengab](https://github.com/jengab) ([\#1091](https://github.com/redis/hiredis/pull/1091))
- Copy OOM errors to redisAsyncContext when finding subscribe callback [@bjosv](https://github.com/bjosv) ([\#1090](https://github.com/redis/hiredis/pull/1090))
- Maintain backward compatibility with our onConnect callback. [@michael-grunder](https://github.com/michael-grunder) ([\#1087](https://github.com/redis/hiredis/pull/1087))
- Fix PUSH handler tests for Redis >= 7.0.5 [@michael-grunder](https://github.com/michael-grunder) ([\#1121](https://github.com/redis/hiredis/pull/1121))
- fix heap-buffer-overflow [@zhangtaoXT5](https://github.com/zhangtaoXT5) ([\#957](https://github.com/redis/hiredis/pull/957))
- Fix heap-buffer-overflow issue in redisvFormatCommad [@bjosv](https://github.com/bjosv) ([\#1097](https://github.com/redis/hiredis/pull/1097))
- Polling adapter requires sockcompat.h [@michael-grunder](https://github.com/michael-grunder) ([\#1095](https://github.com/redis/hiredis/pull/1095))
- Illumos test fixes, error message difference for bad hostname test. [@devnexen](https://github.com/devnexen) ([\#901](https://github.com/redis/hiredis/pull/901))
- Remove semicolon after do-while in \_EL\_CLEANUP [@sundb](https://github.com/sundb) ([\#905](https://github.com/redis/hiredis/pull/905))
- Stability: Support calling redisAsyncCommand and redisAsyncDisconnect from the onConnected callback [@kristjanvalur](https://github.com/kristjanvalur)
([\#931](https://github.com/redis/hiredis/pull/931))
- Fix async connect on Windows [@kristjanvalur](https://github.com/kristjanvalur) ([\#1073](https://github.com/redis/hiredis/pull/1073))
- Fix tests so they work for Redis 7.0 [@michael-grunder](https://github.com/michael-grunder) ([\#1072](https://github.com/redis/hiredis/pull/1072))
- Fix warnings on Win64 [@orgads](https://github.com/orgads) ([\#1058](https://github.com/redis/hiredis/pull/1058))
- Handle push notifications before or after reply. [@yossigo](https://github.com/yossigo) ([\#1062](https://github.com/redis/hiredis/pull/1062))
- Update hiredis sds with improvements found in redis [@bjosv](https://github.com/bjosv) ([\#1045](https://github.com/redis/hiredis/pull/1045))
- Avoid incorrect call to the previous reply's callback [@bjosv](https://github.com/bjosv) ([\#1040](https://github.com/redis/hiredis/pull/1040))
- fix building on AIX and SunOS [\#1031](https://github.com/redis/hiredis/pull/1031) ([@scddev](https://github.com/scddev))
- Allow sending commands after sending an unsubscribe [@bjosv](https://github.com/bjosv) ([\#1036](https://github.com/redis/hiredis/pull/1036))
- Correction for command timeout during pubsub [@bjosv](https://github.com/bjosv) ([\#1038](https://github.com/redis/hiredis/pull/1038))
- Fix adapters/libevent.h compilation for 64-bit Windows [@pbtummillo](https://github.com/pbtummillo) ([\#937](https://github.com/redis/hiredis/pull/937))
- Fix integer overflow when format command larger than 4GB [@sundb](https://github.com/sundb) ([\#1030](https://github.com/redis/hiredis/pull/1030))
- Handle array response during subscribe in RESP3 [@bjosv](https://github.com/bjosv) ([\#1014](https://github.com/redis/hiredis/pull/1014))
- Support PING while subscribing (RESP2) [@bjosv](https://github.com/bjosv) ([\#1027](https://github.com/redis/hiredis/pull/1027))
## 🧰 Maintenance
- CI fixes in preparation of release [@michael-grunder](https://github.com/michael-grunder) ([\#1130](https://github.com/redis/hiredis/pull/1130))
- Add do while(0) (protection for macros [@afcidk](https://github.com/afcidk) [\#959](https://github.com/redis/hiredis/pull/959))
- Fixup of PR734: Coverage of hiredis.c [@bjosv](https://github.com/bjosv) ([\#1124](https://github.com/redis/hiredis/pull/1124))
- CMake corrections for building on Windows [@bjosv](https://github.com/bjosv) ([\#1122](https://github.com/redis/hiredis/pull/1122))
- Install on windows fixes [@bjosv](https://github.com/bjosv) ([\#1117](https://github.com/redis/hiredis/pull/1117))
- Add libhv example to our standard Makefile [@michael-grunder](https://github.com/michael-grunder) ([\#1108](https://github.com/redis/hiredis/pull/1108))
- Additional include directory given by pkg-config [@bjosv](https://github.com/bjosv) ([\#1118](https://github.com/redis/hiredis/pull/1118))
- Use __attribute__ when building with Clang on Windows [@bjosv](https://github.com/bjosv) ([\#1115](https://github.com/redis/hiredis/pull/1115))
- Minor refactor [@michael-grunder](https://github.com/michael-grunder) ([\#1110](https://github.com/redis/hiredis/pull/1110))
- Fix pkgconfig result for hiredis_ssl [@bjosv](https://github.com/bjosv) ([\#1107](https://github.com/redis/hiredis/pull/1107))
- Update documentation to explain redisConnectWithOptions. [@michael-grunder](https://github.com/michael-grunder) ([\#1099](https://github.com/redis/hiredis/pull/1099))
- uvadapter: reduce number of uv_poll_start calls [@noxiouz](https://github.com/noxiouz) ([\#1098](https://github.com/redis/hiredis/pull/1098))
- Regression test for off-by-one parsing error [@bugwz](https://github.com/bugwz) ([\#1092](https://github.com/redis/hiredis/pull/1092))
- CMake: remove dict.c form hiredis_sources [@Lipraxde](https://github.com/Lipraxde) ([\#1055](https://github.com/redis/hiredis/pull/1055))
- Do store command timeout in the context for redisSetTimeout [@catterer](https://github.com/catterer) ([\#593](https://github.com/redis/hiredis/pull/593), [\#1093](https://github.com/redis/hiredis/pull/1093))
- Add GitHub Actions CI workflow for hiredis: Arm, Arm64, 386, windows. [@kristjanvalur](https://github.com/kristjanvalur) ([\#943](https://github.com/redis/hiredis/pull/943))
- CI: bump macOS runner version [@SukkaW](https://github.com/SukkaW) ([\#1079](https://github.com/redis/hiredis/pull/1079))
- Support for generating release notes [@chayim](https://github.com/chayim) ([\#1083](https://github.com/redis/hiredis/pull/1083))
- Improve example for SSL initialization in README.md [@stanhu](https://github.com/stanhu) ([\#1084](https://github.com/redis/hiredis/pull/1084))
- Fix README typos [@bjosv](https://github.com/bjosv) ([\#1080](https://github.com/redis/hiredis/pull/1080))
- fix cmake version [@smmir-cent](https://github.com/@smmircent) ([\#1050](https://github.com/redis/hiredis/pull/1050))
- Use the same name for static and shared libraries [@orgads](https://github.com/orgads) ([\#1057](https://github.com/redis/hiredis/pull/1057))
- Embed debug information in windows static .lib file [@kristjanvalur](https://github.com/kristjanvalur) ([\#1054](https://github.com/redis/hiredis/pull/1054))
- Improved async documentation [@kristjanvalur](https://github.com/kristjanvalur) ([\#1074](https://github.com/redis/hiredis/pull/1074))
- Use official repository for redis package. [@yossigo](https://github.com/yossigo) ([\#1061](https://github.com/redis/hiredis/pull/1061))
- Whitelist hiredis repo path in cygwin [@michael-grunder](https://github.com/michael-grunder) ([\#1063](https://github.com/redis/hiredis/pull/1063))
- CentOS 8 is EOL, switch to RockyLinux [@michael-grunder](https://github.com/michael-grunder) ([\#1046](https://github.com/redis/hiredis/pull/1046))
- CMakeLists.txt: allow building without a C++ compiler [@ffontaine](https://github.com/ffontaine) ([\#872](https://github.com/redis/hiredis/pull/872))
- Makefile: move SSL options into a block and refine rules [@pizhenwei](https://github.com/pizhenwei) ([\#997](https://github.com/redis/hiredis/pull/997))
- Update CMakeLists.txt for more portability [@EricDeng1001](https://github.com/EricDeng1001) ([\#1005](https://github.com/redis/hiredis/pull/1005))
- FreeBSD build fixes + CI [@michael-grunder](https://github.com/michael-grunder) ([\#1026](https://github.com/redis/hiredis/pull/1026))
- Add asynchronous test for pubsub using RESP3 [@bjosv](https://github.com/bjosv) ([\#1012](https://github.com/redis/hiredis/pull/1012))
- Trigger CI failure when Valgrind issues are found [@bjosv](https://github.com/bjosv) ([\#1011](https://github.com/redis/hiredis/pull/1011))
- Move to using make directly in Cygwin [@michael-grunder](https://github.com/michael-grunder) ([\#1020](https://github.com/redis/hiredis/pull/1020))
- Add asynchronous API tests [@bjosv](https://github.com/bjosv) ([\#1010](https://github.com/redis/hiredis/pull/1010))
- Correcting the build target `coverage` for enabled SSL [@bjosv](https://github.com/bjosv) ([\#1009](https://github.com/redis/hiredis/pull/1009))
- GH Actions: Run SSL tests during CI [@bjosv](https://github.com/bjosv) ([\#1008](https://github.com/redis/hiredis/pull/1008))
- GH: Actions - Add valgrind and CMake [@michael-grunder](https://github.com/michael-grunder) ([\#1004](https://github.com/redis/hiredis/pull/1004))
- Add Centos8 tests in GH Actions [@michael-grunder](https://github.com/michael-grunder) ([\#1001](https://github.com/redis/hiredis/pull/1001))
- We should run actions on PRs [@michael-grunder](https://github.com/michael-grunder) (([\#1000](https://github.com/redis/hiredis/pull/1000))
- Add Cygwin test in GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#999](https://github.com/redis/hiredis/pull/999))
- Add Windows tests in GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#996](https://github.com/redis/hiredis/pull/996))
- Switch to GitHub actions [@michael-grunder](https://github.com/michael-grunder) ([\#995](https://github.com/redis/hiredis/pull/995))
- Minor refactor of CVE-2021-32765 fix. [@michael-grunder](https://github.com/michael-grunder) ([\#993](https://github.com/redis/hiredis/pull/993))
- Remove extra comma from CMake var. [@xkszltl](https://github.com/xkszltl) ([\#988](https://github.com/redis/hiredis/pull/988))
- Add REDIS\_OPT\_PREFER\_UNSPEC [@michael-grunder](https://github.com/michael-grunder) ([\#1101](https://github.com/redis/hiredis/pull/1101))
## Contributors
We'd like to thank all the contributors who worked on this release!
<a href="https://github.com/EricDeng1001"><img src="https://github.com/EricDeng1001.png" width="32" height="32"></a>
<a href="https://github.com/Lipraxde"><img src="https://github.com/Lipraxde.png" width="32" height="32"></a>
<a href="https://github.com/MichaelSuen-thePointer"><img src="https://github.com/MichaelSuen-thePointer.png" width="32" height="32"></a>
<a href="https://github.com/SukkaW"><img src="https://github.com/SukkaW.png" width="32" height="32"></a>
<a href="https://github.com/adobeturchenko"><img src="https://github.com/adobeturchenko.png" width="32" height="32"></a>
<a href="https://github.com/afcidk"><img src="https://github.com/afcidk.png" width="32" height="32"></a>
<a href="https://github.com/bit0fun"><img src="https://github.com/bit0fun.png" width="32" height="32"></a>
<a href="https://github.com/bjosv"><img src="https://github.com/bjosv.png" width="32" height="32"></a>
<a href="https://github.com/bugwz"><img src="https://github.com/bugwz.png" width="32" height="32"></a>
<a href="https://github.com/catterer"><img src="https://github.com/catterer.png" width="32" height="32"></a>
<a href="https://github.com/chayim"><img src="https://github.com/chayim.png" width="32" height="32"></a>
<a href="https://github.com/devnexen"><img src="https://github.com/devnexen.png" width="32" height="32"></a>
<a href="https://github.com/ffontaine"><img src="https://github.com/ffontaine.png" width="32" height="32"></a>
<a href="https://github.com/ithewei"><img src="https://github.com/ithewei.png" width="32" height="32"></a>
<a href="https://github.com/jengab"><img src="https://github.com/jengab.png" width="32" height="32"></a>
<a href="https://github.com/kristjanvalur"><img src="https://github.com/kristjanvalur.png" width="32" height="32"></a>
<a href="https://github.com/michael-grunder"><img src="https://github.com/michael-grunder.png" width="32" height="32"></a>
<a href="https://github.com/noxiouz"><img src="https://github.com/noxiouz.png" width="32" height="32"></a>
<a href="https://github.com/mtdxc"><img src="https://github.com/mtdxc.png" width="32" height="32"></a>
<a href="https://github.com/orgads"><img src="https://github.com/orgads.png" width="32" height="32"></a>
<a href="https://github.com/pbtummillo"><img src="https://github.com/pbtummillo.png" width="32" height="32"></a>
<a href="https://github.com/pizhenwei"><img src="https://github.com/pizhenwei.png" width="32" height="32"></a>
<a href="https://github.com/scddev"><img src="https://github.com/scddev.png" width="32" height="32"></a>
<a href="https://github.com/smmir-cent"><img src="https://github.com/smmir-cent.png" width="32" height="32"></a>
<a href="https://github.com/stanhu"><img src="https://github.com/stanhu.png" width="32" height="32"></a>
<a href="https://github.com/sundb"><img src="https://github.com/sundb.png" width="32" height="32"></a>
<a href="https://github.com/vturchenko"><img src="https://github.com/vturchenko.png" width="32" height="32"></a>
<a href="https://github.com/xkszltl"><img src="https://github.com/xkszltl.png" width="32" height="32"></a>
<a href="https://github.com/yossigo"><img src="https://github.com/yossigo.png" width="32" height="32"></a>
<a href="https://github.com/zhangtaoXT5"><img src="https://github.com/zhangtaoXT5.png" width="32" height="32"></a>
<a href="https://github.com/zuiderkwast"><img src="https://github.com/zuiderkwast.png" width="32" height="32"></a>
## [1.0.2](https://github.com/redis/hiredis/tree/v1.0.2) - (2021-10-07)
Announcing Hiredis v1.0.2, which fixes CVE-2021-32765 but returns the SONAME to the correct value of `1.0.0`.
- [Revert SONAME bump](https://github.com/redis/hiredis/commit/d4e6f109a064690cde64765c654e679fea1d3548)
([Michael Grunder](https://github.com/michael-grunder))
## [1.0.1](https://github.com/redis/hiredis/tree/v1.0.1) - (2021-10-04)
<span style="color:red">This release erroneously bumped the SONAME, please use [1.0.2](https://github.com/redis/hiredis/tree/v1.0.2)</span>
Announcing Hiredis v1.0.1, a security release fixing CVE-2021-32765
- Fix for [CVE-2021-32765](https://github.com/redis/hiredis/security/advisories/GHSA-hfm9-39pp-55p2)
[commit](https://github.com/redis/hiredis/commit/76a7b10005c70babee357a7d0f2becf28ec7ed1e)
([Yossi Gottlieb](https://github.com/yossigo))
_Thanks to [Yossi Gottlieb](https://github.com/yossigo) for the security fix and to [Microsoft Security Vulnerability Research](https://www.microsoft.com/en-us/msrc/msvr) for finding the bug._ :sparkling_heart:
## [1.0.0](https://github.com/redis/hiredis/tree/v1.0.0) - (2020-08-03)
Announcing Hiredis v1.0.0, which adds support for RESP3, SSL connections, allocator injection, and better Windows support! :tada:
_A big thanks to everyone who helped with this release. The following list includes everyone who contributed at least five lines, sorted by lines contributed._ :sparkling_heart:
[Michael Grunder](https://github.com/michael-grunder), [Yossi Gottlieb](https://github.com/yossigo),
[Mark Nunberg](https://github.com/mnunberg), [Marcus Geelnard](https://github.com/mbitsnbites),
[Justin Brewer](https://github.com/justinbrewer), [Valentino Geron](https://github.com/valentinogeron),
[Minun Dragonation](https://github.com/dragonation), [Omri Steiner](https://github.com/OmriSteiner),
[Sangmoon Yi](https://github.com/jman-krafton), [Jinjiazh](https://github.com/jinjiazhang),
[Odin Hultgren Van Der Horst](https://github.com/Miniwoffer), [Muhammad Zahalqa](https://github.com/tryfinally),
[Nick Rivera](https://github.com/heronr), [Qi Yang](https://github.com/movebean),
[kevin1018](https://github.com/kevin1018)
[Full Changelog](https://github.com/redis/hiredis/compare/v0.14.1...v1.0.0)
**BREAKING CHANGES**:
* `redisOptions` now has two timeout fields. One for connecting, and one for commands. If you're presently using `options->timeout` you will need to change it to use `options->connect_timeout`. (See [example](https://github.com/redis/hiredis/commit/38b5ae543f5c99eb4ccabbe277770fc6bc81226f#diff-86ba39d37aa829c8c82624cce4f049fbL36))
* Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now protocol errors. This is consistent
with the RESP specification. On 32-bit platforms, the upper bound is lowered to `SIZE_MAX`.
* `redisReplyObjectFunctions.createArray` now takes `size_t` for its length parameter.
**New features:**
- Support for RESP3
[\#697](https://github.com/redis/hiredis/pull/697),
[\#805](https://github.com/redis/hiredis/pull/805),
[\#819](https://github.com/redis/hiredis/pull/819),
[\#841](https://github.com/redis/hiredis/pull/841)
([Yossi Gottlieb](https://github.com/yossigo), [Michael Grunder](https://github.com/michael-grunder))
- Support for SSL connections
[\#645](https://github.com/redis/hiredis/pull/645),
[\#699](https://github.com/redis/hiredis/pull/699),
[\#702](https://github.com/redis/hiredis/pull/702),
[\#708](https://github.com/redis/hiredis/pull/708),
[\#711](https://github.com/redis/hiredis/pull/711),
[\#821](https://github.com/redis/hiredis/pull/821),
[more](https://github.com/redis/hiredis/pulls?q=is%3Apr+is%3Amerged+SSL)
([Mark Nunberg](https://github.com/mnunberg), [Yossi Gottlieb](https://github.com/yossigo))
- Run-time allocator injection
[\#800](https://github.com/redis/hiredis/pull/800)
([Michael Grunder](https://github.com/michael-grunder))
- Improved Windows support (including MinGW and Windows CI)
[\#652](https://github.com/redis/hiredis/pull/652),
[\#663](https://github.com/redis/hiredis/pull/663)
([Marcus Geelnard](https://www.bitsnbites.eu/author/m/))
- Adds support for distinct connect and command timeouts
[\#839](https://github.com/redis/hiredis/pull/839),
[\#829](https://github.com/redis/hiredis/pull/829)
([Valentino Geron](https://github.com/valentinogeron))
- Add generic pointer and destructor to `redisContext` that users can use for context.
[\#855](https://github.com/redis/hiredis/pull/855)
([Michael Grunder](https://github.com/michael-grunder))
**Closed issues (that involved code changes):**
- Makefile does not install TLS libraries [\#809](https://github.com/redis/hiredis/issues/809)
- redisConnectWithOptions should not set command timeout [\#722](https://github.com/redis/hiredis/issues/722), [\#829](https://github.com/redis/hiredis/pull/829) ([valentinogeron](https://github.com/valentinogeron))
- Fix integer overflow in `sdsrange` [\#827](https://github.com/redis/hiredis/issues/827)
- INFO & CLUSTER commands failed when using RESP3 [\#802](https://github.com/redis/hiredis/issues/802)
- Windows compatibility patches [\#687](https://github.com/redis/hiredis/issues/687), [\#838](https://github.com/redis/hiredis/issues/838), [\#842](https://github.com/redis/hiredis/issues/842)
- RESP3 PUSH messages incorrectly use pending callback [\#825](https://github.com/redis/hiredis/issues/825)
- Asynchronous PSUBSCRIBE command fails when using RESP3 [\#815](https://github.com/redis/hiredis/issues/815)
- New SSL API [\#804](https://github.com/redis/hiredis/issues/804), [\#813](https://github.com/redis/hiredis/issues/813)
- Hard-coded limit of nested reply depth [\#794](https://github.com/redis/hiredis/issues/794)
- Fix TCP_NODELAY in Windows/OSX [\#679](https://github.com/redis/hiredis/issues/679), [\#690](https://github.com/redis/hiredis/issues/690), [\#779](https://github.com/redis/hiredis/issues/779), [\#785](https://github.com/redis/hiredis/issues/785),
- Added timers to libev adapter. [\#778](https://github.com/redis/hiredis/issues/778), [\#795](https://github.com/redis/hiredis/pull/795)
- Initialization discards const qualifier [\#777](https://github.com/redis/hiredis/issues/777)
- \[BUG\]\[MinGW64\] Error setting socket timeout [\#775](https://github.com/redis/hiredis/issues/775)
- undefined reference to hi_malloc [\#769](https://github.com/redis/hiredis/issues/769)
- hiredis pkg-config file incorrectly ignores multiarch libdir spec'n [\#767](https://github.com/redis/hiredis/issues/767)
- Don't use -G to build shared object on Solaris [\#757](https://github.com/redis/hiredis/issues/757)
- error when make USE\_SSL=1 [\#748](https://github.com/redis/hiredis/issues/748)
- Allow to change SSL Mode [\#646](https://github.com/redis/hiredis/issues/646)
- hiredis/adapters/libevent.h memleak [\#618](https://github.com/redis/hiredis/issues/618)
- redisLibuvPoll crash when server closes the connetion [\#545](https://github.com/redis/hiredis/issues/545)
- about redisAsyncDisconnect question [\#518](https://github.com/redis/hiredis/issues/518)
- hiredis adapters libuv error for help [\#508](https://github.com/redis/hiredis/issues/508)
- API/ABI changes analysis [\#506](https://github.com/redis/hiredis/issues/506)
- Memory leak patch in Redis [\#502](https://github.com/redis/hiredis/issues/502)
- Remove the depth limitation [\#421](https://github.com/redis/hiredis/issues/421)
**Merged pull requests:**
- Move SSL management to a distinct private pointer [\#855](https://github.com/redis/hiredis/pull/855) ([michael-grunder](https://github.com/michael-grunder))
- Move include to sockcompat.h to maintain style [\#850](https://github.com/redis/hiredis/pull/850) ([michael-grunder](https://github.com/michael-grunder))
- Remove erroneous tag and add license to push example [\#849](https://github.com/redis/hiredis/pull/849) ([michael-grunder](https://github.com/michael-grunder))
- fix windows compiling with mingw [\#848](https://github.com/redis/hiredis/pull/848) ([rmalizia44](https://github.com/rmalizia44))
- Some Windows quality of life improvements. [\#846](https://github.com/redis/hiredis/pull/846) ([michael-grunder](https://github.com/michael-grunder))
- Use \_WIN32 define instead of WIN32 [\#845](https://github.com/redis/hiredis/pull/845) ([michael-grunder](https://github.com/michael-grunder))
- Non Linux CI fixes [\#844](https://github.com/redis/hiredis/pull/844) ([michael-grunder](https://github.com/michael-grunder))
- Resp3 oob push support [\#841](https://github.com/redis/hiredis/pull/841) ([michael-grunder](https://github.com/michael-grunder))
- fix \#785: defer TCP\_NODELAY in async tcp connections [\#836](https://github.com/redis/hiredis/pull/836) ([OmriSteiner](https://github.com/OmriSteiner))
- sdsrange overflow fix [\#830](https://github.com/redis/hiredis/pull/830) ([michael-grunder](https://github.com/michael-grunder))
- Use explicit pointer casting for c++ compatibility [\#826](https://github.com/redis/hiredis/pull/826) ([aureus1](https://github.com/aureus1))
- Document allocator injection and completeness fix in test.c [\#824](https://github.com/redis/hiredis/pull/824) ([michael-grunder](https://github.com/michael-grunder))
- Use unique names for allocator struct members [\#823](https://github.com/redis/hiredis/pull/823) ([michael-grunder](https://github.com/michael-grunder))
- New SSL API to replace redisSecureConnection\(\). [\#821](https://github.com/redis/hiredis/pull/821) ([yossigo](https://github.com/yossigo))
- Add logic to handle RESP3 push messages [\#819](https://github.com/redis/hiredis/pull/819) ([michael-grunder](https://github.com/michael-grunder))
- Use standrad isxdigit instead of custom helper function. [\#814](https://github.com/redis/hiredis/pull/814) ([tryfinally](https://github.com/tryfinally))
- Fix missing SSL build/install options. [\#812](https://github.com/redis/hiredis/pull/812) ([yossigo](https://github.com/yossigo))
- Add link to ABI tracker [\#808](https://github.com/redis/hiredis/pull/808) ([michael-grunder](https://github.com/michael-grunder))
- Resp3 verbatim string support [\#805](https://github.com/redis/hiredis/pull/805) ([michael-grunder](https://github.com/michael-grunder))
- Allow users to replace allocator and handle OOM everywhere. [\#800](https://github.com/redis/hiredis/pull/800) ([michael-grunder](https://github.com/michael-grunder))
- Remove nested depth limitation. [\#797](https://github.com/redis/hiredis/pull/797) ([michael-grunder](https://github.com/michael-grunder))
- Attempt to fix compilation on Solaris [\#796](https://github.com/redis/hiredis/pull/796) ([michael-grunder](https://github.com/michael-grunder))
- Support timeouts in libev adapater [\#795](https://github.com/redis/hiredis/pull/795) ([michael-grunder](https://github.com/michael-grunder))
- Fix pkgconfig when installing to a custom lib dir [\#793](https://github.com/redis/hiredis/pull/793) ([michael-grunder](https://github.com/michael-grunder))
- Fix USE\_SSL=1 make/cmake on OSX and CMake tests [\#789](https://github.com/redis/hiredis/pull/789) ([michael-grunder](https://github.com/michael-grunder))
- Use correct libuv call on Windows [\#784](https://github.com/redis/hiredis/pull/784) ([michael-grunder](https://github.com/michael-grunder))
- Added CMake package config and fixed hiredis\_ssl on Windows [\#783](https://github.com/redis/hiredis/pull/783) ([michael-grunder](https://github.com/michael-grunder))
- CMake: Set hiredis\_ssl shared object version. [\#780](https://github.com/redis/hiredis/pull/780) ([yossigo](https://github.com/yossigo))
- Win32 tests and timeout fix [\#776](https://github.com/redis/hiredis/pull/776) ([michael-grunder](https://github.com/michael-grunder))
- Provides an optional cleanup callback for async data. [\#768](https://github.com/redis/hiredis/pull/768) ([heronr](https://github.com/heronr))
- Housekeeping fixes [\#764](https://github.com/redis/hiredis/pull/764) ([michael-grunder](https://github.com/michael-grunder))
- install alloc.h [\#756](https://github.com/redis/hiredis/pull/756) ([ch1aki](https://github.com/ch1aki))
- fix spelling mistakes [\#746](https://github.com/redis/hiredis/pull/746) ([ShooterIT](https://github.com/ShooterIT))
- Free the reply in redisGetReply when passed NULL [\#741](https://github.com/redis/hiredis/pull/741) ([michael-grunder](https://github.com/michael-grunder))
- Fix dead code in sslLogCallback relating to should\_log variable. [\#737](https://github.com/redis/hiredis/pull/737) ([natoscott](https://github.com/natoscott))
- Fix typo in dict.c. [\#731](https://github.com/redis/hiredis/pull/731) ([Kevin-Xi](https://github.com/Kevin-Xi))
- Adding an option to DISABLE\_TESTS [\#727](https://github.com/redis/hiredis/pull/727) ([pbotros](https://github.com/pbotros))
- Update README with SSL support. [\#720](https://github.com/redis/hiredis/pull/720) ([yossigo](https://github.com/yossigo))
- Fixes leaks in unit tests [\#715](https://github.com/redis/hiredis/pull/715) ([michael-grunder](https://github.com/michael-grunder))
- SSL Tests [\#711](https://github.com/redis/hiredis/pull/711) ([yossigo](https://github.com/yossigo))
- SSL Reorganization [\#708](https://github.com/redis/hiredis/pull/708) ([yossigo](https://github.com/yossigo))
- Fix MSVC build. [\#706](https://github.com/redis/hiredis/pull/706) ([yossigo](https://github.com/yossigo))
- SSL: Properly report SSL\_connect\(\) errors. [\#702](https://github.com/redis/hiredis/pull/702) ([yossigo](https://github.com/yossigo))
- Silent SSL trace to stdout by default. [\#699](https://github.com/redis/hiredis/pull/699) ([yossigo](https://github.com/yossigo))
- Port RESP3 support from Redis. [\#697](https://github.com/redis/hiredis/pull/697) ([yossigo](https://github.com/yossigo))
- Removed whitespace before newline [\#691](https://github.com/redis/hiredis/pull/691) ([Miniwoffer](https://github.com/Miniwoffer))
- Add install adapters header files [\#688](https://github.com/redis/hiredis/pull/688) ([kevin1018](https://github.com/kevin1018))
- Remove unnecessary null check before free [\#684](https://github.com/redis/hiredis/pull/684) ([qlyoung](https://github.com/qlyoung))
- redisReaderGetReply leak memory [\#671](https://github.com/redis/hiredis/pull/671) ([movebean](https://github.com/movebean))
- fix timeout code in windows [\#670](https://github.com/redis/hiredis/pull/670) ([jman-krafton](https://github.com/jman-krafton))
- test: fix errstr matching for musl libc [\#665](https://github.com/redis/hiredis/pull/665) ([ghost](https://github.com/ghost))
- Windows: MinGW fixes and Windows Travis builders [\#663](https://github.com/redis/hiredis/pull/663) ([mbitsnbites](https://github.com/mbitsnbites))
- The setsockopt and getsockopt API diffs from BSD socket and WSA one [\#662](https://github.com/redis/hiredis/pull/662) ([dragonation](https://github.com/dragonation))
- Fix Compile Error On Windows \(Visual Studio\) [\#658](https://github.com/redis/hiredis/pull/658) ([jinjiazhang](https://github.com/jinjiazhang))
- Fix NXDOMAIN test case [\#653](https://github.com/redis/hiredis/pull/653) ([michael-grunder](https://github.com/michael-grunder))
- Add MinGW support [\#652](https://github.com/redis/hiredis/pull/652) ([mbitsnbites](https://github.com/mbitsnbites))
- SSL Support [\#645](https://github.com/redis/hiredis/pull/645) ([mnunberg](https://github.com/mnunberg))
- Fix Invalid argument after redisAsyncConnectUnix [\#644](https://github.com/redis/hiredis/pull/644) ([codehz](https://github.com/codehz))
- Makefile: use predefined AR [\#632](https://github.com/redis/hiredis/pull/632) ([Mic92](https://github.com/Mic92))
- FreeBSD build fix [\#628](https://github.com/redis/hiredis/pull/628) ([devnexen](https://github.com/devnexen))
- Fix errors not propagating properly with libuv.h. [\#624](https://github.com/redis/hiredis/pull/624) ([yossigo](https://github.com/yossigo))
- Update README.md [\#621](https://github.com/redis/hiredis/pull/621) ([Crunsher](https://github.com/Crunsher))
- Fix redisBufferRead documentation [\#620](https://github.com/redis/hiredis/pull/620) ([hacst](https://github.com/hacst))
- Add CPPFLAGS to REAL\_CFLAGS [\#614](https://github.com/redis/hiredis/pull/614) ([thomaslee](https://github.com/thomaslee))
- Update createArray to take size\_t [\#597](https://github.com/redis/hiredis/pull/597) ([justinbrewer](https://github.com/justinbrewer))
- fix common realloc mistake and add null check more [\#580](https://github.com/redis/hiredis/pull/580) ([charsyam](https://github.com/charsyam))
- Proper error reporting for connect failures [\#578](https://github.com/redis/hiredis/pull/578) ([mnunberg](https://github.com/mnunberg))
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
## [1.0.0-rc1](https://github.com/redis/hiredis/tree/v1.0.0-rc1) - (2020-07-29)
_Note: There were no changes to code between v1.0.0-rc1 and v1.0.0 so see v1.0.0 for changelog_
### 0.14.1 (2020-03-13)
* Adds safe allocation wrappers (CVE-2020-7105, #747, #752) (Michael Grunder)
### 0.14.0 (2018-09-25)
**BREAKING CHANGES**:
* Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well.
If it was used to compare to other values, casting might be necessary or can be removed, if casting was applied before.
* Make string2ll static to fix conflict with Redis (Tom Lee [c3188b])
* Use -dynamiclib instead of -shared for OSX (Ryan Schmidt [a65537])
* Use string2ll from Redis w/added tests (Michael Grunder [7bef04, 60f622])
* Makefile - OSX compilation fixes (Ryan Schmidt [881fcb, 0e9af8])
* Remove redundant NULL checks (Justin Brewer [54acc8, 58e6b8])
* Fix bulk and multi-bulk length truncation (Justin Brewer [109197])
* Fix SIGSEGV in OpenBSD by checking for NULL before calling freeaddrinfo (Justin Brewer [546d94])
* Several POSIX compatibility fixes (Justin Brewer [bbeab8, 49bbaa, d1c1b6])
* Makefile - Compatibility fixes (Dimitri Vorobiev [3238cf, 12a9d1])
* Makefile - Fix make install on FreeBSD (Zach Shipko [a2ef2b])
* Makefile - don't assume $(INSTALL) is cp (Igor Gnatenko [725a96])
* Separate side-effect causing function from assert and small cleanup (amallia [b46413, 3c3234])
* Don't send negative values to `__redisAsyncCommand` (Frederik Deweerdt [706129])
* Fix leak if setsockopt fails (Frederik Deweerdt [e21c9c])
* Fix libevent leak (zfz [515228])
* Clean up GCC warning (Ichito Nagata [2ec774])
* Keep track of errno in `__redisSetErrorFromErrno()` as snprintf may use it (Jin Qing [25cd88])
* Solaris compilation fix (Donald Whyte [41b07d])
* Reorder linker arguments when building examples (Tustfarm-heart [06eedd])
* Keep track of subscriptions in case of rapid subscribe/unsubscribe (Hyungjin Kim [073dc8, be76c5, d46999])
* libuv use after free fix (Paul Scott [cbb956])
* Properly close socket fd on reconnect attempt (WSL [64d1ec])
* Skip valgrind in OSX tests (Jan-Erik Rediger [9deb78])
* Various updates for Travis testing OSX (Ted Nyman [fa3774, 16a459, bc0ea5])
* Update libevent (Chris Xin [386802])
* Change sds.h for building in C++ projects (Ali Volkan ATLI [f5b32e])
* Use proper format specifier in redisFormatSdsCommandArgv (Paulino Huerta, Jan-Erik Rediger [360a06, 8655a6])
* Better handling of NULL reply in example code (Jan-Erik Rediger [1b8ed3])
* Prevent overflow when formatting an error (Jan-Erik Rediger [0335cb])
* Compatibility fix for strerror_r (Tom Lee [bb1747])
* Properly detect integer parse/overflow errors (Justin Brewer [93421f])
* Adds CI for Windows and cygwin fixes (owent, [6c53d6, 6c3e40])
* Catch a buffer overflow when formatting the error message
* Import latest upstream sds. This breaks applications that are linked against the old hiredis v0.13
* Fix warnings, when compiled with -Wshadow
* Make hiredis compile in Cygwin on Windows, now CI-tested
* Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to `SIZE_MAX`.
* Remove backwards compatibility macro's
This removes the following old function aliases, use the new name now:
| Old | New |
| --------------------------- | ---------------------- |
| redisReplyReaderCreate | redisReaderCreate |
| redisReplyReaderCreate | redisReaderCreate |
| redisReplyReaderFree | redisReaderFree |
| redisReplyReaderFeed | redisReaderFeed |
| redisReplyReaderGetReply | redisReaderGetReply |
| redisReplyReaderSetPrivdata | redisReaderSetPrivdata |
| redisReplyReaderGetObject | redisReaderGetObject |
| redisReplyReaderGetError | redisReaderGetError |
* The `DEBUG` variable in the Makefile was renamed to `DEBUG_FLAGS`
Previously it broke some builds for people that had `DEBUG` set to some arbitrary value,
due to debugging other software.
By renaming we avoid unintentional name clashes.
Simply rename `DEBUG` to `DEBUG_FLAGS` in your environment to make it working again.
### 0.13.3 (2015-09-16)
* Revert "Clear `REDIS_CONNECTED` flag when connection is closed".
* Make tests pass on FreeBSD (Thanks, Giacomo Olgeni)
If the `REDIS_CONNECTED` flag is cleared,
the async onDisconnect callback function will never be called.
This causes problems as the disconnect is never reported back to the user.
### 0.13.2 (2015-08-25)
* Prevent crash on pending replies in async code (Thanks, @switch-st)
* Clear `REDIS_CONNECTED` flag when connection is closed (Thanks, Jerry Jacobs)
* Add MacOS X addapter (Thanks, @dizzus)
* Add Qt adapter (Thanks, Pietro Cerutti)
* Add Ivykis adapter (Thanks, Gergely Nagy)
All adapters are provided as is and are only tested where possible.
### 0.13.1 (2015-05-03)
This is a bug fix release.
The new `reconnect` method introduced new struct members, which clashed with pre-defined names in pre-C99 code.
Another commit forced C99 compilation just to make it work, but of course this is not desirable for outside projects.
Other non-C99 code can now use hiredis as usual again.
Sorry for the inconvenience.
* Fix memory leak in async reply handling (Salvatore Sanfilippo)
* Rename struct member to avoid name clash with pre-c99 code (Alex Balashov, ncopa)
### 0.13.0 (2015-04-16)
This release adds a minimal Windows compatibility layer.
The parser, standalone since v0.12.0, can now be compiled on Windows
(and thus used in other client libraries as well)
* Windows compatibility layer for parser code (tzickel)
* Properly escape data printed to PKGCONF file (Dan Skorupski)
* Fix tests when assert() undefined (Keith Bennett, Matt Stancliff)
* Implement a reconnect method for the client context, this changes the structure of `redisContext` (Aaron Bedra)
### 0.12.1 (2015-01-26)
* Fix `make install`: DESTDIR support, install all required files, install PKGCONF in proper location
* Fix `make test` as 32 bit build on 64 bit platform
### 0.12.0 (2015-01-22)
* Add optional KeepAlive support
* Try again on EINTR errors
* Add libuv adapter
* Add IPv6 support
* Remove possibility of multiple close on same fd
* Add ability to bind source address on connect
* Add redisConnectFd() and redisFreeKeepFd()
* Fix getaddrinfo() memory leak
* Free string if it is unused (fixes memory leak)
* Improve redisAppendCommandArgv performance 2.5x
* Add support for SO_REUSEADDR
* Fix redisvFormatCommand format parsing
* Add GLib 2.0 adapter
* Refactor reading code into read.c
* Fix errno error buffers to not clobber errors
* Generate pkgconf during build
* Silence _BSD_SOURCE warnings
* Improve digit counting for multibulk creation
### 0.11.0
* Increase the maximum multi-bulk reply depth to 7.
* Increase the read buffer size from 2k to 16k.
* Use poll(2) instead of select(2) to support large fds (>= 1024).
### 0.10.1
* Makefile overhaul. Important to check out if you override one or more
variables using environment variables or via arguments to the "make" tool.
* Issue #45: Fix potential memory leak for a multi bulk reply with 0 elements
being created by the default reply object functions.
* Issue #43: Don't crash in an asynchronous context when Redis returns an error
reply after the connection has been made (this happens when the maximum
number of connections is reached).
### 0.10.0
* See commit log.
+237
View File
@@ -0,0 +1,237 @@
CMAKE_MINIMUM_REQUIRED(VERSION 3.0.0)
OPTION(BUILD_SHARED_LIBS "Build shared libraries" ON)
OPTION(ENABLE_SSL "Build hiredis_ssl for SSL support" OFF)
OPTION(DISABLE_TESTS "If tests should be compiled or not" OFF)
OPTION(ENABLE_SSL_TESTS "Should we test SSL connections" OFF)
OPTION(ENABLE_EXAMPLES "Enable building hiredis examples" OFF)
OPTION(ENABLE_ASYNC_TESTS "Should we run all asynchronous API tests" OFF)
MACRO(getVersionBit name)
SET(VERSION_REGEX "^#define ${name} (.+)$")
FILE(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/hiredis.h"
VERSION_BIT REGEX ${VERSION_REGEX})
STRING(REGEX REPLACE ${VERSION_REGEX} "\\1" ${name} "${VERSION_BIT}")
ENDMACRO(getVersionBit)
getVersionBit(HIREDIS_MAJOR)
getVersionBit(HIREDIS_MINOR)
getVersionBit(HIREDIS_PATCH)
getVersionBit(HIREDIS_SONAME)
SET(VERSION "${HIREDIS_MAJOR}.${HIREDIS_MINOR}.${HIREDIS_PATCH}")
MESSAGE("Detected version: ${VERSION}")
PROJECT(hiredis LANGUAGES "C" VERSION "${VERSION}")
INCLUDE(GNUInstallDirs)
# Hiredis requires C99
SET(CMAKE_C_STANDARD 99)
SET(CMAKE_DEBUG_POSTFIX d)
SET(hiredis_sources
alloc.c
async.c
hiredis.c
net.c
read.c
sds.c
sockcompat.c)
SET(hiredis_sources ${hiredis_sources})
IF(WIN32)
ADD_DEFINITIONS(-D_CRT_SECURE_NO_WARNINGS -DWIN32_LEAN_AND_MEAN)
ENDIF()
ADD_LIBRARY(hiredis ${hiredis_sources})
ADD_LIBRARY(hiredis::hiredis ALIAS hiredis)
set(hiredis_export_name hiredis CACHE STRING "Name of the exported target")
set_target_properties(hiredis PROPERTIES EXPORT_NAME ${hiredis_export_name})
SET_TARGET_PROPERTIES(hiredis
PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS TRUE
VERSION "${HIREDIS_SONAME}")
IF(MSVC)
SET_TARGET_PROPERTIES(hiredis
PROPERTIES COMPILE_FLAGS /Z7)
ENDIF()
IF(WIN32)
TARGET_LINK_LIBRARIES(hiredis PUBLIC ws2_32 crypt32)
ELSEIF(CMAKE_SYSTEM_NAME MATCHES "FreeBSD")
TARGET_LINK_LIBRARIES(hiredis PUBLIC m)
ELSEIF(CMAKE_SYSTEM_NAME MATCHES "SunOS")
TARGET_LINK_LIBRARIES(hiredis PUBLIC socket)
ENDIF()
TARGET_INCLUDE_DIRECTORIES(hiredis PUBLIC $<INSTALL_INTERFACE:include> $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>)
CONFIGURE_FILE(hiredis.pc.in hiredis.pc @ONLY)
set(CPACK_PACKAGE_VENDOR "Redis")
set(CPACK_PACKAGE_DESCRIPTION "\
Hiredis is a minimalistic C client library for the Redis database.
It is minimalistic because it just adds minimal support for the protocol, \
but at the same time it uses a high level printf-alike API in order to make \
it much higher level than otherwise suggested by its minimal code base and the \
lack of explicit bindings for every Redis command.
Apart from supporting sending commands and receiving replies, it comes with a \
reply parser that is decoupled from the I/O layer. It is a stream parser designed \
for easy reusability, which can for instance be used in higher level language bindings \
for efficient reply parsing.
Hiredis only supports the binary-safe Redis protocol, so you can use it with any Redis \
version >= 1.2.0.
The library comes with multiple APIs. There is the synchronous API, the asynchronous API \
and the reply parsing API.")
set(CPACK_PACKAGE_HOMEPAGE_URL "https://github.com/redis/hiredis")
set(CPACK_PACKAGE_CONTACT "michael dot grunder at gmail dot com")
set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
set(CPACK_RPM_PACKAGE_AUTOREQPROV ON)
include(CPack)
INSTALL(TARGETS hiredis
EXPORT hiredis-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
if (MSVC AND BUILD_SHARED_LIBS)
INSTALL(FILES $<TARGET_PDB_FILE:hiredis>
DESTINATION ${CMAKE_INSTALL_BINDIR}
CONFIGURATIONS Debug RelWithDebInfo)
endif()
# For NuGet packages
INSTALL(FILES hiredis.targets
DESTINATION build/native)
INSTALL(FILES hiredis.h read.h sds.h async.h alloc.h sockcompat.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(DIRECTORY adapters
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
export(EXPORT hiredis-targets
FILE "${CMAKE_CURRENT_BINARY_DIR}/hiredis-targets.cmake"
NAMESPACE hiredis::)
if(WIN32)
SET(CMAKE_CONF_INSTALL_DIR share/hiredis)
else()
SET(CMAKE_CONF_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR}/cmake/hiredis)
endif()
SET(INCLUDE_INSTALL_DIR include)
include(CMakePackageConfigHelpers)
write_basic_package_version_file("${CMAKE_CURRENT_BINARY_DIR}/hiredis-config-version.cmake"
COMPATIBILITY SameMajorVersion)
configure_package_config_file(hiredis-config.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/hiredis-config.cmake
INSTALL_DESTINATION ${CMAKE_CONF_INSTALL_DIR}
PATH_VARS INCLUDE_INSTALL_DIR)
INSTALL(EXPORT hiredis-targets
FILE hiredis-targets.cmake
NAMESPACE hiredis::
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis-config.cmake
${CMAKE_CURRENT_BINARY_DIR}/hiredis-config-version.cmake
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
IF(ENABLE_SSL)
IF (NOT OPENSSL_ROOT_DIR)
IF (APPLE)
SET(OPENSSL_ROOT_DIR "/usr/local/opt/openssl")
ENDIF()
ENDIF()
FIND_PACKAGE(OpenSSL REQUIRED)
SET(hiredis_ssl_sources
ssl.c)
ADD_LIBRARY(hiredis_ssl ${hiredis_ssl_sources})
ADD_LIBRARY(hiredis::hiredis_ssl ALIAS hiredis_ssl)
IF (APPLE AND BUILD_SHARED_LIBS)
SET_PROPERTY(TARGET hiredis_ssl PROPERTY LINK_FLAGS "-Wl,-undefined -Wl,dynamic_lookup")
ENDIF()
SET_TARGET_PROPERTIES(hiredis_ssl
PROPERTIES
WINDOWS_EXPORT_ALL_SYMBOLS TRUE
VERSION "${HIREDIS_SONAME}")
IF(MSVC)
SET_TARGET_PROPERTIES(hiredis_ssl
PROPERTIES COMPILE_FLAGS /Z7)
ENDIF()
TARGET_LINK_LIBRARIES(hiredis_ssl PRIVATE OpenSSL::SSL)
IF(WIN32)
TARGET_LINK_LIBRARIES(hiredis_ssl PRIVATE hiredis)
ENDIF()
CONFIGURE_FILE(hiredis_ssl.pc.in hiredis_ssl.pc @ONLY)
INSTALL(TARGETS hiredis_ssl
EXPORT hiredis_ssl-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
if (MSVC AND BUILD_SHARED_LIBS)
INSTALL(FILES $<TARGET_PDB_FILE:hiredis_ssl>
DESTINATION ${CMAKE_INSTALL_BINDIR}
CONFIGURATIONS Debug RelWithDebInfo)
endif()
INSTALL(FILES hiredis_ssl.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
export(EXPORT hiredis_ssl-targets
FILE "${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-targets.cmake"
NAMESPACE hiredis::)
if(WIN32)
SET(CMAKE_CONF_INSTALL_DIR share/hiredis_ssl)
else()
SET(CMAKE_CONF_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR}/cmake/hiredis_ssl)
endif()
configure_package_config_file(hiredis_ssl-config.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-config.cmake
INSTALL_DESTINATION ${CMAKE_CONF_INSTALL_DIR}
PATH_VARS INCLUDE_INSTALL_DIR)
INSTALL(EXPORT hiredis_ssl-targets
FILE hiredis_ssl-targets.cmake
NAMESPACE hiredis::
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl-config.cmake
DESTINATION ${CMAKE_CONF_INSTALL_DIR})
ENDIF()
IF(NOT DISABLE_TESTS)
ENABLE_TESTING()
ADD_EXECUTABLE(hiredis-test test.c)
TARGET_LINK_LIBRARIES(hiredis-test hiredis)
IF(ENABLE_SSL_TESTS)
ADD_DEFINITIONS(-DHIREDIS_TEST_SSL=1)
TARGET_LINK_LIBRARIES(hiredis-test hiredis_ssl)
ENDIF()
IF(ENABLE_ASYNC_TESTS)
ADD_DEFINITIONS(-DHIREDIS_TEST_ASYNC=1)
TARGET_LINK_LIBRARIES(hiredis-test event)
ENDIF()
ADD_TEST(NAME hiredis-test
COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/test.sh)
ENDIF()
# Add examples
IF(ENABLE_EXAMPLES)
ADD_SUBDIRECTORY(examples)
ENDIF(ENABLE_EXAMPLES)
+29
View File
@@ -0,0 +1,29 @@
Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Redis nor the names of its contributors may be used
to endorse or promote products derived from this software without specific
prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+354
View File
@@ -0,0 +1,354 @@
# Hiredis Makefile
# Copyright (C) 2010-2011 Salvatore Sanfilippo <antirez at gmail dot com>
# Copyright (C) 2010-2011 Pieter Noordhuis <pcnoordhuis at gmail dot com>
# This file is released under the BSD license, see the COPYING file
OBJ=alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev hiredis-example-glib hiredis-example-push hiredis-example-poll
TESTS=hiredis-test
LIBNAME=libhiredis
PKGCONFNAME=hiredis.pc
HIREDIS_MAJOR=$(shell grep HIREDIS_MAJOR hiredis.h | awk '{print $$3}')
HIREDIS_MINOR=$(shell grep HIREDIS_MINOR hiredis.h | awk '{print $$3}')
HIREDIS_PATCH=$(shell grep HIREDIS_PATCH hiredis.h | awk '{print $$3}')
HIREDIS_SONAME=$(shell grep HIREDIS_SONAME hiredis.h | awk '{print $$3}')
# Installation related variables and target
PREFIX?=/usr/local
INCLUDE_PATH?=include/hiredis
LIBRARY_PATH?=lib
PKGCONF_PATH?=pkgconfig
INSTALL_INCLUDE_PATH= $(DESTDIR)$(PREFIX)/$(INCLUDE_PATH)
INSTALL_LIBRARY_PATH= $(DESTDIR)$(PREFIX)/$(LIBRARY_PATH)
INSTALL_PKGCONF_PATH= $(INSTALL_LIBRARY_PATH)/$(PKGCONF_PATH)
# redis-server configuration used for testing
REDIS_PORT=56379
REDIS_SERVER=redis-server
define REDIS_TEST_CONFIG
daemonize yes
pidfile /tmp/hiredis-test-redis.pid
port $(REDIS_PORT)
bind 127.0.0.1
unixsocket /tmp/hiredis-test-redis.sock
endef
export REDIS_TEST_CONFIG
# Fallback to gcc when $CC is not in $PATH.
CC:=$(shell sh -c 'type $${CC%% *} >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
CXX:=$(shell sh -c 'type $${CXX%% *} >/dev/null 2>/dev/null && echo $(CXX) || echo g++')
OPTIMIZATION?=-O3
WARNINGS=-Wall -Wextra -Werror -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers
DEBUG_FLAGS?= -g -ggdb
REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CPPFLAGS) $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS) $(PLATFORM_FLAGS)
REAL_LDFLAGS=$(LDFLAGS)
DYLIBSUFFIX=so
STLIBSUFFIX=a
DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
DYLIB_MAJOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
DYLIBNAME=$(LIBNAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) $(PLATFORM_FLAGS) -shared -Wl,-soname,$(DYLIB_MINOR_NAME)
STLIBNAME=$(LIBNAME).$(STLIBSUFFIX)
STLIB_MAKE_CMD=$(AR) rcs
#################### SSL variables start ####################
SSL_OBJ=ssl.o
SSL_LIBNAME=libhiredis_ssl
SSL_PKGCONFNAME=hiredis_ssl.pc
SSL_INSTALLNAME=install-ssl
SSL_DYLIB_MINOR_NAME=$(SSL_LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
SSL_DYLIB_MAJOR_NAME=$(SSL_LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
SSL_DYLIBNAME=$(SSL_LIBNAME).$(DYLIBSUFFIX)
SSL_STLIBNAME=$(SSL_LIBNAME).$(STLIBSUFFIX)
SSL_DYLIB_MAKE_CMD=$(CC) $(PLATFORM_FLAGS) -shared -Wl,-soname,$(SSL_DYLIB_MINOR_NAME)
USE_SSL?=0
ifeq ($(USE_SSL),1)
# This is required for test.c only
CFLAGS+=-DHIREDIS_TEST_SSL
EXAMPLES+=hiredis-example-ssl hiredis-example-libevent-ssl
SSL_STLIB=$(SSL_STLIBNAME)
SSL_DYLIB=$(SSL_DYLIBNAME)
SSL_PKGCONF=$(SSL_PKGCONFNAME)
SSL_INSTALL=$(SSL_INSTALLNAME)
else
SSL_STLIB=
SSL_DYLIB=
SSL_PKGCONF=
SSL_INSTALL=
endif
##################### SSL variables end #####################
# Platform-specific overrides
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
# This is required for test.c only
ifeq ($(TEST_ASYNC),1)
export CFLAGS+=-DHIREDIS_TEST_ASYNC
endif
ifeq ($(USE_SSL),1)
ifndef OPENSSL_PREFIX
ifeq ($(uname_S),Darwin)
SEARCH_PATH1=/opt/homebrew/opt/openssl
SEARCH_PATH2=/usr/local/opt/openssl
ifneq ($(wildcard $(SEARCH_PATH1)),)
OPENSSL_PREFIX=$(SEARCH_PATH1)
else ifneq ($(wildcard $(SEARCH_PATH2)),)
OPENSSL_PREFIX=$(SEARCH_PATH2)
endif
endif
endif
ifdef OPENSSL_PREFIX
CFLAGS+=-I$(OPENSSL_PREFIX)/include
SSL_LDFLAGS+=-L$(OPENSSL_PREFIX)/lib
endif
SSL_LDFLAGS+=-lssl -lcrypto
endif
ifeq ($(uname_S),FreeBSD)
LDFLAGS+=-lm
IS_GCC=$(shell sh -c '$(CC) --version 2>/dev/null |egrep -i -c "gcc"')
ifeq ($(IS_GCC),1)
REAL_CFLAGS+=-pedantic
endif
else
REAL_CFLAGS+=-pedantic
endif
ifeq ($(uname_S),SunOS)
IS_SUN_CC=$(shell sh -c '$(CC) -V 2>&1 |egrep -i -c "sun|studio"')
ifeq ($(IS_SUN_CC),1)
SUN_SHARED_FLAG=-G
else
SUN_SHARED_FLAG=-shared
endif
REAL_LDFLAGS+= -ldl -lnsl -lsocket
DYLIB_MAKE_CMD=$(CC) $(SUN_SHARED_FLAG) -o $(DYLIBNAME) -h $(DYLIB_MINOR_NAME) $(LDFLAGS)
SSL_DYLIB_MAKE_CMD=$(CC) $(SUN_SHARED_FLAG) -o $(SSL_DYLIBNAME) -h $(SSL_DYLIB_MINOR_NAME) $(LDFLAGS) $(SSL_LDFLAGS)
endif
ifeq ($(uname_S),Darwin)
DYLIBSUFFIX=dylib
DYLIB_MINOR_NAME=$(LIBNAME).$(HIREDIS_SONAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) -dynamiclib -Wl,-install_name,$(PREFIX)/$(LIBRARY_PATH)/$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
SSL_DYLIB_MAKE_CMD=$(CC) -dynamiclib -Wl,-install_name,$(PREFIX)/$(LIBRARY_PATH)/$(SSL_DYLIB_MINOR_NAME) -o $(SSL_DYLIBNAME) $(LDFLAGS) $(SSL_LDFLAGS)
DYLIB_PLUGIN=-Wl,-undefined -Wl,dynamic_lookup
endif
all: dynamic static hiredis-test pkgconfig
dynamic: $(DYLIBNAME) $(SSL_DYLIB)
static: $(STLIBNAME) $(SSL_STLIB)
pkgconfig: $(PKGCONFNAME) $(SSL_PKGCONF)
# Deps (use make dep to generate this)
alloc.o: alloc.c fmacros.h alloc.h
async.o: async.c fmacros.h alloc.h async.h hiredis.h read.h sds.h net.h dict.c dict.h win32.h async_private.h
dict.o: dict.c fmacros.h alloc.h dict.h
hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h alloc.h net.h async.h win32.h
net.o: net.c fmacros.h net.h hiredis.h read.h sds.h alloc.h sockcompat.h win32.h
read.o: read.c fmacros.h alloc.h read.h sds.h win32.h
sds.o: sds.c sds.h sdsalloc.h alloc.h
sockcompat.o: sockcompat.c sockcompat.h
test.o: test.c fmacros.h hiredis.h read.h sds.h alloc.h net.h sockcompat.h win32.h
$(DYLIBNAME): $(OBJ)
$(DYLIB_MAKE_CMD) -o $(DYLIBNAME) $(OBJ) $(REAL_LDFLAGS)
$(STLIBNAME): $(OBJ)
$(STLIB_MAKE_CMD) $(STLIBNAME) $(OBJ)
#################### SSL building rules start ####################
$(SSL_DYLIBNAME): $(SSL_OBJ)
$(SSL_DYLIB_MAKE_CMD) $(DYLIB_PLUGIN) -o $(SSL_DYLIBNAME) $(SSL_OBJ) $(REAL_LDFLAGS) $(LDFLAGS) $(SSL_LDFLAGS)
$(SSL_STLIBNAME): $(SSL_OBJ)
$(STLIB_MAKE_CMD) $(SSL_STLIBNAME) $(SSL_OBJ)
$(SSL_OBJ): ssl.c hiredis.h read.h sds.h alloc.h async.h win32.h async_private.h
#################### SSL building rules end ####################
# Binaries:
hiredis-example-libevent: examples/example-libevent.c adapters/libevent.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libevent-ssl: examples/example-libevent-ssl.c adapters/libevent.h $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
hiredis-example-libev: examples/example-libev.c adapters/libev.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -lev $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libhv: examples/example-libhv.c adapters/libhv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -lhv $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-glib: examples/example-glib.c adapters/glib.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(shell pkg-config --cflags --libs glib-2.0) $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-ivykis: examples/example-ivykis.c adapters/ivykis.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -livykis $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-macosx: examples/example-macosx.c adapters/macosx.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-ssl: examples/example-ssl.c $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
hiredis-example-poll: examples/example-poll.c adapters/poll.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
ifndef AE_DIR
hiredis-example-ae:
@echo "Please specify AE_DIR (e.g. <redis repository>/src)"
@false
else
hiredis-example-ae: examples/example-ae.c adapters/ae.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(AE_DIR) $< $(AE_DIR)/ae.o $(AE_DIR)/zmalloc.o $(AE_DIR)/../deps/jemalloc/lib/libjemalloc.a -pthread $(STLIBNAME)
endif
ifndef LIBUV_DIR
# dynamic link libuv.so
hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. -I$(LIBUV_DIR)/include $< -luv -lpthread -lrt $(STLIBNAME) $(REAL_LDFLAGS)
else
# use user provided static lib
hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME) $(REAL_LDFLAGS)
endif
ifeq ($(and $(QT_MOC),$(QT_INCLUDE_DIR),$(QT_LIBRARY_DIR)),)
hiredis-example-qt:
@echo "Please specify QT_MOC, QT_INCLUDE_DIR AND QT_LIBRARY_DIR"
@false
else
hiredis-example-qt: examples/example-qt.cpp adapters/qt.h $(STLIBNAME)
$(QT_MOC) adapters/qt.h -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore | \
$(CXX) -x c++ -o qt-adapter-moc.o -c - $(REAL_CFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore
$(QT_MOC) examples/example-qt.h -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore | \
$(CXX) -x c++ -o qt-example-moc.o -c - $(REAL_CFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore
$(CXX) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(QT_INCLUDE_DIR) -I$(QT_INCLUDE_DIR)/QtCore -L$(QT_LIBRARY_DIR) qt-adapter-moc.o qt-example-moc.o $< -pthread $(STLIBNAME) -lQtCore
endif
hiredis-example: examples/example.c $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-push: examples/example-push.c $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
examples: $(EXAMPLES)
TEST_LIBS = $(STLIBNAME) $(SSL_STLIB)
TEST_LDFLAGS = $(SSL_LDFLAGS)
ifeq ($(USE_SSL),1)
TEST_LDFLAGS += -pthread
endif
ifeq ($(TEST_ASYNC),1)
TEST_LDFLAGS += -levent
endif
hiredis-test: test.o $(TEST_LIBS)
$(CC) -o $@ $(REAL_CFLAGS) -I. $^ $(REAL_LDFLAGS) $(TEST_LDFLAGS)
hiredis-%: %.o $(STLIBNAME)
$(CC) $(REAL_CFLAGS) -o $@ $< $(TEST_LIBS) $(REAL_LDFLAGS)
test: hiredis-test
./hiredis-test
check: hiredis-test
TEST_SSL=$(USE_SSL) ./test.sh
.c.o:
$(CC) -std=c99 -c $(REAL_CFLAGS) $<
clean:
rm -rf $(DYLIBNAME) $(STLIBNAME) $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
dep:
$(CC) $(CPPFLAGS) $(CFLAGS) -MM *.c
INSTALL?= cp -pPR
$(PKGCONFNAME): hiredis.h
@echo "Generating $@ for pkgconfig..."
@echo prefix=$(PREFIX) > $@
@echo exec_prefix=\$${prefix} >> $@
@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
@echo includedir=$(PREFIX)/include >> $@
@echo pkgincludedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo >> $@
@echo Name: hiredis >> $@
@echo Description: Minimalistic C client library for Redis. >> $@
@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
@echo Libs: -L\$${libdir} -lhiredis >> $@
@echo Cflags: -I\$${pkgincludedir} -I\$${includedir} -D_FILE_OFFSET_BITS=64 >> $@
$(SSL_PKGCONFNAME): hiredis_ssl.h
@echo "Generating $@ for pkgconfig..."
@echo prefix=$(PREFIX) > $@
@echo exec_prefix=\$${prefix} >> $@
@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
@echo includedir=$(PREFIX)/include >> $@
@echo pkgincludedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo >> $@
@echo Name: hiredis_ssl >> $@
@echo Description: SSL Support for hiredis. >> $@
@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
@echo Requires: hiredis >> $@
@echo Libs: -L\$${libdir} -lhiredis_ssl >> $@
@echo Libs.private: -lssl -lcrypto >> $@
install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME) $(SSL_INSTALL)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_INCLUDE_PATH)/adapters $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis.h async.h read.h sds.h alloc.h sockcompat.h $(INSTALL_INCLUDE_PATH)
$(INSTALL) adapters/*.h $(INSTALL_INCLUDE_PATH)/adapters
$(INSTALL) $(DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(DYLIB_MINOR_NAME)
cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIBNAME) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIB_MAJOR_NAME)
$(INSTALL) $(STLIBNAME) $(INSTALL_LIBRARY_PATH)
mkdir -p $(INSTALL_PKGCONF_PATH)
$(INSTALL) $(PKGCONFNAME) $(INSTALL_PKGCONF_PATH)
install-ssl: $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(SSL_PKGCONFNAME)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis_ssl.h $(INSTALL_INCLUDE_PATH)
$(INSTALL) $(SSL_DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(SSL_DYLIB_MINOR_NAME)
cd $(INSTALL_LIBRARY_PATH) && ln -sf $(SSL_DYLIB_MINOR_NAME) $(SSL_DYLIBNAME) && ln -sf $(SSL_DYLIB_MINOR_NAME) $(SSL_DYLIB_MAJOR_NAME)
$(INSTALL) $(SSL_STLIBNAME) $(INSTALL_LIBRARY_PATH)
mkdir -p $(INSTALL_PKGCONF_PATH)
$(INSTALL) $(SSL_PKGCONFNAME) $(INSTALL_PKGCONF_PATH)
32bit:
@echo ""
@echo "WARNING: if this fails under Linux you probably need to install libc6-dev-i386"
@echo ""
$(MAKE) CFLAGS="-m32" LDFLAGS="-m32"
32bit-vars:
$(eval CFLAGS=-m32)
$(eval LDFLAGS=-m32)
gprof:
$(MAKE) CFLAGS="-pg" LDFLAGS="-pg"
gcov:
$(MAKE) CFLAGS+="-fprofile-arcs -ftest-coverage" LDFLAGS="-fprofile-arcs"
coverage: gcov
make check
mkdir -p tmp/lcov
lcov -d . -c --exclude '/usr*' -o tmp/lcov/hiredis.info
lcov -q -l tmp/lcov/hiredis.info
genhtml --legend -q -o tmp/lcov/report tmp/lcov/hiredis.info
noopt:
$(MAKE) OPTIMIZATION=""
.PHONY: all test check clean dep install 32bit 32bit-vars gprof gcov noopt
+821
View File
@@ -0,0 +1,821 @@
[![Build Status](https://github.com/redis/hiredis/actions/workflows/build.yml/badge.svg)](https://github.com/redis/hiredis/actions/workflows/build.yml)
**This Readme reflects the latest changed in the master branch. See [v1.0.0](https://github.com/redis/hiredis/tree/v1.0.0) for the Readme and documentation for the latest release ([API/ABI history](https://abi-laboratory.pro/?view=timeline&l=hiredis)).**
# HIREDIS
Hiredis is a minimalistic C client library for the [Redis](https://redis.io/) database.
It is minimalistic because it just adds minimal support for the protocol, but
at the same time it uses a high level printf-alike API in order to make it
much higher level than otherwise suggested by its minimal code base and the
lack of explicit bindings for every Redis command.
Apart from supporting sending commands and receiving replies, it comes with
a reply parser that is decoupled from the I/O layer. It
is a stream parser designed for easy reusability, which can for instance be used
in higher level language bindings for efficient reply parsing.
Hiredis only supports the binary-safe Redis protocol, so you can use it with any
Redis version >= 1.2.0.
The library comes with multiple APIs. There is the
*synchronous API*, the *asynchronous API* and the *reply parsing API*.
## Upgrading to `1.1.0`
Almost all users will simply need to recompile their applications against the newer version of hiredis.
**NOTE**: Hiredis can now return `nan` in addition to `-inf` and `inf` in a `REDIS_REPLY_DOUBLE`.
Applications that deal with `RESP3` doubles should make sure to account for this.
## Upgrading to `1.0.2`
<span style="color:red">NOTE: v1.0.1 erroneously bumped SONAME, which is why it is skipped here.</span>
Version 1.0.2 is simply 1.0.0 with a fix for [CVE-2021-32765](https://github.com/redis/hiredis/security/advisories/GHSA-hfm9-39pp-55p2). They are otherwise identical.
## Upgrading to `1.0.0`
Version 1.0.0 marks the first stable release of Hiredis.
It includes some minor breaking changes, mostly to make the exposed API more uniform and self-explanatory.
It also bundles the updated `sds` library, to sync up with upstream and Redis.
For code changes see the [Changelog](CHANGELOG.md).
_Note: As described below, a few member names have been changed but most applications should be able to upgrade with minor code changes and recompiling._
## IMPORTANT: Breaking changes from `0.14.1` -> `1.0.0`
* `redisContext` has two additional members (`free_privdata`, and `privctx`).
* `redisOptions.timeout` has been renamed to `redisOptions.connect_timeout`, and we've added `redisOptions.command_timeout`.
* `redisReplyObjectFunctions.createArray` now takes `size_t` instead of `int` for its length parameter.
## IMPORTANT: Breaking changes when upgrading from 0.13.x -> 0.14.x
Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to `SIZE_MAX`.
Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well. If it was used to
compare to other values, casting might be necessary or can be removed, if
casting was applied before.
## Upgrading from `<0.9.0`
Version 0.9.0 is a major overhaul of hiredis in every aspect. However, upgrading existing
code using hiredis should not be a big pain. The key thing to keep in mind when
upgrading is that hiredis >= 0.9.0 uses a `redisContext*` to keep state, in contrast to
the stateless 0.0.1 that only has a file descriptor to work with.
## Synchronous API
To consume the synchronous API, there are only a few function calls that need to be introduced:
```c
redisContext *redisConnect(const char *ip, int port);
void *redisCommand(redisContext *c, const char *format, ...);
void freeReplyObject(void *reply);
```
### Connecting
The function `redisConnect` is used to create a so-called `redisContext`. The
context is where Hiredis holds state for a connection. The `redisContext`
struct has an integer `err` field that is non-zero when the connection is in
an error state. The field `errstr` will contain a string with a description of
the error. More information on errors can be found in the **Errors** section.
After trying to connect to Redis using `redisConnect` you should
check the `err` field to see if establishing the connection was successful:
```c
redisContext *c = redisConnect("127.0.0.1", 6379);
if (c == NULL || c->err) {
if (c) {
printf("Error: %s\n", c->errstr);
// handle error
} else {
printf("Can't allocate redis context\n");
}
}
```
One can also use `redisConnectWithOptions` which takes a `redisOptions` argument
that can be configured with endpoint information as well as many different flags
to change how the `redisContext` will be configured.
```c
redisOptions opt = {0};
/* One can set the endpoint with one of our helper macros */
if (tcp) {
REDIS_OPTIONS_SET_TCP(&opt, "localhost", 6379);
} else {
REDIS_OPTIONS_SET_UNIX(&opt, "/tmp/redis.sock");
}
/* And privdata can be specified with another helper */
REDIS_OPTIONS_SET_PRIVDATA(&opt, myPrivData, myPrivDataDtor);
/* Finally various options may be set via the `options` member, as described below */
opt->options |= REDIS_OPT_PREFER_IPV4;
```
If a connection is lost, `int redisReconnect(redisContext *c)` can be used to restore the connection using the same endpoint and options as the given context.
### Configurable redisOptions flags
There are several flags you may set in the `redisOptions` struct to change default behavior. You can specify the flags via the `redisOptions->options` member.
| Flag | Description |
| --- | --- |
| REDIS\_OPT\_NONBLOCK | Tells hiredis to make a non-blocking connection. |
| REDIS\_OPT\_REUSEADDR | Tells hiredis to set the [SO_REUSEADDR](https://man7.org/linux/man-pages/man7/socket.7.html) socket option |
| REDIS\_OPT\_PREFER\_IPV4<br>REDIS\_OPT\_PREFER_IPV6<br>REDIS\_OPT\_PREFER\_IP\_UNSPEC | Informs hiredis to either prefer IPv4 or IPv6 when invoking [getaddrinfo](https://man7.org/linux/man-pages/man3/gai_strerror.3.html). `REDIS_OPT_PREFER_IP_UNSPEC` will cause hiredis to specify `AF_UNSPEC` in the getaddrinfo call, which means both IPv4 and IPv6 addresses will be searched simultaneously.<br>Hiredis prefers IPv4 by default. |
| REDIS\_OPT\_NO\_PUSH\_AUTOFREE | Tells hiredis to not install the default RESP3 PUSH handler (which just intercepts and frees the replies). This is useful in situations where you want to process these messages in-band. |
| REDIS\_OPT\_NOAUTOFREEREPLIES | **ASYNC**: tells hiredis not to automatically invoke `freeReplyObject` after executing the reply callback. |
| REDIS\_OPT\_NOAUTOFREE | **ASYNC**: Tells hiredis not to automatically free the `redisAsyncContext` on connection/communication failure, but only if the user makes an explicit call to `redisAsyncDisconnect` or `redisAsyncFree` |
*Note: A `redisContext` is not thread-safe.*
### Other configuration using socket options
The following socket options are applied directly to the underlying socket.
The values are not stored in the `redisContext`, so they are not automatically applied when reconnecting using `redisReconnect()`.
These functions return `REDIS_OK` on success.
On failure, `REDIS_ERR` is returned and the underlying connection is closed.
To configure these for an asyncronous context (see *Asynchronous API* below), use `ac->c` to get the redisContext out of an asyncRedisContext.
```C
int redisEnableKeepAlive(redisContext *c);
int redisEnableKeepAliveWithInterval(redisContext *c, int interval);
```
Enables TCP keepalive by setting the following socket options (with some variations depending on OS):
* `SO_KEEPALIVE`;
* `TCP_KEEPALIVE` or `TCP_KEEPIDLE`, value configurable using the `interval` parameter, default 15 seconds;
* `TCP_KEEPINTVL` set to 1/3 of `interval`;
* `TCP_KEEPCNT` set to 3.
```C
int redisSetTcpUserTimeout(redisContext *c, unsigned int timeout);
```
Set the `TCP_USER_TIMEOUT` Linux-specific socket option which is as described in the `tcp` man page:
> When the value is greater than 0, it specifies the maximum amount of time in milliseconds that trans mitted data may remain unacknowledged before TCP will forcibly close the corresponding connection and return ETIMEDOUT to the application.
> If the option value is specified as 0, TCP will use the system default.
### Sending commands
There are several ways to issue commands to Redis. The first that will be introduced is
`redisCommand`. This function takes a format similar to printf. In the simplest form,
it is used like this:
```c
reply = redisCommand(context, "SET foo bar");
```
The specifier `%s` interpolates a string in the command, and uses `strlen` to
determine the length of the string:
```c
reply = redisCommand(context, "SET foo %s", value);
```
When you need to pass binary safe strings in a command, the `%b` specifier can be
used. Together with a pointer to the string, it requires a `size_t` length argument
of the string:
```c
reply = redisCommand(context, "SET foo %b", value, (size_t) valuelen);
```
Internally, Hiredis splits the command in different arguments and will
convert it to the protocol used to communicate with Redis.
One or more spaces separates arguments, so you can use the specifiers
anywhere in an argument:
```c
reply = redisCommand(context, "SET key:%s %s", myid, value);
```
### Using replies
The return value of `redisCommand` holds a reply when the command was
successfully executed. When an error occurs, the return value is `NULL` and
the `err` field in the context will be set (see section on **Errors**).
Once an error is returned the context cannot be reused and you should set up
a new connection.
The standard replies that `redisCommand` are of the type `redisReply`. The
`type` field in the `redisReply` should be used to test what kind of reply
was received:
### RESP2
* **`REDIS_REPLY_STATUS`**:
* The command replied with a status reply. The status string can be accessed using `reply->str`.
The length of this string can be accessed using `reply->len`.
* **`REDIS_REPLY_ERROR`**:
* The command replied with an error. The error string can be accessed identical to `REDIS_REPLY_STATUS`.
* **`REDIS_REPLY_INTEGER`**:
* The command replied with an integer. The integer value can be accessed using the
`reply->integer` field of type `long long`.
* **`REDIS_REPLY_NIL`**:
* The command replied with a **nil** object. There is no data to access.
* **`REDIS_REPLY_STRING`**:
* A bulk (string) reply. The value of the reply can be accessed using `reply->str`.
The length of this string can be accessed using `reply->len`.
* **`REDIS_REPLY_ARRAY`**:
* A multi bulk reply. The number of elements in the multi bulk reply is stored in
`reply->elements`. Every element in the multi bulk reply is a `redisReply` object as well
and can be accessed via `reply->element[..index..]`.
Redis may reply with nested arrays but this is fully supported.
### RESP3
Hiredis also supports every new `RESP3` data type which are as follows. For more information about the protocol see the `RESP3` [specification.](https://github.com/antirez/RESP3/blob/master/spec.md)
* **`REDIS_REPLY_DOUBLE`**:
* The command replied with a double-precision floating point number.
The value is stored as a string in the `str` member, and can be converted with `strtod` or similar.
* **`REDIS_REPLY_BOOL`**:
* A boolean true/false reply.
The value is stored in the `integer` member and will be either `0` or `1`.
* **`REDIS_REPLY_MAP`**:
* An array with the added invariant that there will always be an even number of elements.
The MAP is functionally equivalent to `REDIS_REPLY_ARRAY` except for the previously mentioned invariant.
* **`REDIS_REPLY_SET`**:
* An array response where each entry is unique.
Like the MAP type, the data is identical to an array response except there are no duplicate values.
* **`REDIS_REPLY_PUSH`**:
* An array that can be generated spontaneously by Redis.
This array response will always contain at least two subelements. The first contains the type of `PUSH` message (e.g. `message`, or `invalidate`), and the second being a sub-array with the `PUSH` payload itself.
* **`REDIS_REPLY_ATTR`**:
* An array structurally identical to a `MAP` but intended as meta-data about a reply.
_As of Redis 6.0.6 this reply type is not used in Redis_
* **`REDIS_REPLY_BIGNUM`**:
* A string representing an arbitrarily large signed or unsigned integer value.
The number will be encoded as a string in the `str` member of `redisReply`.
* **`REDIS_REPLY_VERB`**:
* A verbatim string, intended to be presented to the user without modification.
The string payload is stored in the `str` member, and type data is stored in the `vtype` member (e.g. `txt` for raw text or `md` for markdown).
Replies should be freed using the `freeReplyObject()` function.
Note that this function will take care of freeing sub-reply objects
contained in arrays and nested arrays, so there is no need for the user to
free the sub replies (it is actually harmful and will corrupt the memory).
**Important:** the current version of hiredis (1.0.0) frees replies when the
asynchronous API is used. This means you should not call `freeReplyObject` when
you use this API. The reply is cleaned up by hiredis _after_ the callback
returns. We may introduce a flag to make this configurable in future versions of the library.
### Cleaning up
To disconnect and free the context the following function can be used:
```c
void redisFree(redisContext *c);
```
This function immediately closes the socket and then frees the allocations done in
creating the context.
### Sending commands (cont'd)
Together with `redisCommand`, the function `redisCommandArgv` can be used to issue commands.
It has the following prototype:
```c
void *redisCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
```
It takes the number of arguments `argc`, an array of strings `argv` and the lengths of the
arguments `argvlen`. For convenience, `argvlen` may be set to `NULL` and the function will
use `strlen(3)` on every argument to determine its length. Obviously, when any of the arguments
need to be binary safe, the entire array of lengths `argvlen` should be provided.
The return value has the same semantic as `redisCommand`.
### Pipelining
To explain how Hiredis supports pipelining in a blocking connection, there needs to be
understanding of the internal execution flow.
When any of the functions in the `redisCommand` family is called, Hiredis first formats the
command according to the Redis protocol. The formatted command is then put in the output buffer
of the context. This output buffer is dynamic, so it can hold any number of commands.
After the command is put in the output buffer, `redisGetReply` is called. This function has the
following two execution paths:
1. The input buffer is non-empty:
* Try to parse a single reply from the input buffer and return it
* If no reply could be parsed, continue at *2*
2. The input buffer is empty:
* Write the **entire** output buffer to the socket
* Read from the socket until a single reply could be parsed
The function `redisGetReply` is exported as part of the Hiredis API and can be used when a reply
is expected on the socket. To pipeline commands, the only thing that needs to be done is
filling up the output buffer. For this cause, two commands can be used that are identical
to the `redisCommand` family, apart from not returning a reply:
```c
void redisAppendCommand(redisContext *c, const char *format, ...);
void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
```
After calling either function one or more times, `redisGetReply` can be used to receive the
subsequent replies. The return value for this function is either `REDIS_OK` or `REDIS_ERR`, where
the latter means an error occurred while reading a reply. Just as with the other commands,
the `err` field in the context can be used to find out what the cause of this error is.
The following examples shows a simple pipeline (resulting in only a single call to `write(2)` and
a single call to `read(2)`):
```c
redisReply *reply;
redisAppendCommand(context,"SET foo bar");
redisAppendCommand(context,"GET foo");
redisGetReply(context,(void**)&reply); // reply for SET
freeReplyObject(reply);
redisGetReply(context,(void**)&reply); // reply for GET
freeReplyObject(reply);
```
This API can also be used to implement a blocking subscriber:
```c
reply = redisCommand(context,"SUBSCRIBE foo");
freeReplyObject(reply);
while(redisGetReply(context,(void *)&reply) == REDIS_OK) {
// consume message
freeReplyObject(reply);
}
```
### Errors
When a function call is not successful, depending on the function either `NULL` or `REDIS_ERR` is
returned. The `err` field inside the context will be non-zero and set to one of the
following constants:
* **`REDIS_ERR_IO`**:
There was an I/O error while creating the connection, trying to write
to the socket or read from the socket. If you included `errno.h` in your
application, you can use the global `errno` variable to find out what is
wrong.
* **`REDIS_ERR_EOF`**:
The server closed the connection which resulted in an empty read.
* **`REDIS_ERR_PROTOCOL`**:
There was an error while parsing the protocol.
* **`REDIS_ERR_OTHER`**:
Any other error. Currently, it is only used when a specified hostname to connect
to cannot be resolved.
In every case, the `errstr` field in the context will be set to hold a string representation
of the error.
## Asynchronous API
Hiredis comes with an asynchronous API that works easily with any event library.
Examples are bundled that show using Hiredis with [libev](http://software.schmorp.de/pkg/libev.html)
and [libevent](http://monkey.org/~provos/libevent/).
### Connecting
The function `redisAsyncConnect` can be used to establish a non-blocking connection to
Redis. It returns a pointer to the newly created `redisAsyncContext` struct. The `err` field
should be checked after creation to see if there were errors creating the connection.
Because the connection that will be created is non-blocking, the kernel is not able to
instantly return if the specified host and port is able to accept a connection.
In case of error, it is the caller's responsibility to free the context using `redisAsyncFree()`
*Note: A `redisAsyncContext` is not thread-safe.*
An application function creating a connection might look like this:
```c
void appConnect(myAppData *appData)
{
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
if (c->err) {
printf("Error: %s\n", c->errstr);
// handle error
redisAsyncFree(c);
c = NULL;
} else {
appData->context = c;
appData->connecting = 1;
c->data = appData; /* store application pointer for the callbacks */
redisAsyncSetConnectCallback(c, appOnConnect);
redisAsyncSetDisconnectCallback(c, appOnDisconnect);
}
}
```
The asynchronous context _should_ hold a *connect* callback function that is called when the connection
attempt completes, either successfully or with an error.
It _can_ also hold a *disconnect* callback function that is called when the
connection is disconnected (either because of an error or per user request). Both callbacks should
have the following prototype:
```c
void(const redisAsyncContext *c, int status);
```
On a *connect*, the `status` argument is set to `REDIS_OK` if the connection attempt succeeded. In this
case, the context is ready to accept commands. If it is called with `REDIS_ERR` then the
connection attempt failed. The `err` field in the context can be accessed to find out the cause of the error.
After a failed connection attempt, the context object is automatically freed by the library after calling
the connect callback. This may be a good point to create a new context and retry the connection.
On a disconnect, the `status` argument is set to `REDIS_OK` when disconnection was initiated by the
user, or `REDIS_ERR` when the disconnection was caused by an error. When it is `REDIS_ERR`, the `err`
field in the context can be accessed to find out the cause of the error.
The context object is always freed after the disconnect callback fired. When a reconnect is needed,
the disconnect callback is a good point to do so.
Setting the connect or disconnect callbacks can only be done once per context. For subsequent calls the
api will return `REDIS_ERR`. The function to set the callbacks have the following prototype:
```c
/* Alternatively you can use redisAsyncSetConnectCallbackNC which will be passed a non-const
redisAsyncContext* on invocation (e.g. allowing writes to the privdata member). */
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
```
`ac->data` may be used to pass user data to both callbacks. A typical implementation
might look something like this:
```c
void appOnConnect(redisAsyncContext *c, int status)
{
myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
appData->connecting = 0;
if (status == REDIS_OK) {
appData->connected = 1;
} else {
appData->connected = 0;
appData->err = c->err;
appData->context = NULL; /* avoid stale pointer when callback returns */
}
appAttemptReconnect();
}
void appOnDisconnect(redisAsyncContext *c, int status)
{
myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
appData->connected = 0;
appData->err = c->err;
appData->context = NULL; /* avoid stale pointer when callback returns */
if (status == REDIS_OK) {
appNotifyDisconnectCompleted(mydata);
} else {
appNotifyUnexpectedDisconnect(mydata);
appAttemptReconnect();
}
}
```
### Sending commands and their callbacks
In an asynchronous context, commands are automatically pipelined due to the nature of an event loop.
Therefore, unlike the synchronous API, there is only a single way to send commands.
Because commands are sent to Redis asynchronously, issuing a command requires a callback function
that is called when the reply is received. Reply callbacks should have the following prototype:
```c
void(redisAsyncContext *c, void *reply, void *privdata);
```
The `privdata` argument can be used to curry arbitrary data to the callback from the point where
the command is initially queued for execution.
The functions that can be used to issue commands in an asynchronous context are:
```c
int redisAsyncCommand(
redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
const char *format, ...);
int redisAsyncCommandArgv(
redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
int argc, const char **argv, const size_t *argvlen);
```
Both functions work like their blocking counterparts. The return value is `REDIS_OK` when the command
was successfully added to the output buffer and `REDIS_ERR` otherwise. Example: when the connection
is being disconnected per user-request, no new commands may be added to the output buffer and `REDIS_ERR` is
returned on calls to the `redisAsyncCommand` family.
If the reply for a command with a `NULL` callback is read, it is immediately freed. When the callback
for a command is non-`NULL`, the memory is freed immediately following the callback: the reply is only
valid for the duration of the callback.
All pending callbacks are called with a `NULL` reply when the context encountered an error.
For every command issued, with the exception of **SUBSCRIBE** and **PSUBSCRIBE**, the callback is
called exactly once. Even if the context object id disconnected or deleted, every pending callback
will be called with a `NULL` reply.
For **SUBSCRIBE** and **PSUBSCRIBE**, the callbacks may be called repeatedly until an `unsubscribe`
message arrives. This will be the last invocation of the callback. In case of error, the callbacks
may receive a final `NULL` reply instead.
### Disconnecting
An asynchronous connection can be terminated using:
```c
void redisAsyncDisconnect(redisAsyncContext *ac);
```
When this function is called, the connection is **not** immediately terminated. Instead, new
commands are no longer accepted and the connection is only terminated when all pending commands
have been written to the socket, their respective replies have been read and their respective
callbacks have been executed. After this, the disconnection callback is executed with the
`REDIS_OK` status and the context object is freed.
The connection can be forcefully disconnected using
```c
void redisAsyncFree(redisAsyncContext *ac);
```
In this case, nothing more is written to the socket, all pending callbacks are called with a `NULL`
reply and the disconnection callback is called with `REDIS_OK`, after which the context object
is freed.
### Hooking it up to event library *X*
There are a few hooks that need to be set on the context object after it is created.
See the `adapters/` directory for bindings to *libev* and *libevent*.
## Reply parsing API
Hiredis comes with a reply parsing API that makes it easy for writing higher
level language bindings.
The reply parsing API consists of the following functions:
```c
redisReader *redisReaderCreate(void);
void redisReaderFree(redisReader *reader);
int redisReaderFeed(redisReader *reader, const char *buf, size_t len);
int redisReaderGetReply(redisReader *reader, void **reply);
```
The same set of functions are used internally by hiredis when creating a
normal Redis context, the above API just exposes it to the user for a direct
usage.
### Usage
The function `redisReaderCreate` creates a `redisReader` structure that holds a
buffer with unparsed data and state for the protocol parser.
Incoming data -- most likely from a socket -- can be placed in the internal
buffer of the `redisReader` using `redisReaderFeed`. This function will make a
copy of the buffer pointed to by `buf` for `len` bytes. This data is parsed
when `redisReaderGetReply` is called. This function returns an integer status
and a reply object (as described above) via `void **reply`. The returned status
can be either `REDIS_OK` or `REDIS_ERR`, where the latter means something went
wrong (either a protocol error, or an out of memory error).
The parser limits the level of nesting for multi bulk payloads to 7. If the
multi bulk nesting level is higher than this, the parser returns an error.
### Customizing replies
The function `redisReaderGetReply` creates `redisReply` and makes the function
argument `reply` point to the created `redisReply` variable. For instance, if
the response of type `REDIS_REPLY_STATUS` then the `str` field of `redisReply`
will hold the status as a vanilla C string. However, the functions that are
responsible for creating instances of the `redisReply` can be customized by
setting the `fn` field on the `redisReader` struct. This should be done
immediately after creating the `redisReader`.
For example, [hiredis-rb](https://github.com/pietern/hiredis-rb/blob/master/ext/hiredis_ext/reader.c)
uses customized reply object functions to create Ruby objects.
### Reader max buffer
Both when using the Reader API directly or when using it indirectly via a
normal Redis context, the redisReader structure uses a buffer in order to
accumulate data from the server.
Usually this buffer is destroyed when it is empty and is larger than 16
KiB in order to avoid wasting memory in unused buffers
However when working with very big payloads destroying the buffer may slow
down performances considerably, so it is possible to modify the max size of
an idle buffer changing the value of the `maxbuf` field of the reader structure
to the desired value. The special value of 0 means that there is no maximum
value for an idle buffer, so the buffer will never get freed.
For instance if you have a normal Redis context you can set the maximum idle
buffer to zero (unlimited) just with:
```c
context->reader->maxbuf = 0;
```
This should be done only in order to maximize performances when working with
large payloads. The context should be set back to `REDIS_READER_MAX_BUF` again
as soon as possible in order to prevent allocation of useless memory.
### Reader max array elements
By default the hiredis reply parser sets the maximum number of multi-bulk elements
to 2^32 - 1 or 4,294,967,295 entries. If you need to process multi-bulk replies
with more than this many elements you can set the value higher or to zero, meaning
unlimited with:
```c
context->reader->maxelements = 0;
```
## SSL/TLS Support
### Building
SSL/TLS support is not built by default and requires an explicit flag:
make USE_SSL=1
This requires OpenSSL development package (e.g. including header files to be
available.
When enabled, SSL/TLS support is built into extra `libhiredis_ssl.a` and
`libhiredis_ssl.so` static/dynamic libraries. This leaves the original libraries
unaffected so no additional dependencies are introduced.
### Using it
First, you'll need to make sure you include the SSL header file:
```c
#include <hiredis/hiredis.h>
#include <hiredis/hiredis_ssl.h>
```
You will also need to link against `libhiredis_ssl`, **in addition** to
`libhiredis` and add `-lssl -lcrypto` to satisfy its dependencies.
Hiredis implements SSL/TLS on top of its normal `redisContext` or
`redisAsyncContext`, so you will need to establish a connection first and then
initiate an SSL/TLS handshake.
#### Hiredis OpenSSL Wrappers
Before Hiredis can negotiate an SSL/TLS connection, it is necessary to
initialize OpenSSL and create a context. You can do that in two ways:
1. Work directly with the OpenSSL API to initialize the library's global context
and create `SSL_CTX *` and `SSL *` contexts. With an `SSL *` object you can
call `redisInitiateSSL()`.
2. Work with a set of Hiredis-provided wrappers around OpenSSL, create a
`redisSSLContext` object to hold configuration and use
`redisInitiateSSLWithContext()` to initiate the SSL/TLS handshake.
```c
/* An Hiredis SSL context. It holds SSL configuration and can be reused across
* many contexts.
*/
redisSSLContext *ssl_context;
/* An error variable to indicate what went wrong, if the context fails to
* initialize.
*/
redisSSLContextError ssl_error = REDIS_SSL_CTX_NONE;
/* Initialize global OpenSSL state.
*
* You should call this only once when your app initializes, and only if
* you don't explicitly or implicitly initialize OpenSSL it elsewhere.
*/
redisInitOpenSSL();
/* Create SSL context */
ssl_context = redisCreateSSLContext(
"cacertbundle.crt", /* File name of trusted CA/ca bundle file, optional */
"/path/to/certs", /* Path of trusted certificates, optional */
"client_cert.pem", /* File name of client certificate file, optional */
"client_key.pem", /* File name of client private key, optional */
"redis.mydomain.com", /* Server name to request (SNI), optional */
&ssl_error);
if(ssl_context == NULL || ssl_error != REDIS_SSL_CTX_NONE) {
/* Handle error and abort... */
/* e.g.
printf("SSL error: %s\n",
(ssl_error != REDIS_SSL_CTX_NONE) ?
redisSSLContextGetError(ssl_error) : "Unknown error");
// Abort
*/
}
/* Create Redis context and establish connection */
c = redisConnect("localhost", 6443);
if (c == NULL || c->err) {
/* Handle error and abort... */
}
/* Negotiate SSL/TLS */
if (redisInitiateSSLWithContext(c, ssl_context) != REDIS_OK) {
/* Handle error, in c->err / c->errstr */
}
```
## RESP3 PUSH replies
Redis 6.0 introduced PUSH replies with the reply-type `>`. These messages are generated spontaneously and can arrive at any time, so must be handled using callbacks.
### Default behavior
Hiredis installs handlers on `redisContext` and `redisAsyncContext` by default, which will intercept and free any PUSH replies detected. This means existing code will work as-is after upgrading to Redis 6 and switching to `RESP3`.
### Custom PUSH handler prototypes
The callback prototypes differ between `redisContext` and `redisAsyncContext`.
#### redisContext
```c
void my_push_handler(void *privdata, void *reply) {
/* Handle the reply */
/* Note: We need to free the reply in our custom handler for
blocking contexts. This lets us keep the reply if
we want. */
freeReplyObject(reply);
}
```
#### redisAsyncContext
```c
void my_async_push_handler(redisAsyncContext *ac, void *reply) {
/* Handle the reply */
/* Note: Because async hiredis always frees replies, you should
not call freeReplyObject in an async push callback. */
}
```
### Installing a custom handler
There are two ways to set your own PUSH handlers.
1. Set `push_cb` or `async_push_cb` in the `redisOptions` struct and connect with `redisConnectWithOptions` or `redisAsyncConnectWithOptions`.
```c
redisOptions = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
options->push_cb = my_push_handler;
redisContext *context = redisConnectWithOptions(&options);
```
2. Call `redisSetPushCallback` or `redisAsyncSetPushCallback` on a connected context.
```c
redisContext *context = redisConnect("127.0.0.1", 6379);
redisSetPushCallback(context, my_push_handler);
```
_Note `redisSetPushCallback` and `redisAsyncSetPushCallback` both return any currently configured handler, making it easy to override and then return to the old value._
### Specifying no handler
If you have a unique use-case where you don't want hiredis to automatically intercept and free PUSH replies, you will want to configure no handler at all. This can be done in two ways.
1. Set the `REDIS_OPT_NO_PUSH_AUTOFREE` flag in `redisOptions` and leave the callback function pointer `NULL`.
```c
redisOptions = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
options->options |= REDIS_OPT_NO_PUSH_AUTOFREE;
redisContext *context = redisConnectWithOptions(&options);
```
3. Call `redisSetPushCallback` with `NULL` once connected.
```c
redisContext *context = redisConnect("127.0.0.1", 6379);
redisSetPushCallback(context, NULL);
```
_Note: With no handler configured, calls to `redisCommand` may generate more than one reply, so this strategy is only applicable when there's some kind of blocking `redisGetReply()` loop (e.g. `MONITOR` or `SUBSCRIBE` workloads)._
## Allocator injection
Hiredis uses a pass-thru structure of function pointers defined in [alloc.h](https://github.com/redis/hiredis/blob/f5d25850/alloc.h#L41) that contain the currently configured allocation and deallocation functions. By default they just point to libc (`malloc`, `calloc`, `realloc`, etc).
### Overriding
One can override the allocators like so:
```c
hiredisAllocFuncs myfuncs = {
.mallocFn = my_malloc,
.callocFn = my_calloc,
.reallocFn = my_realloc,
.strdupFn = my_strdup,
.freeFn = my_free,
};
// Override allocators (function returns current allocators if needed)
hiredisAllocFuncs orig = hiredisSetAllocators(&myfuncs);
```
To reset the allocators to their default libc function simply call:
```c
hiredisResetAllocators();
```
## AUTHORS
Salvatore Sanfilippo (antirez at gmail),\
Pieter Noordhuis (pcnoordhuis at gmail)\
Michael Grunder (michael dot grunder at gmail)
_Hiredis is released under the BSD license._
@@ -28,95 +28,89 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef KV_ADAPTERS_AE_H
#define KV_ADAPTERS_AE_H
#include "../async.h"
#include "../cluster.h"
#include "../kv.h"
#include <ae.h>
#ifndef __HIREDIS_AE_H__
#define __HIREDIS_AE_H__
#include <sys/types.h>
#include <ae.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct kvAeEvents {
kvAsyncContext *context;
typedef struct redisAeEvents {
redisAsyncContext *context;
aeEventLoop *loop;
int fd;
int reading, writing;
} kvAeEvents;
} redisAeEvents;
static void kvAeReadEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
((void)el);
((void)fd);
((void)mask);
static void redisAeReadEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
((void)el); ((void)fd); ((void)mask);
kvAeEvents *e = (kvAeEvents *)privdata;
kvAsyncHandleRead(e->context);
redisAeEvents *e = (redisAeEvents*)privdata;
redisAsyncHandleRead(e->context);
}
static void kvAeWriteEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
((void)el);
((void)fd);
((void)mask);
static void redisAeWriteEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
((void)el); ((void)fd); ((void)mask);
kvAeEvents *e = (kvAeEvents *)privdata;
kvAsyncHandleWrite(e->context);
redisAeEvents *e = (redisAeEvents*)privdata;
redisAsyncHandleWrite(e->context);
}
static void kvAeAddRead(void *privdata) {
kvAeEvents *e = (kvAeEvents *)privdata;
static void redisAeAddRead(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
aeEventLoop *loop = e->loop;
if (!e->reading) {
e->reading = 1;
aeCreateFileEvent(loop, e->fd, AE_READABLE, kvAeReadEvent, e);
aeCreateFileEvent(loop,e->fd,AE_READABLE,redisAeReadEvent,e);
}
}
static void kvAeDelRead(void *privdata) {
kvAeEvents *e = (kvAeEvents *)privdata;
static void redisAeDelRead(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
aeEventLoop *loop = e->loop;
if (e->reading) {
e->reading = 0;
aeDeleteFileEvent(loop, e->fd, AE_READABLE);
aeDeleteFileEvent(loop,e->fd,AE_READABLE);
}
}
static void kvAeAddWrite(void *privdata) {
kvAeEvents *e = (kvAeEvents *)privdata;
static void redisAeAddWrite(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
aeEventLoop *loop = e->loop;
if (!e->writing) {
e->writing = 1;
aeCreateFileEvent(loop, e->fd, AE_WRITABLE, kvAeWriteEvent, e);
aeCreateFileEvent(loop,e->fd,AE_WRITABLE,redisAeWriteEvent,e);
}
}
static void kvAeDelWrite(void *privdata) {
kvAeEvents *e = (kvAeEvents *)privdata;
static void redisAeDelWrite(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
aeEventLoop *loop = e->loop;
if (e->writing) {
e->writing = 0;
aeDeleteFileEvent(loop, e->fd, AE_WRITABLE);
aeDeleteFileEvent(loop,e->fd,AE_WRITABLE);
}
}
static void kvAeCleanup(void *privdata) {
kvAeEvents *e = (kvAeEvents *)privdata;
kvAeDelRead(privdata);
kvAeDelWrite(privdata);
vk_free(e);
static void redisAeCleanup(void *privdata) {
redisAeEvents *e = (redisAeEvents*)privdata;
redisAeDelRead(privdata);
redisAeDelWrite(privdata);
hi_free(e);
}
static int kvAeAttach(aeEventLoop *loop, kvAsyncContext *ac) {
kvContext *c = &(ac->c);
kvAeEvents *e;
static int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisAeEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return KV_ERR;
return REDIS_ERR;
/* Create container for context and r/w events */
e = (kvAeEvents *)vk_malloc(sizeof(*e));
e = (redisAeEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return KV_ERR;
return REDIS_ERR;
e->context = ac;
e->loop = loop;
@@ -124,31 +118,13 @@ static int kvAeAttach(aeEventLoop *loop, kvAsyncContext *ac) {
e->reading = e->writing = 0;
/* Register functions to start/stop listening for events */
ac->ev.addRead = kvAeAddRead;
ac->ev.delRead = kvAeDelRead;
ac->ev.addWrite = kvAeAddWrite;
ac->ev.delWrite = kvAeDelWrite;
ac->ev.cleanup = kvAeCleanup;
ac->ev.addRead = redisAeAddRead;
ac->ev.delRead = redisAeDelRead;
ac->ev.addWrite = redisAeAddWrite;
ac->ev.delWrite = redisAeDelWrite;
ac->ev.cleanup = redisAeCleanup;
ac->ev.data = e;
return KV_OK;
return REDIS_OK;
}
/* Internal adapter function with correct function signature. */
static int kvAeAttachAdapter(kvAsyncContext *ac, void *loop) {
return kvAeAttach((aeEventLoop *)loop, ac);
}
KV_UNUSED
static int kvClusterOptionsUseAe(kvClusterOptions *options,
aeEventLoop *loop) {
if (options == NULL || loop == NULL) {
return KV_ERR;
}
options->attach_fn = kvAeAttachAdapter;
options->attach_data = loop;
return KV_OK;
}
#endif /* KV_ADAPTERS_AE_H */
#endif
+156
View File
@@ -0,0 +1,156 @@
#ifndef __HIREDIS_GLIB_H__
#define __HIREDIS_GLIB_H__
#include <glib.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct
{
GSource source;
redisAsyncContext *ac;
GPollFD poll_fd;
} RedisSource;
static void
redis_source_add_read (gpointer data)
{
RedisSource *source = (RedisSource *)data;
g_return_if_fail(source);
source->poll_fd.events |= G_IO_IN;
g_main_context_wakeup(g_source_get_context((GSource *)data));
}
static void
redis_source_del_read (gpointer data)
{
RedisSource *source = (RedisSource *)data;
g_return_if_fail(source);
source->poll_fd.events &= ~G_IO_IN;
g_main_context_wakeup(g_source_get_context((GSource *)data));
}
static void
redis_source_add_write (gpointer data)
{
RedisSource *source = (RedisSource *)data;
g_return_if_fail(source);
source->poll_fd.events |= G_IO_OUT;
g_main_context_wakeup(g_source_get_context((GSource *)data));
}
static void
redis_source_del_write (gpointer data)
{
RedisSource *source = (RedisSource *)data;
g_return_if_fail(source);
source->poll_fd.events &= ~G_IO_OUT;
g_main_context_wakeup(g_source_get_context((GSource *)data));
}
static void
redis_source_cleanup (gpointer data)
{
RedisSource *source = (RedisSource *)data;
g_return_if_fail(source);
redis_source_del_read(source);
redis_source_del_write(source);
/*
* It is not our responsibility to remove ourself from the
* current main loop. However, we will remove the GPollFD.
*/
if (source->poll_fd.fd >= 0) {
g_source_remove_poll((GSource *)data, &source->poll_fd);
source->poll_fd.fd = -1;
}
}
static gboolean
redis_source_prepare (GSource *source,
gint *timeout_)
{
RedisSource *redis = (RedisSource *)source;
*timeout_ = -1;
return !!(redis->poll_fd.events & redis->poll_fd.revents);
}
static gboolean
redis_source_check (GSource *source)
{
RedisSource *redis = (RedisSource *)source;
return !!(redis->poll_fd.events & redis->poll_fd.revents);
}
static gboolean
redis_source_dispatch (GSource *source,
GSourceFunc callback,
gpointer user_data)
{
RedisSource *redis = (RedisSource *)source;
if ((redis->poll_fd.revents & G_IO_OUT)) {
redisAsyncHandleWrite(redis->ac);
redis->poll_fd.revents &= ~G_IO_OUT;
}
if ((redis->poll_fd.revents & G_IO_IN)) {
redisAsyncHandleRead(redis->ac);
redis->poll_fd.revents &= ~G_IO_IN;
}
if (callback) {
return callback(user_data);
}
return TRUE;
}
static void
redis_source_finalize (GSource *source)
{
RedisSource *redis = (RedisSource *)source;
if (redis->poll_fd.fd >= 0) {
g_source_remove_poll(source, &redis->poll_fd);
redis->poll_fd.fd = -1;
}
}
static GSource *
redis_source_new (redisAsyncContext *ac)
{
static GSourceFuncs source_funcs = {
.prepare = redis_source_prepare,
.check = redis_source_check,
.dispatch = redis_source_dispatch,
.finalize = redis_source_finalize,
};
redisContext *c = &ac->c;
RedisSource *source;
g_return_val_if_fail(ac != NULL, NULL);
source = (RedisSource *)g_source_new(&source_funcs, sizeof *source);
if (source == NULL)
return NULL;
source->ac = ac;
source->poll_fd.fd = c->fd;
source->poll_fd.events = 0;
source->poll_fd.revents = 0;
g_source_add_poll((GSource *)source, &source->poll_fd);
ac->ev.addRead = redis_source_add_read;
ac->ev.delRead = redis_source_del_read;
ac->ev.addWrite = redis_source_add_write;
ac->ev.delWrite = redis_source_del_write;
ac->ev.cleanup = redis_source_cleanup;
ac->ev.data = source;
return (GSource *)source;
}
#endif /* __HIREDIS_GLIB_H__ */
+84
View File
@@ -0,0 +1,84 @@
#ifndef __HIREDIS_IVYKIS_H__
#define __HIREDIS_IVYKIS_H__
#include <iv.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct redisIvykisEvents {
redisAsyncContext *context;
struct iv_fd fd;
} redisIvykisEvents;
static void redisIvykisReadEvent(void *arg) {
redisAsyncContext *context = (redisAsyncContext *)arg;
redisAsyncHandleRead(context);
}
static void redisIvykisWriteEvent(void *arg) {
redisAsyncContext *context = (redisAsyncContext *)arg;
redisAsyncHandleWrite(context);
}
static void redisIvykisAddRead(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_set_handler_in(&e->fd, redisIvykisReadEvent);
}
static void redisIvykisDelRead(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_set_handler_in(&e->fd, NULL);
}
static void redisIvykisAddWrite(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_set_handler_out(&e->fd, redisIvykisWriteEvent);
}
static void redisIvykisDelWrite(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_set_handler_out(&e->fd, NULL);
}
static void redisIvykisCleanup(void *privdata) {
redisIvykisEvents *e = (redisIvykisEvents*)privdata;
iv_fd_unregister(&e->fd);
hi_free(e);
}
static int redisIvykisAttach(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisIvykisEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisIvykisEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e->context = ac;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisIvykisAddRead;
ac->ev.delRead = redisIvykisDelRead;
ac->ev.addWrite = redisIvykisAddWrite;
ac->ev.delWrite = redisIvykisDelWrite;
ac->ev.cleanup = redisIvykisCleanup;
ac->ev.data = e;
/* Initialize and install read/write events */
IV_FD_INIT(&e->fd);
e->fd.fd = c->fd;
e->fd.handler_in = redisIvykisReadEvent;
e->fd.handler_out = redisIvykisWriteEvent;
e->fd.handler_err = NULL;
e->fd.cookie = e->context;
iv_fd_register(&e->fd);
return REDIS_OK;
}
#endif
@@ -28,140 +28,138 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef KV_ADAPTERS_LIBEV_H
#define KV_ADAPTERS_LIBEV_H
#include "../async.h"
#include "../cluster.h"
#include "../kv.h"
#include <ev.h>
#ifndef __HIREDIS_LIBEV_H__
#define __HIREDIS_LIBEV_H__
#include <stdlib.h>
#include <sys/types.h>
#include <ev.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct kvLibevEvents {
kvAsyncContext *context;
typedef struct redisLibevEvents {
redisAsyncContext *context;
struct ev_loop *loop;
int reading, writing;
ev_io rev, wev;
ev_timer timer;
} kvLibevEvents;
} redisLibevEvents;
static void kvLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
static void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
#endif
((void)revents);
kvLibevEvents *e = (kvLibevEvents *)watcher->data;
kvAsyncHandleRead(e->context);
redisLibevEvents *e = (redisLibevEvents*)watcher->data;
redisAsyncHandleRead(e->context);
}
static void kvLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
static void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
#endif
((void)revents);
kvLibevEvents *e = (kvLibevEvents *)watcher->data;
kvAsyncHandleWrite(e->context);
redisLibevEvents *e = (redisLibevEvents*)watcher->data;
redisAsyncHandleWrite(e->context);
}
static void kvLibevAddRead(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevAddRead(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (!e->reading) {
e->reading = 1;
ev_io_start(EV_A_ & e->rev);
ev_io_start(EV_A_ &e->rev);
}
}
static void kvLibevDelRead(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevDelRead(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (e->reading) {
e->reading = 0;
ev_io_stop(EV_A_ & e->rev);
ev_io_stop(EV_A_ &e->rev);
}
}
static void kvLibevAddWrite(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevAddWrite(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (!e->writing) {
e->writing = 1;
ev_io_start(EV_A_ & e->wev);
ev_io_start(EV_A_ &e->wev);
}
}
static void kvLibevDelWrite(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevDelWrite(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (e->writing) {
e->writing = 0;
ev_io_stop(EV_A_ & e->wev);
ev_io_stop(EV_A_ &e->wev);
}
}
static void kvLibevStopTimer(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevStopTimer(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
ev_timer_stop(EV_A_ & e->timer);
ev_timer_stop(EV_A_ &e->timer);
}
static void kvLibevCleanup(void *privdata) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
kvLibevDelRead(privdata);
kvLibevDelWrite(privdata);
kvLibevStopTimer(privdata);
vk_free(e);
static void redisLibevCleanup(void *privdata) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
redisLibevDelRead(privdata);
redisLibevDelWrite(privdata);
redisLibevStopTimer(privdata);
hi_free(e);
}
static void kvLibevTimeout(EV_P_ ev_timer *timer, int revents) {
static void redisLibevTimeout(EV_P_ ev_timer *timer, int revents) {
#if EV_MULTIPLICITY
((void)EV_A);
#endif
((void)revents);
kvLibevEvents *e = (kvLibevEvents *)timer->data;
kvAsyncHandleTimeout(e->context);
redisLibevEvents *e = (redisLibevEvents*)timer->data;
redisAsyncHandleTimeout(e->context);
}
static void kvLibevSetTimeout(void *privdata, struct timeval tv) {
kvLibevEvents *e = (kvLibevEvents *)privdata;
static void redisLibevSetTimeout(void *privdata, struct timeval tv) {
redisLibevEvents *e = (redisLibevEvents*)privdata;
#if EV_MULTIPLICITY
struct ev_loop *loop = e->loop;
#endif
if (!ev_is_active(&e->timer)) {
ev_init(&e->timer, kvLibevTimeout);
ev_init(&e->timer, redisLibevTimeout);
e->timer.data = e;
}
e->timer.repeat = tv.tv_sec + tv.tv_usec / 1000000.00;
ev_timer_again(EV_A_ & e->timer);
ev_timer_again(EV_A_ &e->timer);
}
static int kvLibevAttach(EV_P_ kvAsyncContext *ac) {
kvContext *c = &(ac->c);
kvLibevEvents *e;
static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisLibevEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return KV_ERR;
return REDIS_ERR;
/* Create container for context and r/w events */
e = (kvLibevEvents *)vk_calloc(1, sizeof(*e));
e = (redisLibevEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return KV_ERR;
return REDIS_ERR;
e->context = ac;
#if EV_MULTIPLICITY
@@ -173,35 +171,18 @@ static int kvLibevAttach(EV_P_ kvAsyncContext *ac) {
e->wev.data = e;
/* Register functions to start/stop listening for events */
ac->ev.addRead = kvLibevAddRead;
ac->ev.delRead = kvLibevDelRead;
ac->ev.addWrite = kvLibevAddWrite;
ac->ev.delWrite = kvLibevDelWrite;
ac->ev.cleanup = kvLibevCleanup;
ac->ev.scheduleTimer = kvLibevSetTimeout;
ac->ev.addRead = redisLibevAddRead;
ac->ev.delRead = redisLibevDelRead;
ac->ev.addWrite = redisLibevAddWrite;
ac->ev.delWrite = redisLibevDelWrite;
ac->ev.cleanup = redisLibevCleanup;
ac->ev.scheduleTimer = redisLibevSetTimeout;
ac->ev.data = e;
/* Initialize read/write events */
ev_io_init(&e->rev, kvLibevReadEvent, c->fd, EV_READ);
ev_io_init(&e->wev, kvLibevWriteEvent, c->fd, EV_WRITE);
return KV_OK;
ev_io_init(&e->rev,redisLibevReadEvent,c->fd,EV_READ);
ev_io_init(&e->wev,redisLibevWriteEvent,c->fd,EV_WRITE);
return REDIS_OK;
}
/* Internal adapter function with correct function signature. */
static int kvLibevAttachAdapter(kvAsyncContext *ac, void *loop) {
return kvLibevAttach((struct ev_loop *)loop, ac);
}
KV_UNUSED
static int kvClusterOptionsUseLibev(kvClusterOptions *options,
struct ev_loop *loop) {
if (options == NULL || loop == NULL) {
return KV_ERR;
}
options->attach_fn = kvLibevAttachAdapter;
options->attach_data = loop;
return KV_OK;
}
#endif /* KV_ADAPTERS_LIBEV_H */
#endif
+175
View File
@@ -0,0 +1,175 @@
/*
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_LIBEVENT_H__
#define __HIREDIS_LIBEVENT_H__
#include <event2/event.h>
#include "../hiredis.h"
#include "../async.h"
#define REDIS_LIBEVENT_DELETED 0x01
#define REDIS_LIBEVENT_ENTERED 0x02
typedef struct redisLibeventEvents {
redisAsyncContext *context;
struct event *ev;
struct event_base *base;
struct timeval tv;
short flags;
short state;
} redisLibeventEvents;
static void redisLibeventDestroy(redisLibeventEvents *e) {
hi_free(e);
}
static void redisLibeventHandler(evutil_socket_t fd, short event, void *arg) {
((void)fd);
redisLibeventEvents *e = (redisLibeventEvents*)arg;
e->state |= REDIS_LIBEVENT_ENTERED;
#define CHECK_DELETED() if (e->state & REDIS_LIBEVENT_DELETED) {\
redisLibeventDestroy(e);\
return; \
}
if ((event & EV_TIMEOUT) && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleTimeout(e->context);
CHECK_DELETED();
}
if ((event & EV_READ) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleRead(e->context);
CHECK_DELETED();
}
if ((event & EV_WRITE) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleWrite(e->context);
CHECK_DELETED();
}
e->state &= ~REDIS_LIBEVENT_ENTERED;
#undef CHECK_DELETED
}
static void redisLibeventUpdate(void *privdata, short flag, int isRemove) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
const struct timeval *tv = e->tv.tv_sec || e->tv.tv_usec ? &e->tv : NULL;
if (isRemove) {
if ((e->flags & flag) == 0) {
return;
} else {
e->flags &= ~flag;
}
} else {
if (e->flags & flag) {
return;
} else {
e->flags |= flag;
}
}
event_del(e->ev);
event_assign(e->ev, e->base, e->context->c.fd, e->flags | EV_PERSIST,
redisLibeventHandler, privdata);
event_add(e->ev, tv);
}
static void redisLibeventAddRead(void *privdata) {
redisLibeventUpdate(privdata, EV_READ, 0);
}
static void redisLibeventDelRead(void *privdata) {
redisLibeventUpdate(privdata, EV_READ, 1);
}
static void redisLibeventAddWrite(void *privdata) {
redisLibeventUpdate(privdata, EV_WRITE, 0);
}
static void redisLibeventDelWrite(void *privdata) {
redisLibeventUpdate(privdata, EV_WRITE, 1);
}
static void redisLibeventCleanup(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
if (!e) {
return;
}
event_del(e->ev);
event_free(e->ev);
e->ev = NULL;
if (e->state & REDIS_LIBEVENT_ENTERED) {
e->state |= REDIS_LIBEVENT_DELETED;
} else {
redisLibeventDestroy(e);
}
}
static void redisLibeventSetTimeout(void *privdata, struct timeval tv) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
short flags = e->flags;
e->flags = 0;
e->tv = tv;
redisLibeventUpdate(e, flags, 0);
}
static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
redisContext *c = &(ac->c);
redisLibeventEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibeventEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e->context = ac;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisLibeventAddRead;
ac->ev.delRead = redisLibeventDelRead;
ac->ev.addWrite = redisLibeventAddWrite;
ac->ev.delWrite = redisLibeventDelWrite;
ac->ev.cleanup = redisLibeventCleanup;
ac->ev.scheduleTimer = redisLibeventSetTimeout;
ac->ev.data = e;
/* Initialize and install read/write events */
e->ev = event_new(base, c->fd, EV_READ | EV_WRITE, redisLibeventHandler, e);
e->base = base;
return REDIS_OK;
}
#endif
+123
View File
@@ -0,0 +1,123 @@
#ifndef __HIREDIS_LIBHV_H__
#define __HIREDIS_LIBHV_H__
#include <hv/hloop.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct redisLibhvEvents {
hio_t *io;
htimer_t *timer;
} redisLibhvEvents;
static void redisLibhvHandleEvents(hio_t* io) {
redisAsyncContext* context = (redisAsyncContext*)hevent_userdata(io);
int events = hio_events(io);
int revents = hio_revents(io);
if (context && (events & HV_READ) && (revents & HV_READ)) {
redisAsyncHandleRead(context);
}
if (context && (events & HV_WRITE) && (revents & HV_WRITE)) {
redisAsyncHandleWrite(context);
}
}
static void redisLibhvAddRead(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_add(events->io, redisLibhvHandleEvents, HV_READ);
}
static void redisLibhvDelRead(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_del(events->io, HV_READ);
}
static void redisLibhvAddWrite(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_add(events->io, redisLibhvHandleEvents, HV_WRITE);
}
static void redisLibhvDelWrite(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
hio_del(events->io, HV_WRITE);
}
static void redisLibhvCleanup(void *privdata) {
redisLibhvEvents* events = (redisLibhvEvents*)privdata;
if (events->timer)
htimer_del(events->timer);
hio_close(events->io);
hevent_set_userdata(events->io, NULL);
hi_free(events);
}
static void redisLibhvTimeout(htimer_t* timer) {
hio_t* io = (hio_t*)hevent_userdata(timer);
redisAsyncHandleTimeout((redisAsyncContext*)hevent_userdata(io));
}
static void redisLibhvSetTimeout(void *privdata, struct timeval tv) {
redisLibhvEvents* events;
uint32_t millis;
hloop_t* loop;
events = (redisLibhvEvents*)privdata;
millis = tv.tv_sec * 1000 + tv.tv_usec / 1000;
if (millis == 0) {
/* Libhv disallows zero'd timers so treat this as a delete or NO OP */
if (events->timer) {
htimer_del(events->timer);
events->timer = NULL;
}
} else if (events->timer == NULL) {
/* Add new timer */
loop = hevent_loop(events->io);
events->timer = htimer_add(loop, redisLibhvTimeout, millis, 1);
hevent_set_userdata(events->timer, events->io);
} else {
/* Update existing timer */
htimer_reset(events->timer, millis);
}
}
static int redisLibhvAttach(redisAsyncContext* ac, hloop_t* loop) {
redisContext *c = &(ac->c);
redisLibhvEvents *events;
hio_t* io = NULL;
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
/* Create container struct to keep track of our io and any timer */
events = (redisLibhvEvents*)hi_malloc(sizeof(*events));
if (events == NULL) {
return REDIS_ERR;
}
io = hio_get(loop, c->fd);
if (io == NULL) {
hi_free(events);
return REDIS_ERR;
}
hevent_set_userdata(io, ac);
events->io = io;
events->timer = NULL;
ac->ev.addRead = redisLibhvAddRead;
ac->ev.delRead = redisLibhvDelRead;
ac->ev.addWrite = redisLibhvAddWrite;
ac->ev.delWrite = redisLibhvDelWrite;
ac->ev.cleanup = redisLibhvCleanup;
ac->ev.scheduleTimer = redisLibhvSetTimeout;
ac->ev.data = events;
return REDIS_OK;
}
#endif
+177
View File
@@ -0,0 +1,177 @@
#ifndef HIREDIS_LIBSDEVENT_H
#define HIREDIS_LIBSDEVENT_H
#include <systemd/sd-event.h>
#include "../hiredis.h"
#include "../async.h"
#define REDIS_LIBSDEVENT_DELETED 0x01
#define REDIS_LIBSDEVENT_ENTERED 0x02
typedef struct redisLibsdeventEvents {
redisAsyncContext *context;
struct sd_event *event;
struct sd_event_source *fdSource;
struct sd_event_source *timerSource;
int fd;
short flags;
short state;
} redisLibsdeventEvents;
static void redisLibsdeventDestroy(redisLibsdeventEvents *e) {
if (e->fdSource) {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
if (e->timerSource) {
e->timerSource = sd_event_source_disable_unref(e->timerSource);
}
sd_event_unref(e->event);
hi_free(e);
}
static int redisLibsdeventTimeoutHandler(sd_event_source *s, uint64_t usec, void *userdata) {
((void)s);
((void)usec);
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
redisAsyncHandleTimeout(e->context);
return 0;
}
static int redisLibsdeventHandler(sd_event_source *s, int fd, uint32_t event, void *userdata) {
((void)s);
((void)fd);
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->state |= REDIS_LIBSDEVENT_ENTERED;
#define CHECK_DELETED() if (e->state & REDIS_LIBSDEVENT_DELETED) {\
redisLibsdeventDestroy(e);\
return 0; \
}
if ((event & EPOLLIN) && e->context && (e->state & REDIS_LIBSDEVENT_DELETED) == 0) {
redisAsyncHandleRead(e->context);
CHECK_DELETED();
}
if ((event & EPOLLOUT) && e->context && (e->state & REDIS_LIBSDEVENT_DELETED) == 0) {
redisAsyncHandleWrite(e->context);
CHECK_DELETED();
}
e->state &= ~REDIS_LIBSDEVENT_ENTERED;
#undef CHECK_DELETED
return 0;
}
static void redisLibsdeventAddRead(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (e->flags & EPOLLIN) {
return;
}
e->flags |= EPOLLIN;
if (e->flags & EPOLLOUT) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
sd_event_add_io(e->event, &e->fdSource, e->fd, e->flags, redisLibsdeventHandler, e);
}
}
static void redisLibsdeventDelRead(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->flags &= ~EPOLLIN;
if (e->flags) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
}
static void redisLibsdeventAddWrite(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (e->flags & EPOLLOUT) {
return;
}
e->flags |= EPOLLOUT;
if (e->flags & EPOLLIN) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
sd_event_add_io(e->event, &e->fdSource, e->fd, e->flags, redisLibsdeventHandler, e);
}
}
static void redisLibsdeventDelWrite(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
e->flags &= ~EPOLLOUT;
if (e->flags) {
sd_event_source_set_io_events(e->fdSource, e->flags);
} else {
e->fdSource = sd_event_source_disable_unref(e->fdSource);
}
}
static void redisLibsdeventCleanup(void *userdata) {
redisLibsdeventEvents *e = (redisLibsdeventEvents*)userdata;
if (!e) {
return;
}
if (e->state & REDIS_LIBSDEVENT_ENTERED) {
e->state |= REDIS_LIBSDEVENT_DELETED;
} else {
redisLibsdeventDestroy(e);
}
}
static void redisLibsdeventSetTimeout(void *userdata, struct timeval tv) {
redisLibsdeventEvents *e = (redisLibsdeventEvents *)userdata;
uint64_t usec = tv.tv_sec * 1000000 + tv.tv_usec;
if (!e->timerSource) {
sd_event_add_time_relative(e->event, &e->timerSource, CLOCK_MONOTONIC, usec, 1, redisLibsdeventTimeoutHandler, e);
} else {
sd_event_source_set_time_relative(e->timerSource, usec);
}
}
static int redisLibsdeventAttach(redisAsyncContext *ac, struct sd_event *event) {
redisContext *c = &(ac->c);
redisLibsdeventEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibsdeventEvents*)hi_calloc(1, sizeof(*e));
if (e == NULL)
return REDIS_ERR;
/* Initialize and increase event refcount */
e->context = ac;
e->event = event;
e->fd = c->fd;
sd_event_ref(event);
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisLibsdeventAddRead;
ac->ev.delRead = redisLibsdeventDelRead;
ac->ev.addWrite = redisLibsdeventAddWrite;
ac->ev.delWrite = redisLibsdeventDelWrite;
ac->ev.cleanup = redisLibsdeventCleanup;
ac->ev.scheduleTimer = redisLibsdeventSetTimeout;
ac->ev.data = e;
return REDIS_OK;
}
#endif
+171
View File
@@ -0,0 +1,171 @@
#ifndef __HIREDIS_LIBUV_H__
#define __HIREDIS_LIBUV_H__
#include <stdlib.h>
#include <uv.h>
#include "../hiredis.h"
#include "../async.h"
#include <string.h>
typedef struct redisLibuvEvents {
redisAsyncContext* context;
uv_poll_t handle;
uv_timer_t timer;
int events;
} redisLibuvEvents;
static void redisLibuvPoll(uv_poll_t* handle, int status, int events) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
int ev = (status ? p->events : events);
if (p->context != NULL && (ev & UV_READABLE)) {
redisAsyncHandleRead(p->context);
}
if (p->context != NULL && (ev & UV_WRITABLE)) {
redisAsyncHandleWrite(p->context);
}
}
static void redisLibuvAddRead(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
if (p->events & UV_READABLE) {
return;
}
p->events |= UV_READABLE;
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
}
static void redisLibuvDelRead(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->events &= ~UV_READABLE;
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
}
static void redisLibuvAddWrite(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
if (p->events & UV_WRITABLE) {
return;
}
p->events |= UV_WRITABLE;
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
}
static void redisLibuvDelWrite(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->events &= ~UV_WRITABLE;
if (p->events) {
uv_poll_start(&p->handle, p->events, redisLibuvPoll);
} else {
uv_poll_stop(&p->handle);
}
}
static void on_timer_close(uv_handle_t *handle) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
p->timer.data = NULL;
if (!p->handle.data) {
// both timer and handle are closed
hi_free(p);
}
// else, wait for `on_handle_close`
}
static void on_handle_close(uv_handle_t *handle) {
redisLibuvEvents* p = (redisLibuvEvents*)handle->data;
p->handle.data = NULL;
if (!p->timer.data) {
// timer never started, or timer already destroyed
hi_free(p);
}
// else, wait for `on_timer_close`
}
// libuv removed `status` parameter since v0.11.23
// see: https://github.com/libuv/libuv/blob/v0.11.23/include/uv.h
#if (UV_VERSION_MAJOR == 0 && UV_VERSION_MINOR < 11) || \
(UV_VERSION_MAJOR == 0 && UV_VERSION_MINOR == 11 && UV_VERSION_PATCH < 23)
static void redisLibuvTimeout(uv_timer_t *timer, int status) {
(void)status; // unused
#else
static void redisLibuvTimeout(uv_timer_t *timer) {
#endif
redisLibuvEvents *e = (redisLibuvEvents*)timer->data;
redisAsyncHandleTimeout(e->context);
}
static void redisLibuvSetTimeout(void *privdata, struct timeval tv) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
uint64_t millsec = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
if (!p->timer.data) {
// timer is uninitialized
if (uv_timer_init(p->handle.loop, &p->timer) != 0) {
return;
}
p->timer.data = p;
}
// updates the timeout if the timer has already started
// or start the timer
uv_timer_start(&p->timer, redisLibuvTimeout, millsec, 0);
}
static void redisLibuvCleanup(void *privdata) {
redisLibuvEvents* p = (redisLibuvEvents*)privdata;
p->context = NULL; // indicate that context might no longer exist
if (p->timer.data) {
uv_close((uv_handle_t*)&p->timer, on_timer_close);
}
uv_close((uv_handle_t*)&p->handle, on_handle_close);
}
static int redisLibuvAttach(redisAsyncContext* ac, uv_loop_t* loop) {
redisContext *c = &(ac->c);
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
ac->ev.addRead = redisLibuvAddRead;
ac->ev.delRead = redisLibuvDelRead;
ac->ev.addWrite = redisLibuvAddWrite;
ac->ev.delWrite = redisLibuvDelWrite;
ac->ev.cleanup = redisLibuvCleanup;
ac->ev.scheduleTimer = redisLibuvSetTimeout;
redisLibuvEvents* p = (redisLibuvEvents*)hi_malloc(sizeof(*p));
if (p == NULL)
return REDIS_ERR;
memset(p, 0, sizeof(*p));
if (uv_poll_init_socket(loop, &p->handle, c->fd) != 0) {
return REDIS_ERR;
}
ac->ev.data = p;
p->handle.data = p;
p->context = ac;
return REDIS_OK;
}
#endif
+115
View File
@@ -0,0 +1,115 @@
//
// Created by Дмитрий Бахвалов on 13.07.15.
// Copyright (c) 2015 Dmitry Bakhvalov. All rights reserved.
//
#ifndef __HIREDIS_MACOSX_H__
#define __HIREDIS_MACOSX_H__
#include <CoreFoundation/CoreFoundation.h>
#include "../hiredis.h"
#include "../async.h"
typedef struct {
redisAsyncContext *context;
CFSocketRef socketRef;
CFRunLoopSourceRef sourceRef;
} RedisRunLoop;
static int freeRedisRunLoop(RedisRunLoop* redisRunLoop) {
if( redisRunLoop != NULL ) {
if( redisRunLoop->sourceRef != NULL ) {
CFRunLoopSourceInvalidate(redisRunLoop->sourceRef);
CFRelease(redisRunLoop->sourceRef);
}
if( redisRunLoop->socketRef != NULL ) {
CFSocketInvalidate(redisRunLoop->socketRef);
CFRelease(redisRunLoop->socketRef);
}
hi_free(redisRunLoop);
}
return REDIS_ERR;
}
static void redisMacOSAddRead(void *privdata) {
RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
CFSocketEnableCallBacks(redisRunLoop->socketRef, kCFSocketReadCallBack);
}
static void redisMacOSDelRead(void *privdata) {
RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
CFSocketDisableCallBacks(redisRunLoop->socketRef, kCFSocketReadCallBack);
}
static void redisMacOSAddWrite(void *privdata) {
RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
CFSocketEnableCallBacks(redisRunLoop->socketRef, kCFSocketWriteCallBack);
}
static void redisMacOSDelWrite(void *privdata) {
RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
CFSocketDisableCallBacks(redisRunLoop->socketRef, kCFSocketWriteCallBack);
}
static void redisMacOSCleanup(void *privdata) {
RedisRunLoop *redisRunLoop = (RedisRunLoop*)privdata;
freeRedisRunLoop(redisRunLoop);
}
static void redisMacOSAsyncCallback(CFSocketRef __unused s, CFSocketCallBackType callbackType, CFDataRef __unused address, const void __unused *data, void *info) {
redisAsyncContext* context = (redisAsyncContext*) info;
switch (callbackType) {
case kCFSocketReadCallBack:
redisAsyncHandleRead(context);
break;
case kCFSocketWriteCallBack:
redisAsyncHandleWrite(context);
break;
default:
break;
}
}
static int redisMacOSAttach(redisAsyncContext *redisAsyncCtx, CFRunLoopRef runLoop) {
redisContext *redisCtx = &(redisAsyncCtx->c);
/* Nothing should be attached when something is already attached */
if( redisAsyncCtx->ev.data != NULL ) return REDIS_ERR;
RedisRunLoop* redisRunLoop = (RedisRunLoop*) hi_calloc(1, sizeof(RedisRunLoop));
if (redisRunLoop == NULL)
return REDIS_ERR;
/* Setup redis stuff */
redisRunLoop->context = redisAsyncCtx;
redisAsyncCtx->ev.addRead = redisMacOSAddRead;
redisAsyncCtx->ev.delRead = redisMacOSDelRead;
redisAsyncCtx->ev.addWrite = redisMacOSAddWrite;
redisAsyncCtx->ev.delWrite = redisMacOSDelWrite;
redisAsyncCtx->ev.cleanup = redisMacOSCleanup;
redisAsyncCtx->ev.data = redisRunLoop;
/* Initialize and install read/write events */
CFSocketContext socketCtx = { 0, redisAsyncCtx, NULL, NULL, NULL };
redisRunLoop->socketRef = CFSocketCreateWithNative(NULL, redisCtx->fd,
kCFSocketReadCallBack | kCFSocketWriteCallBack,
redisMacOSAsyncCallback,
&socketCtx);
if( !redisRunLoop->socketRef ) return freeRedisRunLoop(redisRunLoop);
redisRunLoop->sourceRef = CFSocketCreateRunLoopSource(NULL, redisRunLoop->socketRef, 0);
if( !redisRunLoop->sourceRef ) return freeRedisRunLoop(redisRunLoop);
CFRunLoopAddSource(runLoop, redisRunLoop->sourceRef, kCFRunLoopDefaultMode);
return REDIS_OK;
}
#endif
@@ -1,43 +1,41 @@
#ifndef KV_ADAPTERS_POLL_H
#define KV_ADAPTERS_POLL_H
#ifndef HIREDIS_POLL_H
#define HIREDIS_POLL_H
#include "../async.h"
#include "../cluster.h"
#include "../sockcompat.h"
#include <errno.h>
#include <string.h> // for memset
#include <errno.h>
/* Values to return from kvPollTick */
#define KV_POLL_HANDLED_READ 1
#define KV_POLL_HANDLED_WRITE 2
#define KV_POLL_HANDLED_TIMEOUT 4
/* Values to return from redisPollTick */
#define REDIS_POLL_HANDLED_READ 1
#define REDIS_POLL_HANDLED_WRITE 2
#define REDIS_POLL_HANDLED_TIMEOUT 4
/* An adapter to allow manual polling of the async context by checking the state
* of the underlying file descriptor. Useful in cases where there is no formal
* IO event loop but regular ticking can be used, such as in game engines. */
typedef struct kvPollEvents {
kvAsyncContext *context;
kvFD fd;
typedef struct redisPollEvents {
redisAsyncContext *context;
redisFD fd;
char reading, writing;
char in_tick;
char deleted;
double deadline;
} kvPollEvents;
} redisPollEvents;
static double kvPollTimevalToDouble(struct timeval *tv) {
static double redisPollTimevalToDouble(struct timeval *tv) {
if (tv == NULL)
return 0.0;
return tv->tv_sec + tv->tv_usec / 1000000.00;
}
static double kvPollGetNow(void) {
static double redisPollGetNow(void) {
#ifndef _MSC_VER
struct timeval tv;
gettimeofday(&tv, NULL);
return kvPollTimevalToDouble(&tv);
gettimeofday(&tv,NULL);
return redisPollTimevalToDouble(&tv);
#else
FILETIME ft;
ULARGE_INTEGER li;
@@ -51,14 +49,14 @@ static double kvPollGetNow(void) {
/* Poll for io, handling any pending callbacks. The timeout argument can be
* positive to wait for a maximum given time for IO, zero to poll, or negative
* to wait forever */
static int kvPollTick(kvAsyncContext *ac, double timeout) {
static int redisPollTick(redisAsyncContext *ac, double timeout) {
int reading, writing;
struct pollfd pfd;
int handled;
int ns;
int itimeout;
kvPollEvents *e = (kvPollEvents *)ac->ev.data;
redisPollEvents *e = (redisPollEvents*)ac->ev.data;
if (!e)
return 0;
@@ -71,7 +69,7 @@ static int kvPollTick(kvAsyncContext *ac, double timeout) {
pfd.fd = e->fd;
pfd.events = 0;
if (reading)
pfd.events = POLLIN;
pfd.events = POLLIN;
if (writing)
pfd.events |= POLLOUT;
@@ -88,94 +86,95 @@ static int kvPollTick(kvAsyncContext *ac, double timeout) {
return ns;
ns = 0;
}
handled = 0;
e->in_tick = 1;
if (ns) {
if (reading && (pfd.revents & POLLIN)) {
kvAsyncHandleRead(ac);
handled |= KV_POLL_HANDLED_READ;
redisAsyncHandleRead(ac);
handled |= REDIS_POLL_HANDLED_READ;
}
/* on Windows, connection failure is indicated with the Exception fdset.
* handle it the same as writable. */
if (writing && (pfd.revents & (POLLOUT | POLLERR))) {
/* context Read callback may have caused context to be deleted, e.g.
by doing a kvAsyncDisconnect() */
by doing an redisAsyncDisconnect() */
if (!e->deleted) {
kvAsyncHandleWrite(ac);
handled |= KV_POLL_HANDLED_WRITE;
redisAsyncHandleWrite(ac);
handled |= REDIS_POLL_HANDLED_WRITE;
}
}
}
/* perform timeouts */
if (!e->deleted && e->deadline != 0.0) {
double now = kvPollGetNow();
double now = redisPollGetNow();
if (now >= e->deadline) {
/* deadline has passed. disable timeout and perform callback */
e->deadline = 0.0;
kvAsyncHandleTimeout(ac);
handled |= KV_POLL_HANDLED_TIMEOUT;
redisAsyncHandleTimeout(ac);
handled |= REDIS_POLL_HANDLED_TIMEOUT;
}
}
/* do a delayed cleanup if required */
if (e->deleted)
vk_free(e);
hi_free(e);
else
e->in_tick = 0;
return handled;
}
static void kvPollAddRead(void *data) {
kvPollEvents *e = (kvPollEvents *)data;
static void redisPollAddRead(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->reading = 1;
}
static void kvPollDelRead(void *data) {
kvPollEvents *e = (kvPollEvents *)data;
static void redisPollDelRead(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->reading = 0;
}
static void kvPollAddWrite(void *data) {
kvPollEvents *e = (kvPollEvents *)data;
static void redisPollAddWrite(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->writing = 1;
}
static void kvPollDelWrite(void *data) {
kvPollEvents *e = (kvPollEvents *)data;
static void redisPollDelWrite(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
e->writing = 0;
}
static void kvPollCleanup(void *data) {
kvPollEvents *e = (kvPollEvents *)data;
static void redisPollCleanup(void *data) {
redisPollEvents *e = (redisPollEvents*)data;
/* if we are currently processing a tick, postpone deletion */
if (e->in_tick)
e->deleted = 1;
else
vk_free(e);
hi_free(e);
}
static void kvPollScheduleTimer(void *data, struct timeval tv) {
kvPollEvents *e = (kvPollEvents *)data;
double now = kvPollGetNow();
e->deadline = now + kvPollTimevalToDouble(&tv);
static void redisPollScheduleTimer(void *data, struct timeval tv)
{
redisPollEvents *e = (redisPollEvents*)data;
double now = redisPollGetNow();
e->deadline = now + redisPollTimevalToDouble(&tv);
}
static int kvPollAttach(kvAsyncContext *ac) {
kvContext *c = &(ac->c);
kvPollEvents *e;
static int redisPollAttach(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisPollEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return KV_ERR;
return REDIS_ERR;
/* Create container for context and r/w events */
e = (kvPollEvents *)vk_malloc(sizeof(*e));
e = (redisPollEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return KV_ERR;
return REDIS_ERR;
memset(e, 0, sizeof(*e));
e->context = ac;
@@ -185,30 +184,14 @@ static int kvPollAttach(kvAsyncContext *ac) {
e->deadline = 0.0;
/* Register functions to start/stop listening for events */
ac->ev.addRead = kvPollAddRead;
ac->ev.delRead = kvPollDelRead;
ac->ev.addWrite = kvPollAddWrite;
ac->ev.delWrite = kvPollDelWrite;
ac->ev.scheduleTimer = kvPollScheduleTimer;
ac->ev.cleanup = kvPollCleanup;
ac->ev.addRead = redisPollAddRead;
ac->ev.delRead = redisPollDelRead;
ac->ev.addWrite = redisPollAddWrite;
ac->ev.delWrite = redisPollDelWrite;
ac->ev.scheduleTimer = redisPollScheduleTimer;
ac->ev.cleanup = redisPollCleanup;
ac->ev.data = e;
return KV_OK;
return REDIS_OK;
}
/* Internal adapter function with correct function signature. */
static int kvPollAttachAdapter(kvAsyncContext *ac, KV_UNUSED void *unused) {
return kvPollAttach(ac);
}
KV_UNUSED
static int kvClusterOptionsUsePoll(kvClusterOptions *options) {
if (options == NULL) {
return KV_ERR;
}
options->attach_fn = kvPollAttachAdapter;
return KV_OK;
}
#endif /* KV_ADAPTERS_POLL_H */
#endif /* HIREDIS_POLL_H */
+135
View File
@@ -0,0 +1,135 @@
/*-
* Copyright (C) 2014 Pietro Cerutti <gahr@gahr.ch>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#ifndef __HIREDIS_QT_H__
#define __HIREDIS_QT_H__
#include <QSocketNotifier>
#include "../async.h"
static void RedisQtAddRead(void *);
static void RedisQtDelRead(void *);
static void RedisQtAddWrite(void *);
static void RedisQtDelWrite(void *);
static void RedisQtCleanup(void *);
class RedisQtAdapter : public QObject {
Q_OBJECT
friend
void RedisQtAddRead(void * adapter) {
RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
a->addRead();
}
friend
void RedisQtDelRead(void * adapter) {
RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
a->delRead();
}
friend
void RedisQtAddWrite(void * adapter) {
RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
a->addWrite();
}
friend
void RedisQtDelWrite(void * adapter) {
RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
a->delWrite();
}
friend
void RedisQtCleanup(void * adapter) {
RedisQtAdapter * a = static_cast<RedisQtAdapter *>(adapter);
a->cleanup();
}
public:
RedisQtAdapter(QObject * parent = 0)
: QObject(parent), m_ctx(0), m_read(0), m_write(0) { }
~RedisQtAdapter() {
if (m_ctx != 0) {
m_ctx->ev.data = NULL;
}
}
int setContext(redisAsyncContext * ac) {
if (ac->ev.data != NULL) {
return REDIS_ERR;
}
m_ctx = ac;
m_ctx->ev.data = this;
m_ctx->ev.addRead = RedisQtAddRead;
m_ctx->ev.delRead = RedisQtDelRead;
m_ctx->ev.addWrite = RedisQtAddWrite;
m_ctx->ev.delWrite = RedisQtDelWrite;
m_ctx->ev.cleanup = RedisQtCleanup;
return REDIS_OK;
}
private:
void addRead() {
if (m_read) return;
m_read = new QSocketNotifier(m_ctx->c.fd, QSocketNotifier::Read, 0);
connect(m_read, SIGNAL(activated(int)), this, SLOT(read()));
}
void delRead() {
if (!m_read) return;
delete m_read;
m_read = 0;
}
void addWrite() {
if (m_write) return;
m_write = new QSocketNotifier(m_ctx->c.fd, QSocketNotifier::Write, 0);
connect(m_write, SIGNAL(activated(int)), this, SLOT(write()));
}
void delWrite() {
if (!m_write) return;
delete m_write;
m_write = 0;
}
void cleanup() {
delRead();
delWrite();
}
private slots:
void read() { redisAsyncHandleRead(m_ctx); }
void write() { redisAsyncHandleWrite(m_ctx); }
private:
redisAsyncContext * m_ctx;
QSocketNotifier * m_read;
QSocketNotifier * m_write;
};
#endif /* !__HIREDIS_QT_H__ */
+144
View File
@@ -0,0 +1,144 @@
#ifndef __HIREDIS_REDISMODULEAPI_H__
#define __HIREDIS_REDISMODULEAPI_H__
#include "redismodule.h"
#include "../async.h"
#include "../hiredis.h"
#include <sys/types.h>
typedef struct redisModuleEvents {
redisAsyncContext *context;
RedisModuleCtx *module_ctx;
int fd;
int reading, writing;
int timer_active;
RedisModuleTimerID timer_id;
} redisModuleEvents;
static inline void redisModuleReadEvent(int fd, void *privdata, int mask) {
(void) fd;
(void) mask;
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisAsyncHandleRead(e->context);
}
static inline void redisModuleWriteEvent(int fd, void *privdata, int mask) {
(void) fd;
(void) mask;
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisAsyncHandleWrite(e->context);
}
static inline void redisModuleAddRead(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (!e->reading) {
e->reading = 1;
RedisModule_EventLoopAdd(e->fd, REDISMODULE_EVENTLOOP_READABLE, redisModuleReadEvent, e);
}
}
static inline void redisModuleDelRead(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->reading) {
e->reading = 0;
RedisModule_EventLoopDel(e->fd, REDISMODULE_EVENTLOOP_READABLE);
}
}
static inline void redisModuleAddWrite(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (!e->writing) {
e->writing = 1;
RedisModule_EventLoopAdd(e->fd, REDISMODULE_EVENTLOOP_WRITABLE, redisModuleWriteEvent, e);
}
}
static inline void redisModuleDelWrite(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->writing) {
e->writing = 0;
RedisModule_EventLoopDel(e->fd, REDISMODULE_EVENTLOOP_WRITABLE);
}
}
static inline void redisModuleStopTimer(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
if (e->timer_active) {
RedisModule_StopTimer(e->module_ctx, e->timer_id, NULL);
}
e->timer_active = 0;
}
static inline void redisModuleCleanup(void *privdata) {
redisModuleEvents *e = (redisModuleEvents*)privdata;
redisModuleDelRead(privdata);
redisModuleDelWrite(privdata);
redisModuleStopTimer(privdata);
hi_free(e);
}
static inline void redisModuleTimeout(RedisModuleCtx *ctx, void *privdata) {
(void) ctx;
redisModuleEvents *e = (redisModuleEvents*)privdata;
e->timer_active = 0;
redisAsyncHandleTimeout(e->context);
}
static inline void redisModuleSetTimeout(void *privdata, struct timeval tv) {
redisModuleEvents* e = (redisModuleEvents*)privdata;
redisModuleStopTimer(privdata);
mstime_t millis = tv.tv_sec * 1000 + tv.tv_usec / 1000.0;
e->timer_id = RedisModule_CreateTimer(e->module_ctx, millis, redisModuleTimeout, e);
e->timer_active = 1;
}
/* Check if Redis version is compatible with the adapter. */
static inline int redisModuleCompatibilityCheck(void) {
if (!RedisModule_EventLoopAdd ||
!RedisModule_EventLoopDel ||
!RedisModule_CreateTimer ||
!RedisModule_StopTimer) {
return REDIS_ERR;
}
return REDIS_OK;
}
static inline int redisModuleAttach(redisAsyncContext *ac, RedisModuleCtx *module_ctx) {
redisContext *c = &(ac->c);
redisModuleEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisModuleEvents*)hi_malloc(sizeof(*e));
if (e == NULL)
return REDIS_ERR;
e->context = ac;
e->module_ctx = module_ctx;
e->fd = c->fd;
e->reading = e->writing = 0;
e->timer_active = 0;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisModuleAddRead;
ac->ev.delRead = redisModuleDelRead;
ac->ev.addWrite = redisModuleAddWrite;
ac->ev.delWrite = redisModuleDelWrite;
ac->ev.cleanup = redisModuleCleanup;
ac->ev.scheduleTimer = redisModuleSetTimeout;
ac->ev.data = e;
return REDIS_OK;
}
#endif
+18 -20
View File
@@ -29,13 +29,11 @@
*/
#include "fmacros.h"
#include "alloc.h"
#include <stdlib.h>
#include <string.h>
#include <stdlib.h>
kvAllocFuncs kvAllocFns = {
hiredisAllocFuncs hiredisAllocFns = {
.mallocFn = malloc,
.callocFn = calloc,
.reallocFn = realloc,
@@ -43,18 +41,18 @@ kvAllocFuncs kvAllocFns = {
.freeFn = free,
};
/* Override kv' allocators with ones supplied by the user */
kvAllocFuncs kvSetAllocators(kvAllocFuncs *override) {
kvAllocFuncs orig = kvAllocFns;
/* Override hiredis' allocators with ones supplied by the user */
hiredisAllocFuncs hiredisSetAllocators(hiredisAllocFuncs *override) {
hiredisAllocFuncs orig = hiredisAllocFns;
kvAllocFns = *override;
hiredisAllocFns = *override;
return orig;
}
/* Reset allocators to use libc defaults */
void kvResetAllocators(void) {
kvAllocFns = (kvAllocFuncs){
void hiredisResetAllocators(void) {
hiredisAllocFns = (hiredisAllocFuncs) {
.mallocFn = malloc,
.callocFn = calloc,
.reallocFn = realloc,
@@ -65,28 +63,28 @@ void kvResetAllocators(void) {
#ifdef _WIN32
void *vk_malloc(size_t size) {
return kvAllocFns.mallocFn(size);
void *hi_malloc(size_t size) {
return hiredisAllocFns.mallocFn(size);
}
void *vk_calloc(size_t nmemb, size_t size) {
void *hi_calloc(size_t nmemb, size_t size) {
/* Overflow check as the user can specify any arbitrary allocator */
if (SIZE_MAX / size < nmemb)
return NULL;
return kvAllocFns.callocFn(nmemb, size);
return hiredisAllocFns.callocFn(nmemb, size);
}
void *vk_realloc(void *ptr, size_t size) {
return kvAllocFns.reallocFn(ptr, size);
void *hi_realloc(void *ptr, size_t size) {
return hiredisAllocFns.reallocFn(ptr, size);
}
char *vk_strdup(const char *str) {
return kvAllocFns.strdupFn(str);
char *hi_strdup(const char *str) {
return hiredisAllocFns.strdupFn(str);
}
void vk_free(void *ptr) {
kvAllocFns.freeFn(ptr);
void hi_free(void *ptr) {
hiredisAllocFns.freeFn(ptr);
}
#endif
@@ -28,9 +28,8 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef KV_ALLOC_H
#define KV_ALLOC_H
#include "visibility.h"
#ifndef HIREDIS_ALLOC_H
#define HIREDIS_ALLOC_H
#include <stddef.h> /* for size_t */
#include <stdint.h>
@@ -40,53 +39,53 @@ extern "C" {
#endif
/* Structure pointing to our actually configured allocators */
typedef struct kvAllocFuncs {
typedef struct hiredisAllocFuncs {
void *(*mallocFn)(size_t);
void *(*callocFn)(size_t, size_t);
void *(*reallocFn)(void *, size_t);
char *(*strdupFn)(const char *);
void (*freeFn)(void *);
} kvAllocFuncs;
void *(*callocFn)(size_t,size_t);
void *(*reallocFn)(void*,size_t);
char *(*strdupFn)(const char*);
void (*freeFn)(void*);
} hiredisAllocFuncs;
LIBKV_API kvAllocFuncs kvSetAllocators(kvAllocFuncs *fns);
LIBKV_API void kvResetAllocators(void);
hiredisAllocFuncs hiredisSetAllocators(hiredisAllocFuncs *ha);
void hiredisResetAllocators(void);
#ifndef _WIN32
/* kv' configured allocator function pointer struct */
LIBKV_API extern kvAllocFuncs kvAllocFns;
/* Hiredis' configured allocator function pointer struct */
extern hiredisAllocFuncs hiredisAllocFns;
static inline void *vk_malloc(size_t size) {
return kvAllocFns.mallocFn(size);
static inline void *hi_malloc(size_t size) {
return hiredisAllocFns.mallocFn(size);
}
static inline void *vk_calloc(size_t nmemb, size_t size) {
static inline void *hi_calloc(size_t nmemb, size_t size) {
/* Overflow check as the user can specify any arbitrary allocator */
if (SIZE_MAX / size < nmemb)
return NULL;
return kvAllocFns.callocFn(nmemb, size);
return hiredisAllocFns.callocFn(nmemb, size);
}
static inline void *vk_realloc(void *ptr, size_t size) {
return kvAllocFns.reallocFn(ptr, size);
static inline void *hi_realloc(void *ptr, size_t size) {
return hiredisAllocFns.reallocFn(ptr, size);
}
static inline char *vk_strdup(const char *str) {
return kvAllocFns.strdupFn(str);
static inline char *hi_strdup(const char *str) {
return hiredisAllocFns.strdupFn(str);
}
static inline void vk_free(void *ptr) {
kvAllocFns.freeFn(ptr);
static inline void hi_free(void *ptr) {
hiredisAllocFns.freeFn(ptr);
}
#else
LIBKV_API void *vk_malloc(size_t size);
LIBKV_API void *vk_calloc(size_t nmemb, size_t size);
LIBKV_API void *vk_realloc(void *ptr, size_t size);
LIBKV_API char *vk_strdup(const char *str);
LIBKV_API void vk_free(void *ptr);
void *hi_malloc(size_t size);
void *hi_calloc(size_t nmemb, size_t size);
void *hi_realloc(void *ptr, size_t size);
char *hi_strdup(const char *str);
void hi_free(void *ptr);
#endif
@@ -94,4 +93,4 @@ LIBKV_API void vk_free(void *ptr);
}
#endif
#endif /* KV_ALLOC_H */
#endif /* HIREDIS_ALLOC_H */
+24
View File
@@ -0,0 +1,24 @@
# Appveyor configuration file for CI build of hiredis on Windows (under Cygwin)
environment:
matrix:
- CYG_BASH: C:\cygwin64\bin\bash
CC: gcc
- CYG_BASH: C:\cygwin\bin\bash
CC: gcc
CFLAGS: -m32
CXXFLAGS: -m32
LDFLAGS: -m32
clone_depth: 1
# Attempt to ensure we don't try to convert line endings to Win32 CRLF as this will cause build to fail
init:
- git config --global core.autocrlf input
# Install needed build dependencies
install:
- '%CYG_BASH% -lc "cygcheck -dc cygwin"'
build_script:
- 'echo building...'
- '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; mkdir build && cd build && cmake .. -G \"Unix Makefiles\" && make VERBOSE=1"'
+1034
View File
File diff suppressed because it is too large Load Diff
@@ -29,56 +29,48 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef KV_ASYNC_H
#define KV_ASYNC_H
#include "kv.h"
#include "visibility.h"
#ifndef __HIREDIS_ASYNC_H
#define __HIREDIS_ASYNC_H
#include "hiredis.h"
#ifdef __cplusplus
extern "C" {
#endif
/* For the async cluster attach functions. */
#if defined(__GNUC__) || defined(__clang__)
#define KV_UNUSED __attribute__((unused))
#else
#define KV_UNUSED
#endif
struct kvAsyncContext; /* need forward declaration of kvAsyncContext */
struct dict; /* dictionary header is included in async.c */
struct redisAsyncContext; /* need forward declaration of redisAsyncContext */
struct dict; /* dictionary header is included in async.c */
/* Reply callback prototype and container */
typedef void(kvCallbackFn)(struct kvAsyncContext *, void *, void *);
typedef struct kvCallback {
struct kvCallback *next; /* simple singly linked list */
kvCallbackFn *fn;
typedef void (redisCallbackFn)(struct redisAsyncContext*, void*, void*);
typedef struct redisCallback {
struct redisCallback *next; /* simple singly linked list */
redisCallbackFn *fn;
int pending_subs;
int unsubscribe_sent;
void *privdata;
int subscribed;
} kvCallback;
} redisCallback;
/* List of callbacks for either regular replies or pub/sub */
typedef struct kvCallbackList {
kvCallback *head, *tail;
} kvCallbackList;
typedef struct redisCallbackList {
redisCallback *head, *tail;
} redisCallbackList;
/* Connection callback prototypes */
typedef void(kvDisconnectCallback)(const struct kvAsyncContext *, int status);
typedef void(kvConnectCallback)(struct kvAsyncContext *, int status);
typedef void(kvTimerCallback)(void *timer, void *privdata);
typedef void (redisDisconnectCallback)(const struct redisAsyncContext*, int status);
typedef void (redisConnectCallback)(const struct redisAsyncContext*, int status);
typedef void (redisConnectCallbackNC)(struct redisAsyncContext *, int status);
typedef void(redisTimerCallback)(void *timer, void *privdata);
/* Context for an async connection to KV */
typedef struct kvAsyncContext {
/* Context for an async connection to Redis */
typedef struct redisAsyncContext {
/* Hold the regular context, so it can be realloc'ed. */
kvContext c;
redisContext c;
/* Setup error flags so they can be used directly. */
int err;
char *errstr;
/* Not used by libkv */
/* Not used by hiredis */
void *data;
void (*dataCleanup)(void *privdata);
@@ -97,14 +89,15 @@ typedef struct kvAsyncContext {
} ev;
/* Called when either the connection is terminated due to an error or per
* user request. The status is set accordingly (KV_OK, KV_ERR). */
kvDisconnectCallback *onDisconnect;
* user request. The status is set accordingly (REDIS_OK, REDIS_ERR). */
redisDisconnectCallback *onDisconnect;
/* Called when the first write event was received. */
kvConnectCallback *onConnect;
redisConnectCallback *onConnect;
redisConnectCallbackNC *onConnectNC;
/* Regular command callbacks */
kvCallbackList replies;
redisCallbackList replies;
/* Address used for connect() */
struct sockaddr *saddr;
@@ -112,47 +105,48 @@ typedef struct kvAsyncContext {
/* Subscription callbacks */
struct {
kvCallbackList replies;
redisCallbackList replies;
struct dict *channels;
struct dict *patterns;
struct dict *schannels;
int pending_unsubs;
} sub;
/* Any configured RESP3 PUSH handler */
kvAsyncPushFn *push_cb;
} kvAsyncContext;
redisAsyncPushFn *push_cb;
} redisAsyncContext;
LIBKV_API kvAsyncContext *kvAsyncConnectWithOptions(const kvOptions *options);
LIBKV_API kvAsyncContext *kvAsyncConnect(const char *ip, int port);
LIBKV_API kvAsyncContext *kvAsyncConnectBind(const char *ip, int port, const char *source_addr);
LIBKV_API kvAsyncContext *kvAsyncConnectBindWithReuse(const char *ip, int port,
const char *source_addr);
LIBKV_API kvAsyncContext *kvAsyncConnectUnix(const char *path);
LIBKV_API int kvAsyncSetConnectCallback(kvAsyncContext *ac, kvConnectCallback *fn);
LIBKV_API int kvAsyncSetDisconnectCallback(kvAsyncContext *ac, kvDisconnectCallback *fn);
/* Functions that proxy to hiredis */
redisAsyncContext *redisAsyncConnectWithOptions(const redisOptions *options);
redisAsyncContext *redisAsyncConnect(const char *ip, int port);
redisAsyncContext *redisAsyncConnectBind(const char *ip, int port, const char *source_addr);
redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
const char *source_addr);
redisAsyncContext *redisAsyncConnectUnix(const char *path);
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetConnectCallbackNC(redisAsyncContext *ac, redisConnectCallbackNC *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
LIBKV_API kvAsyncPushFn *kvAsyncSetPushCallback(kvAsyncContext *ac, kvAsyncPushFn *fn);
LIBKV_API int kvAsyncSetTimeout(kvAsyncContext *ac, struct timeval tv);
LIBKV_API void kvAsyncDisconnect(kvAsyncContext *ac);
LIBKV_API void kvAsyncFree(kvAsyncContext *ac);
redisAsyncPushFn *redisAsyncSetPushCallback(redisAsyncContext *ac, redisAsyncPushFn *fn);
int redisAsyncSetTimeout(redisAsyncContext *ac, struct timeval tv);
void redisAsyncDisconnect(redisAsyncContext *ac);
void redisAsyncFree(redisAsyncContext *ac);
/* Handle read/write events */
LIBKV_API void kvAsyncHandleRead(kvAsyncContext *ac);
LIBKV_API void kvAsyncHandleWrite(kvAsyncContext *ac);
LIBKV_API void kvAsyncHandleTimeout(kvAsyncContext *ac);
LIBKV_API void kvAsyncRead(kvAsyncContext *ac);
LIBKV_API void kvAsyncWrite(kvAsyncContext *ac);
void redisAsyncHandleRead(redisAsyncContext *ac);
void redisAsyncHandleWrite(redisAsyncContext *ac);
void redisAsyncHandleTimeout(redisAsyncContext *ac);
void redisAsyncRead(redisAsyncContext *ac);
void redisAsyncWrite(redisAsyncContext *ac);
/* Command functions for an async context. Write the command to the
* output buffer and register the provided callback. */
LIBKV_API int kvvAsyncCommand(kvAsyncContext *ac, kvCallbackFn *fn, void *privdata, const char *format, va_list ap);
LIBKV_API int kvAsyncCommand(kvAsyncContext *ac, kvCallbackFn *fn, void *privdata, const char *format, ...);
LIBKV_API int kvAsyncCommandArgv(kvAsyncContext *ac, kvCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);
LIBKV_API int kvAsyncFormattedCommand(kvAsyncContext *ac, kvCallbackFn *fn, void *privdata, const char *cmd, size_t len);
int redisvAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *format, va_list ap);
int redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *format, ...);
int redisAsyncCommandArgv(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, int argc, const char **argv, const size_t *argvlen);
int redisAsyncFormattedCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *cmd, size_t len);
#ifdef __cplusplus
}
#endif
#endif /* KV_ASYNC_H */
#endif

Some files were not shown because too many files have changed in this diff Show More