Redis 2.4.3

Fixed bug in getClientInfoString() that was not rendering the N (no flags) special flag correctly.
Close client connection and log the event when the client input buffer reaches 1GB.
2011-11-21 18:47:39 +01:00 · 2011-11-21 18:37:22 +01:00 · 2011-11-21 18:37:17 +01:00 · 2011-11-21 18:37:05 +01:00 · 2011-11-21 18:37:01 +01:00 · 2011-11-21 18:36:56 +01:00
356 changed files with 17841 additions and 66608 deletions
@@ -1,14 +1,13 @@
-.*.swp
 *.o
+*.rdb
 *.log
-dump.rdb
-redis-benchmark
-redis-check-aof
-redis-check-dump
 redis-cli
-redis-sentinel
 redis-server
+redis-benchmark
+redis-check-dump
+redis-check-aof
 doc-tools
+mkrelease.sh
 release
 misc/*
 src/release.h
@@ -17,11 +16,4 @@ SHORT_TERM_TODO
 release.h
 src/transfer.sh
 src/configs
-redis.ds
-src/redis.conf
-deps/lua/src/lua
-deps/lua/src/luac
-deps/lua/src/liblua.a
-.make-*
-.prerequisites
-*.dSYM
+src/redis-server.dSYM
@@ -1,459 +1,133 @@
-Redis 2.6 release notes
-=======================
+Redis 2.4 release notes

-** IMPORTANT ** Check the 'Migrating from 2.4 to 2.6' section at the end of
-                this file for information about what changed between 2.4 and
-                2.6 and how this may affect your application.
-
--------------------------------------------------------------------------------
-Upgrade urgency levels:
-
-LOW:      No need to upgrade unless there are new features you want to use.
-MODERATE: Program an upgrade of the server, but it's not urgent.
-HIGH:     There is a critical bug that may affect a subset of users. Upgrade!
-CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
--------------------------------------------------------------------------------
-
--[ Redis 2.6.17 ] Release date: 11 Dec 2013
-
-UPGRADE URGENCY: MODERATE.
-
-* [FIX] redis-cli: fix big keys search when the key no longer exist.
-* [FIX] Allow AUTH / PING when disconnected from slave with serve-stale-data on.
-* [FIX] redis-benchmark: update help for new __rand_int__ form.
-* [FIX] Fix broken rdbWriteRaw() return value check in rdb.c (harmless).
-* [FIX] Log to what master a slave is going to connect to.
-* [FIX] Only run the fast active expire cycle if master & enabled.
-* [FIX] Fixed a replication bug involving 32 bit instances and big datasets
-        hard to compress that resulted into more than 2GB of RDB file sent.
-
--[ Redis 2.6.16 ] Release date: 28 Aug 2013
-
-UPGRADE URGENCY: MODERATE.
-
-* [FIX] Stop writes when fork() fails when trying to BGSAVE. BGSAVE errors
-        were trapped only in the case the saving child failed, so the fork()
-        error was no trapped as no child is created at all.
-* [FIX] Fix a serious memory efficiency regression, Redis 2.6.16 is
-        significantly better with large values (>= a few kbytes) and will
-        be able to store two times the amount of data compared to Redis 2.6.15
-        when values are in that range.
-* [FIX] Fixed an issue with Table conversion in the lua-cmsgpack library exposed
-        by the Redis scripting engine.
-* [NEW] DEBUG SDSLEN was introduced to debug memory inefficiencies due to
-        SDS allocation more easily.
-
--[ Redis 2.6.15 ] Release date: 21 Aug 2013
-
-UPGRADE URGENCY: MODERATE, upgrade ASAP only if you experience issues related to
-                 the expired keys collection algorithm, or if you use the
-                 ZUNIONSTORE command.
-
-* [FIX] Improved expired keys collection algorithm. Even under heavy load keys
-        to be expired can't accumulate because of lack of CPU time.
-* [FIX] Fixed Issue #1240, ZUNIONSTORE could lead to wrong result.
-* [NEW] redis-benchmark improvements.
-* [NEW] dict.c API wrong usage detection.
-
-WARNING: redis-benchmark released with 2.6.15 features non backward compatible
-         change in the way the random arguments are specified, use the --help
-         option for more information.
-
--[ Redis 2.6.14 ] Release date: 20 Jun 2013
-
-UPGRADE URGENCY: HIGH because of the following two issues:
-    * Lua scripting + Replication + AOF in slaves problem (see Issue #1164).
-    * AOF + expires possible race condition (see Issue #1079).
-
-* [FIX] AOF bug: expire could be removed from key on AOF rewrite.
-* [FIX] Allow writes from scripts called by AOF loading in read-only slaves.
-* [FIX] Sentinel: parse new version of INFO replication output correctly.
-* [NEW] Reset masterauth if an empty string is configured.
-
--[ Redis 2.6.13 ]
-
-UPGRADE URGENCY: MODERATE, nothing very critical but upgrading is suggested
-                 if you experienced:
-
-                 1) Strange issues with Lua scripting.
-                 2) Not reconfigured reappearing master using Sentinel.
-                 3) Server continusly trying to save on save error.
-
-                 This version of Redis may also help with AOF and slow / busy
-                 disks and latency issues.
-
-* [FIX] Throttle BGSAVE attempt on saving error.
-* [FIX] redis-cli: raise error on bad command line switch.
-* [FIX] Redis/Jemalloc Gitignore were too aggressive.
-* [FIX] Test: fix RDB test checking file permissions.
-* [FIX] Sentinel: always redirect on master->slave transition.
-* [FIX] Lua updated to version 5.1.5. Fixes rare scripting issues.
-* [NEW] AOF: improved latency figures with slow/busy disks.
-* [NEW] Sentinel: turn old master into a slave when it comes back.
-* [NEW] More explicit panic message on out of memory.
-* [NEW] redis-cli: --latency-history mode implemented.
-
--[ Redis 2.6.12 ]
-
-UPGRADE URGENCY: MODERATE, nothing very critical but a few non trivial bugs.
-
-* [BUGFIX]   redis-cli --bigkeys: don't crash with empty DB.
-* [BUGFIX]   stop-writes-on-bgsave-error now works in redis.conf
-* [BUGFIX]   Don't crash at startup if RDB is there but can't be opened.
-* [BUGFIX]   Initial value for master_link_down_since_seconds is now huge.
-* [BUGFIX]   Allow SELECT while loading the DB.
-* [BUGFIX]   Don't replicate/AOF an empty MULTI/EXEC if the transaction
-             is empty or containing just read-only commands.
-* [BUGFIX]   EXPIRE should not be able to resurrect keys (see issue #1026).
-* [IMPROVED] Extended SET back ported from Redis 2.8 / unstable
-             See http://redis.io/commands/set for more information.
-* [IMPROVED] Test suite improved.
-
--[ Redis 2.6.11 ]
-
-UPGRADE URGENCY: LOW, however updating is encouraged if you have many instances
-                 per server and you want to lower the CPU / energy usage.
-
-* [BUGFIX]   Replication: more strict error checking for master PING reply.
-* [BUGFIX]   redis-cli: use keepalive socket option for improved reliability.
-* [BUGFIX]   Allow AUTH while loading the DB in memory.
-* [BUGFIX]   Don't segfault on unbalanced quotes while parsing config file.
-* [BUGFIX]   Don't segfault if command gets propagated to AOF / replication
-             link as another command name that was renamed in redis.conf
-* [IMPROVED] serverCron() frequency is now a runtime parameter (was REDIS_HZ).
-* [IMPROVED] Use a lot less CPU when idle, even with many configured DBs.
-
--[ Redis 2.6.10 ]
-
-UPGRADE URGENCY: MODERATE, this release contains many non-critical fixes
-                 and many small improvements.
-
-* [BUGFIX]   redis-cli --rdb, fixed when the server sends newlines to ping.
-* [BUGFIX]   redis-cli, minor fixes on connection handling, prompt.
-* [BUGFIX]   Slow log: don't log EXEC, just executed commands.
-* [BUGFIX]   On failed shutdown don't try again and again compulsively.
-* [BUGFIX]   Fix build on sunos without backtrace().
-* [BUGFIX]   UNSUBSCRIBE and PUNSUBSCRIBE: always provide a reply (see 742e580)
-* [BUGFIX]   Lua struct library was broken, upgraded.
-* [BUGFIX]   Fix a bug in srandmemberWithCountCommand() with count argument.
-* [BUGFIX]   Test: disable clients timeout to prevent issues on slow systems.
-* [BUGFIX]   Sentinel: don't advertise the promoted slave as master too early.
-* [IMPROVED] Whitelist SIGUSR1, see http://redis.io/topics/signals.
-* [IMPROVED] Simpler to understand redis-cli --bigkeys output.
-* [IMPROVED] Test now works with tclsh > 8.5.
-* [IMPROVED] Added option to turn of the Nagle algorithm in slave socket.
-* [IMPROVED] Optionally use SO_KEEPALIVE to detect dead peers.
-
--[ Redis 2.6.9 ]
-
-UPGRADE URGENCY: MODERATE if you use replication.
-
-* [BUGFIX]   Changing master at runtime (SLAVEOF command) in presence of
-             network problems, or in very rapid succession, could result
-             in non-critical problems (GitHub Issue #828).
-* [IMPROVED] CLINGET GETNAME and SETNAME to set and query connection names
-             reported by CLIENT LIST. Very useful for debugging of
-             problems.
-* [IMPROVED] redis-cli is now able to transfer an RDB file from a remote
-             server to a local file using the --rdb <filename> command
-             line option.
-
--[ Redis 2.6.8 ]
-
-UPGRADE URGENCY: MODERATE if you use Lua scripting. Otherwise LOW.
-
-* [BUGFIX]   Multiple fixes for EVAL (issue #872).
-* [BUGFIX]   Fix overflow in mstime() in redis-cli and benchmark.
-* [BUGFIX]   Fix Linux / PPC64 behavior by correcting endianess detection.
-* [BUGFIX]   Fix NetBSD build by defining _XOPEN_SOURCE appropriately.
-* [BUGFIX]   Added missing license and copyright in a few places.
-* [BUGFIX]   Better error reporting when fd event creation fails.
-
--[ Redis 2.6.7 ]
-
-UPGRADE URGENCY: MODERATE (unless you BLPOP using the same key multiple times).
-
-* [BUGFIX]   Don't crash if BLPOP & co are called with the same key repeated
-             multiple times (Issue #801).
-
--[ Redis 2.6.6 ]
-
-UPGRADE URGENCY: CRITICAL if you experienced one more more crashes.
-                 MODERATE if Redis is running fine for you.
-
-* [BUGFIX]   Jemalloc updated to 3.2.0.
-
--[ Redis 2.6.5 ]
-
-UPGRADE URGENCY: MODERATE
-
-Warning: this release of Redis introduces a different behavior in MULTI/EXEC
-         handling of errors. This was done because the new behavior is safer
-         compared to the old one, and should not break any code targeting
-         Redis 2.6 in a critical way.
-
-         For more information check http://redis.io/topics/transactions
-
-* [IMPROVED] RDB/AOF childern now log amount of additional memory used
-             because of copy on write.
-* [BUGFIX]   MIGRATE non critical fixes (see commits for details).
-* [BUGFIX]   MULTI/EXEC: now EXEC aborts on errors before EXEC.
-* [BUGFIX]   Fix integer overflow in zunionInterGenericCommand resulting
-             into Z[INTER|UNION][STORE] commands to crash under extremely
-             unlikely conditions (almost impossible in real world).
-* [BUGFIX]   EVALSHA is now case insensitive (and will not crash).
-
--[ Redis 2.6.4 ]
-
-UPGRADE URGENCY: LOW
-
-* [IMPROVED] BSD license and copyright notice added to every .c and .h file.
-
--[ Redis 2.6.2 ]
-
-UPGRADE URGENCY: LOW
-
-* [BUGFIX]   The compilation fix for RHLE5 in 2.6.1 was broken. Fixed.
-* [IMPROVED] Linenoise updated, now supports Ctrl+w.
-
---[ Redis 2.6.1 ]
-
-UPGRADE URGENCY: LOW
-
-* [BUGFIX]   Compilation on Linux < 2.6.17 or glibc < 2.6 fixed (RHLE5 & co).
-
---[ Redis 2.6.0 ]
-
-UPGRADE URGENCY: HIGH
-
-* [BUGFIX]   Allow AUTH when server is in -BUSY state because of a slow script.
-* [BUGFIX]   MULTI/EXEC flow now makes sense when observed in MONITOR
-* [BUGFIX]   SCRIPT KILL now uses different error prefixes for different errors.
-* [BUGFIX]   Default memory limit for 32bit archs lowered from 3.5 to 3 GB.
-* [BUGFIX]   redis-check-dump is now compatible with RDB files generated by 2.6.
-* [IMPROVED] New field in INFO: slave_read_only.
-
---[ Redis 2.5.14 (2.6 Release Candidate 8) ]
-
-* [BUGFIX]   Fixed compilation on FreeBSD.
-* [IMPROVED] SRANDMEMBER <count> that returns multiple random elements.
-* [IMPROVED] Sentinel backported to 2.6. It will be taken in sync with 2.8.
-* [IMPROVED] Helper function for scripting to return errors and status replies.
-* [IMPROVED] SORT by nosort [ASC|DESC] uses sorted set elements ordering.
-* [BUGFIX]   Better resistence to system clock skew.
-* [IMPROVED] Warn the user when the configured maxmemory seems odd.
-* [BUGFIX]   Hashing function is now murmurhash2 for security purposes.
-* [IMPROVED] Install script no longer uses a template but redis.conf itself.
-
---[ Redis 2.5.13 (2.6 Release Candidate 7) ]
-
-UPGRADE URGENCY: HIGH
-
-* [BUGFIX]   Theoretical bug in ziplist fixed.
-* [BUGFIX]   Better out of memory handling (Log produced in log file).
-* [BUGFIX]   Incrementally flush RDB file on slave side while performing the
-             first synchronization with the master. This makes Redis less
-             blocking in environments where disk I/O is slow.
-* [BUGFIX]   Don't crash with Lua's redis.call() without arguments.
-* [BUGFIX]   Don't crash after a big number of Lua calls on 32 bit systems
-             because of a failed assertion.
-* [BUGFIX]   Fix SORT behaviour when called from scripting.
-* [BUGFIX]   Adjust slave PING period accordingly to REDIS_HZ define.
-* [BUGFIX]   BITCOUNT: fix crash on overflowing arguments.
-* [BUGFIX]   Return an error when SELECT argument is not an integer.
-* [BUGFIX]   Blocking operations on lists were completely reimplemented for
-             correctness. Now blocking list ops and pushes originated from
-             Lua scripts will play well together and will be replicated
-             and transmitted to the AOF correctly.
-* [IMPROVED] Send async PING before starting replication to avoid blocking if
-             master allows us to connect but it is actually not able to reply.
-* [IMPROVED] Support slave-priority for Redis Sentinel.
-* [IMPROVED] Hiredis library updated.
-
---[ Redis 2.5.12 (2.6 Release Candidate 6) ]
-
-UPGRADE URGENCY: MODERATE.
-
-* [BUGFIX]   Fixed a timing attack on AUTH (Issue #560).
-* [BUGFIX]   Don't assume that "char" is signed.
-* [BUGFIX]   Check that we have connection before enabling pipe mode.
-* [BUGFIX]   Use the optimized version of the function to convert a double to
-             its string representation. Compilation was disabled because of
-             a typo in the #if statement.
-* [IMPROVED} REPLCONF internal command introduced, now INFO shows slaves with
-             correct port numbers. This makes 2.5.12 Redis Sentinel compatible.
-* [IMPROVED] Truncate short write from the AOF for a cleaner restart. On short
-             writes (for instance out of space) Redis will now try to remove
-             the half-written data so that the next restart will work without
-             the need for the "redis-check-aof" utility.
-* [IMPROVED] New in INFO: aof_last_bgrewrite_status
-* [IMPROVED] Allow Pub/Sub in contexts where other commands are blocked.
-* [BUGFIX]   mark fd as writable when EPOLLERR or EPOLLHUP is returned by
-             epoll_wait.
-
---[ Redis 2.5.11 (2.6 Release Candidate 5) ]
-
-UPGRADE URGENCY: HIGH.
-
-* [BUGFIX]   Fixed Hash corruption when loading an RDB file generated by
-             previous versions of Redis that encoded hashes using
-             a different ziplist encoding format for small integers.
-             All the fileds that are integers in the range 0-255 may not
-             be recognized, or duplicated un updates, causing a crash
-             when the ziplist is converted to a real hash. (Issue #547).
-* [BUGFIX]   Fixed the count of memory used by output buffers in the
-             setDeferredMultiBulkLength() function.
-
---[ Redis 2.5.10 (2.6 Release Candidate 4) ]
-
-UPGRADE URGENCY: HIGH.
-
-* [BUGFIX]   Allow PREFIX to be overwritten on "make install".
-* [BUGFIX]   Run the test with just one client if the computer is slow.
-* [BUGFIX]   Event port support in our event driven libray.
-* [BUGFIX]   Jemalloc updated to 3.0.0. This fixes a possibly AOF rewrite issue.
-             See https://github.com/antirez/redis/issues/504 for info.
-* [BUGFIX]   Fixed issue #516: ZINTERSTORE / ZUNIONSTORE with mixed sets/zsets.
-* [BUGFIX]   Set fd to writable when poll(2) detects POLLERR or POLLHUP event.
-* [BUGFIX]   Fixed RESTORE hash failure (Issue #532).
-* [IMPROVED] Allow an AOF rewrite buffer > 2GB (Related to issue #504).
-* [IMPROVED] Server cron function frequency is now configurable (REDIS_HZ).
-* [IMPROVED] Better, less blocking expired keys collection algorithm.
-* [FEATURE]  New commands: BITOP and BITCOUNT.
-* [FEATURE]  redis-cli --pipe for mass import.
-
-What's new in Redis 2.5.9 (aka 2.6 Release Candidate 3)
-=======================================================
-
-UPGRADE URGENCY: critical, upgrade ASAP.
-
-* [BUGFIX] Fix for issue #500 (https://github.com/antirez/redis/pull/500).
-           Redis 2.6-RC1 and RC2 may corrupt ziplist-encoded sorted sets
-           produced by Redis 2.4.x.
-* [BUGFIX] Fixed several bugs in init.d script.
-* [BUGFIX] syncio.c functions modified for speed and correctness. On osx
-           (and possibly other BSD-based systems) the slave would block on
-           replication to send the SYNC command when the master was not
-           available. This is fixed now, but was not affecting Linux installs.
-* Now when slave-serve-stale-data is set to yes and the master is down, instead
-  of reporting a generic error Redis replies with -MASTERDOWN.
-
-What's new in Redis 2.5.8 (aka 2.6 Release Candidate 2)
-=======================================================
-
-UPGRADE URGENCY: high for all the users of the KEYS command, otherwise low.
-
-* [BUGFIX] Fix for KEYS command: if the DB contains keys with expires the KEYS
-           command may return the wrong output, having duplicated or missing
-           keys. See issue #487 and #488 on github for details.
-
-What's new in Redis 2.5.7 (aka 2.6 Release Candidate 1)
-=======================================================
-
-UPGRADE URGENCY: upgrade not recommended because this is an RC release.
-
-* This is the first release candidate for Redis 2.6. We are not aware of
-  bugs, but part of this code is young and was never tested in production
-  environments, so handle with care.
-
-An overview of new features and changes in Redis 2.6.x
-======================================================
-
-* Server side Lua scripting, see http://redis.io/commands/eval
-* Virtual Memory removed (was deprecated in 2.4)
-* Hardcoded limits about max number of clients removed.
-* AOF low level semantics is generally more sane, and especially when used
-  in slaves.
-* Milliseconds resolution expires, also added new commands with milliseconds
-  precision (PEXPIRE, PTTL, ...).
-* Better memory usage for "small" lists, ziplists and hashes when fields or
-  values contain small integers.
-* Read only slaves.
-* New bit opeations: BITCOUNT and BITOP commands.
-* Clients max output buffer soft and hard limits. You can specifiy different
-  limits for different classes of clients (normal,pubsub,slave).
-* More incremental (less blocking) expired keys collection algorithm, in
-  practical terms this means that Redis is more responsive when a very
-  big number of keys expire about at the same time.
-* AOF is now able to rewrite aggregate data types using variadic commands,
-  often producing an AOF that is faster to save, load, and is smaller in size.
-* Every redis.conf directive is now accepted as a command line option for the
-  redis-server binary, with the same name and number of arguments.
-* Hash table seed randomization for protection against collisions attacks.
-* Performances improved when writing large objects to Redis.
-* Integrated memory test, see redis-server --test-memory.
-* INCRBYFLOAT and HINCRBYFLOAT commands.
-* New DUMP, RESTORE, MIGRATE commands (back ported from Redis Cluster to 2.6).
-* CRC64 checksump in RDB files.
-* Better MONITOR output and behavior (now commands are logged before execution).
-* "Software Watchdog" feature to debug latency issues.
-* Significant parts of the core refactored or rewritten. New internal APIs
-  and core changes allowed to develop Redis Cluster on top of the new code,
-  however for 2.6 all the cluster code was removed, and will be released with
-  Redis 3.0 when it is more complete and stable.
-* Redis ASCII art logo added at startup.
-* Crash report on memory violation or failed asserts improved significantly
-  to make debugging of hard to catch bugs simpler.
-* redis-benchmark improvements: ability to run selected tests,
-  CSV output, faster, better help.
-* redis-cli improvements: --eval for comfortable development of Lua scripts.
-* SHUTDOWN now supports two optional arguments: "SAVE" and "NOSAVE".
-* INFO output split into sections, the command is now able to just show 
-  pecific sections.
-* New statistics about how many time a command was called, and how much
-  execution time it used (INFO commandstats).
-* More predictable SORT behavior in edge cases.
-* Better support for big endian and *BSD systems.
-* Build system improved.
-
-Migrating from 2.4 to 2.6
+Migrating from 2.2 to 2.4
 =========================

-Redis 2.4 is mostly a strict subset of 2.6. However there are a few things
-that you should be aware of:
+Redis 2.2 is mostly a strict subset of 2.4.
+The only thing you should be aware is that you can't use .rdb and AOF files
+generated with 2.4 into a 2.2 instance.

-* You can't use .rdb and AOF files generated with 2.6 into a 2.4 instance.
-* 2.6 slaves can be attached to 2.4 masters, but not the contrary, and only
-  for the time needed to perform the version upgrade.
+2.4 slaves can be attached to 2.2 masters, but not the contrary, and only for
+the time needed to perform the version upgrade.

-There are also a few API differences, that are unlikely to cause problems,
-but it is better to keep them in mind:
+From the point of view of the API Redis 2.4 only adds new commands
+(other commands now accepts a variable number of arguments) so you don't need
+to modify your program in order to use Redis 2.4.

-* SORT now will refuse to sort in numerical mode elements that can't be parsed
-  as numbers.
-* EXPIREs now all have millisecond resolution (but this is very unlikely to
-  break code that was not conceived exploting the previous resolution error
-  in some way.)
-* INFO output is a bit different now, and contains empty lines and comments
-  starting with '#'. All the major clients should be already fixed to work
-  with the new INFO format.
-* Slaves are only read-only by default (but you can change this easily
-  setting the "slave-read-only" configuration option to "no" editing your
-  redis.conf or using CONFIG SET.
+---------
+CHANGELOG
+---------

-The following INFO fields were renamed for consistency:
+What's new in Redis 2.4.3
+=========================

-     changes_since_last_save -> rdb_changes_since_last_save
-     bgsave_in_progress -> rdb_bgsave_in_progress
-     last_save_time -> rdb_last_save_time
-     last_bgsave_status -> rdb_last_bgsave_status
-     bgrewriteaof_in_progress -> aof_rewrite_in_progress
-     bgrewriteaof_scheduled -> aof_rewrite_scheduled
+* redis-cli now supports 'single quotes' style strings.
+* It is possible to disable password auth with CONFIG SET.
+* Some section of redis.conf better documented.
+* Default timeout for client connections now is 0 (no timeout).
+* I/O buffer length modified for better performances with big payloads.
+* Fixed crash on SPARC due to improper alighment due to bad assumptions about data types size.
+* CLIENT LIST output improved, code refactored.
+* [BUGFIX] Max log message length set to 4k to avoid truncation in INFO output written in the log file after a crash.
+* [BUGFIX] Close client connection when the query buffer reaches 1GB and log informaiton about the offending client, instead of crashing the instance when the query buffer reaches 2GB (for sds.c string overflow). This is related to issue #141 (github issues) and should fix the effect of this rare bug related to replication, but the cause is still not perfectly clear.

-The following redis.conf and CONFIG GET / SET parameters changed:
+What's new in Redis 2.4.2
+=========================

-    * hash-max-zipmap-entries, now replaced by hash-max-ziplist-entries
-    * hash-max-zipmap-value, now replaced by hash-max-ziplist-value
-    * glueoutputbuf option was now completely removed (was deprecated)
+* [BUGFIX] Unix socket creation mask config directive fixed.
+* [BUGFIX] Fixed a bug that forced Redis to continously rewrite the AOF file
+           when a manual BGREWRITEAOF was issued during a BGSAVE was
+           still in progress.
+* [BUGFIX] Fixed a server crash resulting from bad handling of SLAVEOF NO ONE.
+* [BUGFIX] Fixed a memory leak in redis-cli.
+* Debian/Ubuntu Redis system-wide installation script added under /utils.

--------------------------------------------------------------------------------
+What's new in Redis 2.4.1
+=========================
+
+* [BUGFIX] FLUSHALL was not replicated nor written into the Append Only File.
+* [BUGFIX] FLUSHALL now only performs a sync SAVE if there is at least
+           one save point configured.
+
+What's new in Redis 2.4.0
+=========================
+
+* [BUGFIX] redis-cli segfault with single numerical argument fixed.
+* [BUGFIX] OpenBSD compilation problem fixed.
+* [BUGFIX] More robust Redis test, with better random port selection.
+* [BUGFIX] Fix for bug #128 about the RENAME command.
+* [BUGFIX] Fixed Issue #131. stime/utime reported in INFO was inverted.
+* [BUGFIX] Unlink Unix socket file on shutdown.
+* [BUGFIX] AUTH now returns error if no password is set on the server.
+* [BUGFIX] Exit with Fatal error at startup on RDB loading errors.
+* redis-check-dump: RDB version 2 now supported.
+* More informative error when DEBUG RELOAD fails.
+* Added a config directive for a Unix socket mask.
+* CONFIG SET/GET for loglevel.
+
+What's new in Redis 2.3.11 (2.4 Release Candidate 8)
+====================================================
+
+* [BUGFIX] Fixed a rare but possible AOF race condition that could result into
+           duplicated commands inside the AOF.
+* [BUGFIX] Fixed issue 620, don't segfault on corrupted (by hand) AOF.
+* [BUGFIX] Fixed compilation on Mac/PPC.
+* [BUGFIX] Don't replicate SAVE.
+* LRANGE optimization may drastically improve performances when querying the
+  final part of a long list.
+* redis-cli now implements a --latency mode to monitory Redis delay.
+* Hash type settings removed from INFO (same info is available via config GET)
+* Include port number on error when can't bind.
+* AOF fsync is now performed in background when fsync policy is 'everysec'.
+* AOF performances improved moving in background a possibly slow close(2) call.
+* AOF protocol synthesis speedup.
+
+What's new in Redis 2.3.10 (2.4 Release Candidate 7)
+====================================================
+
+* [BUGFIX] Fixed issue 593 (BRPOPLPUSH related crash).
+* [BUGFIX] Fixed an issue with the networking layer that may prevent Redis from sending the whole reply back to client under extreme conditions.
+
+What's new in Redis 2.3.9 (2.4 Release Candidate 6)
+===================================================
+
+* [BUGFIX] Fixed a bug with the automatic AOF rewrite causing continuous
+  rewrites for AOF files bigger than 4 GB.
+* New maxmemory tests.
+
+What's new in Redis 2.3.8 (2.4 Release Candidate 5)
+===================================================
+
+Compared to Redis 2.3.7 (RC4) there are the following changes:
+
+* [BUGFIX] HDEL: Abort deleting fields when hash is removed.
+* [BUGFIX] Fix adding bulk reply when getcwd fails.
+
+WHAT'S NEW IN REDIS 2.4 compared to the 2.2 version?
+====================================================
+
+* Specially encoded sorted sets, now small sorted sets will use little memory.
+* Native persistence of specially encoded data types (ziplists, zipmaps,
+  intsets). Many data sets will be saved and loaded an order of magnitude
+  faster.
+* Variadic versions of commands: SADD, HDEL, SREM, ZREM, ZADD, L/RPUSH.
+* Jemalloc support (enabled by default for Linux build) in order to
+  avoid fragmentation issues.
+* Reduced memory usage while saving.
+* More info fields (peak memory, fork time, ...)
+* OBJECT command for objects introspection.
+* CLIENT command for clients introspection.
+* Non blocking slave -> master connection.
+* Better redis-cli connection handling. New redis-cli features.
+* Better redis-benchmark, now able to benchmark user provided commands.
+* Colorized Make.
+* VM deprecated. Still supported but with a big warning... don't use it.
+* Many speed optimizations and bug fixes.

 Credits: Where not specified the implementation and design are done by
 Salvatore Sanfilippo and Pieter Noordhuis. Thanks to VMware for making all
 this possible. Also many thanks to all the other contributors and the amazing
 community we have.

-See commit messages for more credits.
-
 Cheers,
 Salvatore
@@ -1 +1 @@
-Plese check https://github.com/antirez/redis/issues
+Plese check http://code.google.com/p/redis/issues/list
@@ -1,34 +1,13 @@
-Note: by contributing code to the Redis project in any form, including sending
-a pull request via Github, a code fragment or patch via private email or
-public discussion groups, you agree to release your code under the terms
-of the BSD license that you can find in the COPYING file included in the Redis
-source distribution. You will include BSD license in the COPYING file within
-each source file that you contribute.
+1. Enter irc.freenode.org #redis and start talking with 'antirez' and/or 'pietern' to check if there is interest for such a feature and to understand the probability of it being merged. We'll try hard to keep Redis simple... so you'll likely encounter an high resistence.

-# IMPORTANT: HOW TO USE REDIS GITHUB ISSUES
+2. Drop a message to the Redis Google Group with a proposal of semantics/API.

-* Github issues SHOULD ONLY BE USED to report bugs, and for DETAILED feature
-  requests. Everything else belongs to the Redis Google Group.
+3. If steps 1 and 2 are ok, use the following procedure to submit a patch:

-  PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
-  bugs in the Github issues system. We'll be very happy to help you and provide
-  all the support in the Redis Google Group.
-
-  Redis Google Group address:
-  
-      https://groups.google.com/forum/?fromgroups#!forum/redis-db
-
-# How to provide a patch for a new feature
-
-1. Drop a message to the Redis Google Group with a proposal of semantics/API.
-
-2. If in steps 1 you get an acknowledge from the project leaders, use the
-   following procedure to submit a patch:
-
-    a. Fork Redis on github ( http://help.github.com/fork-a-repo/ )
+    a. Fork Redis on github
    b. Create a topic branch (git checkout -b my_branch)
    c. Push to your branch (git push origin my_branch)
-    d. Initiate a pull request on github ( http://help.github.com/send-pull-requests/ )
+    d. Create an issue in the Redis google code site with a link to your patch
    e. Done :)

 Thanks!
@@ -1,4 +1,4 @@
-Copyright (c) 2006-2012, Salvatore Sanfilippo
+Copyright (c) 2006-2009, Salvatore Sanfilippo
 All rights reserved.

 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
@@ -1 +1 @@
-See README
+Please check the README file.
@@ -1,67 +0,0 @@
-[Note: this is the Redis manifesto, for general information about
-       installing and running Redis read the README file instead.]
-
-Redis Manifesto
-===============
-
-1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language)
-    that manipulates abstract data types and implemented as a TCP daemon.
-    Commands manipulate a key space where keys are binary-safe strings and
-    values are different kinds of abstract data types. Every data type
-    represents an abstract version of a fundamental data structure. For instance
-    Redis Lists are an abstract representation of linked lists. In Redis, the
-    essence of a data type isn't just the kind of operations that the data types
-    support, but also the space and time complexity of the data type and the
-    operations performed upon it.
-
-2 - Memory storage is #1. The Redis data set, composed of defined key-value
-    pairs, is primarily stored in the computer's memory. The amount of memory in
-    all kinds of computers, including entry-level servers, is increasing
-    significantly each year. Memory is fast, and allows Redis to have very
-    predictable performance. Datasets composed of 10k or 40 millions keys will
-    perform similarly. Complex data types like Redis Sorted Sets are easy to
-    implement and manipulate in memory with good performance, making Redis very
-    simple. Redis will continue to explore alternative options (where data can
-    be optionally stored on disk, say) but the main goal of the project remains
-    the development of an in-memory database.
-
-3 - Fundamental data structures for a fundamental API. The Redis API is a direct
-    consequence of fundamental data structures. APIs can often be arbitrary but
-    not an API that resembles the nature of fundamental data structures. If we
-    ever meet intelligent life forms from another part of the universe, they'll
-    likely know, understand and recognize the same basic data structures we have
-    in our computer science books. Redis will avoid intermediate layers in API,
-    so that the complexity is obvious and more complex operations can be
-    performed as the sum of the basic operations.
-
-4 - Code is like a poem; it's not just something we write to reach some
-    practical result. Sometimes people that are far from the Redis philosophy
-    suggest using other code written by other authors (frequently in other
-    languages) in order to implement something Redis currently lacks. But to us
-    this is like if Shakespeare decided to end Enrico IV using the Paradiso from
-    the Divina Commedia. Is using any external code a bad idea? Not at all. Like
-    in "One Thousand and One Nights" smaller self contained stories are embedded
-    in a bigger story, we'll be happy to use beautiful self contained libraries
-    when needed. At the same time, when writing the Redis story we're trying to
-    write smaller stories that will fit in to other code.
-
-5 - We're against complexity. We believe designing systems is a fight against
-    complexity. We'll accept to fight the complexity when it's worthwhile but
-    we'll try hard to recognize when a small feature is not worth 1000s of lines
-    of code. Most of the time the best way to fight complexity is by not
-    creating it at all.
-
-6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
-    naturally into a distributed version of Redis and 2) a more complex API that
-    supports multi-key operations. Both are useful if used judiciously but
-    there's no way to make the more complex multi-keys API distributed in an
-    opaque way without violating our other principles. We don't want to provide
-    the illusion of something that will work magically when actually it can't in
-    all cases. Instead we'll provide commands to quickly migrate keys from one
-    instance to another to perform multi-key operations and expose the tradeoffs
-    to the user.
-
-7 - We optimize for joy. We believe writing code is a lot of hard work, and the
-    only way it can be worth is by enjoying it. When there is no longer joy in
-    writing code, the best thing to do is stop. To prevent this, we'll avoid
-    taking paths that will make Redis less of a joy to develop.
@@ -1,11 +1,23 @@
 # Top level makefile, the real shit is at src/Makefile

-default: all
+TARGETS=32bit noopt test

-.DEFAULT:
+all:
 	cd src && $(MAKE) $@

-install:
+install: dummy
 	cd src && $(MAKE) $@

-.PHONY: install
+clean:
+	cd src && $(MAKE) $@
+	cd deps/hiredis && $(MAKE) $@
+	cd deps/linenoise && $(MAKE) $@
+	-(cd deps/jemalloc && $(MAKE) distclean)
+
+$(TARGETS):
+	cd src && $(MAKE) $@
+
+src/help.h:
+	@./utils/generate-command-help.rb > $@
+
+dummy:
@@ -7,13 +7,6 @@ documentation at http://redis.io
 Building Redis
 --------------

-Redis can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
-We support big endian and little endian architectures.
-
-It may compile on Solaris derived systems (for instance SmartOS) but our
-support for this platform is "best effort" and Redis is not guaranteed to
-work as well as in Linux, OSX, and *BSD there.
-
 It is as simple as:

    % make
@@ -26,37 +19,25 @@ After building Redis is a good idea to test it, using:

    % make test

-Fixing problems building 32 bit binaries
---------
-
-If after building Redis with a 32 bit target you need to rebuild it
-with a 64 bit target, or the other way around, you need to perform a
-"make distclean" in the root directory of the Redis distribution.
-
-In case of build errors when trying to build a 32 bit binary of Redis, try
-the following steps:
-
-* Install the packages libc6-dev-i386 (also try g++-multilib).
-* Try using the following command line instead of "make 32bit":
-
-    make CFLAGS="-m32 -march=native" LDFLAGS="-m32"
+NOTE: if after building Redis with a 32 bit target you need to rebuild it
+      with a 64 bit target you need to perform a "make clean" in the root
+      directory of the Redis distribution.

 Allocator
 ---------

-Selecting a non-default memory allocator when building Redis is done by setting
-the `MALLOC` environment variable. Redis is compiled and linked against libc
-malloc by default, with the exception of jemalloc being the default on Linux
-systems. This default was picked because jemalloc has proven to have fewer
-fragmentation problems than libc malloc.
+By default Redis compiles and links against jemalloc under Linux, since
+glibc malloc() has memory fragmentation problems.

-To force compiling against libc malloc, use:
+To force a libc malloc() build use:

-    % make MALLOC=libc
+    % make FORCE_LIBC_MALLOC=yes

-To compile against jemalloc on Mac OS X systems, use:
+In all the other non Linux systems the libc malloc() is used by default.

-    % make MALLOC=jemalloc
+On Mac OS X you can force a jemalloc based build using the following:
+
+    % make USE_JEMALLOC=yes

 Verbose build
 -------------
@@ -80,15 +61,6 @@ parameter (the path of the configuration file):
    % cd src
    % ./redis-server /path/to/redis.conf

-It is possible to alter the Redis configuration passing parameters directly
-as options using the command line. Examples:
-
-    % ./redis-server --port 9999 --slaveof 127.0.0.1 6379
-    % ./redis-server /etc/redis/6379.conf --loglevel debug
-
-All the options in redis.conf are also supported as options using the command
-line, with exactly the same name.
-
 Playing with Redis
 ------------------

@@ -139,16 +111,4 @@ system reboots.
 You'll be able to stop and start Redis using the script named
 /etc/init.d/redis_<portnumber>, for instance /etc/init.d/redis_6379.

-Code contributions
---
-
-Note: by contributing code to the Redis project in any form, including sending
-a pull request via Github, a code fragment or patch via private email or
-public discussion groups, you agree to release your code under the terms
-of the BSD license that you can find in the COPYING file included in the Redis
-source distribution.
-
-Please see the CONTRIBUTING file in this source distribution for more
-information.
-
 Enjoy!
@@ -0,0 +1,4 @@
+This is a stable release! No TODO file here.
+Please check the TODO file in the master branch on github.
+
+    https://github.com/antirez/redis/raw/master/TODO
@@ -1,78 +0,0 @@
-# Redis dependency Makefile
-
-uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
-
-CCCOLOR="\033[34m"
-LINKCOLOR="\033[34;1m"
-SRCCOLOR="\033[33m"
-BINCOLOR="\033[37;1m"
-MAKECOLOR="\033[32;1m"
-ENDCOLOR="\033[0m"
-
-default:
-	@echo "Explicit target required"
-
-.PHONY: default
-
-# Prerequisites target
-.make-prerequisites:
-	@touch $@
-
-# Clean everything when CFLAGS is different
-ifneq ($(shell sh -c '[ -f .make-cflags ] && cat .make-cflags || echo none'), $(CFLAGS))
-.make-cflags: distclean
-	-(echo "$(CFLAGS)" > .make-cflags)
-.make-prerequisites: .make-cflags
-endif
-
-# Clean everything when LDFLAGS is different
-ifneq ($(shell sh -c '[ -f .make-ldflags ] && cat .make-ldflags || echo none'), $(LDFLAGS))
-.make-ldflags: distclean
-	-(echo "$(LDFLAGS)" > .make-ldflags)
-.make-prerequisites: .make-ldflags
-endif
-
-distclean:
-	-(cd hiredis && $(MAKE) clean) > /dev/null || true
-	-(cd linenoise && $(MAKE) clean) > /dev/null || true
-	-(cd lua && $(MAKE) clean) > /dev/null || true
-	-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
-	-(rm -f .make-*)
-
-.PHONY: distclean
-
-hiredis: .make-prerequisites
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
-	cd hiredis && $(MAKE) static
-
-.PHONY: hiredis
-
-linenoise: .make-prerequisites
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
-	cd linenoise && $(MAKE)
-
-.PHONY: linenoise
-
-ifeq ($(uname_S),SunOS)
-  # Make isinf() available
-  LUA_CFLAGS= -D__C99FEATURES__=1
-endif
-
-LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI $(CFLAGS)
-LUA_LDFLAGS+= $(LDFLAGS)
-
-lua: .make-prerequisites
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
-	cd lua/src && $(MAKE) all CFLAGS="$(LUA_CFLAGS)" MYLDFLAGS="$(LUA_LDFLAGS)"
-
-.PHONY: lua
-
-JEMALLOC_CFLAGS= -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops $(CFLAGS)
-JEMALLOC_LDFLAGS= $(LDFLAGS)
-
-jemalloc: .make-prerequisites
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
-	cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
-	cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
-
-.PHONY: jemalloc
@@ -1,16 +0,0 @@
-### 0.10.1
-
-* Makefile overhaul. Important to check out if you override one or more
-  variables using environment variables or via arguments to the "make" tool.
-
-* Issue #45: Fix potential memory leak for a multi bulk reply with 0 elements
-  being created by the default reply object functions.
-
-* Issue #43: Don't crash in an asynchronous context when Redis returns an error
-  reply after the connection has been made (this happens when the maximum
-  number of connections is reached).
-
-### 0.10.0
-
-* See commit log.
-
@@ -1,29 +1,10 @@
-Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
-Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
-
+Copyright (c) 2006-2009, Salvatore Sanfilippo
 All rights reserved.

-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

-* Redistributions of source code must retain the above copyright notice,
-  this list of conditions and the following disclaimer.
+    * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+    * Neither the name of Redis nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

-* Redistributions in binary form must reproduce the above copyright notice,
-  this list of conditions and the following disclaimer in the documentation
-  and/or other materials provided with the distribution.
-
-* Neither the name of Redis nor the names of its contributors may be used
-  to endorse or promote products derived from this software without specific
-  prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
-ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
-ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -1,148 +1,115 @@
 # Hiredis Makefile
-# Copyright (C) 2010-2011 Salvatore Sanfilippo <antirez at gmail dot com>
-# Copyright (C) 2010-2011 Pieter Noordhuis <pcnoordhuis at gmail dot com>
+# Copyright (C) 2010 Salvatore Sanfilippo <antirez at gmail dot com>
 # This file is released under the BSD license, see the COPYING file

-OBJ=net.o hiredis.o sds.o async.o
-BINS=hiredis-example hiredis-test
-LIBNAME=libhiredis
+OBJ = net.o hiredis.o sds.o async.o
+BINS = hiredis-example hiredis-test

-HIREDIS_MAJOR=0
-HIREDIS_MINOR=10
-
-# Fallback to gcc when $CC is not in $PATH.
-CC:=$(shell sh -c 'type $(CC) >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
-OPTIMIZATION?=-O3
-WARNINGS=-Wall -W -Wstrict-prototypes -Wwrite-strings
-DEBUG?= -g -ggdb
-REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CFLAGS) $(WARNINGS) $(DEBUG) $(ARCH)
-REAL_LDFLAGS=$(LDFLAGS) $(ARCH)
-
-DYLIBSUFFIX=so
-STLIBSUFFIX=a
-DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR).$(HIREDIS_MINOR)
-DYLIB_MAJOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
-DYLIBNAME=$(LIBNAME).$(DYLIBSUFFIX)
-DYLIB_MAKE_CMD=$(CC) -shared -Wl,-soname,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
-STLIBNAME=$(LIBNAME).$(STLIBSUFFIX)
-STLIB_MAKE_CMD=ar rcs $(STLIBNAME)
-
-# Platform-specific overrides
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
+OPTIMIZATION?=-O3
 ifeq ($(uname_S),SunOS)
-  REAL_LDFLAGS+= -ldl -lnsl -lsocket
-  DYLIB_MAKE_CMD=$(CC) -G -o $(DYLIBNAME) -h $(DYLIB_MINOR_NAME) $(LDFLAGS)
-  INSTALL= cp -r
-endif
+  CFLAGS?=-std=c99 -pedantic $(OPTIMIZATION) -fPIC -Wall -W -D__EXTENSIONS__ -D_XPG6 $(ARCH) $(PROF)
+  CCLINK?=-ldl -lnsl -lsocket -lm -lpthread
+  LDFLAGS?=-L. -Wl,-R,.
+  DYLIBNAME?=libhiredis.so
+  DYLIB_MAKE_CMD?=$(CC) -G -o ${DYLIBNAME} ${OBJ}
+  STLIBNAME?=libhiredis.a
+  STLIB_MAKE_CMD?=ar rcs ${STLIBNAME} ${OBJ}
+else
 ifeq ($(uname_S),Darwin)
-  DYLIBSUFFIX=dylib
-  DYLIB_MINOR_NAME=$(LIBNAME).$(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(DYLIBSUFFIX)
-  DYLIB_MAJOR_NAME=$(LIBNAME).$(HIREDIS_MAJOR).$(DYLIBSUFFIX)
-  DYLIB_MAKE_CMD=$(CC) -shared -Wl,-install_name,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
+  CFLAGS?=-std=c99 -pedantic $(OPTIMIZATION) -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings $(ARCH) $(PROF)
+  CCLINK?=-lm -pthread
+  LDFLAGS?=-L. -Wl,-rpath,.
+  OBJARCH?=-arch i386 -arch x86_64
+  DYLIBNAME?=libhiredis.dylib
+  DYLIB_MAKE_CMD?=libtool -dynamic -o ${DYLIBNAME} -lm ${DEBUG} - ${OBJ}
+  STLIBNAME?=libhiredis.a
+  STLIB_MAKE_CMD?=libtool -static -o ${STLIBNAME} - ${OBJ}
+else
+  CFLAGS?=-std=c99 -pedantic $(OPTIMIZATION) -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings $(ARCH) $(PROF)
+  CCLINK?=-lm -pthread
+  LDFLAGS?=-L. -Wl,-rpath,.
+  DYLIBNAME?=libhiredis.so
+  DYLIB_MAKE_CMD?=gcc -shared -Wl,-soname,${DYLIBNAME} -o ${DYLIBNAME} ${OBJ}
+  STLIBNAME?=libhiredis.a
+  STLIB_MAKE_CMD?=ar rcs ${STLIBNAME} ${OBJ}
+endif
 endif

-all: $(DYLIBNAME) $(BINS)
+CCOPT= $(CFLAGS) $(CCLINK)
+DEBUG?= -g -ggdb
+
+PREFIX?= /usr/local
+INSTALL_INC= $(PREFIX)/include/hiredis
+INSTALL_LIB= $(PREFIX)/lib
+INSTALL= cp -a
+
+all: ${DYLIBNAME} ${BINS}

 # Deps (use make dep to generate this)
-net.o: net.c fmacros.h net.h hiredis.h
-async.o: async.c async.h hiredis.h sds.h dict.c dict.h
+net.o: net.c fmacros.h net.h
+async.o: async.c async.h hiredis.h sds.h util.h dict.c dict.h
 example.o: example.c hiredis.h
-hiredis.o: hiredis.c fmacros.h hiredis.h net.h sds.h
+hiredis.o: hiredis.c hiredis.h net.h sds.h util.h
 sds.o: sds.c sds.h
 test.o: test.c hiredis.h

-$(DYLIBNAME): $(OBJ)
-	$(DYLIB_MAKE_CMD) $(OBJ)
+${DYLIBNAME}: ${OBJ}
+	${DYLIB_MAKE_CMD}

-$(STLIBNAME): $(OBJ)
-	$(STLIB_MAKE_CMD) $(OBJ)
+${STLIBNAME}: ${OBJ}
+	${STLIB_MAKE_CMD}

-dynamic: $(DYLIBNAME)
-static: $(STLIBNAME)
+dynamic: ${DYLIBNAME}
+static: ${STLIBNAME}

 # Binaries:
-hiredis-example-libevent: example-libevent.c adapters/libevent.h $(STLIBNAME)
-	$(CC) -o $@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -levent example-libevent.c $(STLIBNAME)
+hiredis-example-libevent: example-libevent.c adapters/libevent.h ${DYLIBNAME}
+	$(CC) -o $@ $(CCOPT) $(DEBUG) $(LDFLAGS) -lhiredis -levent example-libevent.c

-hiredis-example-libev: example-libev.c adapters/libev.h $(STLIBNAME)
-	$(CC) -o $@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -lev example-libev.c $(STLIBNAME)
+hiredis-example-libev: example-libev.c adapters/libev.h ${DYLIBNAME}
+	$(CC) -o $@ $(CCOPT) $(DEBUG) $(LDFLAGS) -lhiredis -lev example-libev.c

 ifndef AE_DIR
 hiredis-example-ae:
 	@echo "Please specify AE_DIR (e.g. <redis repository>/src)"
 	@false
 else
-hiredis-example-ae: example-ae.c adapters/ae.h $(STLIBNAME)
-	$(CC) -o $@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I$(AE_DIR) $(AE_DIR)/ae.o $(AE_DIR)/zmalloc.o example-ae.c $(STLIBNAME)
+hiredis-example-ae: example-ae.c adapters/ae.h ${DYLIBNAME}
+	$(CC) -o $@ $(CCOPT) $(DEBUG) -I$(AE_DIR) $(LDFLAGS) -lhiredis example-ae.c $(AE_DIR)/ae.o $(AE_DIR)/zmalloc.o
 endif

-hiredis-%: %.o $(STLIBNAME)
-	$(CC) -o $@ $(REAL_LDFLAGS) $< $(STLIBNAME)
+hiredis-%: %.o ${DYLIBNAME}
+	$(CC) -o $@ $(CCOPT) $(DEBUG) $(LDFLAGS) -lhiredis $<

 test: hiredis-test
 	./hiredis-test

-check: hiredis-test
-	echo \
-		"daemonize yes\n" \
-		"pidfile /tmp/hiredis-test-redis.pid\n" \
-		"port 56379\n" \
-		"bind 127.0.0.1\n" \
-		"unixsocket /tmp/hiredis-test-redis.sock" \
-			| redis-server -
-	./hiredis-test -h 127.0.0.1 -p 56379 -s /tmp/hiredis-test-redis.sock || \
-			( kill `cat /tmp/hiredis-test-redis.pid` && false )
-	kill `cat /tmp/hiredis-test-redis.pid`
-
 .c.o:
-	$(CC) -std=c99 -pedantic -c $(REAL_CFLAGS) $<
+	$(CC) -c $(CFLAGS) $(OBJARCH) $(DEBUG) $(COMPILE_TIME) $<

 clean:
-	rm -rf $(DYLIBNAME) $(STLIBNAME) $(BINS) hiredis-example* *.o *.gcda *.gcno *.gcov
+	rm -rf ${DYLIBNAME} ${STLIBNAME} $(BINS) hiredis-example* *.o *.gcda *.gcno *.gcov

 dep:
 	$(CC) -MM *.c

-# Installation related variables and target
-PREFIX?=/usr/local
-INCLUDE_PATH?=include/hiredis
-LIBRARY_PATH?=lib
-INSTALL_INCLUDE_PATH= $(PREFIX)/$(INCLUDE_PATH)
-INSTALL_LIBRARY_PATH= $(PREFIX)/$(LIBRARY_PATH)
-
-ifeq ($(uname_S),SunOS)
-  INSTALL?= cp -r
-endif
-
-INSTALL?= cp -a
-
-install: $(DYLIBNAME) $(STLIBNAME)
-	mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_LIBRARY_PATH)
-	$(INSTALL) hiredis.h async.h adapters $(INSTALL_INCLUDE_PATH)
-	$(INSTALL) $(DYLIBNAME) $(INSTALL_LIBRARY_PATH)/$(DYLIB_MINOR_NAME)
-	cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MINOR_NAME) $(DYLIB_MAJOR_NAME)
-	cd $(INSTALL_LIBRARY_PATH) && ln -sf $(DYLIB_MAJOR_NAME) $(DYLIBNAME)
-	$(INSTALL) $(STLIBNAME) $(INSTALL_LIBRARY_PATH)
+install: ${DYLIBNAME} ${STLIBNAME}
+	mkdir -p $(INSTALL_INC) $(INSTALL_LIB)
+	$(INSTALL) hiredis.h async.h adapters $(INSTALL_INC)
+	$(INSTALL) ${DYLIBNAME} ${STLIBNAME} $(INSTALL_LIB)

 32bit:
 	@echo ""
-	@echo "WARNING: if this fails under Linux you probably need to install libc6-dev-i386"
+	@echo "WARNING: if it fails under Linux you probably need to install libc6-dev-i386"
 	@echo ""
-	$(MAKE) CFLAGS="-m32" LDFLAGS="-m32"
+	$(MAKE) ARCH="-m32"

 gprof:
-	$(MAKE) CFLAGS="-pg" LDFLAGS="-pg"
+	$(MAKE) PROF="-pg"

 gcov:
-	$(MAKE) CFLAGS="-fprofile-arcs -ftest-coverage" LDFLAGS="-fprofile-arcs"
-
-coverage: gcov
-	make check
-	mkdir -p tmp/lcov
-	lcov -d . -c -o tmp/lcov/hiredis.info
-	genhtml --legend -o tmp/lcov/report tmp/lcov/hiredis.info
+	$(MAKE) PROF="-fprofile-arcs -ftest-coverage"

 noopt:
 	$(MAKE) OPTIMIZATION=""
-
-.PHONY: all test check clean dep install 32bit gprof gcov noopt
@@ -73,7 +73,7 @@ convert it to the protocol used to communicate with Redis.
 One or more spaces separates arguments, so you can use the specifiers
 anywhere in an argument:

-    reply = redisCommand(context, "SET key:%s %s", myid, value);
+    reply = redisCommand("SET key:%s %s", myid, value);

 ### Using replies

@@ -116,12 +116,6 @@ Note that this function will take care of freeing sub-replies objects
 contained in arrays and nested arrays, so there is no need for the user to
 free the sub replies (it is actually harmful and will corrupt the memory).

-**Important:** the current version of hiredis (0.10.0) free's replies when the
-asynchronous API is used. This means you should not call `freeReplyObject` when
-you use this API. The reply is cleaned up by hiredis _after_ the callback
-returns. This behavior will probably change in future releases, so make sure to
-keep an eye on the changelog when upgrading (see issue #39).
-
 ### Cleaning up

 To disconnect and free the context the following function can be used:
@@ -286,8 +280,7 @@ is being disconnected per user-request, no new commands may be added to the outp
 returned on calls to the `redisAsyncCommand` family.

 If the reply for a command with a `NULL` callback is read, it is immediately free'd. When the callback
-for a command is non-`NULL`, the memory is free'd immediately following the callback: the reply is only
-valid for the duration of the callback.
+for a command is non-`NULL`, it is responsible for cleaning up the reply.

 All pending callbacks are called with a `NULL` reply when the context encountered an error.

@@ -310,68 +303,7 @@ See the `adapters/` directory for bindings to *libev* and *libevent*.

 ## Reply parsing API

-Hiredis comes with a reply parsing API that makes it easy for writing higher
-level language bindings.
-
-The reply parsing API consists of the following functions:
-
-    redisReader *redisReaderCreate(void);
-    void redisReaderFree(redisReader *reader);
-    int redisReaderFeed(redisReader *reader, const char *buf, size_t len);
-    int redisReaderGetReply(redisReader *reader, void **reply);
-
-The same set of functions are used internally by hiredis when creating a
-normal Redis context, the above API just exposes it to the user for a direct
-usage.
-
-### Usage
-
-The function `redisReaderCreate` creates a `redisReader` structure that holds a
-buffer with unparsed data and state for the protocol parser.
-
-Incoming data -- most likely from a socket -- can be placed in the internal
-buffer of the `redisReader` using `redisReaderFeed`. This function will make a
-copy of the buffer pointed to by `buf` for `len` bytes. This data is parsed
-when `redisReaderGetReply` is called. This function returns an integer status
-and a reply object (as described above) via `void **reply`. The returned status
-can be either `REDIS_OK` or `REDIS_ERR`, where the latter means something went
-wrong (either a protocol error, or an out of memory error).
-
-### Customizing replies
-
-The function `redisReaderGetReply` creates `redisReply` and makes the function
-argument `reply` point to the created `redisReply` variable. For instance, if
-the response of type `REDIS_REPLY_STATUS` then the `str` field of `redisReply`
-will hold the status as a vanilla C string. However, the functions that are
-responsible for creating instances of the `redisReply` can be customized by
-setting the `fn` field on the `redisReader` struct. This should be done
-immediately after creating the `redisReader`.
-
-For example, [hiredis-rb](https://github.com/pietern/hiredis-rb/blob/master/ext/hiredis_ext/reader.c)
-uses customized reply object functions to create Ruby objects.
-
-### Reader max buffer
-
-Both when using the Reader API directly or when using it indirectly via a
-normal Redis context, the redisReader structure uses a buffer in order to
-accumulate data from the server.
-Usually this buffer is destroyed when it is empty and is larger than 16
-kb in order to avoid wasting memory in unused buffers
-
-However when working with very big payloads destroying the buffer may slow
-down performances considerably, so it is possible to modify the max size of
-an idle buffer changing the value of the `maxbuf` field of the reader structure
-to the desired value. The special value of 0 means that there is no maximum
-value for an idle buffer, so the buffer will never get freed.
-
-For instance if you have a normal Redis context you can set the maximum idle
-buffer to zero (unlimited) just with:
-
-    context->reader->maxbuf = 0;
-
-This should be done only in order to maximize performances when working with
-large payloads. The context should be set back to `REDIS_READER_MAX_BUF` again
-as soon as possible in order to prevent allocation of useless memory.
+To be done.

 ## AUTHORS

@@ -0,0 +1,2 @@
+- add redisCommandVector()
+- add support for pipelining
@@ -1,35 +1,3 @@
-/*
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- *
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef __HIREDIS_AE_H__
-#define __HIREDIS_AE_H__
 #include <sys/types.h>
 #include <ae.h>
 #include "../hiredis.h"
@@ -42,21 +10,21 @@ typedef struct redisAeEvents {
    int reading, writing;
 } redisAeEvents;

-static void redisAeReadEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
+void redisAeReadEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
    ((void)el); ((void)fd); ((void)mask);

    redisAeEvents *e = (redisAeEvents*)privdata;
    redisAsyncHandleRead(e->context);
 }

-static void redisAeWriteEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
+void redisAeWriteEvent(aeEventLoop *el, int fd, void *privdata, int mask) {
    ((void)el); ((void)fd); ((void)mask);

    redisAeEvents *e = (redisAeEvents*)privdata;
    redisAsyncHandleWrite(e->context);
 }

-static void redisAeAddRead(void *privdata) {
+void redisAeAddRead(void *privdata) {
    redisAeEvents *e = (redisAeEvents*)privdata;
    aeEventLoop *loop = e->loop;
    if (!e->reading) {
@@ -65,7 +33,7 @@ static void redisAeAddRead(void *privdata) {
    }
 }

-static void redisAeDelRead(void *privdata) {
+void redisAeDelRead(void *privdata) {
    redisAeEvents *e = (redisAeEvents*)privdata;
    aeEventLoop *loop = e->loop;
    if (e->reading) {
@@ -74,7 +42,7 @@ static void redisAeDelRead(void *privdata) {
    }
 }

-static void redisAeAddWrite(void *privdata) {
+void redisAeAddWrite(void *privdata) {
    redisAeEvents *e = (redisAeEvents*)privdata;
    aeEventLoop *loop = e->loop;
    if (!e->writing) {
@@ -83,7 +51,7 @@ static void redisAeAddWrite(void *privdata) {
    }
 }

-static void redisAeDelWrite(void *privdata) {
+void redisAeDelWrite(void *privdata) {
    redisAeEvents *e = (redisAeEvents*)privdata;
    aeEventLoop *loop = e->loop;
    if (e->writing) {
@@ -92,19 +60,19 @@ static void redisAeDelWrite(void *privdata) {
    }
 }

-static void redisAeCleanup(void *privdata) {
+void redisAeCleanup(void *privdata) {
    redisAeEvents *e = (redisAeEvents*)privdata;
    redisAeDelRead(privdata);
    redisAeDelWrite(privdata);
    free(e);
 }

-static int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
+int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
    redisContext *c = &(ac->c);
    redisAeEvents *e;

    /* Nothing should be attached when something is already attached */
-    if (ac->ev.data != NULL)
+    if (ac->_adapter_data != NULL)
        return REDIS_ERR;

    /* Create container for context and r/w events */
@@ -115,13 +83,13 @@ static int redisAeAttach(aeEventLoop *loop, redisAsyncContext *ac) {
    e->reading = e->writing = 0;

    /* Register functions to start/stop listening for events */
-    ac->ev.addRead = redisAeAddRead;
-    ac->ev.delRead = redisAeDelRead;
-    ac->ev.addWrite = redisAeAddWrite;
-    ac->ev.delWrite = redisAeDelWrite;
-    ac->ev.cleanup = redisAeCleanup;
-    ac->ev.data = e;
+    ac->evAddRead = redisAeAddRead;
+    ac->evDelRead = redisAeDelRead;
+    ac->evAddWrite = redisAeAddWrite;
+    ac->evDelWrite = redisAeDelWrite;
+    ac->evCleanup = redisAeCleanup;
+    ac->_adapter_data = e;

    return REDIS_OK;
 }
-#endif
+
@@ -1,36 +1,3 @@
-/*
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- *
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef __HIREDIS_LIBEV_H__
-#define __HIREDIS_LIBEV_H__
-#include <stdlib.h>
 #include <sys/types.h>
 #include <ev.h>
 #include "../hiredis.h"
@@ -43,7 +10,7 @@ typedef struct redisLibevEvents {
    ev_io rev, wev;
 } redisLibevEvents;

-static void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
+void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
 #if EV_MULTIPLICITY
    ((void)loop);
 #endif
@@ -53,7 +20,7 @@ static void redisLibevReadEvent(EV_P_ ev_io *watcher, int revents) {
    redisAsyncHandleRead(e->context);
 }

-static void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
+void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
 #if EV_MULTIPLICITY
    ((void)loop);
 #endif
@@ -63,7 +30,7 @@ static void redisLibevWriteEvent(EV_P_ ev_io *watcher, int revents) {
    redisAsyncHandleWrite(e->context);
 }

-static void redisLibevAddRead(void *privdata) {
+void redisLibevAddRead(void *privdata) {
    redisLibevEvents *e = (redisLibevEvents*)privdata;
    struct ev_loop *loop = e->loop;
    ((void)loop);
@@ -73,7 +40,7 @@ static void redisLibevAddRead(void *privdata) {
    }
 }

-static void redisLibevDelRead(void *privdata) {
+void redisLibevDelRead(void *privdata) {
    redisLibevEvents *e = (redisLibevEvents*)privdata;
    struct ev_loop *loop = e->loop;
    ((void)loop);
@@ -83,7 +50,7 @@ static void redisLibevDelRead(void *privdata) {
    }
 }

-static void redisLibevAddWrite(void *privdata) {
+void redisLibevAddWrite(void *privdata) {
    redisLibevEvents *e = (redisLibevEvents*)privdata;
    struct ev_loop *loop = e->loop;
    ((void)loop);
@@ -93,7 +60,7 @@ static void redisLibevAddWrite(void *privdata) {
    }
 }

-static void redisLibevDelWrite(void *privdata) {
+void redisLibevDelWrite(void *privdata) {
    redisLibevEvents *e = (redisLibevEvents*)privdata;
    struct ev_loop *loop = e->loop;
    ((void)loop);
@@ -103,19 +70,19 @@ static void redisLibevDelWrite(void *privdata) {
    }
 }

-static void redisLibevCleanup(void *privdata) {
+void redisLibevCleanup(void *privdata) {
    redisLibevEvents *e = (redisLibevEvents*)privdata;
    redisLibevDelRead(privdata);
    redisLibevDelWrite(privdata);
    free(e);
 }

-static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
+int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
    redisContext *c = &(ac->c);
    redisLibevEvents *e;

    /* Nothing should be attached when something is already attached */
-    if (ac->ev.data != NULL)
+    if (ac->_adapter_data != NULL)
        return REDIS_ERR;

    /* Create container for context and r/w events */
@@ -131,12 +98,12 @@ static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
    e->wev.data = e;

    /* Register functions to start/stop listening for events */
-    ac->ev.addRead = redisLibevAddRead;
-    ac->ev.delRead = redisLibevDelRead;
-    ac->ev.addWrite = redisLibevAddWrite;
-    ac->ev.delWrite = redisLibevDelWrite;
-    ac->ev.cleanup = redisLibevCleanup;
-    ac->ev.data = e;
+    ac->evAddRead = redisLibevAddRead;
+    ac->evDelRead = redisLibevDelRead;
+    ac->evAddWrite = redisLibevAddWrite;
+    ac->evDelWrite = redisLibevDelWrite;
+    ac->evCleanup = redisLibevCleanup;
+    ac->_adapter_data = e;

    /* Initialize read/write events */
    ev_io_init(&e->rev,redisLibevReadEvent,c->fd,EV_READ);
@@ -144,4 +111,3 @@ static int redisLibevAttach(EV_P_ redisAsyncContext *ac) {
    return REDIS_OK;
 }

-#endif
@@ -1,35 +1,4 @@
-/*
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
- *
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *   * Redistributions of source code must retain the above copyright notice,
- *     this list of conditions and the following disclaimer.
- *   * Redistributions in binary form must reproduce the above copyright
- *     notice, this list of conditions and the following disclaimer in the
- *     documentation and/or other materials provided with the distribution.
- *   * Neither the name of Redis nor the names of its contributors may be used
- *     to endorse or promote products derived from this software without
- *     specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef __HIREDIS_LIBEVENT_H__
-#define __HIREDIS_LIBEVENT_H__
+#include <sys/types.h>
 #include <event.h>
 #include "../hiredis.h"
 #include "../async.h"
@@ -39,51 +8,51 @@ typedef struct redisLibeventEvents {
    struct event rev, wev;
 } redisLibeventEvents;

-static void redisLibeventReadEvent(int fd, short event, void *arg) {
+void redisLibeventReadEvent(int fd, short event, void *arg) {
    ((void)fd); ((void)event);
    redisLibeventEvents *e = (redisLibeventEvents*)arg;
    redisAsyncHandleRead(e->context);
 }

-static void redisLibeventWriteEvent(int fd, short event, void *arg) {
+void redisLibeventWriteEvent(int fd, short event, void *arg) {
    ((void)fd); ((void)event);
    redisLibeventEvents *e = (redisLibeventEvents*)arg;
    redisAsyncHandleWrite(e->context);
 }

-static void redisLibeventAddRead(void *privdata) {
+void redisLibeventAddRead(void *privdata) {
    redisLibeventEvents *e = (redisLibeventEvents*)privdata;
    event_add(&e->rev,NULL);
 }

-static void redisLibeventDelRead(void *privdata) {
+void redisLibeventDelRead(void *privdata) {
    redisLibeventEvents *e = (redisLibeventEvents*)privdata;
    event_del(&e->rev);
 }

-static void redisLibeventAddWrite(void *privdata) {
+void redisLibeventAddWrite(void *privdata) {
    redisLibeventEvents *e = (redisLibeventEvents*)privdata;
    event_add(&e->wev,NULL);
 }

-static void redisLibeventDelWrite(void *privdata) {
+void redisLibeventDelWrite(void *privdata) {
    redisLibeventEvents *e = (redisLibeventEvents*)privdata;
    event_del(&e->wev);
 }

-static void redisLibeventCleanup(void *privdata) {
+void redisLibeventCleanup(void *privdata) {
    redisLibeventEvents *e = (redisLibeventEvents*)privdata;
    event_del(&e->rev);
    event_del(&e->wev);
    free(e);
 }

-static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
+int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
    redisContext *c = &(ac->c);
    redisLibeventEvents *e;

    /* Nothing should be attached when something is already attached */
-    if (ac->ev.data != NULL)
+    if (ac->_adapter_data != NULL)
        return REDIS_ERR;

    /* Create container for context and r/w events */
@@ -91,12 +60,12 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
    e->context = ac;

    /* Register functions to start/stop listening for events */
-    ac->ev.addRead = redisLibeventAddRead;
-    ac->ev.delRead = redisLibeventDelRead;
-    ac->ev.addWrite = redisLibeventAddWrite;
-    ac->ev.delWrite = redisLibeventDelWrite;
-    ac->ev.cleanup = redisLibeventCleanup;
-    ac->ev.data = e;
+    ac->evAddRead = redisLibeventAddRead;
+    ac->evDelRead = redisLibeventDelRead;
+    ac->evAddWrite = redisLibeventAddWrite;
+    ac->evDelWrite = redisLibeventDelWrite;
+    ac->evCleanup = redisLibeventCleanup;
+    ac->_adapter_data = e;

    /* Initialize and install read/write events */
    event_set(&e->rev,c->fd,EV_READ,redisLibeventReadEvent,e);
@@ -105,4 +74,3 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
    event_base_set(base,&e->wev);
    return REDIS_OK;
 }
-#endif
@@ -1,6 +1,6 @@
 /*
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
+ * Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
 *
 * All rights reserved.
 *
@@ -29,33 +29,14 @@
 * POSSIBILITY OF SUCH DAMAGE.
 */

-#include "fmacros.h"
-#include <stdlib.h>
 #include <string.h>
 #include <strings.h>
 #include <assert.h>
 #include <ctype.h>
-#include <errno.h>
 #include "async.h"
-#include "net.h"
 #include "dict.c"
 #include "sds.h"
-
-#define _EL_ADD_READ(ctx) do { \
-        if ((ctx)->ev.addRead) (ctx)->ev.addRead((ctx)->ev.data); \
-    } while(0)
-#define _EL_DEL_READ(ctx) do { \
-        if ((ctx)->ev.delRead) (ctx)->ev.delRead((ctx)->ev.data); \
-    } while(0)
-#define _EL_ADD_WRITE(ctx) do { \
-        if ((ctx)->ev.addWrite) (ctx)->ev.addWrite((ctx)->ev.data); \
-    } while(0)
-#define _EL_DEL_WRITE(ctx) do { \
-        if ((ctx)->ev.delWrite) (ctx)->ev.delWrite((ctx)->ev.data); \
-    } while(0)
-#define _EL_CLEANUP(ctx) do { \
-        if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
-    } while(0);
+#include "util.h"

 /* Forward declaration of function in hiredis.c */
 void __redisAppendCommand(redisContext *c, char *cmd, size_t len);
@@ -155,6 +136,11 @@ redisAsyncContext *redisAsyncConnectUnix(const char *path) {
    return ac;
 }

+int redisAsyncSetReplyObjectFunctions(redisAsyncContext *ac, redisReplyObjectFunctions *fn) {
+    redisContext *c = &(ac->c);
+    return redisSetReplyObjectFunctions(c,fn);
+}
+
 int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn) {
    if (ac->onConnect == NULL) {
        ac->onConnect = fn;
@@ -162,7 +148,7 @@ int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn
        /* The common way to detect an established connection is to wait for
         * the first write event to be fired. This assumes the related event
         * library functions are already set. */
-        _EL_ADD_WRITE(ac);
+        if (ac->ev.addWrite) ac->ev.addWrite(ac->ev.data);
        return REDIS_OK;
    }
    return REDIS_ERR;
@@ -182,6 +168,7 @@ static int __redisPushCallback(redisCallbackList *list, redisCallback *source) {

    /* Copy callback from stack to heap */
    cb = malloc(sizeof(*cb));
+    if (!cb) redisOOM();
    if (source != NULL) {
        memcpy(cb,source,sizeof(*cb));
        cb->next = NULL;
@@ -250,7 +237,7 @@ static void __redisAsyncFree(redisAsyncContext *ac) {
    dictRelease(ac->sub.patterns);

    /* Signal event lib to clean up */
-    _EL_CLEANUP(ac);
+    if (ac->ev.cleanup) ac->ev.cleanup(ac->ev.data);

    /* Execute disconnect callback. When redisAsyncFree() initiated destroying
     * this context, the status will always be REDIS_OK. */
@@ -372,11 +359,6 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
                __redisAsyncDisconnect(ac);
                return;
            }
-            
-            /* If monitor mode, repush callback */
-            if(c->flags & REDIS_MONITORING) {
-                __redisPushCallback(&ac->replies,&cb);
-            }

            /* When the connection is not being disconnected, simply stop
             * trying to get replies and wait for the next loop tick. */
@@ -386,36 +368,14 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
        /* Even if the context is subscribed, pending regular callbacks will
         * get a reply before pub/sub messages arrive. */
        if (__redisShiftCallback(&ac->replies,&cb) != REDIS_OK) {
-            /*
-             * A spontaneous reply in a not-subscribed context can be the error
-             * reply that is sent when a new connection exceeds the maximum
-             * number of allowed connections on the server side.
-             *
-             * This is seen as an error instead of a regular reply because the
-             * server closes the connection after sending it.
-             *
-             * To prevent the error from being overwritten by an EOF error the
-             * connection is closed here. See issue #43.
-             *
-             * Another possibility is that the server is loading its dataset.
-             * In this case we also want to close the connection, and have the
-             * user wait until the server is ready to take our request.
-             */
-            if (((redisReply*)reply)->type == REDIS_REPLY_ERROR) {
-                c->err = REDIS_ERR_OTHER;
-                snprintf(c->errstr,sizeof(c->errstr),"%s",((redisReply*)reply)->str);
-                __redisAsyncDisconnect(ac);
-                return;
-            }
-            /* No more regular callbacks and no errors, the context *must* be subscribed or monitoring. */
-            assert((c->flags & REDIS_SUBSCRIBED || c->flags & REDIS_MONITORING));
-            if(c->flags & REDIS_SUBSCRIBED)
-                __redisGetSubscribeCallback(ac,reply,&cb);
+            /* No more regular callbacks, the context *must* be subscribed. */
+            assert(c->flags & REDIS_SUBSCRIBED);
+            __redisGetSubscribeCallback(ac,reply,&cb);
        }

        if (cb.fn != NULL) {
            __redisRunCallback(ac,&cb,reply);
-            c->reader->fn->freeObject(reply);
+            c->fn->freeObject(reply);

            /* Proceed with free'ing when redisAsyncFree() was called. */
            if (c->flags & REDIS_FREEING) {
@@ -427,7 +387,7 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
             * or there were no callbacks to begin with. Either way, don't
             * abort with an error, but simply ignore it because the client
             * doesn't know what the server will spit out over the wire. */
-            c->reader->fn->freeObject(reply);
+            c->fn->freeObject(reply);
        }
    }

@@ -436,48 +396,17 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
        __redisAsyncDisconnect(ac);
 }

-/* Internal helper function to detect socket status the first time a read or
- * write event fires. When connecting was not succesful, the connect callback
- * is called with a REDIS_ERR status and the context is free'd. */
-static int __redisAsyncHandleConnect(redisAsyncContext *ac) {
-    redisContext *c = &(ac->c);
-
-    if (redisCheckSocketError(c,c->fd) == REDIS_ERR) {
-        /* Try again later when connect(2) is still in progress. */
-        if (errno == EINPROGRESS)
-            return REDIS_OK;
-
-        if (ac->onConnect) ac->onConnect(ac,REDIS_ERR);
-        __redisAsyncDisconnect(ac);
-        return REDIS_ERR;
-    }
-
-    /* Mark context as connected. */
-    c->flags |= REDIS_CONNECTED;
-    if (ac->onConnect) ac->onConnect(ac,REDIS_OK);
-    return REDIS_OK;
-}
-
 /* This function should be called when the socket is readable.
 * It processes all replies that can be read and executes their callbacks.
 */
 void redisAsyncHandleRead(redisAsyncContext *ac) {
    redisContext *c = &(ac->c);

-    if (!(c->flags & REDIS_CONNECTED)) {
-        /* Abort connect was not successful. */
-        if (__redisAsyncHandleConnect(ac) != REDIS_OK)
-            return;
-        /* Try again later when the context is still not connected. */
-        if (!(c->flags & REDIS_CONNECTED))
-            return;
-    }
-
    if (redisBufferRead(c) == REDIS_ERR) {
        __redisAsyncDisconnect(ac);
    } else {
        /* Always re-schedule reads */
-        _EL_ADD_READ(ac);
+        if (ac->ev.addRead) ac->ev.addRead(ac->ev.data);
        redisProcessCallbacks(ac);
    }
 }
@@ -486,26 +415,24 @@ void redisAsyncHandleWrite(redisAsyncContext *ac) {
    redisContext *c = &(ac->c);
    int done = 0;

-    if (!(c->flags & REDIS_CONNECTED)) {
-        /* Abort connect was not successful. */
-        if (__redisAsyncHandleConnect(ac) != REDIS_OK)
-            return;
-        /* Try again later when the context is still not connected. */
-        if (!(c->flags & REDIS_CONNECTED))
-            return;
-    }
-
    if (redisBufferWrite(c,&done) == REDIS_ERR) {
        __redisAsyncDisconnect(ac);
    } else {
        /* Continue writing when not done, stop writing otherwise */
-        if (!done)
-            _EL_ADD_WRITE(ac);
-        else
-            _EL_DEL_WRITE(ac);
+        if (!done) {
+            if (ac->ev.addWrite) ac->ev.addWrite(ac->ev.data);
+        } else {
+            if (ac->ev.delWrite) ac->ev.delWrite(ac->ev.data);
+        }

        /* Always schedule reads after writes */
-        _EL_ADD_READ(ac);
+        if (ac->ev.addRead) ac->ev.addRead(ac->ev.data);
+
+        /* Fire onConnect when this is the first write event. */
+        if (!(c->flags & REDIS_CONNECTED)) {
+            c->flags |= REDIS_CONNECTED;
+            if (ac->onConnect) ac->onConnect(ac);
+        }
    }
 }

@@ -571,10 +498,6 @@ static int __redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void
        /* (P)UNSUBSCRIBE does not have its own response: every channel or
         * pattern that is unsubscribed will receive a message. This means we
         * should not append a callback function for this command. */
-     } else if(strncasecmp(cstr,"monitor\r\n",9) == 0) {
-         /* Set monitor flag and push callback */
-         c->flags |= REDIS_MONITORING;
-         __redisPushCallback(&ac->replies,&cb);
    } else {
        if (c->flags & REDIS_SUBSCRIBED)
            /* This will likely result in an error reply, but it needs to be
@@ -587,7 +510,7 @@ static int __redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void
    __redisAppendCommand(c,cmd,len);

    /* Always schedule a write when the write buffer is non-empty */
-    _EL_ADD_WRITE(ac);
+    if (ac->ev.addWrite) ac->ev.addWrite(ac->ev.data);

    return REDIS_OK;
 }
@@ -1,6 +1,6 @@
 /*
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
+ * Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
 *
 * All rights reserved.
 *
@@ -55,7 +55,7 @@ typedef struct redisCallbackList {

 /* Connection callback prototypes */
 typedef void (redisDisconnectCallback)(const struct redisAsyncContext*, int status);
-typedef void (redisConnectCallback)(const struct redisAsyncContext*, int status);
+typedef void (redisConnectCallback)(const struct redisAsyncContext*);

 /* Context for an async connection to Redis */
 typedef struct redisAsyncContext {
@@ -103,6 +103,7 @@ typedef struct redisAsyncContext {
 /* Functions that proxy to hiredis */
 redisAsyncContext *redisAsyncConnect(const char *ip, int port);
 redisAsyncContext *redisAsyncConnectUnix(const char *path);
+int redisAsyncSetReplyObjectFunctions(redisAsyncContext *ac, redisReplyObjectFunctions *fn);
 int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
 int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
 void redisAsyncDisconnect(redisAsyncContext *ac);
@@ -18,20 +18,17 @@ void getCallback(redisAsyncContext *c, void *r, void *privdata) {
    redisAsyncDisconnect(c);
 }

-void connectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Connected...\n");
+void connectCallback(const redisAsyncContext *c) {
+    ((void)c);
+    printf("connected...\n");
 }

 void disconnectCallback(const redisAsyncContext *c, int status) {
    if (status != REDIS_OK) {
        printf("Error: %s\n", c->errstr);
-        return;
    }
-    printf("Disconnected...\n");
+    printf("disconnected...\n");
+    aeStop(loop);
 }

 int main (int argc, char **argv) {
@@ -15,20 +15,16 @@ void getCallback(redisAsyncContext *c, void *r, void *privdata) {
    redisAsyncDisconnect(c);
 }

-void connectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Connected...\n");
+void connectCallback(const redisAsyncContext *c) {
+    ((void)c);
+    printf("connected...\n");
 }

 void disconnectCallback(const redisAsyncContext *c, int status) {
    if (status != REDIS_OK) {
        printf("Error: %s\n", c->errstr);
-        return;
    }
-    printf("Disconnected...\n");
+    printf("disconnected...\n");
 }

 int main (int argc, char **argv) {
@@ -15,20 +15,16 @@ void getCallback(redisAsyncContext *c, void *r, void *privdata) {
    redisAsyncDisconnect(c);
 }

-void connectCallback(const redisAsyncContext *c, int status) {
-    if (status != REDIS_OK) {
-        printf("Error: %s\n", c->errstr);
-        return;
-    }
-    printf("Connected...\n");
+void connectCallback(const redisAsyncContext *c) {
+    ((void)c);
+    printf("connected...\n");
 }

 void disconnectCallback(const redisAsyncContext *c, int status) {
    if (status != REDIS_OK) {
        printf("Error: %s\n", c->errstr);
-        return;
    }
-    printf("Disconnected...\n");
+    printf("disconnected...\n");
 }

 int main (int argc, char **argv) {
@@ -1,14 +1,12 @@
 #ifndef __HIREDIS_FMACRO_H
 #define __HIREDIS_FMACRO_H

-#if !defined(_BSD_SOURCE)
+#ifndef _BSD_SOURCE
 #define _BSD_SOURCE
 #endif

-#if defined(__sun__)
-#define _POSIX_C_SOURCE 200112L
-#elif defined(__linux__) || defined(__OpenBSD__) || defined(__NetBSD__)
-#define _XOPEN_SOURCE 600
+#ifdef __linux__
+#define _XOPEN_SOURCE 700
 #else
 #define _XOPEN_SOURCE
 #endif
@@ -1,6 +1,6 @@
 /*
- * Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
+ * Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
 *
 * All rights reserved.
 *
@@ -36,8 +36,8 @@
 #include <sys/time.h> /* for struct timeval */

 #define HIREDIS_MAJOR 0
-#define HIREDIS_MINOR 10
-#define HIREDIS_PATCH 1
+#define HIREDIS_MINOR 9
+#define HIREDIS_PATCH 2

 #define REDIS_ERR -1
 #define REDIS_OK 0
@@ -46,11 +46,10 @@
 * error that occured. REDIS_ERR_IO means there was an I/O error and you
 * should use the "errno" variable to find out what is wrong.
 * For other values, the "errstr" field will hold a description. */
-#define REDIS_ERR_IO 1 /* Error in read or write */
-#define REDIS_ERR_EOF 3 /* End of file */
-#define REDIS_ERR_PROTOCOL 4 /* Protocol error */
-#define REDIS_ERR_OOM 5 /* Out of memory */
-#define REDIS_ERR_OTHER 2 /* Everything else... */
+#define REDIS_ERR_IO 1 /* error in read or write */
+#define REDIS_ERR_EOF 3 /* eof */
+#define REDIS_ERR_PROTOCOL 4 /* protocol error */
+#define REDIS_ERR_OTHER 2 /* something else */

 /* Connection type can be blocking or non-blocking and is set in the
 * least significant bit of the flags field in redisContext. */
@@ -76,9 +75,6 @@
 /* Flag that is set when the async context has one or more subscriptions. */
 #define REDIS_SUBSCRIBED 0x20

-/* Flag that is set when monitor mode is active */
-#define REDIS_MONITORING 0x40
-
 #define REDIS_REPLY_STRING 1
 #define REDIS_REPLY_ARRAY 2
 #define REDIS_REPLY_INTEGER 3
@@ -86,8 +82,6 @@
 #define REDIS_REPLY_STATUS 5
 #define REDIS_REPLY_ERROR 6

-#define REDIS_READER_MAX_BUF (1024*16)  /* Default max unused reader buffer. */
-
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -119,57 +113,36 @@ typedef struct redisReplyObjectFunctions {
    void (*freeObject)(void*);
 } redisReplyObjectFunctions;

-/* State for the protocol parser */
-typedef struct redisReader {
+struct redisContext; /* need forward declaration of redisContext */
+
+/* Context for a connection to Redis */
+typedef struct redisContext {
+    int fd;
+    int flags;
+    char *obuf; /* Write buffer */
    int err; /* Error flags, 0 when there is no error */
-    char errstr[128]; /* String representation of error when applicable */
-
-    char *buf; /* Read buffer */
-    size_t pos; /* Buffer cursor */
-    size_t len; /* Buffer length */
-    size_t maxbuf; /* Max length of unused buffer */
-
-    redisReadTask rstack[9];
-    int ridx; /* Index of current read task */
-    void *reply; /* Temporary reply pointer */
+    char *errstr; /* String representation of error when applicable */

+    /* Function set for reply buildup and reply reader */
    redisReplyObjectFunctions *fn;
-    void *privdata;
-} redisReader;
+    void *reader;
+} redisContext;

-/* Public API for the protocol parser. */
-redisReader *redisReaderCreate(void);
-void redisReaderFree(redisReader *r);
-int redisReaderFeed(redisReader *r, const char *buf, size_t len);
-int redisReaderGetReply(redisReader *r, void **reply);
-
-/* Backwards compatibility, can be removed on big version bump. */
-#define redisReplyReaderCreate redisReaderCreate
-#define redisReplyReaderFree redisReaderFree
-#define redisReplyReaderFeed redisReaderFeed
-#define redisReplyReaderGetReply redisReaderGetReply
-#define redisReplyReaderSetPrivdata(_r, _p) (int)(((redisReader*)(_r))->privdata = (_p))
-#define redisReplyReaderGetObject(_r) (((redisReader*)(_r))->reply)
-#define redisReplyReaderGetError(_r) (((redisReader*)(_r))->errstr)
-
-/* Function to free the reply objects hiredis returns by default. */
 void freeReplyObject(void *reply);
+void *redisReplyReaderCreate(void);
+int redisReplyReaderSetReplyObjectFunctions(void *reader, redisReplyObjectFunctions *fn);
+int redisReplyReaderSetPrivdata(void *reader, void *privdata);
+void *redisReplyReaderGetObject(void *reader);
+char *redisReplyReaderGetError(void *reader);
+void redisReplyReaderFree(void *ptr);
+void redisReplyReaderFeed(void *reader, const char *buf, size_t len);
+int redisReplyReaderGetReply(void *reader, void **reply);

 /* Functions to format a command according to the protocol. */
 int redisvFormatCommand(char **target, const char *format, va_list ap);
 int redisFormatCommand(char **target, const char *format, ...);
 int redisFormatCommandArgv(char **target, int argc, const char **argv, const size_t *argvlen);

-/* Context for a connection to Redis */
-typedef struct redisContext {
-    int err; /* Error flags, 0 when there is no error */
-    char errstr[128]; /* String representation of error when applicable */
-    int fd;
-    int flags;
-    char *obuf; /* Write buffer */
-    redisReader *reader; /* Protocol reader */
-} redisContext;
-
 redisContext *redisConnect(const char *ip, int port);
 redisContext *redisConnectWithTimeout(const char *ip, int port, struct timeval tv);
 redisContext *redisConnectNonBlock(const char *ip, int port);
@@ -177,6 +150,7 @@ redisContext *redisConnectUnix(const char *path);
 redisContext *redisConnectUnixWithTimeout(const char *path, struct timeval tv);
 redisContext *redisConnectUnixNonBlock(const char *path);
 int redisSetTimeout(redisContext *c, struct timeval tv);
+int redisSetReplyObjectFunctions(redisContext *c, redisReplyObjectFunctions *fn);
 void redisFree(redisContext *c);
 int redisBufferRead(redisContext *c);
 int redisBufferWrite(redisContext *c, int *done);
@@ -190,9 +164,9 @@ int redisGetReplyFromReader(redisContext *c, void **reply);

 /* Write a command to the output buffer. Use these functions in blocking mode
 * to get a pipeline of commands. */
-int redisvAppendCommand(redisContext *c, const char *format, va_list ap);
-int redisAppendCommand(redisContext *c, const char *format, ...);
-int redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);
+void redisvAppendCommand(redisContext *c, const char *format, va_list ap);
+void redisAppendCommand(redisContext *c, const char *format, ...);
+void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);

 /* Issue a command to Redis. In a blocking context, it is identical to calling
 * redisAppendCommand, followed by redisGetReply. The function will return
@@ -1,7 +1,7 @@
 /* Extracted from anet.c to work properly with Hiredis error reporting.
 *
- * Copyright (c) 2006-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
+ * Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
 *
 * All rights reserved.
 *
@@ -45,43 +45,23 @@
 #include <errno.h>
 #include <stdarg.h>
 #include <stdio.h>
-#include <poll.h>
-#include <limits.h>

 #include "net.h"
 #include "sds.h"

-/* Defined in hiredis.c */
-void __redisSetError(redisContext *c, int type, const char *str);
-
-static void __redisSetErrorFromErrno(redisContext *c, int type, const char *prefix) {
-    char buf[128];
-    size_t len = 0;
-
-    if (prefix != NULL)
-        len = snprintf(buf,sizeof(buf),"%s: ",prefix);
-    strerror_r(errno,buf+len,sizeof(buf)-len);
-    __redisSetError(c,type,buf);
-}
-
-static int redisSetReuseAddr(redisContext *c, int fd) {
-    int on = 1;
-    if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
-        close(fd);
-        return REDIS_ERR;
-    }
-    return REDIS_OK;
-}
+/* Forward declaration */
+void __redisSetError(redisContext *c, int type, sds err);

 static int redisCreateSocket(redisContext *c, int type) {
-    int s;
+    int s, on = 1;
    if ((s = socket(type, SOCK_STREAM, 0)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
+        __redisSetError(c,REDIS_ERR_IO,NULL);
        return REDIS_ERR;
    }
    if (type == AF_INET) {
-        if (redisSetReuseAddr(c,s) == REDIS_ERR) {
+        if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) {
+            __redisSetError(c,REDIS_ERR_IO,NULL);
+            close(s);
            return REDIS_ERR;
        }
    }
@@ -95,7 +75,8 @@ static int redisSetBlocking(redisContext *c, int fd, int blocking) {
     * Note that fcntl(2) for F_GETFL and F_SETFL can't be
     * interrupted by a signal. */
    if ((flags = fcntl(fd, F_GETFL)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"fcntl(F_GETFL)");
+        __redisSetError(c,REDIS_ERR_IO,
+            sdscatprintf(sdsempty(), "fcntl(F_GETFL): %s", strerror(errno)));
        close(fd);
        return REDIS_ERR;
    }
@@ -106,7 +87,8 @@ static int redisSetBlocking(redisContext *c, int fd, int blocking) {
        flags |= O_NONBLOCK;

    if (fcntl(fd, F_SETFL, flags) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"fcntl(F_SETFL)");
+        __redisSetError(c,REDIS_ERR_IO,
+            sdscatprintf(sdsempty(), "fcntl(F_SETFL): %s", strerror(errno)));
        close(fd);
        return REDIS_ERR;
    }
@@ -116,148 +98,127 @@ static int redisSetBlocking(redisContext *c, int fd, int blocking) {
 static int redisSetTcpNoDelay(redisContext *c, int fd) {
    int yes = 1;
    if (setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &yes, sizeof(yes)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(TCP_NODELAY)");
+        __redisSetError(c,REDIS_ERR_IO,
+            sdscatprintf(sdsempty(), "setsockopt(TCP_NODELAY): %s", strerror(errno)));
        close(fd);
        return REDIS_ERR;
    }
    return REDIS_OK;
 }

-#define __MAX_MSEC (((LONG_MAX) - 999) / 1000)
-
 static int redisContextWaitReady(redisContext *c, int fd, const struct timeval *timeout) {
-    struct pollfd   wfd[1];
-    long msec;
-
-    msec          = -1;
-    wfd[0].fd     = fd;
-    wfd[0].events = POLLOUT;
+    struct timeval to;
+    struct timeval *toptr = NULL;
+    fd_set wfd;
+    int err;
+    socklen_t errlen;

    /* Only use timeout when not NULL. */
    if (timeout != NULL) {
-        if (timeout->tv_usec > 1000000 || timeout->tv_sec > __MAX_MSEC) {
-            close(fd);
-            return REDIS_ERR;
-        }
-
-        msec = (timeout->tv_sec * 1000) + ((timeout->tv_usec + 999) / 1000);
-
-        if (msec < 0 || msec > INT_MAX) {
-            msec = INT_MAX;
-        }
+        to = *timeout;
+        toptr = &to;
    }

    if (errno == EINPROGRESS) {
-        int res;
+        FD_ZERO(&wfd);
+        FD_SET(fd, &wfd);

-        if ((res = poll(wfd, 1, msec)) == -1) {
-            __redisSetErrorFromErrno(c, REDIS_ERR_IO, "poll(2)");
-            close(fd);
-            return REDIS_ERR;
-        } else if (res == 0) {
-            errno = ETIMEDOUT;
-            __redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
+        if (select(FD_SETSIZE, NULL, &wfd, NULL, toptr) == -1) {
+            __redisSetError(c,REDIS_ERR_IO,
+                sdscatprintf(sdsempty(), "select(2): %s", strerror(errno)));
            close(fd);
            return REDIS_ERR;
        }

-        if (redisCheckSocketError(c, fd) != REDIS_OK)
+        if (!FD_ISSET(fd, &wfd)) {
+            errno = ETIMEDOUT;
+            __redisSetError(c,REDIS_ERR_IO,NULL);
+            close(fd);
            return REDIS_ERR;
+        }
+
+        err = 0;
+        errlen = sizeof(err);
+        if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &err, &errlen) == -1) {
+            __redisSetError(c,REDIS_ERR_IO,
+                sdscatprintf(sdsempty(), "getsockopt(SO_ERROR): %s", strerror(errno)));
+            close(fd);
+            return REDIS_ERR;
+        }
+
+        if (err) {
+            errno = err;
+            __redisSetError(c,REDIS_ERR_IO,NULL);
+            close(fd);
+            return REDIS_ERR;
+        }

        return REDIS_OK;
    }

-    __redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
+    __redisSetError(c,REDIS_ERR_IO,NULL);
    close(fd);
    return REDIS_ERR;
 }

-int redisCheckSocketError(redisContext *c, int fd) {
-    int err = 0;
-    socklen_t errlen = sizeof(err);
-
-    if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &err, &errlen) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"getsockopt(SO_ERROR)");
-        close(fd);
-        return REDIS_ERR;
-    }
-
-    if (err) {
-        errno = err;
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
-        close(fd);
-        return REDIS_ERR;
-    }
-
-    return REDIS_OK;
-}
-
 int redisContextSetTimeout(redisContext *c, struct timeval tv) {
    if (setsockopt(c->fd,SOL_SOCKET,SO_RCVTIMEO,&tv,sizeof(tv)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(SO_RCVTIMEO)");
+        __redisSetError(c,REDIS_ERR_IO,
+            sdscatprintf(sdsempty(), "setsockopt(SO_RCVTIMEO): %s", strerror(errno)));
        return REDIS_ERR;
    }
    if (setsockopt(c->fd,SOL_SOCKET,SO_SNDTIMEO,&tv,sizeof(tv)) == -1) {
-        __redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(SO_SNDTIMEO)");
+        __redisSetError(c,REDIS_ERR_IO,
+            sdscatprintf(sdsempty(), "setsockopt(SO_SNDTIMEO): %s", strerror(errno)));
        return REDIS_ERR;
    }
    return REDIS_OK;
 }

 int redisContextConnectTcp(redisContext *c, const char *addr, int port, struct timeval *timeout) {
-    int s, rv;
-    char _port[6];  /* strlen("65535"); */
-    struct addrinfo hints, *servinfo, *p;
+    int s;
    int blocking = (c->flags & REDIS_BLOCK);
+    struct sockaddr_in sa;

-    snprintf(_port, 6, "%d", port);
-    memset(&hints,0,sizeof(hints));
-    hints.ai_family = AF_INET;
-    hints.ai_socktype = SOCK_STREAM;
-
-    if ((rv = getaddrinfo(addr,_port,&hints,&servinfo)) != 0) {
-        __redisSetError(c,REDIS_ERR_OTHER,gai_strerror(rv));
+    if ((s = redisCreateSocket(c,AF_INET)) < 0)
+        return REDIS_ERR;
+    if (redisSetBlocking(c,s,0) != REDIS_OK)
        return REDIS_ERR;
-    }
-    for (p = servinfo; p != NULL; p = p->ai_next) {
-        if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
-            continue;

-        if (redisSetBlocking(c,s,0) != REDIS_OK)
-            goto error;
-        if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
-            if (errno == EHOSTUNREACH) {
-                close(s);
-                continue;
-            } else if (errno == EINPROGRESS && !blocking) {
-                /* This is ok. */
-            } else {
-                if (redisContextWaitReady(c,s,timeout) != REDIS_OK)
-                    goto error;
-            }
+    sa.sin_family = AF_INET;
+    sa.sin_port = htons(port);
+    if (inet_aton(addr, &sa.sin_addr) == 0) {
+        struct hostent *he;
+
+        he = gethostbyname(addr);
+        if (he == NULL) {
+            __redisSetError(c,REDIS_ERR_OTHER,
+                sdscatprintf(sdsempty(),"Can't resolve: %s",addr));
+            close(s);
+            return REDIS_ERR;
        }
-        if (blocking && redisSetBlocking(c,s,1) != REDIS_OK)
-            goto error;
-        if (redisSetTcpNoDelay(c,s) != REDIS_OK)
-            goto error;
-
-        c->fd = s;
-        c->flags |= REDIS_CONNECTED;
-        rv = REDIS_OK;
-        goto end;
-    }
-    if (p == NULL) {
-        char buf[128];
-        snprintf(buf,sizeof(buf),"Can't create socket: %s",strerror(errno));
-        __redisSetError(c,REDIS_ERR_OTHER,buf);
-        goto error;
+        memcpy(&sa.sin_addr, he->h_addr, sizeof(struct in_addr));
    }

-error:
-    rv = REDIS_ERR;
-end:
-    freeaddrinfo(servinfo);
-    return rv;  // Need to return REDIS_OK if alright
+    if (connect(s, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
+        if (errno == EINPROGRESS && !blocking) {
+            /* This is ok. */
+        } else {
+            if (redisContextWaitReady(c,s,timeout) != REDIS_OK)
+                return REDIS_ERR;
+        }
+    }
+
+    /* Reset socket to be blocking after connect(2). */
+    if (blocking && redisSetBlocking(c,s,1) != REDIS_OK)
+        return REDIS_ERR;
+
+    if (redisSetTcpNoDelay(c,s) != REDIS_OK)
+        return REDIS_ERR;
+
+    c->fd = s;
+    c->flags |= REDIS_CONNECTED;
+    return REDIS_OK;
 }

 int redisContextConnectUnix(redisContext *c, const char *path, struct timeval *timeout) {
@@ -1,7 +1,7 @@
 /* Extracted from anet.c to work properly with Hiredis error reporting.
 *
- * Copyright (c) 2006-2011, Salvatore Sanfilippo <antirez at gmail dot com>
- * Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
+ * Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
 *
 * All rights reserved.
 *
@@ -39,7 +39,6 @@
 #define AF_LOCAL AF_UNIX
 #endif

-int redisCheckSocketError(redisContext *c, int fd);
 int redisContextSetTimeout(redisContext *c, struct timeval tv);
 int redisContextConnectTcp(redisContext *c, const char *addr, int port, struct timeval *timeout);
 int redisContextConnectUnix(redisContext *c, const char *path, struct timeval *timeout);
@@ -28,18 +28,18 @@
 * POSSIBILITY OF SUCH DAMAGE.
 */

+#define SDS_ABORT_ON_OOM
+
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <ctype.h>
 #include "sds.h"

-#ifdef SDS_ABORT_ON_OOM
 static void sdsOomAbort(void) {
    fprintf(stderr,"SDS: Out Of Memory (SDS_ABORT_ON_OOM defined)\n");
    abort();
 }
-#endif

 sds sdsnewlen(const void *init, size_t initlen) {
    struct sdshdr *sh;
@@ -378,19 +378,17 @@ sds sdsfromlonglong(long long value) {

 sds sdscatrepr(sds s, char *p, size_t len) {
    s = sdscatlen(s,"\"",1);
-    if (s == NULL) return NULL;
-
    while(len--) {
        switch(*p) {
        case '\\':
        case '"':
            s = sdscatprintf(s,"\\%c",*p);
            break;
-        case '\n': s = sdscatlen(s,"\\n",2); break;
-        case '\r': s = sdscatlen(s,"\\r",2); break;
-        case '\t': s = sdscatlen(s,"\\t",2); break;
-        case '\a': s = sdscatlen(s,"\\a",2); break;
-        case '\b': s = sdscatlen(s,"\\b",2); break;
+        case '\n': s = sdscatlen(s,"\\n",1); break;
+        case '\r': s = sdscatlen(s,"\\r",1); break;
+        case '\t': s = sdscatlen(s,"\\t",1); break;
+        case '\a': s = sdscatlen(s,"\\a",1); break;
+        case '\b': s = sdscatlen(s,"\\b",1); break;
        default:
            if (isprint(*p))
                s = sdscatprintf(s,"%c",*p);
@@ -399,7 +397,6 @@ sds sdscatrepr(sds s, char *p, size_t len) {
            break;
        }
        p++;
-        if (s == NULL) return NULL;
    }
    return sdscatlen(s,"\"",1);
 }
@@ -419,7 +416,7 @@ sds sdscatrepr(sds s, char *p, size_t len) {
 sds *sdssplitargs(char *line, int *argc) {
    char *p = line;
    char *current = NULL;
-    char **vector = NULL, **_vector = NULL;
+    char **vector = NULL;

    *argc = 0;
    while(1) {
@@ -430,11 +427,7 @@ sds *sdssplitargs(char *line, int *argc) {
            int inq=0; /* set to 1 if we are in "quotes" */
            int done=0;

-            if (current == NULL) {
-                current = sdsempty();
-                if (current == NULL) goto err;
-            }
-
+            if (current == NULL) current = sdsempty();
            while(!done) {
                if (inq) {
                    if (*p == '\\' && *(p+1)) {
@@ -478,13 +471,9 @@ sds *sdssplitargs(char *line, int *argc) {
                    }
                }
                if (*p) p++;
-                if (current == NULL) goto err;
            }
            /* add the token to the vector */
-            _vector = realloc(vector,((*argc)+1)*sizeof(char*));
-            if (_vector == NULL) goto err;
-
-            vector = _vector;
+            vector = realloc(vector,((*argc)+1)*sizeof(char*));
            vector[*argc] = current;
            (*argc)++;
            current = NULL;
@@ -496,8 +485,8 @@ sds *sdssplitargs(char *line, int *argc) {
 err:
    while((*argc)--)
        sdsfree(vector[*argc]);
-    if (vector != NULL) free(vector);
-    if (current != NULL) sdsfree(current);
+    free(vector);
+    if (current) sdsfree(current);
    return NULL;
 }

@@ -1,4 +1,3 @@
-#include "fmacros.h"
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -11,28 +10,10 @@

 #include "hiredis.h"

-enum connection_type {
-    CONN_TCP,
-    CONN_UNIX
-};
-
-struct config {
-    enum connection_type type;
-
-    struct {
-        const char *host;
-        int port;
-    } tcp;
-
-    struct {
-        const char *path;
-    } unix;
-};
-
 /* The following lines make up our testing "framework" :) */
 static int tests = 0, fails = 0;
 #define test(_s) { printf("#%02d ", ++tests); printf(_s); }
-#define test_cond(_c) if(_c) printf("\033[0;32mPASSED\033[0;0m\n"); else {printf("\033[0;31mFAILED\033[0;0m\n"); fails++;}
+#define test_cond(_c) if(_c) printf("PASSED\n"); else {printf("FAILED\n"); fails++;}

 static long long usec(void) {
    struct timeval tv;
@@ -40,60 +21,15 @@ static long long usec(void) {
    return (((long long)tv.tv_sec)*1000000)+tv.tv_usec;
 }

-static redisContext *select_database(redisContext *c) {
-    redisReply *reply;
-
-    /* Switch to DB 9 for testing, now that we know we can chat. */
-    reply = redisCommand(c,"SELECT 9");
-    assert(reply != NULL);
-    freeReplyObject(reply);
-
-    /* Make sure the DB is emtpy */
-    reply = redisCommand(c,"DBSIZE");
-    assert(reply != NULL);
-    if (reply->type == REDIS_REPLY_INTEGER && reply->integer == 0) {
-        /* Awesome, DB 9 is empty and we can continue. */
-        freeReplyObject(reply);
-    } else {
-        printf("Database #9 is not empty, test can not continue\n");
+static int use_unix = 0;
+static redisContext *blocking_context = NULL;
+static void __connect(redisContext **target) {
+    *target = blocking_context = (use_unix ?
+        redisConnectUnix("/tmp/redis.sock") : redisConnect((char*)"127.0.0.1", 6379));
+    if (blocking_context->err) {
+        printf("Connection error: %s\n", blocking_context->errstr);
        exit(1);
    }
-
-    return c;
-}
-
-static void disconnect(redisContext *c) {
-    redisReply *reply;
-
-    /* Make sure we're on DB 9. */
-    reply = redisCommand(c,"SELECT 9");
-    assert(reply != NULL);
-    freeReplyObject(reply);
-    reply = redisCommand(c,"FLUSHDB");
-    assert(reply != NULL);
-    freeReplyObject(reply);
-
-    /* Free the context as well. */
-    redisFree(c);
-}
-
-static redisContext *connect(struct config config) {
-    redisContext *c = NULL;
-
-    if (config.type == CONN_TCP) {
-        c = redisConnect(config.tcp.host, config.tcp.port);
-    } else if (config.type == CONN_UNIX) {
-        c = redisConnectUnix(config.unix.path);
-    } else {
-        assert(NULL);
-    }
-
-    if (c->err) {
-        printf("Connection error: %s\n", c->errstr);
-        exit(1);
-    }
-
-    return select_database(c);
 }

 static void test_format_commands(void) {
@@ -142,43 +78,29 @@ static void test_format_commands(void) {
        len == 4+4+(3+2)+4+(1+2)+4+(1+2));
    free(cmd);

-    /* Vararg width depends on the type. These tests make sure that the
-     * width is correctly determined using the format and subsequent varargs
-     * can correctly be interpolated. */
-#define INTEGER_WIDTH_TEST(fmt, type) do {                                                \
-    type value = 123;                                                                     \
-    test("Format command with printf-delegation (" #type "): ");                          \
-    len = redisFormatCommand(&cmd,"key:%08" fmt " str:%s", value, "hello");               \
-    test_cond(strncmp(cmd,"*2\r\n$12\r\nkey:00000123\r\n$9\r\nstr:hello\r\n",len) == 0 && \
-        len == 4+5+(12+2)+4+(9+2));                                                       \
-    free(cmd);                                                                            \
-} while(0)
+    test("Format command with printf-delegation (long long): ");
+    len = redisFormatCommand(&cmd,"key:%08lld",1234ll);
+    test_cond(strncmp(cmd,"*1\r\n$12\r\nkey:00001234\r\n",len) == 0 &&
+        len == 4+5+(12+2));
+    free(cmd);

-#define FLOAT_WIDTH_TEST(type) do {                                                       \
-    type value = 123.0;                                                                   \
-    test("Format command with printf-delegation (" #type "): ");                          \
-    len = redisFormatCommand(&cmd,"key:%08.3f str:%s", value, "hello");                   \
-    test_cond(strncmp(cmd,"*2\r\n$12\r\nkey:0123.000\r\n$9\r\nstr:hello\r\n",len) == 0 && \
-        len == 4+5+(12+2)+4+(9+2));                                                       \
-    free(cmd);                                                                            \
-} while(0)
+    test("Format command with printf-delegation (float): ");
+    len = redisFormatCommand(&cmd,"v:%06.1f",12.34f);
+    test_cond(strncmp(cmd,"*1\r\n$8\r\nv:0012.3\r\n",len) == 0 &&
+        len == 4+4+(8+2));
+    free(cmd);

-    INTEGER_WIDTH_TEST("d", int);
-    INTEGER_WIDTH_TEST("hhd", char);
-    INTEGER_WIDTH_TEST("hd", short);
-    INTEGER_WIDTH_TEST("ld", long);
-    INTEGER_WIDTH_TEST("lld", long long);
-    INTEGER_WIDTH_TEST("u", unsigned int);
-    INTEGER_WIDTH_TEST("hhu", unsigned char);
-    INTEGER_WIDTH_TEST("hu", unsigned short);
-    INTEGER_WIDTH_TEST("lu", unsigned long);
-    INTEGER_WIDTH_TEST("llu", unsigned long long);
-    FLOAT_WIDTH_TEST(float);
-    FLOAT_WIDTH_TEST(double);
+    test("Format command with printf-delegation and extra interpolation: ");
+    len = redisFormatCommand(&cmd,"key:%d %b",1234,"foo",3);
+    test_cond(strncmp(cmd,"*2\r\n$8\r\nkey:1234\r\n$3\r\nfoo\r\n",len) == 0 &&
+        len == 4+4+(8+2)+4+(3+2));
+    free(cmd);

-    test("Format command with invalid printf format: ");
-    len = redisFormatCommand(&cmd,"key:%08p %b",(void*)1234,"foo",3);
-    test_cond(len == -1);
+    test("Format command with wrong printf format and extra interpolation: ");
+    len = redisFormatCommand(&cmd,"key:%08p %b",1234,"foo",3);
+    test_cond(strncmp(cmd,"*2\r\n$6\r\nkey:8p\r\n$3\r\nfoo\r\n",len) == 0 &&
+        len == 4+4+(6+2)+4+(3+2));
+    free(cmd);

    const char *argv[3];
    argv[0] = "SET";
@@ -200,117 +122,42 @@ static void test_format_commands(void) {
    free(cmd);
 }

-static void test_reply_reader(void) {
-    redisReader *reader;
-    void *reply;
-    int ret;
-
-    test("Error handling in reply parser: ");
-    reader = redisReaderCreate();
-    redisReaderFeed(reader,(char*)"@foo\r\n",6);
-    ret = redisReaderGetReply(reader,NULL);
-    test_cond(ret == REDIS_ERR &&
-              strcasecmp(reader->errstr,"Protocol error, got \"@\" as reply type byte") == 0);
-    redisReaderFree(reader);
-
-    /* when the reply already contains multiple items, they must be free'd
-     * on an error. valgrind will bark when this doesn't happen. */
-    test("Memory cleanup in reply parser: ");
-    reader = redisReaderCreate();
-    redisReaderFeed(reader,(char*)"*2\r\n",4);
-    redisReaderFeed(reader,(char*)"$5\r\nhello\r\n",11);
-    redisReaderFeed(reader,(char*)"@foo\r\n",6);
-    ret = redisReaderGetReply(reader,NULL);
-    test_cond(ret == REDIS_ERR &&
-              strcasecmp(reader->errstr,"Protocol error, got \"@\" as reply type byte") == 0);
-    redisReaderFree(reader);
-
-    test("Set error on nested multi bulks with depth > 2: ");
-    reader = redisReaderCreate();
-    redisReaderFeed(reader,(char*)"*1\r\n",4);
-    redisReaderFeed(reader,(char*)"*1\r\n",4);
-    redisReaderFeed(reader,(char*)"*1\r\n",4);
-    redisReaderFeed(reader,(char*)"*1\r\n",4);
-    ret = redisReaderGetReply(reader,NULL);
-    test_cond(ret == REDIS_ERR &&
-              strncasecmp(reader->errstr,"No support for",14) == 0);
-    redisReaderFree(reader);
-
-    test("Works with NULL functions for reply: ");
-    reader = redisReaderCreate();
-    reader->fn = NULL;
-    redisReaderFeed(reader,(char*)"+OK\r\n",5);
-    ret = redisReaderGetReply(reader,&reply);
-    test_cond(ret == REDIS_OK && reply == (void*)REDIS_REPLY_STATUS);
-    redisReaderFree(reader);
-
-    test("Works when a single newline (\\r\\n) covers two calls to feed: ");
-    reader = redisReaderCreate();
-    reader->fn = NULL;
-    redisReaderFeed(reader,(char*)"+OK\r",4);
-    ret = redisReaderGetReply(reader,&reply);
-    assert(ret == REDIS_OK && reply == NULL);
-    redisReaderFeed(reader,(char*)"\n",1);
-    ret = redisReaderGetReply(reader,&reply);
-    test_cond(ret == REDIS_OK && reply == (void*)REDIS_REPLY_STATUS);
-    redisReaderFree(reader);
-
-    test("Don't reset state after protocol error: ");
-    reader = redisReaderCreate();
-    reader->fn = NULL;
-    redisReaderFeed(reader,(char*)"x",1);
-    ret = redisReaderGetReply(reader,&reply);
-    assert(ret == REDIS_ERR);
-    ret = redisReaderGetReply(reader,&reply);
-    test_cond(ret == REDIS_ERR && reply == NULL);
-    redisReaderFree(reader);
-
-    /* Regression test for issue #45 on GitHub. */
-    test("Don't do empty allocation for empty multi bulk: ");
-    reader = redisReaderCreate();
-    redisReaderFeed(reader,(char*)"*0\r\n",4);
-    ret = redisReaderGetReply(reader,&reply);
-    test_cond(ret == REDIS_OK &&
-        ((redisReply*)reply)->type == REDIS_REPLY_ARRAY &&
-        ((redisReply*)reply)->elements == 0);
-    freeReplyObject(reply);
-    redisReaderFree(reader);
-}
-
-static void test_blocking_connection_errors(void) {
+static void test_blocking_connection(void) {
    redisContext *c;
+    redisReply *reply;
+    int major, minor;

    test("Returns error when host cannot be resolved: ");
    c = redisConnect((char*)"idontexist.local", 6379);
    test_cond(c->err == REDIS_ERR_OTHER &&
-        (strcmp(c->errstr,"Name or service not known") == 0 ||
-         strcmp(c->errstr,"Can't resolve: idontexist.local") == 0));
+        strcmp(c->errstr,"Can't resolve: idontexist.local") == 0);
    redisFree(c);

    test("Returns error when the port is not open: ");
-    c = redisConnect((char*)"localhost", 1);
+    c = redisConnect((char*)"localhost", 56380);
    test_cond(c->err == REDIS_ERR_IO &&
        strcmp(c->errstr,"Connection refused") == 0);
    redisFree(c);

-    test("Returns error when the unix socket path doesn't accept connections: ");
-    c = redisConnectUnix((char*)"/tmp/idontexist.sock");
-    test_cond(c->err == REDIS_ERR_IO); /* Don't care about the message... */
-    redisFree(c);
-}
-
-static void test_blocking_connection(struct config config) {
-    redisContext *c;
-    redisReply *reply;
-
-    c = connect(config);
-
+    __connect(&c);
    test("Is able to deliver commands: ");
    reply = redisCommand(c,"PING");
    test_cond(reply->type == REDIS_REPLY_STATUS &&
        strcasecmp(reply->str,"pong") == 0)
    freeReplyObject(reply);

+    /* Switch to DB 9 for testing, now that we know we can chat. */
+    reply = redisCommand(c,"SELECT 9");
+    freeReplyObject(reply);
+
+    /* Make sure the DB is emtpy */
+    reply = redisCommand(c,"DBSIZE");
+    if (reply->type != REDIS_REPLY_INTEGER || reply->integer != 0) {
+        printf("Database #9 is not empty, test can not continue\n");
+        exit(1);
+    }
+    freeReplyObject(reply);
+
    test("Is a able to send commands verbatim: ");
    reply = redisCommand(c,"SET foo bar");
    test_cond (reply->type == REDIS_REPLY_STATUS &&
@@ -374,17 +221,6 @@ static void test_blocking_connection(struct config config) {
              strcasecmp(reply->element[1]->str,"pong") == 0);
    freeReplyObject(reply);

-    disconnect(c);
-}
-
-static void test_blocking_io_errors(struct config config) {
-    redisContext *c;
-    redisReply *reply;
-    void *_reply;
-    int major, minor;
-
-    /* Connect to target given by config. */
-    c = connect(config);
    {
        /* Find out Redis version to determine the path for the next test */
        const char *field = "redis_version:";
@@ -404,7 +240,7 @@ static void test_blocking_io_errors(struct config config) {
        /* > 2.0 returns OK on QUIT and read() should be issued once more
         * to know the descriptor is at EOF. */
        test_cond(strcasecmp(reply->str,"OK") == 0 &&
-            redisGetReply(c,&_reply) == REDIS_ERR);
+            redisGetReply(c,(void**)&reply) == REDIS_ERR);
        freeReplyObject(reply);
    } else {
        test_cond(reply == NULL);
@@ -419,20 +255,91 @@ static void test_blocking_io_errors(struct config config) {
        strcmp(c->errstr,"Server closed the connection") == 0);
    redisFree(c);

-    c = connect(config);
+    __connect(&c);
    test("Returns I/O error on socket timeout: ");
    struct timeval tv = { 0, 1000 };
    assert(redisSetTimeout(c,tv) == REDIS_OK);
-    test_cond(redisGetReply(c,&_reply) == REDIS_ERR &&
+    test_cond(redisGetReply(c,(void**)&reply) == REDIS_ERR &&
        c->err == REDIS_ERR_IO && errno == EAGAIN);
    redisFree(c);
+
+    /* Context should be connected */
+    __connect(&c);
 }

-static void test_throughput(struct config config) {
-    redisContext *c = connect(config);
-    redisReply **replies;
+static void test_reply_reader(void) {
+    void *reader;
+    void *reply;
+    char *err;
+    int ret;
+
+    test("Error handling in reply parser: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderFeed(reader,(char*)"@foo\r\n",6);
+    ret = redisReplyReaderGetReply(reader,NULL);
+    err = redisReplyReaderGetError(reader);
+    test_cond(ret == REDIS_ERR &&
+              strcasecmp(err,"Protocol error, got \"@\" as reply type byte") == 0);
+    redisReplyReaderFree(reader);
+
+    /* when the reply already contains multiple items, they must be free'd
+     * on an error. valgrind will bark when this doesn't happen. */
+    test("Memory cleanup in reply parser: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderFeed(reader,(char*)"*2\r\n",4);
+    redisReplyReaderFeed(reader,(char*)"$5\r\nhello\r\n",11);
+    redisReplyReaderFeed(reader,(char*)"@foo\r\n",6);
+    ret = redisReplyReaderGetReply(reader,NULL);
+    err = redisReplyReaderGetError(reader);
+    test_cond(ret == REDIS_ERR &&
+              strcasecmp(err,"Protocol error, got \"@\" as reply type byte") == 0);
+    redisReplyReaderFree(reader);
+
+    test("Set error on nested multi bulks with depth > 1: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderFeed(reader,(char*)"*1\r\n",4);
+    redisReplyReaderFeed(reader,(char*)"*1\r\n",4);
+    redisReplyReaderFeed(reader,(char*)"*1\r\n",4);
+    ret = redisReplyReaderGetReply(reader,NULL);
+    err = redisReplyReaderGetError(reader);
+    test_cond(ret == REDIS_ERR &&
+              strncasecmp(err,"No support for",14) == 0);
+    redisReplyReaderFree(reader);
+
+    test("Works with NULL functions for reply: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderSetReplyObjectFunctions(reader,NULL);
+    redisReplyReaderFeed(reader,(char*)"+OK\r\n",5);
+    ret = redisReplyReaderGetReply(reader,&reply);
+    test_cond(ret == REDIS_OK && reply == (void*)REDIS_REPLY_STATUS);
+    redisReplyReaderFree(reader);
+
+    test("Works when a single newline (\\r\\n) covers two calls to feed: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderSetReplyObjectFunctions(reader,NULL);
+    redisReplyReaderFeed(reader,(char*)"+OK\r",4);
+    ret = redisReplyReaderGetReply(reader,&reply);
+    assert(ret == REDIS_OK && reply == NULL);
+    redisReplyReaderFeed(reader,(char*)"\n",1);
+    ret = redisReplyReaderGetReply(reader,&reply);
+    test_cond(ret == REDIS_OK && reply == (void*)REDIS_REPLY_STATUS);
+    redisReplyReaderFree(reader);
+
+    test("Properly reset state after protocol error: ");
+    reader = redisReplyReaderCreate();
+    redisReplyReaderSetReplyObjectFunctions(reader,NULL);
+    redisReplyReaderFeed(reader,(char*)"x",1);
+    ret = redisReplyReaderGetReply(reader,&reply);
+    assert(ret == REDIS_ERR);
+    ret = redisReplyReaderGetReply(reader,&reply);
+    test_cond(ret == REDIS_OK && reply == NULL)
+}
+
+static void test_throughput(void) {
    int i, num;
    long long t1, t2;
+    redisContext *c = blocking_context;
+    redisReply **replies;

    test("Throughput:\n");
    for (i = 0; i < 500; i++)
@@ -489,8 +396,18 @@ static void test_throughput(struct config config) {
    for (i = 0; i < num; i++) freeReplyObject(replies[i]);
    free(replies);
    printf("\t(%dx LRANGE with 500 elements (pipelined): %.3fs)\n", num, (t2-t1)/1000000.0);
+}

-    disconnect(c);
+static void cleanup(void) {
+    redisContext *c = blocking_context;
+    redisReply *reply;
+
+    /* Make sure we're on DB 9 */
+    reply = redisCommand(c,"SELECT 9");
+    assert(reply != NULL); freeReplyObject(reply);
+    reply = redisCommand(c,"FLUSHDB");
+    assert(reply != NULL); freeReplyObject(reply);
+    redisFree(c);
 }

 // static long __test_callback_flags = 0;
@@ -512,7 +429,7 @@ static void test_throughput(struct config config) {
 // static redisContext *__connect_nonblock() {
 //     /* Reset callback flags */
 //     __test_callback_flags = 0;
-//     return redisConnectNonBlock("127.0.0.1", port, NULL);
+//     return redisConnectNonBlock("127.0.0.1", 6379, NULL);
 // }
 //
 // static void test_nonblocking_connection() {
@@ -593,62 +510,23 @@ static void test_throughput(struct config config) {
 // }

 int main(int argc, char **argv) {
-    struct config cfg = {
-        .tcp = {
-            .host = "127.0.0.1",
-            .port = 6379
-        },
-        .unix = {
-            .path = "/tmp/redis.sock"
-        }
-    };
-    int throughput = 1;
+    if (argc > 1) {
+        if (strcmp(argv[1],"-s") == 0)
+            use_unix = 1;
+    }

-    /* Ignore broken pipe signal (for I/O error tests). */
    signal(SIGPIPE, SIG_IGN);
-
-    /* Parse command line options. */
-    argv++; argc--;
-    while (argc) {
-        if (argc >= 2 && !strcmp(argv[0],"-h")) {
-            argv++; argc--;
-            cfg.tcp.host = argv[0];
-        } else if (argc >= 2 && !strcmp(argv[0],"-p")) {
-            argv++; argc--;
-            cfg.tcp.port = atoi(argv[0]);
-        } else if (argc >= 2 && !strcmp(argv[0],"-s")) {
-            argv++; argc--;
-            cfg.unix.path = argv[0];
-        } else if (argc >= 1 && !strcmp(argv[0],"--skip-throughput")) {
-            throughput = 0;
-        } else {
-            fprintf(stderr, "Invalid argument: %s\n", argv[0]);
-            exit(1);
-        }
-        argv++; argc--;
-    }
-
    test_format_commands();
+    test_blocking_connection();
    test_reply_reader();
-    test_blocking_connection_errors();
+    // test_nonblocking_connection();
+    test_throughput();
+    cleanup();

-    printf("\nTesting against TCP connection (%s:%d):\n", cfg.tcp.host, cfg.tcp.port);
-    cfg.type = CONN_TCP;
-    test_blocking_connection(cfg);
-    test_blocking_io_errors(cfg);
-    if (throughput) test_throughput(cfg);
-
-    printf("\nTesting against Unix socket connection (%s):\n", cfg.unix.path);
-    cfg.type = CONN_UNIX;
-    test_blocking_connection(cfg);
-    test_blocking_io_errors(cfg);
-    if (throughput) test_throughput(cfg);
-
-    if (fails) {
+    if (fails == 0) {
+        printf("ALL TESTS PASSED\n");
+    } else {
        printf("*** %d TESTS FAILED ***\n", fails);
-        return 1;
    }
-
-    printf("ALL TESTS PASSED\n");
    return 0;
 }
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
+ * Copyright (c) 2009-2010, Salvatore Sanfilippo <antirez at gmail dot com>
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
@@ -27,12 +27,14 @@
 * POSSIBILITY OF SUCH DAMAGE.
 */

-#ifndef REDIS_RANDOM_H
-#define REDIS_RANDOM_H
+#ifndef __UTIL_H
+#define __UTIL_H
+#include <stdlib.h>

-int32_t redisLrand48();
-void redisSrand48(int32_t seedval);
-
-#define REDIS_LRAND48_MAX INT32_MAX
+/* Abort on out of memory */
+static void redisOOM(void) {
+    fprintf(stderr,"Out of memory in hiredis");
+    exit(1);
+}

 #endif
@@ -1,20 +1,15 @@
-/autom4te.cache/
-/config.stamp
-/config.log
-/config.status
-/doc/html.xsl
-/doc/manpages.xsl
-/doc/jemalloc.xml
-/lib/
 /Makefile
-/include/jemalloc/internal/jemalloc_internal\.h
-/include/jemalloc/internal/size_classes\.h
-/include/jemalloc/jemalloc\.h
-/include/jemalloc/jemalloc_defs\.h
-/src/*.[od]
-/test/*.[od]
-/test/*.out
-/test/[a-zA-Z_]*
-!test/*.c
-!test/*.exp
-/bin/jemalloc.sh
+/config.stamp
+/config.status
+
+/doc/html.xsl
+/doc/jemalloc.xml
+/doc/manpages.xsl
+
+/src/*.d
+/lib
+
+# The following headers are generated
+/include/jemalloc/*.h
+/include/jemalloc/internal/jemalloc_internal.h
+/test/jemalloc_test.h
@@ -1,10 +1,9 @@
 Unless otherwise specified, files in the jemalloc source distribution are
-subject to the following license:
+subject to the following licenses:
 --------------------------------------------------------------------------------
-Copyright (C) 2002-2012 Jason Evans <jasone@canonware.com>.
+Copyright (C) 2002-2010 Jason Evans <jasone@canonware.com>.
 All rights reserved.
-Copyright (C) 2007-2012 Mozilla Foundation.  All rights reserved.
-Copyright (C) 2009-2012 Facebook, Inc.  All rights reserved.
+Copyright (C) 2007-2010 Mozilla Foundation.  All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:
@@ -25,3 +24,28 @@ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
 OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 --------------------------------------------------------------------------------
+Copyright (C) 2009-2010 Facebook, Inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright notice, this
+  list of conditions and the following disclaimer in the documentation and/or
+  other materials provided with the distribution.
+* Neither the name of Facebook, Inc. nor the names of its contributors may be
+  used to endorse or promote products derived from this software without
+  specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+--------------------------------------------------------------------------------
@@ -6,173 +6,6 @@ found in the git revision history:
    http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git
    git://canonware.com/jemalloc.git

-* 3.2.0 (November 9, 2012)
-
-  In addition to a couple of bug fixes, this version modifies page run
-  allocation and dirty page purging algorithms in order to better control
-  page-level virtual memory fragmentation.
-
-  Incompatible changes:
-  - Change the "opt.lg_dirty_mult" default from 5 to 3 (32:1 to 8:1).
-
-  Bug fixes:
-  - Fix dss/mmap allocation precedence code to use recyclable mmap memory only
-    after primary dss allocation fails.
-  - Fix deadlock in the "arenas.purge" mallctl.  This regression was introduced
-    in 3.1.0 by the addition of the "arena.<i>.purge" mallctl.
-
-* 3.1.0 (October 16, 2012)
-
-  New features:
-  - Auto-detect whether running inside Valgrind, thus removing the need to
-    manually specify MALLOC_CONF=valgrind:true.
-  - Add the "arenas.extend" mallctl, which allows applications to create
-    manually managed arenas.
-  - Add the ALLOCM_ARENA() flag for {,r,d}allocm().
-  - Add the "opt.dss", "arena.<i>.dss", and "stats.arenas.<i>.dss" mallctls,
-    which provide control over dss/mmap precedence.
-  - Add the "arena.<i>.purge" mallctl, which obsoletes "arenas.purge".
-  - Define LG_QUANTUM for hppa.
-
-  Incompatible changes:
-  - Disable tcache by default if running inside Valgrind, in order to avoid
-    making unallocated objects appear reachable to Valgrind.
-  - Drop const from malloc_usable_size() argument on Linux.
-
-  Bug fixes:
-  - Fix heap profiling crash if sampled object is freed via realloc(p, 0).
-  - Remove const from __*_hook variable declarations, so that glibc can modify
-    them during process forking.
-  - Fix mlockall(2)/madvise(2) interaction.
-  - Fix fork(2)-related deadlocks.
-  - Fix error return value for "thread.tcache.enabled" mallctl.
-
-* 3.0.0 (May 11, 2012)
-
-  Although this version adds some major new features, the primary focus is on
-  internal code cleanup that facilitates maintainability and portability, most
-  of which is not reflected in the ChangeLog.  This is the first release to
-  incorporate substantial contributions from numerous other developers, and the
-  result is a more broadly useful allocator (see the git revision history for
-  contribution details).  Note that the license has been unified, thanks to
-  Facebook granting a license under the same terms as the other copyright
-  holders (see COPYING).
-
-  New features:
-  - Implement Valgrind support, redzones, and quarantine.
-  - Add support for additional platforms:
-    + FreeBSD
-    + Mac OS X Lion
-    + MinGW
-    + Windows (no support yet for replacing the system malloc)
-  - Add support for additional architectures:
-    + MIPS
-    + SH4
-    + Tilera
-  - Add support for cross compiling.
-  - Add nallocm(), which rounds a request size up to the nearest size class
-    without actually allocating.
-  - Implement aligned_alloc() (blame C11).
-  - Add the "thread.tcache.enabled" mallctl.
-  - Add the "opt.prof_final" mallctl.
-  - Update pprof (from gperftools 2.0).
-  - Add the --with-mangling option.
-  - Add the --disable-experimental option.
-  - Add the --disable-munmap option, and make it the default on Linux.
-  - Add the --enable-mremap option, which disables use of mremap(2) by default.
-
-  Incompatible changes:
-  - Enable stats by default.
-  - Enable fill by default.
-  - Disable lazy locking by default.
-  - Rename the "tcache.flush" mallctl to "thread.tcache.flush".
-  - Rename the "arenas.pagesize" mallctl to "arenas.page".
-  - Change the "opt.lg_prof_sample" default from 0 to 19 (1 B to 512 KiB).
-  - Change the "opt.prof_accum" default from true to false.
-
-  Removed features:
-  - Remove the swap feature, including the "config.swap", "swap.avail",
-    "swap.prezeroed", "swap.nfds", and "swap.fds" mallctls.
-  - Remove highruns statistics, including the
-    "stats.arenas.<i>.bins.<j>.highruns" and
-    "stats.arenas.<i>.lruns.<j>.highruns" mallctls.
-  - As part of small size class refactoring, remove the "opt.lg_[qc]space_max",
-    "arenas.cacheline", "arenas.subpage", "arenas.[tqcs]space_{min,max}", and
-    "arenas.[tqcs]bins" mallctls.
-  - Remove the "arenas.chunksize" mallctl.
-  - Remove the "opt.lg_prof_tcmax" option.
-  - Remove the "opt.lg_prof_bt_max" option.
-  - Remove the "opt.lg_tcache_gc_sweep" option.
-  - Remove the --disable-tiny option, including the "config.tiny" mallctl.
-  - Remove the --enable-dynamic-page-shift configure option.
-  - Remove the --enable-sysv configure option.
-
-  Bug fixes:
-  - Fix a statistics-related bug in the "thread.arena" mallctl that could cause
-    invalid statistics and crashes.
-  - Work around TLS deallocation via free() on Linux.  This bug could cause
-    write-after-free memory corruption.
-  - Fix a potential deadlock that could occur during interval- and
-    growth-triggered heap profile dumps.
-  - Fix large calloc() zeroing bugs due to dropping chunk map unzeroed flags.
-  - Fix chunk_alloc_dss() to stop claiming memory is zeroed.  This bug could
-    cause memory corruption and crashes with --enable-dss specified.
-  - Fix fork-related bugs that could cause deadlock in children between fork
-    and exec.
-  - Fix malloc_stats_print() to honor 'b' and 'l' in the opts parameter.
-  - Fix realloc(p, 0) to act like free(p).
-  - Do not enforce minimum alignment in memalign().
-  - Check for NULL pointer in malloc_usable_size().
-  - Fix an off-by-one heap profile statistics bug that could be observed in
-    interval- and growth-triggered heap profiles.
-  - Fix the "epoch" mallctl to update cached stats even if the passed in epoch
-    is 0.
-  - Fix bin->runcur management to fix a layout policy bug.  This bug did not
-    affect correctness.
-  - Fix a bug in choose_arena_hard() that potentially caused more arenas to be
-    initialized than necessary.
-  - Add missing "opt.lg_tcache_max" mallctl implementation.
-  - Use glibc allocator hooks to make mixed allocator usage less likely.
-  - Fix build issues for --disable-tcache.
-  - Don't mangle pthread_create() when --with-private-namespace is specified.
-
-* 2.2.5 (November 14, 2011)
-
-  Bug fixes:
-  - Fix huge_ralloc() race when using mremap(2).  This is a serious bug that
-    could cause memory corruption and/or crashes.
-  - Fix huge_ralloc() to maintain chunk statistics.
-  - Fix malloc_stats_print(..., "a") output.
-
-* 2.2.4 (November 5, 2011)
-
-  Bug fixes:
-  - Initialize arenas_tsd before using it.  This bug existed for 2.2.[0-3], as
-    well as for --disable-tls builds in earlier releases.
-  - Do not assume a 4 KiB page size in test/rallocm.c.
-
-* 2.2.3 (August 31, 2011)
-
-  This version fixes numerous bugs related to heap profiling.
-
-  Bug fixes:
-  - Fix a prof-related race condition.  This bug could cause memory corruption,
-    but only occurred in non-default configurations (prof_accum:false).
-  - Fix off-by-one backtracing issues (make sure that prof_alloc_prep() is
-    excluded from backtraces).
-  - Fix a prof-related bug in realloc() (only triggered by OOM errors).
-  - Fix prof-related bugs in allocm() and rallocm().
-  - Fix prof_tdata_cleanup() for --disable-tls builds.
-  - Fix a relative include path, to fix objdir builds.
-
-* 2.2.2 (July 30, 2011)
-
-  Bug fixes:
-  - Fix a build error for --disable-tcache.
-  - Fix assertions in arena_purge() (for real this time).
-  - Add the --with-private-namespace option.  This is a workaround for symbol
-    conflicts that can inadvertently arise when using static libraries.
-
 * 2.2.1 (March 30, 2011)

  Bug fixes:
@@ -26,19 +26,6 @@ any of the following arguments (not a definitive list) to 'configure':
    Embed one or more library paths, so that libjemalloc can find the libraries
    it is linked to.  This works only on ELF-based systems.

--with-mangling=<map>
-    Mangle public symbols specified in <map> which is a comma-separated list of
-    name:mangled pairs.
-
-    For example, to use ld's --wrap option as an alternative method for
-    overriding libc's malloc implementation, specify something like:
-
-      --with-mangling=malloc:__wrap_malloc,free:__wrap_free[...]
-
-    Note that mangling happens prior to application of the prefix specified by
-    --with-jemalloc-prefix, and mangled symbols are then ignored when applying
-    the prefix.
-
 --with-jemalloc-prefix=<prefix>
    Prefix all public APIs with <prefix>.  For example, if <prefix> is
    "prefix_", API changes like the following occur:
@@ -55,12 +42,6 @@ any of the following arguments (not a definitive list) to 'configure':
    jemalloc overlays the default malloc zone, but makes no attempt to actually
    replace the "malloc", "calloc", etc. symbols.

--with-private-namespace=<prefix>
-    Prefix all library-private APIs with <prefix>.  For shared libraries,
-    symbol visibility mechanisms prevent these symbols from being exported, but
-    for static libraries, naming collisions are a real possibility.  By
-    default, the prefix is "" (empty string).
-
 --with-install-suffix=<suffix>
    Append <suffix> to the base name of all installed files, such that multiple
    versions of jemalloc can coexist in the same installation directory.  For
@@ -75,8 +56,8 @@ any of the following arguments (not a definitive list) to 'configure':
    Enable assertions and validation code.  This incurs a substantial
    performance hit, but is very useful during application development.

--disable-stats
-    Disable statistics gathering functionality.  See the "opt.stats_print"
+--enable-stats
+    Enable statistics gathering functionality.  See the "opt.stats_print"
    option documentation for usage details.

 --enable-prof
@@ -103,50 +84,51 @@ any of the following arguments (not a definitive list) to 'configure':
    Statically link against the specified libunwind.a rather than dynamically
    linking with -lunwind.

+--disable-tiny
+    Disable tiny (sub-quantum-sized) object support.  Technically it is not
+    legal for a malloc implementation to allocate objects with less than
+    quantum alignment (8 or 16 bytes, depending on architecture), but in
+    practice it never causes any problems if, for example, 4-byte allocations
+    are 4-byte-aligned.
+
 --disable-tcache
    Disable thread-specific caches for small objects.  Objects are cached and
    released in bulk, thus reducing the total number of mutex operations.  See
    the "opt.tcache" option for usage details.

--enable-mremap
-    Enable huge realloc() via mremap(2).  mremap() is disabled by default
-    because the flavor used is specific to Linux, which has a quirk in its
-    virtual memory allocation algorithm that causes semi-permanent VM map holes
-    under normal jemalloc operation.
-
--disable-munmap
-    Disable virtual memory deallocation via munmap(2); instead keep track of
-    the virtual memory for later use.  munmap() is disabled by default (i.e.
-    --disable-munmap is implied) on Linux, which has a quirk in its virtual
-    memory allocation algorithm that causes semi-permanent VM map holes under
-    normal jemalloc operation.
+--enable-swap
+    Enable mmap()ed swap file support.  When this feature is built in, it is
+    possible to specify one or more files that act as backing store.  This
+    effectively allows for per application swap files.

 --enable-dss
    Enable support for page allocation/deallocation via sbrk(2), in addition to
    mmap(2).

--disable-fill
-    Disable support for junk/zero filling of memory, quarantine, and redzones.
-    See the "opt.junk", "opt.zero", "opt.quarantine", and "opt.redzone" option
-    documentation for usage details.
-
--disable-valgrind
-    Disable support for Valgrind.
-
--disable-experimental
-    Disable support for the experimental API (*allocm()).
-
--enable-utrace
-    Enable utrace(2)-based allocation tracing.  This feature is not broadly
-    portable (FreeBSD has it, but Linux and OS X do not).
+--enable-fill
+    Enable support for junk/zero filling of memory.  See the "opt.junk"/
+    "opt.zero" option documentation for usage details.

 --enable-xmalloc
    Enable support for optional immediate termination due to out-of-memory
    errors, as is commonly implemented by "xmalloc" wrapper function for malloc.
    See the "opt.xmalloc" option documentation for usage details.

--enable-lazy-lock
-    Enable code that wraps pthread_create() to detect when an application
+--enable-sysv
+    Enable support for System V semantics, wherein malloc(0) returns NULL
+    rather than a minimal allocation.  See the "opt.sysv" option documentation
+    for usage details.
+
+--enable-dynamic-page-shift
+    Under most conditions, the system page size never changes (usually 4KiB or
+    8KiB, depending on architecture and configuration), and unless this option
+    is enabled, jemalloc assumes that page size can safely be determined during
+    configuration and hard-coded.  Enabling dynamic page size determination has
+    a measurable impact on performance, since the compiler is forced to load
+    the page size from memory rather than embedding immediate values.
+
+--disable-lazy-lock
+    Disable code that wraps pthread_create() to detect when an application
    switches from single-threaded to multi-threaded mode, so that it can avoid
    mutex locking/unlocking operations while in single-threaded mode.  In
    practice, this feature usually has little impact on performance unless
@@ -193,24 +175,11 @@ PATH="?"

 === Advanced compilation =======================================================

-To build only parts of jemalloc, use the following targets:
-
-    build_lib_shared
-    build_lib_static
-    build_lib
-    build_doc_html
-    build_doc_man
-    build_doc
-
 To install only parts of jemalloc, use the following targets:

    install_bin
    install_include
-    install_lib_shared
-    install_lib_static
    install_lib
-    install_doc_html
-    install_doc_man
    install_doc

 To clean up build results to varying degrees, use the following make targets:
@@ -273,6 +242,10 @@ directory, issue configuration and build commands:

 The manual page is generated in both html and roff formats.  Any web browser
 can be used to view the html manual.  The roff manual page can be formatted
-prior to installation via the following command:
+prior to installation via any of the following commands:

    nroff -man -t doc/jemalloc.3
+
+    groff -man -t -Tps doc/jemalloc.3 | ps2pdf - doc/jemalloc.3.pdf
+
+    (cd doc; groff -man -man-ext -t -Thtml jemalloc.3 > jemalloc.3.html)
@@ -17,185 +17,130 @@ INCLUDEDIR := $(DESTDIR)@INCLUDEDIR@
 LIBDIR := $(DESTDIR)@LIBDIR@
 DATADIR := $(DESTDIR)@DATADIR@
 MANDIR := $(DESTDIR)@MANDIR@
-srcroot := @srcroot@
-objroot := @objroot@
-abs_srcroot := @abs_srcroot@
-abs_objroot := @abs_objroot@

 # Build parameters.
-CPPFLAGS := @CPPFLAGS@ -I$(srcroot)include -I$(objroot)include
+CPPFLAGS := @CPPFLAGS@ -I@srcroot@include -I@objroot@include
 CFLAGS := @CFLAGS@
+ifeq (macho, @abi@)
+CFLAGS += -dynamic
+endif
 LDFLAGS := @LDFLAGS@
-EXTRA_LDFLAGS := @EXTRA_LDFLAGS@
 LIBS := @LIBS@
 RPATH_EXTRA := @RPATH_EXTRA@
-SO := @so@
-IMPORTLIB := @importlib@
-O := @o@
-A := @a@
-EXE := @exe@
-LIBPREFIX := @libprefix@
-REV := @rev@
-install_suffix := @install_suffix@
-ABI := @abi@
-XSLTPROC := @XSLTPROC@
-AUTOCONF := @AUTOCONF@
-_RPATH = @RPATH@
-RPATH = $(if $(1),$(call _RPATH,$(1)))
-cfghdrs_in := @cfghdrs_in@
-cfghdrs_out := @cfghdrs_out@
-cfgoutputs_in := @cfgoutputs_in@
-cfgoutputs_out := @cfgoutputs_out@
-enable_autogen := @enable_autogen@
-enable_experimental := @enable_experimental@
-DSO_LDFLAGS = @DSO_LDFLAGS@
-SOREV = @SOREV@
-PIC_CFLAGS = @PIC_CFLAGS@
-CTARGET = @CTARGET@
-LDTARGET = @LDTARGET@
-MKLIB = @MKLIB@
-CC_MM = @CC_MM@
-
-ifeq (macho, $(ABI))
-TEST_LIBRARY_PATH := DYLD_FALLBACK_LIBRARY_PATH="$(objroot)lib"
+ifeq (macho, @abi@)
+SO := dylib
+WL_SONAME := dylib_install_name
 else
-ifeq (pecoff, $(ABI))
-TEST_LIBRARY_PATH := PATH="$(PATH):$(objroot)lib"
+SO := so
+WL_SONAME := soname
+endif
+REV := 1
+ifeq (macho, @abi@)
+TEST_LIBRARY_PATH := DYLD_FALLBACK_LIBRARY_PATH=@objroot@lib
 else
 TEST_LIBRARY_PATH :=
 endif
-endif
-
-LIBJEMALLOC := $(LIBPREFIX)jemalloc$(install_suffix)

 # Lists of files.
-BINS := $(srcroot)bin/pprof $(objroot)bin/jemalloc.sh
-CHDRS := $(objroot)include/jemalloc/jemalloc$(install_suffix).h \
-	$(objroot)include/jemalloc/jemalloc_defs$(install_suffix).h
-CSRCS := $(srcroot)src/jemalloc.c $(srcroot)src/arena.c $(srcroot)src/atomic.c \
-	$(srcroot)src/base.c $(srcroot)src/bitmap.c $(srcroot)src/chunk.c \
-	$(srcroot)src/chunk_dss.c $(srcroot)src/chunk_mmap.c \
-	$(srcroot)src/ckh.c $(srcroot)src/ctl.c $(srcroot)src/extent.c \
-	$(srcroot)src/hash.c $(srcroot)src/huge.c $(srcroot)src/mb.c \
-	$(srcroot)src/mutex.c $(srcroot)src/prof.c $(srcroot)src/quarantine.c \
-	$(srcroot)src/rtree.c $(srcroot)src/stats.c $(srcroot)src/tcache.c \
-	$(srcroot)src/util.c $(srcroot)src/tsd.c
-ifeq (macho, $(ABI))
-CSRCS += $(srcroot)src/zone.c
+BINS := @srcroot@bin/pprof
+CHDRS := @objroot@include/jemalloc/jemalloc@install_suffix@.h \
+	@objroot@include/jemalloc/jemalloc_defs@install_suffix@.h
+CSRCS := @srcroot@src/jemalloc.c @srcroot@src/arena.c @srcroot@src/atomic.c \
+	@srcroot@src/base.c @srcroot@src/bitmap.c @srcroot@src/chunk.c \
+	@srcroot@src/chunk_dss.c @srcroot@src/chunk_mmap.c \
+	@srcroot@src/chunk_swap.c @srcroot@src/ckh.c @srcroot@src/ctl.c \
+	@srcroot@src/extent.c @srcroot@src/hash.c @srcroot@src/huge.c \
+	@srcroot@src/mb.c @srcroot@src/mutex.c @srcroot@src/prof.c \
+	@srcroot@src/rtree.c @srcroot@src/stats.c @srcroot@src/tcache.c
+ifeq (macho, @abi@)
+CSRCS += @srcroot@src/zone.c
 endif
-ifeq ($(IMPORTLIB),$(SO))
-STATIC_LIBS := $(objroot)lib/$(LIBJEMALLOC).$(A)
-endif
-ifdef PIC_CFLAGS
-STATIC_LIBS += $(objroot)lib/$(LIBJEMALLOC)_pic.$(A)
-else
-STATIC_LIBS += $(objroot)lib/$(LIBJEMALLOC)_s.$(A)
-endif
-DSOS := $(objroot)lib/$(LIBJEMALLOC).$(SOREV)
-ifneq ($(SOREV),$(SO))
-DSOS += $(objroot)lib/$(LIBJEMALLOC).$(SO)
-endif
-MAN3 := $(objroot)doc/jemalloc$(install_suffix).3
-DOCS_XML := $(objroot)doc/jemalloc$(install_suffix).xml
-DOCS_HTML := $(DOCS_XML:$(objroot)%.xml=$(srcroot)%.html)
-DOCS_MAN3 := $(DOCS_XML:$(objroot)%.xml=$(srcroot)%.3)
+STATIC_LIBS :=	@objroot@lib/libjemalloc@install_suffix@.a
+DSOS := @objroot@lib/libjemalloc@install_suffix@.$(SO).$(REV) \
+	@objroot@lib/libjemalloc@install_suffix@.$(SO) \
+	@objroot@lib/libjemalloc@install_suffix@_pic.a
+MAN3 := @objroot@doc/jemalloc@install_suffix@.3
+DOCS_XML := @objroot@doc/jemalloc@install_suffix@.xml
+DOCS_HTML := $(DOCS_XML:@objroot@%.xml=@srcroot@%.html)
+DOCS_MAN3 := $(DOCS_XML:@objroot@%.xml=@srcroot@%.3)
 DOCS := $(DOCS_HTML) $(DOCS_MAN3)
-CTESTS := $(srcroot)test/aligned_alloc.c $(srcroot)test/allocated.c \
-	$(srcroot)test/ALLOCM_ARENA.c $(srcroot)test/bitmap.c \
-	$(srcroot)test/mremap.c $(srcroot)test/posix_memalign.c \
-	$(srcroot)test/thread_arena.c $(srcroot)test/thread_tcache_enabled.c
-ifeq ($(enable_experimental), 1)
-CTESTS += $(srcroot)test/allocm.c $(srcroot)test/rallocm.c
-endif
-
-COBJS := $(CSRCS:$(srcroot)%.c=$(objroot)%.$(O))
-CPICOBJS := $(CSRCS:$(srcroot)%.c=$(objroot)%.pic.$(O))
-CTESTOBJS := $(CTESTS:$(srcroot)%.c=$(objroot)%.$(O))
+CTESTS := @srcroot@test/allocated.c @srcroot@test/allocm.c \
+	@srcroot@test/bitmap.c @srcroot@test/mremap.c \
+	@srcroot@test/posix_memalign.c @srcroot@test/rallocm.c \
+	@srcroot@test/thread_arena.c

 .PHONY: all dist doc_html doc_man doc
 .PHONY: install_bin install_include install_lib
 .PHONY: install_html install_man install_doc install
 .PHONY: tests check clean distclean relclean

-.SECONDARY : $(CTESTOBJS)
+.SECONDARY : $(CTESTS:@srcroot@%.c=@objroot@%.o)

 # Default target.
-all: build
+all: $(DSOS) $(STATIC_LIBS)

-dist: build_doc
+dist: doc

-$(srcroot)doc/%.html : $(objroot)doc/%.xml $(srcroot)doc/stylesheet.xsl $(objroot)doc/html.xsl
-	$(XSLTPROC) -o $@ $(objroot)doc/html.xsl $<
+@srcroot@doc/%.html : @objroot@doc/%.xml @srcroot@doc/stylesheet.xsl @objroot@doc/html.xsl
+	@XSLTPROC@ -o $@ @objroot@doc/html.xsl $<

-$(srcroot)doc/%.3 : $(objroot)doc/%.xml $(srcroot)doc/stylesheet.xsl $(objroot)doc/manpages.xsl
-	$(XSLTPROC) -o $@ $(objroot)doc/manpages.xsl $<
+@srcroot@doc/%.3 : @objroot@doc/%.xml @srcroot@doc/stylesheet.xsl @objroot@doc/manpages.xsl
+	@XSLTPROC@ -o $@ @objroot@doc/manpages.xsl $<

-build_doc_html: $(DOCS_HTML)
-build_doc_man: $(DOCS_MAN3)
-build_doc: $(DOCS)
+doc_html: $(DOCS_HTML)
+doc_man: $(DOCS_MAN3)
+doc: $(DOCS)

 #
 # Include generated dependency files.
 #
-ifdef CC_MM
-include $(COBJS:%.$(O)=%.d)
-include $(CPICOBJS:%.$(O)=%.d)
-include $(CTESTOBJS:%.$(O)=%.d)
-endif
+-include $(CSRCS:@srcroot@%.c=@objroot@%.d)
+-include $(CSRCS:@srcroot@%.c=@objroot@%.pic.d)
+-include $(CTESTS:@srcroot@%.c=@objroot@%.d)

-$(COBJS): $(objroot)src/%.$(O): $(srcroot)src/%.c
-$(CPICOBJS): $(objroot)src/%.pic.$(O): $(srcroot)src/%.c
-$(CPICOBJS): CFLAGS += $(PIC_CFLAGS)
-$(CTESTOBJS): $(objroot)test/%.$(O): $(srcroot)test/%.c
-$(CTESTOBJS): CPPFLAGS += -I$(objroot)test
-ifneq ($(IMPORTLIB),$(SO))
-$(COBJS): CPPFLAGS += -DDLLEXPORT
-endif
-
-ifndef CC_MM
-# Dependencies
-HEADER_DIRS = $(srcroot)include/jemalloc/internal \
-	$(objroot)include/jemalloc $(objroot)include/jemalloc/internal
-HEADERS = $(wildcard $(foreach dir,$(HEADER_DIRS),$(dir)/*.h))
-$(COBJS) $(CPICOBJS) $(CTESTOBJS): $(HEADERS)
-$(CTESTOBJS): $(objroot)test/jemalloc_test.h
-endif
-
-$(COBJS) $(CPICOBJS) $(CTESTOBJS): %.$(O):
+@objroot@src/%.o: @srcroot@src/%.c
 	@mkdir -p $(@D)
-	$(CC) $(CFLAGS) -c $(CPPFLAGS) $(CTARGET) $<
-ifdef CC_MM
-	@$(CC) -MM $(CPPFLAGS) -MT $@ -o $(@:%.$(O)=%.d) $<
-endif
+	$(CC) $(CFLAGS) -c $(CPPFLAGS) -o $@ $<
+	@$(SHELL) -ec "$(CC) -MM $(CPPFLAGS) $< | sed \"s/\($(subst /,\/,$(notdir $(basename $@)))\)\.o\([ :]*\)/$(subst /,\/,$(strip $(dir $@)))\1.o \2/g\" > $(@:%.o=%.d)"

-ifneq ($(SOREV),$(SO))
-%.$(SO) : %.$(SOREV)
+@objroot@src/%.pic.o: @srcroot@src/%.c
+	@mkdir -p $(@D)
+	$(CC) $(CFLAGS) -fPIC -DPIC -c $(CPPFLAGS) -o $@ $<
+	@$(SHELL) -ec "$(CC) -MM $(CPPFLAGS) $< | sed \"s/\($(subst /,\/,$(notdir $(basename $(basename $@))))\)\.o\([ :]*\)/$(subst /,\/,$(strip $(dir $@)))\1.pic.o \2/g\" > $(@:%.o=%.d)"
+
+%.$(SO) : %.$(SO).$(REV)
 	@mkdir -p $(@D)
 	ln -sf $(<F) $@
+
+@objroot@lib/libjemalloc@install_suffix@.$(SO).$(REV) : $(CSRCS:@srcroot@%.c=@objroot@%.pic.o)
+	@mkdir -p $(@D)
+	$(CC) -shared -Wl,-$(WL_SONAME),$(@F) $(RPATH_EXTRA:%=@RPATH@%) -o $@ $+ $(LDFLAGS) $(LIBS)
+
+@objroot@lib/libjemalloc@install_suffix@_pic.a : $(CSRCS:@srcroot@%.c=@objroot@%.pic.o)
+	@mkdir -p $(@D)
+	ar crus $@ $+
+
+@objroot@lib/libjemalloc@install_suffix@.a : $(CSRCS:@srcroot@%.c=@objroot@%.o)
+	@mkdir -p $(@D)
+	ar crus $@ $+
+
+@objroot@test/%.o: @srcroot@test/%.c
+	@mkdir -p $(@D)
+	$(CC) $(CFLAGS) -c $(CPPFLAGS) -I@objroot@test -o $@ $<
+	@$(SHELL) -ec "$(CC) -MM $(CPPFLAGS) -I@objroot@test $< | sed \"s/\($(subst /,\/,$(notdir $(basename $@)))\)\.o\([ :]*\)/$(subst /,\/,$(strip $(dir $@)))\1.o \2/g\" > $(@:%.o=%.d)"
+
+# Automatic dependency generation misses #include "*.c".
+@objroot@test/bitmap.o : @objroot@src/bitmap.o
+
+@objroot@test/%: @objroot@test/%.o \
+		 @objroot@lib/libjemalloc@install_suffix@.$(SO)
+	@mkdir -p $(@D)
+ifneq (@RPATH@, )
+	$(CC) -o $@ $< @RPATH@@objroot@lib -L@objroot@lib -ljemalloc@install_suffix@
+else
+	$(CC) -o $@ $< -L@objroot@lib -ljemalloc@install_suffix@
 endif

-$(objroot)lib/$(LIBJEMALLOC).$(SOREV) : $(if $(PIC_CFLAGS),$(CPICOBJS),$(COBJS))
-	@mkdir -p $(@D)
-	$(CC) $(DSO_LDFLAGS) $(call RPATH,$(RPATH_EXTRA)) $(LDTARGET) $+ $(LDFLAGS) $(LIBS) $(EXTRA_LDFLAGS)
-
-$(objroot)lib/$(LIBJEMALLOC)_pic.$(A) : $(CPICOBJS)
-$(objroot)lib/$(LIBJEMALLOC).$(A) : $(COBJS)
-$(objroot)lib/$(LIBJEMALLOC)_s.$(A) : $(COBJS)
-
-$(STATIC_LIBS):
-	@mkdir -p $(@D)
-	$(MKLIB) $+
-
-$(objroot)test/bitmap$(EXE): $(objroot)src/bitmap.$(O)
-
-$(objroot)test/%$(EXE): $(objroot)test/%.$(O) $(objroot)src/util.$(O) $(DSOS)
-	@mkdir -p $(@D)
-	$(CC) $(LDTARGET) $(filter %.$(O),$^) $(call RPATH,$(objroot)lib) $(objroot)lib/$(LIBJEMALLOC).$(IMPORTLIB) $(filter -lpthread,$(LIBS)) $(EXTRA_LDFLAGS)
-
-build_lib_shared: $(DSOS)
-build_lib_static: $(STATIC_LIBS)
-build: build_lib_shared build_lib_static
-
 install_bin:
 	install -d $(BINDIR)
 	@for b in $(BINS); do \
@@ -210,55 +155,46 @@ install_include:
 	install -m 644 $$h $(INCLUDEDIR)/jemalloc; \
 done

-install_lib_shared: $(DSOS)
+install_lib: $(DSOS) $(STATIC_LIBS)
 	install -d $(LIBDIR)
-	install -m 755 $(objroot)lib/$(LIBJEMALLOC).$(SOREV) $(LIBDIR)
-ifneq ($(SOREV),$(SO))
-	ln -sf $(LIBJEMALLOC).$(SOREV) $(LIBDIR)/$(LIBJEMALLOC).$(SO)
-endif
+	install -m 755 @objroot@lib/libjemalloc@install_suffix@.$(SO).$(REV) $(LIBDIR)
+	ln -sf libjemalloc@install_suffix@.$(SO).$(REV) $(LIBDIR)/libjemalloc@install_suffix@.$(SO)
+	install -m 755 @objroot@lib/libjemalloc@install_suffix@_pic.a $(LIBDIR)
+	install -m 755 @objroot@lib/libjemalloc@install_suffix@.a $(LIBDIR)

-install_lib_static: $(STATIC_LIBS)
-	install -d $(LIBDIR)
-	@for l in $(STATIC_LIBS); do \
-	echo "install -m 755 $$l $(LIBDIR)"; \
-	install -m 755 $$l $(LIBDIR); \
-done
-
-install_lib: install_lib_shared install_lib_static
-
-install_doc_html:
-	install -d $(DATADIR)/doc/jemalloc$(install_suffix)
+install_html:
+	install -d $(DATADIR)/doc/jemalloc@install_suffix@
 	@for d in $(DOCS_HTML); do \
-	echo "install -m 644 $$d $(DATADIR)/doc/jemalloc$(install_suffix)"; \
-	install -m 644 $$d $(DATADIR)/doc/jemalloc$(install_suffix); \
+	echo "install -m 644 $$d $(DATADIR)/doc/jemalloc@install_suffix@"; \
+	install -m 644 $$d $(DATADIR)/doc/jemalloc@install_suffix@; \
 done

-install_doc_man:
+install_man:
 	install -d $(MANDIR)/man3
 	@for d in $(DOCS_MAN3); do \
 	echo "install -m 644 $$d $(MANDIR)/man3"; \
 	install -m 644 $$d $(MANDIR)/man3; \
 done

-install_doc: install_doc_html install_doc_man
+install_doc: install_html install_man

 install: install_bin install_include install_lib install_doc

-tests: $(CTESTS:$(srcroot)%.c=$(objroot)%$(EXE))
+tests: $(CTESTS:@srcroot@%.c=@objroot@%)

 check: tests
-	@mkdir -p $(objroot)test
+	@mkdir -p @objroot@test
 	@$(SHELL) -c 'total=0; \
 		failures=0; \
 		echo "========================================="; \
-		for t in $(CTESTS:$(srcroot)%.c=$(objroot)%); do \
+		for t in $(CTESTS:@srcroot@%.c=@objroot@%); do \
 			total=`expr $$total + 1`; \
 			/bin/echo -n "$${t} ... "; \
-			$(TEST_LIBRARY_PATH) $${t}$(EXE) $(abs_srcroot) \
-			  $(abs_objroot) > $(objroot)$${t}.out 2>&1; \
-			if test -e "$(srcroot)$${t}.exp"; then \
-				diff -w -u $(srcroot)$${t}.exp \
-				  $(objroot)$${t}.out >/dev/null 2>&1; \
+			$(TEST_LIBRARY_PATH) $${t} @abs_srcroot@ @abs_objroot@ \
+			  > @objroot@$${t}.out 2>&1; \
+			if test -e "@srcroot@$${t}.exp"; then \
+				diff -u @srcroot@$${t}.exp \
+				  @objroot@$${t}.out >/dev/null 2>&1; \
 				fail=$$?; \
 				if test "$${fail}" -eq "1" ; then \
 					failures=`expr $${failures} + 1`; \
@@ -275,49 +211,49 @@ check: tests
 		echo "Failures: $${failures}/$${total}"'

 clean:
-	rm -f $(COBJS)
-	rm -f $(CPICOBJS)
-	rm -f $(COBJS:%.$(O)=%.d)
-	rm -f $(CPICOBJS:%.$(O)=%.d)
-	rm -f $(CTESTOBJS:%.$(O)=%$(EXE))
-	rm -f $(CTESTOBJS)
-	rm -f $(CTESTOBJS:%.$(O)=%.d)
-	rm -f $(CTESTOBJS:%.$(O)=%.out)
+	rm -f $(CSRCS:@srcroot@%.c=@objroot@%.o)
+	rm -f $(CSRCS:@srcroot@%.c=@objroot@%.pic.o)
+	rm -f $(CSRCS:@srcroot@%.c=@objroot@%.d)
+	rm -f $(CSRCS:@srcroot@%.c=@objroot@%.pic.d)
+	rm -f $(CTESTS:@srcroot@%.c=@objroot@%)
+	rm -f $(CTESTS:@srcroot@%.c=@objroot@%.o)
+	rm -f $(CTESTS:@srcroot@%.c=@objroot@%.d)
+	rm -f $(CTESTS:@srcroot@%.c=@objroot@%.out)
 	rm -f $(DSOS) $(STATIC_LIBS)

 distclean: clean
-	rm -rf $(objroot)autom4te.cache
-	rm -f $(objroot)config.log
-	rm -f $(objroot)config.status
-	rm -f $(objroot)config.stamp
-	rm -f $(cfghdrs_out)
-	rm -f $(cfgoutputs_out)
+	rm -rf @objroot@autom4te.cache
+	rm -f @objroot@config.log
+	rm -f @objroot@config.status
+	rm -f @objroot@config.stamp
+	rm -f @cfghdrs_out@
+	rm -f @cfgoutputs_out@

 relclean: distclean
-	rm -f $(objroot)configure
-	rm -f $(srcroot)VERSION
+	rm -f @objroot@configure
+	rm -f @srcroot@VERSION
 	rm -f $(DOCS_HTML)
 	rm -f $(DOCS_MAN3)

 #===============================================================================
 # Re-configuration rules.

-ifeq ($(enable_autogen), 1)
-$(srcroot)configure : $(srcroot)configure.ac
-	cd ./$(srcroot) && $(AUTOCONF)
+ifeq (@enable_autogen@, 1)
+@srcroot@configure : @srcroot@configure.ac
+	cd ./@srcroot@ && @AUTOCONF@

-$(objroot)config.status : $(srcroot)configure
-	./$(objroot)config.status --recheck
+@objroot@config.status : @srcroot@configure
+	./@objroot@config.status --recheck

-$(srcroot)config.stamp.in : $(srcroot)configure.ac
-	echo stamp > $(srcroot)config.stamp.in
+@srcroot@config.stamp.in : @srcroot@configure.ac
+	echo stamp > @srcroot@config.stamp.in

-$(objroot)config.stamp : $(cfgoutputs_in) $(cfghdrs_in) $(srcroot)configure
-	./$(objroot)config.status
+@objroot@config.stamp : @cfgoutputs_in@ @cfghdrs_in@ @srcroot@configure
+	./@objroot@config.status
 	@touch $@

 # There must be some action in order for make to re-read Makefile when it is
 # out of date.
-$(cfgoutputs_out) $(cfghdrs_out) : $(objroot)config.stamp
+@cfgoutputs_out@ @cfghdrs_out@ : @objroot@config.stamp
 	@true
 endif
@@ -1,10 +1,10 @@
 jemalloc is a general-purpose scalable concurrent malloc(3) implementation.
-This distribution is a "portable" implementation that currently targets
-FreeBSD, Linux, Apple OS X, and MinGW.  jemalloc is included as the default
-allocator in the FreeBSD and NetBSD operating systems, and it is used by the
-Mozilla Firefox web browser on Microsoft Windows-related platforms.  Depending
-on your needs, one of the other divergent versions may suit your needs better
-than this distribution.
+This distribution is a stand-alone "portable" implementation that currently
+targets Linux and Apple OS X.  jemalloc is included as the default allocator in
+the FreeBSD and NetBSD operating systems, and it is used by the Mozilla Firefox
+web browser on Microsoft Windows-related platforms.  Depending on your needs,
+one of the other divergent versions may suit your needs better than this
+distribution.

 The COPYING file contains copyright and licensing information.

@@ -1 +1 @@
-3.2.0-0-g87499f6748ebe4817571e817e9f680ccb5bf54a9
+2.2.1-0-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862
@@ -1,9 +0,0 @@
-#!/bin/sh
-
-prefix=@prefix@
-exec_prefix=@exec_prefix@
-libdir=@libdir@
-
-@LD_PRELOAD_VAR@=${libdir}/libjemalloc.@SOREV@
-export @LD_PRELOAD_VAR@
-exec "$@"
@@ -1,10 +1,9 @@
 #! /bin/sh
 # Configuration validation subroutine script.
 #   Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-#   2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
-#   2011, 2012 Free Software Foundation, Inc.
+#   2000, 2001, 2002, 2003 Free Software Foundation, Inc.

-timestamp='2012-02-10'
+timestamp='2004-02-23'

 # This file is (in principle) common to ALL GNU software.
 # The presence of a machine in this file suggests that SOME GNU software
@@ -21,25 +20,23 @@ timestamp='2012-02-10'
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
-# along with this program; if not, see <http://www.gnu.org/licenses/>.
-#
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
 # As a special exception to the GNU General Public License, if you
 # distribute this file as part of a program that contains a
 # configuration script generated by Autoconf, you may include it under
 # the same distribution terms that you use for the rest of that program.

-
 # Please send patches to <config-patches@gnu.org>.  Submit a context
-# diff and a properly formatted GNU ChangeLog entry.
+# diff and a properly formatted ChangeLog entry.
 #
 # Configuration subroutine to validate and canonicalize a configuration type.
 # Supply the specified configuration type as an argument.
 # If it is invalid, we print an error message on stderr and exit with code 1.
 # Otherwise, we print the canonical config type on stdout and succeed.

-# You can get the latest version of this script from:
-# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD
-
 # This file is supposed to be the same for all GNU packages
 # and recognize all the CPU types, system types and aliases
 # that are meaningful with *any* GNU software.
@@ -73,8 +70,7 @@ Report bugs and patches to <config-patches@gnu.org>."
 version="\
 GNU config.sub ($timestamp)

-Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,
-2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012
+Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001
 Free Software Foundation, Inc.

 This is free software; see the source for copying conditions.  There is NO
@@ -87,11 +83,11 @@ Try \`$me --help' for more information."
 while test $# -gt 0 ; do
  case $1 in
    --time-stamp | --time* | -t )
-       echo "$timestamp" ; exit ;;
+       echo "$timestamp" ; exit 0 ;;
    --version | -v )
-       echo "$version" ; exit ;;
+       echo "$version" ; exit 0 ;;
    --help | --h* | -h )
-       echo "$usage"; exit ;;
+       echo "$usage"; exit 0 ;;
    -- )     # Stop option processing
       shift; break ;;
    - )	# Use stdin as input.
@@ -103,7 +99,7 @@ while test $# -gt 0 ; do
    *local*)
       # First pass through any local machine types.
       echo $1
-       exit ;;
+       exit 0;;

    * )
       break ;;
@@ -122,18 +118,11 @@ esac
 # Here we must recognize all the valid KERNEL-OS combinations.
 maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'`
 case $maybe_os in
-  nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \
-  linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \
-  knetbsd*-gnu* | netbsd*-gnu* | \
-  kopensolaris*-gnu* | \
-  storm-chaos* | os2-emx* | rtmk-nova*)
+  nto-qnx* | linux-gnu* | linux-dietlibc | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | \
+  kfreebsd*-gnu* | knetbsd*-gnu* | netbsd*-gnu* | storm-chaos* | os2-emx* | rtmk-nova*)
    os=-$maybe_os
    basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`
    ;;
-  android-linux)
-    os=-linux-android
-    basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown
-    ;;
  *)
    basic_machine=`echo $1 | sed 's/-[^-]*$//'`
    if [ $basic_machine != $1 ]
@@ -156,13 +145,10 @@ case $os in
 	-convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\
 	-c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \
 	-harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \
-	-apple | -axis | -knuth | -cray | -microblaze)
+	-apple | -axis)
 		os=
 		basic_machine=$1
 		;;
-	-bluegene*)
-		os=-cnk
-		;;
 	-sim | -cisco | -oki | -wec | -winbond)
 		os=
 		basic_machine=$1
@@ -177,17 +163,13 @@ case $os in
 		os=-chorusos
 		basic_machine=$1
 		;;
-	-chorusrdb)
-		os=-chorusrdb
+ 	-chorusrdb)
+ 		os=-chorusrdb
 		basic_machine=$1
-		;;
+ 		;;
 	-hiux*)
 		os=-hiuxwe2
 		;;
-	-sco6)
-		os=-sco5v6
-		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
-		;;
 	-sco5)
 		os=-sco3.2v5
 		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
@@ -204,10 +186,6 @@ case $os in
 		# Don't forget version if it is 3.2v4 or newer.
 		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
 		;;
-	-sco5v6*)
-		# Don't forget version if it is 3.2v4 or newer.
-		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
-		;;
 	-sco*)
 		os=-sco3.2v2
 		basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`
@@ -249,36 +227,25 @@ case $basic_machine in
 	# Some are omitted here because they have special meanings below.
 	1750a | 580 \
 	| a29k \
-	| aarch64 | aarch64_be \
 	| alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \
 	| alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \
 	| am33_2.0 \
-	| arc | arm | arm[bl]e | arme[lb] | armv[2345] | armv[345][lb] | avr | avr32 \
-        | be32 | be64 \
-	| bfin \
+	| arc | arm | arm[bl]e | arme[lb] | armv[2345] | armv[345][lb] | avr \
 	| c4x | clipper \
 	| d10v | d30v | dlx | dsp16xx \
-	| epiphany \
-	| fido | fr30 | frv \
+	| fr30 | frv \
 	| h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \
-	| hexagon \
 	| i370 | i860 | i960 | ia64 \
 	| ip2k | iq2000 \
-	| le32 | le64 \
-	| lm32 \
-	| m32c | m32r | m32rle | m68000 | m68k | m88k \
-	| maxq | mb | microblaze | mcore | mep | metag \
+	| m32r | m68000 | m68k | m88k | mcore \
 	| mips | mipsbe | mipseb | mipsel | mipsle \
 	| mips16 \
 	| mips64 | mips64el \
-	| mips64octeon | mips64octeonel \
-	| mips64orion | mips64orionel \
-	| mips64r5900 | mips64r5900el \
 	| mips64vr | mips64vrel \
+	| mips64orion | mips64orionel \
 	| mips64vr4100 | mips64vr4100el \
 	| mips64vr4300 | mips64vr4300el \
 	| mips64vr5000 | mips64vr5000el \
-	| mips64vr5900 | mips64vr5900el \
 	| mipsisa32 | mipsisa32el \
 	| mipsisa32r2 | mipsisa32r2el \
 	| mipsisa64 | mipsisa64el \
@@ -287,65 +254,30 @@ case $basic_machine in
 	| mipsisa64sr71k | mipsisa64sr71kel \
 	| mipstx39 | mipstx39el \
 	| mn10200 | mn10300 \
-	| moxie \
-	| mt \
 	| msp430 \
-	| nds32 | nds32le | nds32be \
-	| nios | nios2 \
 	| ns16k | ns32k \
-	| open8 \
-	| or32 \
+	| openrisc | or32 \
 	| pdp10 | pdp11 | pj | pjl \
-	| powerpc | powerpc64 | powerpc64le | powerpcle \
+	| powerpc | powerpc64 | powerpc64le | powerpcle | ppcbe \
 	| pyramid \
-	| rl78 | rx \
-	| score \
-	| sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \
+	| sh | sh[1234] | sh[23]e | sh[34]eb | shbe | shle | sh[1234]le | sh3ele \
 	| sh64 | sh64le \
-	| sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \
-	| sparcv8 | sparcv9 | sparcv9b | sparcv9v \
-	| spu \
-	| tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \
-	| ubicom32 \
-	| v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \
+	| sparc | sparc64 | sparc86x | sparclet | sparclite | sparcv9 | sparcv9b \
+	| strongarm \
+	| tahoe | thumb | tic4x | tic80 | tron \
+	| v850 | v850e \
 	| we32k \
-	| x86 | xc16x | xstormy16 | xtensa \
-	| z8k | z80)
+	| x86 | xscale | xstormy16 | xtensa \
+	| z8k)
 		basic_machine=$basic_machine-unknown
 		;;
-	c54x)
-		basic_machine=tic54x-unknown
-		;;
-	c55x)
-		basic_machine=tic55x-unknown
-		;;
-	c6x)
-		basic_machine=tic6x-unknown
-		;;
-	m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | picochip)
+	m6811 | m68hc11 | m6812 | m68hc12)
+		# Motorola 68HC11/12.
 		basic_machine=$basic_machine-unknown
 		os=-none
 		;;
 	m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k)
 		;;
-	ms1)
-		basic_machine=mt-unknown
-		;;
-
-	strongarm | thumb | xscale)
-		basic_machine=arm-unknown
-		;;
-	xgate)
-		basic_machine=$basic_machine-unknown
-		os=-none
-		;;
-	xscaleeb)
-		basic_machine=armeb-unknown
-		;;
-
-	xscaleel)
-		basic_machine=armel-unknown
-		;;

 	# We use `pc' rather than `unknown'
 	# because (1) that's what they normally are, and
@@ -361,40 +293,32 @@ case $basic_machine in
 	# Recognize the basic CPU types with company name.
 	580-* \
 	| a29k-* \
-	| aarch64-* | aarch64_be-* \
 	| alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \
 	| alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \
 	| alphapca5[67]-* | alpha64pca5[67]-* | arc-* \
 	| arm-*  | armbe-* | armle-* | armeb-* | armv*-* \
-	| avr-* | avr32-* \
-	| be32-* | be64-* \
-	| bfin-* | bs2000-* \
-	| c[123]* | c30-* | [cjt]90-* | c4x-* \
-	| clipper-* | craynv-* | cydra-* \
+	| avr-* \
+	| bs2000-* \
+	| c[123]* | c30-* | [cjt]90-* | c4x-* | c54x-* | c55x-* | c6x-* \
+	| clipper-* | cydra-* \
 	| d10v-* | d30v-* | dlx-* \
 	| elxsi-* \
-	| f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \
+	| f30[01]-* | f700-* | fr30-* | frv-* | fx80-* \
 	| h8300-* | h8500-* \
 	| hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \
-	| hexagon-* \
 	| i*86-* | i860-* | i960-* | ia64-* \
 	| ip2k-* | iq2000-* \
-	| le32-* | le64-* \
-	| lm32-* \
-	| m32c-* | m32r-* | m32rle-* \
+	| m32r-* \
 	| m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \
-	| m88110-* | m88k-* | maxq-* | mcore-* | metag-* | microblaze-* \
+	| m88110-* | m88k-* | mcore-* \
 	| mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \
 	| mips16-* \
 	| mips64-* | mips64el-* \
-	| mips64octeon-* | mips64octeonel-* \
-	| mips64orion-* | mips64orionel-* \
-	| mips64r5900-* | mips64r5900el-* \
 	| mips64vr-* | mips64vrel-* \
+	| mips64orion-* | mips64orionel-* \
 	| mips64vr4100-* | mips64vr4100el-* \
 	| mips64vr4300-* | mips64vr4300el-* \
 	| mips64vr5000-* | mips64vr5000el-* \
-	| mips64vr5900-* | mips64vr5900el-* \
 	| mipsisa32-* | mipsisa32el-* \
 	| mipsisa32r2-* | mipsisa32r2el-* \
 	| mipsisa64-* | mipsisa64el-* \
@@ -402,39 +326,26 @@ case $basic_machine in
 	| mipsisa64sb1-* | mipsisa64sb1el-* \
 	| mipsisa64sr71k-* | mipsisa64sr71kel-* \
 	| mipstx39-* | mipstx39el-* \
-	| mmix-* \
-	| mt-* \
 	| msp430-* \
-	| nds32-* | nds32le-* | nds32be-* \
-	| nios-* | nios2-* \
-	| none-* | np1-* | ns16k-* | ns32k-* \
-	| open8-* \
+	| none-* | np1-* | nv1-* | ns16k-* | ns32k-* \
 	| orion-* \
 	| pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \
-	| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \
+	| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* | ppcbe-* \
 	| pyramid-* \
-	| rl78-* | romp-* | rs6000-* | rx-* \
-	| sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \
+	| romp-* | rs6000-* \
+	| sh-* | sh[1234]-* | sh[23]e-* | sh[34]eb-* | shbe-* \
 	| shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \
-	| sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \
-	| sparclite-* \
-	| sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx?-* \
-	| tahoe-* \
+	| sparc-* | sparc64-* | sparc86x-* | sparclet-* | sparclite-* \
+	| sparcv9-* | sparcv9b-* | strongarm-* | sv1-* | sx?-* \
+	| tahoe-* | thumb-* \
 	| tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \
-	| tile*-* \
 	| tron-* \
-	| ubicom32-* \
-	| v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \
-	| vax-* \
+	| v850-* | v850e-* | vax-* \
 	| we32k-* \
-	| x86-* | x86_64-* | xc16x-* | xps100-* \
-	| xstormy16-* | xtensa*-* \
+	| x86-* | x86_64-* | xps100-* | xscale-* | xstormy16-* \
+	| xtensa-* \
 	| ymp-* \
-	| z8k-* | z80-*)
-		;;
-	# Recognize the basic CPU types without company name, with glob match.
-	xtensa*)
-		basic_machine=$basic_machine-unknown
+	| z8k-*)
 		;;
 	# Recognize the various machine names and aliases which stand
 	# for a CPU type and a company and sometimes even an OS.
@@ -452,7 +363,7 @@ case $basic_machine in
 		basic_machine=a29k-amd
 		os=-udi
 		;;
-	abacus)
+    	abacus)
 		basic_machine=abacus-unknown
 		;;
 	adobe68k)
@@ -498,10 +409,6 @@ case $basic_machine in
 		basic_machine=m68k-apollo
 		os=-bsd
 		;;
-	aros)
-		basic_machine=i386-pc
-		os=-aros
-		;;
 	aux)
 		basic_machine=m68k-apple
 		os=-aux
@@ -510,35 +417,10 @@ case $basic_machine in
 		basic_machine=ns32k-sequent
 		os=-dynix
 		;;
-	blackfin)
-		basic_machine=bfin-unknown
-		os=-linux
-		;;
-	blackfin-*)
-		basic_machine=bfin-`echo $basic_machine | sed 's/^[^-]*-//'`
-		os=-linux
-		;;
-	bluegene*)
-		basic_machine=powerpc-ibm
-		os=-cnk
-		;;
-	c54x-*)
-		basic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'`
-		;;
-	c55x-*)
-		basic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'`
-		;;
-	c6x-*)
-		basic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'`
-		;;
 	c90)
 		basic_machine=c90-cray
 		os=-unicos
 		;;
-	cegcc)
-		basic_machine=arm-unknown
-		os=-cegcc
-		;;
 	convex-c1)
 		basic_machine=c1-convex
 		os=-bsd
@@ -563,20 +445,13 @@ case $basic_machine in
 		basic_machine=j90-cray
 		os=-unicos
 		;;
-	craynv)
-		basic_machine=craynv-cray
-		os=-unicosmp
-		;;
-	cr16 | cr16-*)
-		basic_machine=cr16-unknown
+	cr16c)
+		basic_machine=cr16c-unknown
 		os=-elf
 		;;
 	crds | unos)
 		basic_machine=m68k-crds
 		;;
-	crisv32 | crisv32-* | etraxfs*)
-		basic_machine=crisv32-axis
-		;;
 	cris | cris-* | etrax*)
 		basic_machine=cris-axis
 		;;
@@ -606,14 +481,6 @@ case $basic_machine in
 		basic_machine=m88k-motorola
 		os=-sysv3
 		;;
-	dicos)
-		basic_machine=i686-pc
-		os=-dicos
-		;;
-	djgpp)
-		basic_machine=i586-pc
-		os=-msdosdjgpp
-		;;
 	dpx20 | dpx20-*)
 		basic_machine=rs6000-bull
 		os=-bosx
@@ -725,6 +592,7 @@ case $basic_machine in
 	i370-ibm* | ibm*)
 		basic_machine=i370-ibm
 		;;
+# I'm not sure what "Sysv32" means.  Should this be sysv3.2?
 	i*86v32)
 		basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`
 		os=-sysv32
@@ -763,14 +631,6 @@ case $basic_machine in
 		basic_machine=m68k-isi
 		os=-sysv
 		;;
-	m68knommu)
-		basic_machine=m68k-unknown
-		os=-linux
-		;;
-	m68knommu-*)
-		basic_machine=m68k-`echo $basic_machine | sed 's/^[^-]*-//'`
-		os=-linux
-		;;
 	m88k-omron*)
 		basic_machine=m88k-omron
 		;;
@@ -782,17 +642,10 @@ case $basic_machine in
 		basic_machine=ns32k-utek
 		os=-sysv
 		;;
-	microblaze)
-		basic_machine=microblaze-xilinx
-		;;
 	mingw32)
 		basic_machine=i386-pc
 		os=-mingw32
 		;;
-	mingw32ce)
-		basic_machine=arm-unknown
-		os=-mingw32ce
-		;;
 	miniframe)
 		basic_machine=m68000-convergent
 		;;
@@ -806,6 +659,10 @@ case $basic_machine in
 	mips3*)
 		basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown
 		;;
+	mmix*)
+		basic_machine=mmix-knuth
+		os=-mmixware
+		;;
 	monitor)
 		basic_machine=m68k-rom68k
 		os=-coff
@@ -818,21 +675,10 @@ case $basic_machine in
 		basic_machine=i386-pc
 		os=-msdos
 		;;
-	ms1-*)
-		basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'`
-		;;
-	msys)
-		basic_machine=i386-pc
-		os=-msys
-		;;
 	mvs)
 		basic_machine=i370-ibm
 		os=-mvs
 		;;
-	nacl)
-		basic_machine=le32-unknown
-		os=-nacl
-		;;
 	ncr3000)
 		basic_machine=i486-ncr
 		os=-sysv4
@@ -897,11 +743,9 @@ case $basic_machine in
 	np1)
 		basic_machine=np1-gould
 		;;
-	neo-tandem)
-		basic_machine=neo-tandem
-		;;
-	nse-tandem)
-		basic_machine=nse-tandem
+	nv1)
+		basic_machine=nv1-cray
+		os=-unicosmp
 		;;
 	nsr-tandem)
 		basic_machine=nsr-tandem
@@ -910,8 +754,9 @@ case $basic_machine in
 		basic_machine=hppa1.1-oki
 		os=-proelf
 		;;
-	openrisc | openrisc-*)
+	or32 | or32-*)
 		basic_machine=or32-unknown
+		os=-coff
 		;;
 	os400)
 		basic_machine=powerpc-ibm
@@ -933,14 +778,6 @@ case $basic_machine in
 		basic_machine=i860-intel
 		os=-osf
 		;;
-	parisc)
-		basic_machine=hppa-unknown
-		os=-linux
-		;;
-	parisc-*)
-		basic_machine=hppa-`echo $basic_machine | sed 's/^[^-]*-//'`
-		os=-linux
-		;;
 	pbd)
 		basic_machine=sparc-tti
 		;;
@@ -950,12 +787,6 @@ case $basic_machine in
 	pc532 | pc532-*)
 		basic_machine=ns32k-pc532
 		;;
-	pc98)
-		basic_machine=i386-pc
-		;;
-	pc98-*)
-		basic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'`
-		;;
 	pentium | p5 | k5 | k6 | nexgen | viac3)
 		basic_machine=i586-pc
 		;;
@@ -985,10 +816,9 @@ case $basic_machine in
 		;;
 	power)	basic_machine=power-ibm
 		;;
-	ppc | ppcbe)	basic_machine=powerpc-unknown
+	ppc)	basic_machine=powerpc-unknown
 		;;
-	ppc-* | ppcbe-*)
-		basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`
+	ppc-*)	basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`
 		;;
 	ppcle | powerpclittle | ppc-le | powerpc-little)
 		basic_machine=powerpcle-unknown
@@ -1013,10 +843,6 @@ case $basic_machine in
 		basic_machine=i586-unknown
 		os=-pw32
 		;;
-	rdos)
-		basic_machine=i386-pc
-		os=-rdos
-		;;
 	rom68k)
 		basic_machine=m68k-rom68k
 		os=-coff
@@ -1043,10 +869,6 @@ case $basic_machine in
 	sb1el)
 		basic_machine=mipsisa64sb1el-unknown
 		;;
-	sde)
-		basic_machine=mipsisa32-sde
-		os=-elf
-		;;
 	sei)
 		basic_machine=mips-sei
 		os=-seiux
@@ -1058,9 +880,6 @@ case $basic_machine in
 		basic_machine=sh-hitachi
 		os=-hms
 		;;
-	sh5el)
-		basic_machine=sh5le-unknown
-		;;
 	sh64)
 		basic_machine=sh64-unknown
 		;;
@@ -1082,9 +901,6 @@ case $basic_machine in
 		basic_machine=i860-stratus
 		os=-sysv4
 		;;
-	strongarm-* | thumb-*)
-		basic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'`
-		;;
 	sun2)
 		basic_machine=m68000-sun
 		;;
@@ -1141,9 +957,17 @@ case $basic_machine in
 		basic_machine=t90-cray
 		os=-unicos
 		;;
-	tile*)
-		basic_machine=$basic_machine-unknown
-		os=-linux-gnu
+	tic54x | c54x*)
+		basic_machine=tic54x-unknown
+		os=-coff
+		;;
+	tic55x | c55x*)
+		basic_machine=tic55x-unknown
+		os=-coff
+		;;
+	tic6x | c6x*)
+		basic_machine=tic6x-unknown
+		os=-coff
 		;;
 	tx39)
 		basic_machine=mipstx39-unknown
@@ -1205,16 +1029,9 @@ case $basic_machine in
 		basic_machine=hppa1.1-winbond
 		os=-proelf
 		;;
-	xbox)
-		basic_machine=i686-pc
-		os=-mingw32
-		;;
 	xps | xps100)
 		basic_machine=xps100-honeywell
 		;;
-	xscale-* | xscalee[bl]-*)
-		basic_machine=`echo $basic_machine | sed 's/^xscale/arm/'`
-		;;
 	ymp)
 		basic_machine=ymp-cray
 		os=-unicos
@@ -1223,10 +1040,6 @@ case $basic_machine in
 		basic_machine=z8k-unknown
 		os=-sim
 		;;
-	z80-*-coff)
-		basic_machine=z80-unknown
-		os=-sim
-		;;
 	none)
 		basic_machine=none-none
 		os=-none
@@ -1246,9 +1059,6 @@ case $basic_machine in
 	romp)
 		basic_machine=romp-ibm
 		;;
-	mmix)
-		basic_machine=mmix-knuth
-		;;
 	rs6000)
 		basic_machine=rs6000-ibm
 		;;
@@ -1265,10 +1075,13 @@ case $basic_machine in
 	we32k)
 		basic_machine=we32k-att
 		;;
-	sh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele)
+	sh3 | sh4 | sh[34]eb | sh[1234]le | sh[23]ele)
 		basic_machine=sh-unknown
 		;;
-	sparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v)
+	sh64)
+		basic_machine=sh64-unknown
+		;;
+	sparc | sparcv9 | sparcv9b)
 		basic_machine=sparc-sun
 		;;
 	cydra)
@@ -1312,12 +1125,9 @@ esac
 if [ x"$os" != x"" ]
 then
 case $os in
-	# First match some system type aliases
-	# that might get confused with valid system types.
+        # First match some system type aliases
+        # that might get confused with valid system types.
 	# -solaris* is a basic system type, with this one exception.
-	-auroraux)
-		os=-auroraux
-		;;
 	-solaris1 | -solaris1.*)
 		os=`echo $os | sed -e 's|solaris1|sunos4|'`
 		;;
@@ -1338,31 +1148,26 @@ case $os in
 	# Each alternative MUST END IN A *, to match a version number.
 	# -sysv* is not here because it comes later, after sysvr4.
 	-gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \
-	      | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\
-	      | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \
-	      | -sym* | -kopensolaris* \
+	      | -*vms* | -sco* | -esix* | -isc* | -aix* | -sunos | -sunos[34]*\
+	      | -hpux* | -unos* | -osf* | -luna* | -dgux* | -solaris* | -sym* \
 	      | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \
-	      | -aos* | -aros* \
+	      | -aos* \
 	      | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \
 	      | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \
-	      | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \
-	      | -openbsd* | -solidbsd* \
+	      | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* | -openbsd* \
 	      | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \
 	      | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \
 	      | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \
 	      | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \
-	      | -chorusos* | -chorusrdb* | -cegcc* \
-	      | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \
-	      | -mingw32* | -linux-gnu* | -linux-android* \
-	      | -linux-newlib* | -linux-uclibc* \
-	      | -uxpv* | -beos* | -mpeix* | -udk* \
+	      | -chorusos* | -chorusrdb* \
+	      | -cygwin* | -pe* | -psos* | -moss* | -proelf* | -rtems* \
+	      | -mingw32* | -linux-gnu* | -linux-uclibc* | -uxpv* | -beos* | -mpeix* | -udk* \
 	      | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \
 	      | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \
 	      | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \
 	      | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \
 	      | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \
-	      | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \
-	      | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es*)
+	      | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly*)
 	# Remember, each alternative MUST END IN *, to match a version number.
 		;;
 	-qnx*)
@@ -1380,7 +1185,7 @@ case $os in
 		os=`echo $os | sed -e 's|nto|nto-qnx|'`
 		;;
 	-sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \
-	      | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \
+	      | -windows* | -osx | -abug | -netware* | -os9* | -beos* \
 	      | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*)
 		;;
 	-mac*)
@@ -1401,7 +1206,7 @@ case $os in
 	-opened*)
 		os=-openedition
 		;;
-	-os400*)
+        -os400*)
 		os=-os400
 		;;
 	-wince*)
@@ -1450,7 +1255,7 @@ case $os in
 	-sinix*)
 		os=-sysv4
 		;;
-	-tpf*)
+        -tpf*)
 		os=-tpf
 		;;
 	-triton*)
@@ -1489,14 +1294,6 @@ case $os in
 	-kaos*)
 		os=-kaos
 		;;
-	-zvmoe)
-		os=-zvmoe
-		;;
-	-dicos*)
-		os=-dicos
-		;;
-	-nacl*)
-		;;
 	-none)
 		;;
 	*)
@@ -1519,12 +1316,6 @@ else
 # system, and we'll never get to this point.

 case $basic_machine in
-	score-*)
-		os=-elf
-		;;
-	spu-*)
-		os=-elf
-		;;
 	*-acorn)
 		os=-riscix1.2
 		;;
@@ -1534,18 +1325,9 @@ case $basic_machine in
 	arm*-semi)
 		os=-aout
 		;;
-	c4x-* | tic4x-*)
-		os=-coff
-		;;
-	tic54x-*)
-		os=-coff
-		;;
-	tic55x-*)
-		os=-coff
-		;;
-	tic6x-*)
-		os=-coff
-		;;
+    c4x-* | tic4x-*)
+        os=-coff
+        ;;
 	# This must come before the *-dec entry.
 	pdp10-*)
 		os=-tops20
@@ -1564,13 +1346,13 @@ case $basic_machine in
 		;;
 	m68000-sun)
 		os=-sunos3
+		# This also exists in the configure program, but was not the
+		# default.
+		# os=-sunos4
 		;;
 	m68*-cisco)
 		os=-aout
 		;;
-	mep-*)
-		os=-elf
-		;;
 	mips*-cisco)
 		os=-elf
 		;;
@@ -1589,15 +1371,9 @@ case $basic_machine in
 	*-be)
 		os=-beos
 		;;
-	*-haiku)
-		os=-haiku
-		;;
 	*-ibm)
 		os=-aix
 		;;
-	*-knuth)
-		os=-mmixware
-		;;
 	*-wec)
 		os=-proelf
 		;;
@@ -1700,7 +1476,7 @@ case $basic_machine in
 			-sunos*)
 				vendor=sun
 				;;
-			-cnk*|-aix*)
+			-aix*)
 				vendor=ibm
 				;;
 			-beos*)
@@ -1763,7 +1539,7 @@ case $basic_machine in
 esac

 echo $basic_machine$os
-exit
+exit 0

 # Local variables:
 # eval: (add-hook 'write-file-hooks 'time-stamp)
@@ -1,13 +1,13 @@
 '\" t
 .\"     Title: JEMALLOC
 .\"    Author: Jason Evans
-.\" Generator: DocBook XSL Stylesheets v1.76.1 <http://docbook.sf.net/>
-.\"      Date: 11/09/2012
+.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
+.\"      Date: 03/30/2011
 .\"    Manual: User Manual
-.\"    Source: jemalloc 3.2.0-0-g87499f6748ebe4817571e817e9f680ccb5bf54a9
+.\"    Source: jemalloc 2.2.1-0-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862
 .\"  Language: English
 .\"
-.TH "JEMALLOC" "3" "11/09/2012" "jemalloc 3.2.0-0-g87499f6748eb" "User Manual"
+.TH "JEMALLOC" "3" "03/30/2011" "jemalloc 2.2.1-0-g5ef7abf6d846" "User Manual"
 .\" -----------------------------------------------------------------
 .\" * Define some portability stuff
 .\" -----------------------------------------------------------------
@@ -31,7 +31,7 @@
 jemalloc \- general purpose memory allocation functions
 .SH "LIBRARY"
 .PP
-This manual describes jemalloc 3\&.2\&.0\-0\-g87499f6748ebe4817571e817e9f680ccb5bf54a9\&. More information can be found at the
+This manual describes jemalloc 2\&.2\&.1\-0\-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862\&. More information can be found at the
 \m[blue]\fBjemalloc website\fR\m[]\&\s-2\u[1]\d\s+2\&.
 .SH "SYNOPSIS"
 .sp
@@ -48,8 +48,6 @@ This manual describes jemalloc 3\&.2\&.0\-0\-g87499f6748ebe4817571e817e9f680ccb5
 .BI "void *calloc(size_t\ " "number" ", size_t\ " "size" ");"
 .HP \w'int\ posix_memalign('u
 .BI "int posix_memalign(void\ **" "ptr" ", size_t\ " "alignment" ", size_t\ " "size" ");"
-.HP \w'void\ *aligned_alloc('u
-.BI "void *aligned_alloc(size_t\ " "alignment" ", size_t\ " "size" ");"
 .HP \w'void\ *realloc('u
 .BI "void *realloc(void\ *" "ptr" ", size_t\ " "size" ");"
 .HP \w'void\ free('u
@@ -78,8 +76,6 @@ const char *\fImalloc_conf\fR;
 .BI "int sallocm(const\ void\ *" "ptr" ", size_t\ *" "rsize" ", int\ " "flags" ");"
 .HP \w'int\ dallocm('u
 .BI "int dallocm(void\ *" "ptr" ", int\ " "flags" ");"
-.HP \w'int\ nallocm('u
-.BI "int nallocm(size_t\ *" "rsize" ", size_t\ " "size" ", int\ " "flags" ");"
 .SH "DESCRIPTION"
 .SS "Standard API"
 .PP
@@ -114,18 +110,6 @@ must be a power of 2 at least as large as
 sizeof(\fBvoid *\fR)\&.
 .PP
 The
-\fBaligned_alloc\fR\fB\fR
-function allocates
-\fIsize\fR
-bytes of memory such that the allocation\*(Aqs base address is an even multiple of
-\fIalignment\fR\&. The requested
-\fIalignment\fR
-must be a power of 2\&. Behavior is undefined if
-\fIsize\fR
-is not an integral multiple of
-\fIalignment\fR\&.
-.PP
-The
 \fBrealloc\fR\fB\fR
 function changes the size of the previously allocated memory referenced by
 \fIptr\fR
@@ -252,16 +236,13 @@ for (i = 0; i < nbins; i++) {
 .\}
 .SS "Experimental API"
 .PP
-The experimental API is subject to change or removal without regard for backward compatibility\&. If
-\fB\-\-disable\-experimental\fR
-is specified during configuration, the experimental API is omitted\&.
+The experimental API is subject to change or removal without regard for backward compatibility\&.
 .PP
 The
 \fBallocm\fR\fB\fR,
 \fBrallocm\fR\fB\fR,
-\fBsallocm\fR\fB\fR,
-\fBdallocm\fR\fB\fR, and
-\fBnallocm\fR\fB\fR
+\fBsallocm\fR\fB\fR, and
+\fBdallocm\fR\fB\fR
 functions all have a
 \fIflags\fR
 argument that can be used to specify options\&. The functions only check the options that are contextually relevant\&. Use bitwise or (|) operations to specify one or more of the following:
@@ -294,14 +275,6 @@ Initialize newly allocated memory to contain zero bytes\&. In the growing reallo
 For reallocation, fail rather than moving the object\&. This constraint can apply to both growth and shrinkage\&.
 .RE
 .PP
-\fBALLOCM_ARENA(\fR\fB\fIa\fR\fR\fB) \fR
-.RS 4
-Use the arena specified by the index
-\fIa\fR\&. This macro does not validate that
-\fIa\fR
-specifies an arena in the valid range\&.
-.RE
-.PP
 The
 \fBallocm\fR\fB\fR
 function allocates at least
@@ -313,10 +286,7 @@ to the base address of the allocation, and sets
 to the real size of the allocation if
 \fIrsize\fR
 is not
-\fBNULL\fR\&. Behavior is undefined if
-\fIsize\fR
-is
-\fB0\fR\&.
+\fBNULL\fR\&.
 .PP
 The
 \fBrallocm\fR\fB\fR
@@ -336,9 +306,6 @@ is not
 is non\-zero, an attempt is made to resize the allocation to be at least
 \fIsize\fR + \fIextra\fR)
 bytes, though inability to allocate the extra byte(s) will not by itself result in failure\&. Behavior is undefined if
-\fIsize\fR
-is
-\fB0\fR, or if
 (\fIsize\fR + \fIextra\fR > \fBSIZE_T_MAX\fR)\&.
 .PP
 The
@@ -352,23 +319,6 @@ The
 function causes the memory referenced by
 \fIptr\fR
 to be made available for future allocations\&.
-.PP
-The
-\fBnallocm\fR\fB\fR
-function allocates no memory, but it performs the same size computation as the
-\fBallocm\fR\fB\fR
-function, and if
-\fIrsize\fR
-is not
-\fBNULL\fR
-it sets
-\fI*rsize\fR
-to the real size of the allocation that would result from the equivalent
-\fBallocm\fR\fB\fR
-function call\&. Behavior is undefined if
-\fIsize\fR
-is
-\fB0\fR\&.
 .SH "TUNING"
 .PP
 Once, when the first call is made to one of the memory allocation routines, the allocator initializes its internals based in part on various options that can be specified at compile\- or run\-time\&.
@@ -396,9 +346,9 @@ Traditionally, allocators have used
 to obtain memory, which is suboptimal for several reasons, including race conditions, increased fragmentation, and artificial limitations on maximum usable memory\&. If
 \fB\-\-enable\-dss\fR
 is specified during configuration, this allocator uses both
-\fBmmap\fR(2)
+\fBsbrk\fR(2)
 and
-\fBsbrk\fR(2), in that order of preference; otherwise only
+\fBmmap\fR(2), in that order of preference; otherwise only
 \fBmmap\fR(2)
 is used\&.
 .PP
@@ -414,8 +364,14 @@ User objects are broken into three categories according to size: small, large, a
 .PP
 Each chunk that is managed by an arena tracks its contents as runs of contiguous pages (unused, backing a set of small objects, or backing one large object)\&. The combination of chunk alignment and chunk page maps makes it possible to determine all metadata regarding small and large allocations in constant time\&.
 .PP
-Small objects are managed in groups by page runs\&. Each run maintains a frontier and free list to track which regions are in use\&. Allocation requests that are no more than half the quantum (8 or 16, depending on architecture) are rounded up to the nearest power of two that is at least
-sizeof(\fBdouble\fR)\&. All other small object size classes are multiples of the quantum, spaced such that internal fragmentation is limited to approximately 25% for all but the smallest size classes\&. Allocation requests that are larger than the maximum small size class, but small enough to fit in an arena\-managed chunk (see the
+Small objects are managed in groups by page runs\&. Each run maintains a frontier and free list to track which regions are in use\&. Unless
+\fB\-\-disable\-tiny\fR
+is specified during configuration, allocation requests that are no more than half the quantum (8 or 16, depending on architecture) are rounded up to the nearest power of two that is at least
+sizeof(\fBvoid *\fR)\&. Allocation requests that are more than half the quantum, but no more than the minimum cacheline\-multiple size class (see the
+"opt\&.lg_qspace_max"
+option) are rounded up to the nearest multiple of the quantum\&. Allocation requests that are more than the minimum cacheline\-multiple size class, but no more than the minimum subpage\-multiple size class (see the
+"opt\&.lg_cspace_max"
+option) are rounded up to the nearest multiple of the cacheline size (64)\&. Allocation requests that are more than the minimum subpage\-multiple size class, but no more than the maximum subpage\-multiple size class are rounded up to the nearest multiple of the subpage size (256)\&. Allocation requests that are more than the maximum subpage\-multiple size class, but small enough to fit in an arena\-managed chunk (see the
 "opt\&.lg_chunk"
 option), are rounded up to the nearest run size\&. Allocation requests that are too large to fit in an arena\-managed chunk are rounded up to the nearest multiple of the chunk size\&.
 .PP
@@ -431,73 +387,51 @@ Table 1\&.
 .B Table\ \&1.\ \&Size classes
 .TS
 allbox tab(:);
-lB rB lB.
+lB lB lB.
 T{
 Category
 T}:T{
-Spacing
+Subcategory
 T}:T{
 Size
 T}
 .T&
-l r l
-^ r l
-^ r l
-^ r l
-^ r l
-^ r l
-^ r l
-l r l
-l r l.
+l l l
+^ l l
+^ l l
+^ l l
+l s l
+l s l.
 T{
 Small
 T}:T{
-lg
+Tiny
 T}:T{
 [8]
 T}
 :T{
-16
+Quantum\-spaced
 T}:T{
 [16, 32, 48, \&.\&.\&., 128]
 T}
 :T{
-32
+Cacheline\-spaced
 T}:T{
-[160, 192, 224, 256]
+[192, 256, 320, \&.\&.\&., 512]
 T}
 :T{
-64
+Subpage\-spaced
 T}:T{
-[320, 384, 448, 512]
-T}
-:T{
-128
-T}:T{
-[640, 768, 896, 1024]
-T}
-:T{
-256
-T}:T{
-[1280, 1536, 1792, 2048]
-T}
-:T{
-512
-T}:T{
-[2560, 3072, 3584]
+[768, 1024, 1280, \&.\&.\&., 3840]
 T}
 T{
 Large
 T}:T{
-4 KiB
-T}:T{
 [4 KiB, 8 KiB, 12 KiB, \&.\&.\&., 4072 KiB]
 T}
 T{
 Huge
 T}:T{
-4 MiB
-T}:T{
 [4 MiB, 8 MiB, 12 MiB, \&.\&.\&.]
 T}
 .TE
@@ -547,6 +481,12 @@ was specified during build configuration\&.
 was specified during build configuration\&.
 .RE
 .PP
+"config\&.dynamic_page_shift" (\fBbool\fR) r\-
+.RS 4
+\fB\-\-enable\-dynamic\-page\-shift\fR
+was specified during build configuration\&.
+.RE
+.PP
 "config\&.fill" (\fBbool\fR) r\-
 .RS 4
 \fB\-\-enable\-fill\fR
@@ -559,18 +499,6 @@ was specified during build configuration\&.
 was specified during build configuration\&.
 .RE
 .PP
-"config\&.mremap" (\fBbool\fR) r\-
-.RS 4
-\fB\-\-enable\-mremap\fR
-was specified during build configuration\&.
-.RE
-.PP
-"config\&.munmap" (\fBbool\fR) r\-
-.RS 4
-\fB\-\-enable\-munmap\fR
-was specified during build configuration\&.
-.RE
-.PP
 "config\&.prof" (\fBbool\fR) r\-
 .RS 4
 \fB\-\-enable\-prof\fR
@@ -595,30 +523,36 @@ was specified during build configuration\&.
 was specified during build configuration\&.
 .RE
 .PP
+"config\&.swap" (\fBbool\fR) r\-
+.RS 4
+\fB\-\-enable\-swap\fR
+was specified during build configuration\&.
+.RE
+.PP
+"config\&.sysv" (\fBbool\fR) r\-
+.RS 4
+\fB\-\-enable\-sysv\fR
+was specified during build configuration\&.
+.RE
+.PP
 "config\&.tcache" (\fBbool\fR) r\-
 .RS 4
 \fB\-\-disable\-tcache\fR
 was not specified during build configuration\&.
 .RE
 .PP
+"config\&.tiny" (\fBbool\fR) r\-
+.RS 4
+\fB\-\-disable\-tiny\fR
+was not specified during build configuration\&.
+.RE
+.PP
 "config\&.tls" (\fBbool\fR) r\-
 .RS 4
 \fB\-\-disable\-tls\fR
 was not specified during build configuration\&.
 .RE
 .PP
-"config\&.utrace" (\fBbool\fR) r\-
-.RS 4
-\fB\-\-enable\-utrace\fR
-was specified during build configuration\&.
-.RE
-.PP
-"config\&.valgrind" (\fBbool\fR) r\-
-.RS 4
-\fB\-\-enable\-valgrind\fR
-was specified during build configuration\&.
-.RE
-.PP
 "config\&.xmalloc" (\fBbool\fR) r\-
 .RS 4
 \fB\-\-enable\-xmalloc\fR
@@ -634,28 +568,31 @@ in these cases\&. This option is disabled by default unless
 is specified during configuration, in which case it is enabled by default\&.
 .RE
 .PP
+"opt\&.lg_qspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Size (log base 2) of the maximum size class that is a multiple of the quantum (8 or 16 bytes, depending on architecture)\&. Above this size, cacheline spacing is used for size classes\&. The default value is 128 bytes (2^7)\&.
+.RE
+.PP
+"opt\&.lg_cspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Size (log base 2) of the maximum size class that is a multiple of the cacheline size (64)\&. Above this size, subpage spacing (256 bytes) is used for size classes\&. The default value is 512 bytes (2^9)\&.
+.RE
+.PP
 "opt\&.lg_chunk" (\fBsize_t\fR) r\-
 .RS 4
 Virtual memory chunk size (log base 2)\&. The default chunk size is 4 MiB (2^22)\&.
 .RE
 .PP
-"opt\&.dss" (\fBconst char *\fR) r\-
-.RS 4
-dss (\fBsbrk\fR(2)) allocation precedence as related to
-\fBmmap\fR(2)
-allocation\&. The following settings are supported: \(lqdisabled\(rq, \(lqprimary\(rq, and \(lqsecondary\(rq (default)\&.
-.RE
-.PP
 "opt\&.narenas" (\fBsize_t\fR) r\-
 .RS 4
-Maximum number of arenas to use for automatic multiplexing of threads and arenas\&. The default is four times the number of CPUs, or one if there is a single CPU\&.
+Maximum number of arenas to use\&. The default maximum number of arenas is four times the number of CPUs, or one if there is a single CPU\&.
 .RE
 .PP
 "opt\&.lg_dirty_mult" (\fBssize_t\fR) r\-
 .RS 4
 Per\-arena minimum ratio (log base 2) of active to dirty pages\&. Some dirty unused pages may be allowed to accumulate, within the limit set by the ratio (or one chunk worth of dirty pages, whichever is greater), before informing the kernel about some of those pages via
 \fBmadvise\fR(2)
-or a similar system call\&. This provides the kernel with sufficient information to recycle dirty pages if physical memory becomes scarce and the pages remain unused\&. The default minimum ratio is 8:1 (2^3:1); an option value of \-1 will disable dirty page purging\&.
+or a similar system call\&. This provides the kernel with sufficient information to recycle dirty pages if physical memory becomes scarce and the pages remain unused\&. The default minimum ratio is 32:1 (2^5:1); an option value of \-1 will disable dirty page purging\&.
 .RE
 .PP
 "opt\&.stats_print" (\fBbool\fR) r\-
@@ -675,24 +612,7 @@ Junk filling enabled/disabled\&. If enabled, each byte of uninitialized allocate
 0xa5\&. All deallocated memory will be initialized to
 0x5a\&. This is intended for debugging and will impact performance negatively\&. This option is disabled by default unless
 \fB\-\-enable\-debug\fR
-is specified during configuration, in which case it is enabled by default unless running inside
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2\&.
-.RE
-.PP
-"opt\&.quarantine" (\fBsize_t\fR) r\- [\fB\-\-enable\-fill\fR]
-.RS 4
-Per thread quarantine size in bytes\&. If non\-zero, each thread maintains a FIFO object quarantine that stores up to the specified number of bytes of memory\&. The quarantined memory is not freed until it is released from quarantine, though it is immediately junk\-filled if the
-"opt\&.junk"
-option is enabled\&. This feature is of particular use in combination with
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2, which can detect attempts to access quarantined objects\&. This is intended for debugging and will impact performance negatively\&. The default quarantine size is 0 unless running inside Valgrind, in which case the default is 16 MiB\&.
-.RE
-.PP
-"opt\&.redzone" (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR]
-.RS 4
-Redzones enabled/disabled\&. If enabled, small allocations have redzones before and after them\&. Furthermore, if the
-"opt\&.junk"
-option is enabled, the redzones are checked for corruption during deallocation\&. However, the primary intended purpose of this feature is to be used in combination with
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2, which needs redzones in order to do effective buffer overflow/underflow detection\&. This option is intended for debugging and will impact performance negatively\&. This option is disabled by default unless running inside Valgrind\&.
+is specified during configuration, in which case it is enabled by default\&.
 .RE
 .PP
 "opt\&.zero" (\fBbool\fR) r\- [\fB\-\-enable\-fill\fR]
@@ -704,17 +624,13 @@ and
 calls do not zero memory that was previously allocated\&. This is intended for debugging and will impact performance negatively\&. This option is disabled by default\&.
 .RE
 .PP
-"opt\&.utrace" (\fBbool\fR) r\- [\fB\-\-enable\-utrace\fR]
+"opt\&.sysv" (\fBbool\fR) r\- [\fB\-\-enable\-sysv\fR]
 .RS 4
-Allocation tracing based on
-\fButrace\fR(2)
-enabled/disabled\&. This option is disabled by default\&.
-.RE
-.PP
-"opt\&.valgrind" (\fBbool\fR) r\- [\fB\-\-enable\-valgrind\fR]
-.RS 4
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2
-support enabled/disabled\&. This option is vestigal because jemalloc auto\-detects whether it is running inside Valgrind\&. This option is disabled by default, unless running inside Valgrind\&.
+If enabled, attempting to allocate zero bytes will return a
+\fBNULL\fR
+pointer instead of a valid pointer\&. (The default behavior is to make a minimal allocation and return a pointer to it\&.) This option is provided for System V compatibility\&. This option is incompatible with the
+"opt\&.xmalloc"
+option\&. This option is disabled by default\&.
 .RE
 .PP
 "opt\&.xmalloc" (\fBbool\fR) r\- [\fB\-\-enable\-xmalloc\fR]
@@ -740,9 +656,15 @@ This option is disabled by default\&.
 "opt\&.tcache" (\fBbool\fR) r\- [\fB\-\-enable\-tcache\fR]
 .RS 4
 Thread\-specific caching enabled/disabled\&. When there are multiple threads, each thread uses a thread\-specific cache for objects up to a certain size\&. Thread\-specific caching allows many allocations to be satisfied without performing any thread synchronization, at the cost of increased memory use\&. See the
+"opt\&.lg_tcache_gc_sweep"
+and
 "opt\&.lg_tcache_max"
-option for related tuning information\&. This option is enabled by default unless running inside
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2\&.
+options for related tuning information\&. This option is enabled by default\&.
+.RE
+.PP
+"opt\&.lg_tcache_gc_sweep" (\fBssize_t\fR) r\- [\fB\-\-enable\-tcache\fR]
+.RS 4
+Approximate interval (log base 2) between full thread\-specific cache garbage collection sweeps, counted in terms of thread\-specific cache allocation/deallocation events\&. Garbage collection is actually performed incrementally, one size class at a time, in order to avoid large collection pauses\&. The default sweep interval is 8192 (2^13); setting this option to \-1 will disable garbage collection\&.
 .RE
 .PP
 "opt\&.lg_tcache_max" (\fBsize_t\fR) r\- [\fB\-\-enable\-tcache\fR]
@@ -752,22 +674,31 @@ Maximum size class (log base 2) to cache in the thread\-specific cache\&. At a m
 .PP
 "opt\&.prof" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR]
 .RS 4
-Memory profiling enabled/disabled\&. If enabled, profile memory allocation activity\&. See the
+Memory profiling enabled/disabled\&. If enabled, profile memory allocation activity, and use an
+\fBatexit\fR(3)
+function to dump final memory usage to a file named according to the pattern
+<prefix>\&.<pid>\&.<seq>\&.f\&.heap, where
+<prefix>
+is controlled by the
+"opt\&.prof_prefix"
+option\&. See the
+"opt\&.lg_prof_bt_max"
+option for backtrace depth control\&. See the
 "opt\&.prof_active"
 option for on\-the\-fly activation/deactivation\&. See the
 "opt\&.lg_prof_sample"
 option for probabilistic sampling control\&. See the
 "opt\&.prof_accum"
 option for control of cumulative sample reporting\&. See the
+"opt\&.lg_prof_tcmax"
+option for control of per thread backtrace caching\&. See the
 "opt\&.lg_prof_interval"
-option for information on interval\-triggered profile dumping, the
+option for information on interval\-triggered profile dumping, and the
 "opt\&.prof_gdump"
-option for information on high\-water\-triggered profile dumping, and the
-"opt\&.prof_final"
-option for final profile dumping\&. Profile output is compatible with the included
+option for information on high\-water\-triggered profile dumping\&. Profile output is compatible with the included
 \fBpprof\fR
 Perl script, which originates from the
-\m[blue]\fBgperftools package\fR\m[]\&\s-2\u[3]\d\s+2\&.
+\m[blue]\fBgoogle\-perftools package\fR\m[]\&\s-2\u[2]\d\s+2\&.
 .RE
 .PP
 "opt\&.prof_prefix" (\fBconst char *\fR) r\- [\fB\-\-enable\-prof\fR]
@@ -776,6 +707,11 @@ Filename prefix for profile dumps\&. If the prefix is set to the empty string, n
 jeprof\&.
 .RE
 .PP
+"opt\&.lg_prof_bt_max" (\fBsize_t\fR) r\- [\fB\-\-enable\-prof\fR]
+.RS 4
+Maximum backtrace depth (log base 2) when profiling memory allocation activity\&. The default is 128 (2^7)\&.
+.RE
+.PP
 "opt\&.prof_active" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR]
 .RS 4
 Profiling activated/deactivated\&. This is a secondary control mechanism that makes it possible to start the application with profiling enabled (see the
@@ -787,12 +723,21 @@ mallctl\&. This option is enabled by default\&.
 .PP
 "opt\&.lg_prof_sample" (\fBssize_t\fR) r\- [\fB\-\-enable\-prof\fR]
 .RS 4
-Average interval (log base 2) between allocation samples, as measured in bytes of allocation activity\&. Increasing the sampling interval decreases profile fidelity, but also decreases the computational overhead\&. The default sample interval is 512 KiB (2^19 B)\&.
+Average interval (log base 2) between allocation samples, as measured in bytes of allocation activity\&. Increasing the sampling interval decreases profile fidelity, but also decreases the computational overhead\&. The default sample interval is 1 (2^0) (i\&.e\&. all allocations are sampled)\&.
 .RE
 .PP
 "opt\&.prof_accum" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR]
 .RS 4
-Reporting of cumulative object/byte counts in profile dumps enabled/disabled\&. If this option is enabled, every unique backtrace must be stored for the duration of execution\&. Depending on the application, this can impose a large memory overhead, and the cumulative counts are not always of interest\&. This option is disabled by default\&.
+Reporting of cumulative object/byte counts in profile dumps enabled/disabled\&. If this option is enabled, every unique backtrace must be stored for the duration of execution\&. Depending on the application, this can impose a large memory overhead, and the cumulative counts are not always of interest\&. See the
+"opt\&.lg_prof_tcmax"
+option for control of per thread backtrace caching, which has important interactions\&. This option is enabled by default\&.
+.RE
+.PP
+"opt\&.lg_prof_tcmax" (\fBssize_t\fR) r\- [\fB\-\-enable\-prof\fR]
+.RS 4
+Maximum per thread backtrace cache (log base 2) used for heap profiling\&. A backtrace can only be discarded if the
+"opt\&.prof_accum"
+option is disabled, and no thread caches currently refer to the backtrace\&. Therefore, a backtrace cache limit should be imposed if the intention is to limit how much memory is used by backtraces\&. By default, no limit is imposed (encoded as \-1)\&.
 .RE
 .PP
 "opt\&.lg_prof_interval" (\fBssize_t\fR) r\- [\fB\-\-enable\-prof\fR]
@@ -815,30 +760,38 @@ is controlled by the
 option\&. This option is disabled by default\&.
 .RE
 .PP
-"opt\&.prof_final" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR]
-.RS 4
-Use an
-\fBatexit\fR(3)
-function to dump final memory usage to a file named according to the pattern
-<prefix>\&.<pid>\&.<seq>\&.f\&.heap, where
-<prefix>
-is controlled by the
-"opt\&.prof_prefix"
-option\&. This option is enabled by default\&.
-.RE
-.PP
 "opt\&.prof_leak" (\fBbool\fR) r\- [\fB\-\-enable\-prof\fR]
 .RS 4
 Leak reporting enabled/disabled\&. If enabled, use an
 \fBatexit\fR(3)
 function to report memory leaks detected by allocation sampling\&. See the
+"opt\&.lg_prof_bt_max"
+option for backtrace depth control\&. See the
 "opt\&.prof"
 option for information on analyzing heap profile output\&. This option is disabled by default\&.
 .RE
 .PP
+"opt\&.overcommit" (\fBbool\fR) r\- [\fB\-\-enable\-swap\fR]
+.RS 4
+Over\-commit enabled/disabled\&. If enabled, over\-commit memory as a side effect of using anonymous
+\fBmmap\fR(2)
+or
+\fBsbrk\fR(2)
+for virtual memory allocation\&. In order for overcommit to be disabled, the
+"swap\&.fds"
+mallctl must have been successfully written to\&. This option is enabled by default\&.
+.RE
+.PP
+"tcache\&.flush" (\fBvoid\fR) \-\- [\fB\-\-enable\-tcache\fR]
+.RS 4
+Flush calling thread\*(Aqs tcache\&. This interface releases all cached objects and internal data structures associated with the calling thread\*(Aqs thread\-specific cache\&. Ordinarily, this interface need not be called, since automatic periodic incremental garbage collection occurs, and the thread cache is automatically discarded when a thread exits\&. However, garbage collection is triggered by allocation activity, so it is possible for a thread that stops allocating/deallocating to retain its cache indefinitely, in which case the developer may find manual flushing useful\&.
+.RE
+.PP
 "thread\&.arena" (\fBunsigned\fR) rw
 .RS 4
-Get or set the arena associated with the calling thread\&. If the specified arena was not initialized beforehand (see the
+Get or set the arena associated with the calling thread\&. The arena index must be less than the maximum number of arenas (see the
+"arenas\&.narenas"
+mallctl)\&. If the specified arena was not initialized beforehand (see the
 "arenas\&.initialized"
 mallctl), it will be automatically initialized as a side effect of calling this interface\&.
 .RE
@@ -871,34 +824,9 @@ mallctl\&. This is useful for avoiding the overhead of repeated
 calls\&.
 .RE
 .PP
-"thread\&.tcache\&.enabled" (\fBbool\fR) rw [\fB\-\-enable\-tcache\fR]
-.RS 4
-Enable/disable calling thread\*(Aqs tcache\&. The tcache is implicitly flushed as a side effect of becoming disabled (see
-"thread\&.tcache\&.flush")\&.
-.RE
-.PP
-"thread\&.tcache\&.flush" (\fBvoid\fR) \-\- [\fB\-\-enable\-tcache\fR]
-.RS 4
-Flush calling thread\*(Aqs tcache\&. This interface releases all cached objects and internal data structures associated with the calling thread\*(Aqs thread\-specific cache\&. Ordinarily, this interface need not be called, since automatic periodic incremental garbage collection occurs, and the thread cache is automatically discarded when a thread exits\&. However, garbage collection is triggered by allocation activity, so it is possible for a thread that stops allocating/deallocating to retain its cache indefinitely, in which case the developer may find manual flushing useful\&.
-.RE
-.PP
-"arena\&.<i>\&.purge" (\fBunsigned\fR) \-\-
-.RS 4
-Purge unused dirty pages for arena <i>, or for all arenas if <i> equals
-"arenas\&.narenas"\&.
-.RE
-.PP
-"arena\&.<i>\&.dss" (\fBconst char *\fR) rw
-.RS 4
-Set the precedence of dss allocation as related to mmap allocation for arena <i>, or for all arenas if <i> equals
-"arenas\&.narenas"\&. See
-"opt\&.dss"
-for supported settings\&.
-.RE
-.PP
 "arenas\&.narenas" (\fBunsigned\fR) r\-
 .RS 4
-Current limit on number of arenas\&.
+Maximum number of arenas\&.
 .RE
 .PP
 "arenas\&.initialized" (\fBbool *\fR) r\-
@@ -913,19 +841,94 @@ booleans\&. Each boolean indicates whether the corresponding arena is initialize
 Quantum size\&.
 .RE
 .PP
-"arenas\&.page" (\fBsize_t\fR) r\-
+"arenas\&.cacheline" (\fBsize_t\fR) r\-
+.RS 4
+Assumed cacheline size\&.
+.RE
+.PP
+"arenas\&.subpage" (\fBsize_t\fR) r\-
+.RS 4
+Subpage size class interval\&.
+.RE
+.PP
+"arenas\&.pagesize" (\fBsize_t\fR) r\-
 .RS 4
 Page size\&.
 .RE
 .PP
+"arenas\&.chunksize" (\fBsize_t\fR) r\-
+.RS 4
+Chunk size\&.
+.RE
+.PP
+"arenas\&.tspace_min" (\fBsize_t\fR) r\-
+.RS 4
+Minimum tiny size class\&. Tiny size classes are powers of two\&.
+.RE
+.PP
+"arenas\&.tspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Maximum tiny size class\&. Tiny size classes are powers of two\&.
+.RE
+.PP
+"arenas\&.qspace_min" (\fBsize_t\fR) r\-
+.RS 4
+Minimum quantum\-spaced size class\&.
+.RE
+.PP
+"arenas\&.qspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Maximum quantum\-spaced size class\&.
+.RE
+.PP
+"arenas\&.cspace_min" (\fBsize_t\fR) r\-
+.RS 4
+Minimum cacheline\-spaced size class\&.
+.RE
+.PP
+"arenas\&.cspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Maximum cacheline\-spaced size class\&.
+.RE
+.PP
+"arenas\&.sspace_min" (\fBsize_t\fR) r\-
+.RS 4
+Minimum subpage\-spaced size class\&.
+.RE
+.PP
+"arenas\&.sspace_max" (\fBsize_t\fR) r\-
+.RS 4
+Maximum subpage\-spaced size class\&.
+.RE
+.PP
 "arenas\&.tcache_max" (\fBsize_t\fR) r\- [\fB\-\-enable\-tcache\fR]
 .RS 4
 Maximum thread\-cached size class\&.
 .RE
 .PP
+"arenas\&.ntbins" (\fBunsigned\fR) r\-
+.RS 4
+Number of tiny bin size classes\&.
+.RE
+.PP
+"arenas\&.nqbins" (\fBunsigned\fR) r\-
+.RS 4
+Number of quantum\-spaced bin size classes\&.
+.RE
+.PP
+"arenas\&.ncbins" (\fBunsigned\fR) r\-
+.RS 4
+Number of cacheline\-spaced bin size classes\&.
+.RE
+.PP
+"arenas\&.nsbins" (\fBunsigned\fR) r\-
+.RS 4
+Number of subpage\-spaced bin size classes\&.
+.RE
+.PP
 "arenas\&.nbins" (\fBunsigned\fR) r\-
 .RS 4
-Number of bin size classes\&.
+Total number of bin size classes\&.
 .RE
 .PP
 "arenas\&.nhbins" (\fBunsigned\fR) r\- [\fB\-\-enable\-tcache\fR]
@@ -963,11 +966,6 @@ Maximum size supported by this large size class\&.
 Purge unused dirty pages for the specified arena, or for all arenas if none is specified\&.
 .RE
 .PP
-"arenas\&.extend" (\fBunsigned\fR) r\-
-.RS 4
-Extend the array of arenas by appending a new arena, and returning the new arena index\&.
-.RE
-.PP
 "prof\&.active" (\fBbool\fR) rw [\fB\-\-enable\-prof\fR]
 .RS 4
 Control whether sampling is currently active\&. See the
@@ -1007,20 +1005,18 @@ Total number of bytes allocated by the application\&.
 "stats\&.active" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
 .RS 4
 Total number of bytes in active pages allocated by the application\&. This is a multiple of the page size, and greater than or equal to
-"stats\&.allocated"\&. This does not include
-"stats\&.arenas\&.<i>\&.pdirty"
-and pages entirely devoted to allocator metadata\&.
+"stats\&.allocated"\&.
 .RE
 .PP
 "stats\&.mapped" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
 .RS 4
 Total number of bytes in chunks mapped on behalf of the application\&. This is a multiple of the chunk size, and is at least as large as
-"stats\&.active"\&. This does not include inactive chunks\&.
+"stats\&.active"\&. This does not include inactive chunks backed by swap files\&. his does not include inactive chunks embedded in the DSS\&.
 .RE
 .PP
 "stats\&.chunks\&.current" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
 .RS 4
-Total number of chunks actively mapped on behalf of the application\&. This does not include inactive chunks\&.
+Total number of chunks actively mapped on behalf of the application\&. This does not include inactive chunks backed by swap files\&. This does not include inactive chunks embedded in the DSS\&.
 .RE
 .PP
 "stats\&.chunks\&.total" (\fBuint64_t\fR) r\- [\fB\-\-enable\-stats\fR]
@@ -1048,15 +1044,6 @@ Cumulative number of huge allocation requests\&.
 Cumulative number of huge deallocation requests\&.
 .RE
 .PP
-"stats\&.arenas\&.<i>\&.dss" (\fBconst char *\fR) r\-
-.RS 4
-dss (\fBsbrk\fR(2)) allocation precedence as related to
-\fBmmap\fR(2)
-allocation\&. See
-"opt\&.dss"
-for details\&.
-.RE
-.PP
 "stats\&.arenas\&.<i>\&.nthreads" (\fBunsigned\fR) r\-
 .RS 4
 Number of threads currently assigned to arena\&.
@@ -1176,6 +1163,11 @@ Cumulative number of runs created\&.
 Cumulative number of times the current run from which to allocate changed\&.
 .RE
 .PP
+"stats\&.arenas\&.<i>\&.bins\&.<j>\&.highruns" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
+.RS 4
+Maximum number of runs at any time thus far\&.
+.RE
+.PP
 "stats\&.arenas\&.<i>\&.bins\&.<j>\&.curruns" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
 .RS 4
 Current number of runs\&.
@@ -1196,10 +1188,44 @@ Cumulative number of deallocation requests for this size class served directly b
 Cumulative number of allocation requests for this size class\&.
 .RE
 .PP
+"stats\&.arenas\&.<i>\&.lruns\&.<j>\&.highruns" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
+.RS 4
+Maximum number of runs at any time thus far for this size class\&.
+.RE
+.PP
 "stats\&.arenas\&.<i>\&.lruns\&.<j>\&.curruns" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats\fR]
 .RS 4
 Current number of runs for this size class\&.
 .RE
+.PP
+"swap\&.avail" (\fBsize_t\fR) r\- [\fB\-\-enable\-stats \-\-enable\-swap\fR]
+.RS 4
+Number of swap file bytes that are currently not associated with any chunk (i\&.e\&. mapped, but otherwise completely unmanaged)\&.
+.RE
+.PP
+"swap\&.prezeroed" (\fBbool\fR) rw [\fB\-\-enable\-swap\fR]
+.RS 4
+If true, the allocator assumes that the swap file(s) contain nothing but nil bytes\&. If this assumption is violated, allocator behavior is undefined\&. This value becomes read\-only after
+"swap\&.fds"
+is successfully written to\&.
+.RE
+.PP
+"swap\&.nfds" (\fBsize_t\fR) r\- [\fB\-\-enable\-swap\fR]
+.RS 4
+Number of file descriptors in use for swap\&.
+.RE
+.PP
+"swap\&.fds" (\fBint *\fR) r\- [\fB\-\-enable\-swap\fR]
+.RS 4
+When written to, the files associated with the specified file descriptors are contiguously mapped via
+\fBmmap\fR(2)\&. The resulting virtual memory region is preferred over anonymous
+\fBmmap\fR(2)
+and
+\fBsbrk\fR(2)
+memory\&. Note that if a file\*(Aqs size is not a multiple of the page size, it is automatically truncated to the nearest page size multiple\&. See the
+"swap\&.prezeroed"
+mallctl for specifying that the files are pre\-zeroed\&.
+.RE
 .SH "DEBUGGING MALLOC PROBLEMS"
 .PP
 When debugging, it is a good idea to configure/build jemalloc with the
@@ -1214,11 +1240,7 @@ option) tends to expose such bugs in the form of obviously incorrect results and
 "opt\&.zero"
 option) eliminates the symptoms of such bugs\&. Between these two options, it is usually possible to quickly detect, diagnose, and eliminate such bugs\&.
 .PP
-This implementation does not provide much detail about the problems it detects, because the performance impact for storing such information would be prohibitive\&. However, jemalloc does integrate with the most excellent
-\m[blue]\fBValgrind\fR\m[]\&\s-2\u[2]\d\s+2
-tool if the
-\fB\-\-enable\-valgrind\fR
-configuration option is enabled\&.
+This implementation does not provide much detail about the problems it detects, because the performance impact for storing such information would be prohibitive\&. There are a number of allocator implementations available on the Internet which focus on detecting and pinpointing problems by trading performance for extra sanity checks and detailed diagnostics\&.
 .SH "DIAGNOSTIC MESSAGES"
 .PP
 If any of the memory allocation/deallocation functions detect an error or warning condition, a message will be printed to file descriptor
@@ -1274,28 +1296,6 @@ Memory allocation error\&.
 .RE
 .PP
 The
-\fBaligned_alloc\fR\fB\fR
-function returns a pointer to the allocated memory if successful; otherwise a
-\fBNULL\fR
-pointer is returned and
-\fIerrno\fR
-is set\&. The
-\fBaligned_alloc\fR\fB\fR
-function will fail if:
-.PP
-EINVAL
-.RS 4
-The
-\fIalignment\fR
-parameter is not a power of 2\&.
-.RE
-.PP
-ENOMEM
-.RS 4
-Memory allocation error\&.
-.RE
-.PP
-The
 \fBrealloc\fR\fB\fR
 function returns a pointer, possibly identical to
 \fIptr\fR, to the allocated memory if successful; otherwise a
@@ -1370,15 +1370,14 @@ read/write processing\&.
 The
 \fBallocm\fR\fB\fR,
 \fBrallocm\fR\fB\fR,
-\fBsallocm\fR\fB\fR,
-\fBdallocm\fR\fB\fR, and
-\fBnallocm\fR\fB\fR
+\fBsallocm\fR\fB\fR, and
+\fBdallocm\fR\fB\fR
 functions return
 \fBALLOCM_SUCCESS\fR
 on success; otherwise they return an error value\&. The
-\fBallocm\fR\fB\fR,
-\fBrallocm\fR\fB\fR, and
-\fBnallocm\fR\fB\fR
+\fBallocm\fR\fB\fR
+and
+\fBrallocm\fR\fB\fR
 functions will fail if:
 .PP
 ALLOCM_ERR_OOM
@@ -1443,7 +1442,6 @@ malloc_conf = "lg_chunk:24";
 \fBmadvise\fR(2),
 \fBmmap\fR(2),
 \fBsbrk\fR(2),
-\fButrace\fR(2),
 \fBalloca\fR(3),
 \fBatexit\fR(3),
 \fBgetpagesize\fR(3)
@@ -1471,12 +1469,7 @@ jemalloc website
 \%http://www.canonware.com/jemalloc/
 .RE
 .IP " 2." 4
-Valgrind
+google-perftools package
 .RS 4
-\%http://valgrind.org/
-.RE
-.IP " 3." 4
-gperftools package
-.RS 4
-\%http://code.google.com/p/gperftools/
+\%http://code.google.com/p/google-perftools/
 .RE
@@ -11,8 +11,22 @@

 #define	atomic_read_uint64(p)	atomic_add_uint64(p, 0)
 #define	atomic_read_uint32(p)	atomic_add_uint32(p, 0)
-#define	atomic_read_z(p)	atomic_add_z(p, 0)
-#define	atomic_read_u(p)	atomic_add_u(p, 0)
+
+#if (LG_SIZEOF_PTR == 3)
+#  define atomic_read_z(p)						\
+    (size_t)atomic_add_uint64((uint64_t *)p, (uint64_t)0)
+#  define atomic_add_z(p, x)						\
+    (size_t)atomic_add_uint64((uint64_t *)p, (uint64_t)x)
+#  define atomic_sub_z(p, x)						\
+    (size_t)atomic_sub_uint64((uint64_t *)p, (uint64_t)x)
+#elif (LG_SIZEOF_PTR == 2)
+#  define atomic_read_z(p)						\
+    (size_t)atomic_add_uint32((uint32_t *)p, (uint32_t)0)
+#  define atomic_add_z(p, x)						\
+    (size_t)atomic_add_uint32((uint32_t *)p, (uint32_t)x)
+#  define atomic_sub_z(p, x)						\
+    (size_t)atomic_sub_uint32((uint32_t *)p, (uint32_t)x)
+#endif

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -23,17 +37,12 @@ uint64_t	atomic_add_uint64(uint64_t *p, uint64_t x);
 uint64_t	atomic_sub_uint64(uint64_t *p, uint64_t x);
 uint32_t	atomic_add_uint32(uint32_t *p, uint32_t x);
 uint32_t	atomic_sub_uint32(uint32_t *p, uint32_t x);
-size_t	atomic_add_z(size_t *p, size_t x);
-size_t	atomic_sub_z(size_t *p, size_t x);
-unsigned	atomic_add_u(unsigned *p, unsigned x);
-unsigned	atomic_sub_u(unsigned *p, unsigned x);
 #endif

 #if (defined(JEMALLOC_ENABLE_INLINE) || defined(JEMALLOC_ATOMIC_C_))
 /******************************************************************************/
 /* 64-bit operations. */
-#if (LG_SIZEOF_PTR == 3 || LG_SIZEOF_INT == 3)
-#  ifdef __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
+#ifdef __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
 JEMALLOC_INLINE uint64_t
 atomic_add_uint64(uint64_t *p, uint64_t x)
 {
@@ -47,20 +56,6 @@ atomic_sub_uint64(uint64_t *p, uint64_t x)

 	return (__sync_sub_and_fetch(p, x));
 }
-#elif (defined(_MSC_VER))
-JEMALLOC_INLINE uint64_t
-atomic_add_uint64(uint64_t *p, uint64_t x)
-{
-
-	return (InterlockedExchangeAdd64(p, x));
-}
-
-JEMALLOC_INLINE uint64_t
-atomic_sub_uint64(uint64_t *p, uint64_t x)
-{
-
-	return (InterlockedExchangeAdd64(p, -((int64_t)x)));
-}
 #elif (defined(JEMALLOC_OSATOMIC))
 JEMALLOC_INLINE uint64_t
 atomic_add_uint64(uint64_t *p, uint64_t x)
@@ -75,7 +70,7 @@ atomic_sub_uint64(uint64_t *p, uint64_t x)

 	return (OSAtomicAdd64(-((int64_t)x), (int64_t *)p));
 }
-#  elif (defined(__amd64__) || defined(__x86_64__))
+#elif (defined(__amd64_) || defined(__x86_64__))
 JEMALLOC_INLINE uint64_t
 atomic_add_uint64(uint64_t *p, uint64_t x)
 {
@@ -102,43 +97,8 @@ atomic_sub_uint64(uint64_t *p, uint64_t x)

 	return (x);
 }
-#  elif (defined(JEMALLOC_ATOMIC9))
-JEMALLOC_INLINE uint64_t
-atomic_add_uint64(uint64_t *p, uint64_t x)
-{
-
-	/*
-	 * atomic_fetchadd_64() doesn't exist, but we only ever use this
-	 * function on LP64 systems, so atomic_fetchadd_long() will do.
-	 */
-	assert(sizeof(uint64_t) == sizeof(unsigned long));
-
-	return (atomic_fetchadd_long(p, (unsigned long)x) + x);
-}
-
-JEMALLOC_INLINE uint64_t
-atomic_sub_uint64(uint64_t *p, uint64_t x)
-{
-
-	assert(sizeof(uint64_t) == sizeof(unsigned long));
-
-	return (atomic_fetchadd_long(p, (unsigned long)(-(long)x)) - x);
-}
-#  elif (defined(JE_FORCE_SYNC_COMPARE_AND_SWAP_8))
-JEMALLOC_INLINE uint64_t
-atomic_add_uint64(uint64_t *p, uint64_t x)
-{
-
-	return (__sync_add_and_fetch(p, x));
-}
-
-JEMALLOC_INLINE uint64_t
-atomic_sub_uint64(uint64_t *p, uint64_t x)
-{
-
-	return (__sync_sub_and_fetch(p, x));
-}
-#  else
+#else
+#  if (LG_SIZEOF_PTR == 3)
 #    error "Missing implementation for 64-bit atomic operations"
 #  endif
 #endif
@@ -159,20 +119,6 @@ atomic_sub_uint32(uint32_t *p, uint32_t x)

 	return (__sync_sub_and_fetch(p, x));
 }
-#elif (defined(_MSC_VER))
-JEMALLOC_INLINE uint32_t
-atomic_add_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (InterlockedExchangeAdd(p, x));
-}
-
-JEMALLOC_INLINE uint32_t
-atomic_sub_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (InterlockedExchangeAdd(p, -((int32_t)x)));
-}
 #elif (defined(JEMALLOC_OSATOMIC))
 JEMALLOC_INLINE uint32_t
 atomic_add_uint32(uint32_t *p, uint32_t x)
@@ -187,7 +133,7 @@ atomic_sub_uint32(uint32_t *p, uint32_t x)

 	return (OSAtomicAdd32(-((int32_t)x), (int32_t *)p));
 }
-#elif (defined(__i386__) || defined(__amd64__) || defined(__x86_64__))
+#elif (defined(__i386__) || defined(__amd64_) || defined(__x86_64__))
 JEMALLOC_INLINE uint32_t
 atomic_add_uint32(uint32_t *p, uint32_t x)
 {
@@ -214,90 +160,9 @@ atomic_sub_uint32(uint32_t *p, uint32_t x)

 	return (x);
 }
-#elif (defined(JEMALLOC_ATOMIC9))
-JEMALLOC_INLINE uint32_t
-atomic_add_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (atomic_fetchadd_32(p, x) + x);
-}
-
-JEMALLOC_INLINE uint32_t
-atomic_sub_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (atomic_fetchadd_32(p, (uint32_t)(-(int32_t)x)) - x);
-}
-#elif (defined(JE_FORCE_SYNC_COMPARE_AND_SWAP_4))
-JEMALLOC_INLINE uint32_t
-atomic_add_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (__sync_add_and_fetch(p, x));
-}
-
-JEMALLOC_INLINE uint32_t
-atomic_sub_uint32(uint32_t *p, uint32_t x)
-{
-
-	return (__sync_sub_and_fetch(p, x));
-}
 #else
 #  error "Missing implementation for 32-bit atomic operations"
 #endif
-
-/******************************************************************************/
-/* size_t operations. */
-JEMALLOC_INLINE size_t
-atomic_add_z(size_t *p, size_t x)
-{
-
-#if (LG_SIZEOF_PTR == 3)
-	return ((size_t)atomic_add_uint64((uint64_t *)p, (uint64_t)x));
-#elif (LG_SIZEOF_PTR == 2)
-	return ((size_t)atomic_add_uint32((uint32_t *)p, (uint32_t)x));
-#endif
-}
-
-JEMALLOC_INLINE size_t
-atomic_sub_z(size_t *p, size_t x)
-{
-
-#if (LG_SIZEOF_PTR == 3)
-	return ((size_t)atomic_add_uint64((uint64_t *)p,
-	    (uint64_t)-((int64_t)x)));
-#elif (LG_SIZEOF_PTR == 2)
-	return ((size_t)atomic_add_uint32((uint32_t *)p,
-	    (uint32_t)-((int32_t)x)));
-#endif
-}
-
-/******************************************************************************/
-/* unsigned operations. */
-JEMALLOC_INLINE unsigned
-atomic_add_u(unsigned *p, unsigned x)
-{
-
-#if (LG_SIZEOF_INT == 3)
-	return ((unsigned)atomic_add_uint64((uint64_t *)p, (uint64_t)x));
-#elif (LG_SIZEOF_INT == 2)
-	return ((unsigned)atomic_add_uint32((uint32_t *)p, (uint32_t)x));
-#endif
-}
-
-JEMALLOC_INLINE unsigned
-atomic_sub_u(unsigned *p, unsigned x)
-{
-
-#if (LG_SIZEOF_INT == 3)
-	return ((unsigned)atomic_add_uint64((uint64_t *)p,
-	    (uint64_t)-((int64_t)x)));
-#elif (LG_SIZEOF_INT == 2)
-	return ((unsigned)atomic_add_uint32((uint32_t *)p,
-	    (uint32_t)-((int32_t)x)));
-#endif
-}
-/******************************************************************************/
 #endif

 #endif /* JEMALLOC_H_INLINES */
@@ -9,14 +9,12 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

+extern malloc_mutex_t	base_mtx;
+
 void	*base_alloc(size_t size);
-void	*base_calloc(size_t number, size_t size);
 extent_node_t *base_node_alloc(void);
 void	base_node_dealloc(extent_node_t *node);
 bool	base_boot(void);
-void	base_prefork(void);
-void	base_postfork_parent(void);
-void	base_postfork_child(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -28,14 +28,20 @@
 #ifdef JEMALLOC_H_EXTERNS

 extern size_t		opt_lg_chunk;
-extern const char	*opt_dss;
+#ifdef JEMALLOC_SWAP
+extern bool		opt_overcommit;
+#endif

+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
 /* Protects stats_chunks; currently not used for any other purpose. */
 extern malloc_mutex_t	chunks_mtx;
 /* Chunk statistics. */
 extern chunk_stats_t	stats_chunks;
+#endif

+#ifdef JEMALLOC_IVSALLOC
 extern rtree_t		*chunks_rtree;
+#endif

 extern size_t		chunksize;
 extern size_t		chunksize_mask; /* (chunksize - 1). */
@@ -43,14 +49,9 @@ extern size_t		chunk_npages;
 extern size_t		map_bias; /* Number of arena chunk header pages. */
 extern size_t		arena_maxclass; /* Max size class for arenas. */

-void	*chunk_alloc(size_t size, size_t alignment, bool base, bool *zero,
-    dss_prec_t dss_prec);
-void	chunk_unmap(void *chunk, size_t size);
-void	chunk_dealloc(void *chunk, size_t size, bool unmap);
+void	*chunk_alloc(size_t size, bool base, bool *zero);
+void	chunk_dealloc(void *chunk, size_t size);
 bool	chunk_boot(void);
-void	chunk_prefork(void);
-void	chunk_postfork_parent(void);
-void	chunk_postfork_child(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -59,5 +60,6 @@ void	chunk_postfork_child(void);
 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/

+#include "jemalloc/internal/chunk_swap.h"
 #include "jemalloc/internal/chunk_dss.h"
 #include "jemalloc/internal/chunk_mmap.h"
@@ -1,34 +1,25 @@
+#ifdef JEMALLOC_DSS
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

-typedef enum {
-	dss_prec_disabled  = 0,
-	dss_prec_primary   = 1,
-	dss_prec_secondary = 2,
-
-	dss_prec_limit     = 3
-} dss_prec_t ;
-#define	DSS_PREC_DEFAULT	dss_prec_secondary
-#define	DSS_DEFAULT		"secondary"
-
 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
 #ifdef JEMALLOC_H_STRUCTS

-extern const char *dss_prec_names[];
-
 #endif /* JEMALLOC_H_STRUCTS */
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

-dss_prec_t	chunk_dss_prec_get(void);
-bool	chunk_dss_prec_set(dss_prec_t dss_prec);
-void	*chunk_alloc_dss(size_t size, size_t alignment, bool *zero);
+/*
+ * Protects sbrk() calls.  This avoids malloc races among threads, though it
+ * does not protect against races with threads that call sbrk() directly.
+ */
+extern malloc_mutex_t	dss_mtx;
+
+void	*chunk_alloc_dss(size_t size, bool *zero);
 bool	chunk_in_dss(void *chunk);
+bool	chunk_dealloc_dss(void *chunk, size_t size);
 bool	chunk_dss_boot(void);
-void	chunk_dss_prefork(void);
-void	chunk_dss_postfork_parent(void);
-void	chunk_dss_postfork_child(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -36,3 +27,4 @@ void	chunk_dss_postfork_child(void);

 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/
+#endif /* JEMALLOC_DSS */
@@ -9,10 +9,11 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

-bool	pages_purge(void *addr, size_t length);
+void	*chunk_alloc_mmap(size_t size);
+void	*chunk_alloc_mmap_noreserve(size_t size);
+void	chunk_dealloc_mmap(void *chunk, size_t size);

-void	*chunk_alloc_mmap(size_t size, size_t alignment, bool *zero);
-bool	chunk_dealloc_mmap(void *chunk, size_t size);
+bool	chunk_mmap_boot(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -0,0 +1,34 @@
+#ifdef JEMALLOC_SWAP
+/******************************************************************************/
+#ifdef JEMALLOC_H_TYPES
+
+#endif /* JEMALLOC_H_TYPES */
+/******************************************************************************/
+#ifdef JEMALLOC_H_STRUCTS
+
+#endif /* JEMALLOC_H_STRUCTS */
+/******************************************************************************/
+#ifdef JEMALLOC_H_EXTERNS
+
+extern malloc_mutex_t	swap_mtx;
+extern bool		swap_enabled;
+extern bool		swap_prezeroed;
+extern size_t		swap_nfds;
+extern int		*swap_fds;
+#ifdef JEMALLOC_STATS
+extern size_t		swap_avail;
+#endif
+
+void	*chunk_alloc_swap(size_t size, bool *zero);
+bool	chunk_in_swap(void *chunk);
+bool	chunk_dealloc_swap(void *chunk, size_t size);
+bool	chunk_swap_enable(const int *fds, unsigned nfds, bool prezeroed);
+bool	chunk_swap_boot(void);
+
+#endif /* JEMALLOC_H_EXTERNS */
+/******************************************************************************/
+#ifdef JEMALLOC_H_INLINES
+
+#endif /* JEMALLOC_H_INLINES */
+/******************************************************************************/
+#endif /* JEMALLOC_SWAP */
@@ -30,6 +30,11 @@ struct ckhc_s {
 };

 struct ckh_s {
+#ifdef JEMALLOC_DEBUG
+#define	CKH_MAGIC	0x3af2489d
+	uint32_t	magic;
+#endif
+
 #ifdef CKH_COUNT
 	/* Counters used to get an idea of performance. */
 	uint64_t	ngrows;
@@ -42,7 +47,7 @@ struct ckh_s {
 	/* Used for pseudo-random number generation. */
 #define	CKH_A		1103515241
 #define	CKH_C		12347
-	uint32_t	prng_state;
+	uint32_t	prn_state;

 	/* Total number of items. */
 	size_t		count;
@@ -2,8 +2,6 @@
 #ifdef JEMALLOC_H_TYPES

 typedef struct ctl_node_s ctl_node_t;
-typedef struct ctl_named_node_s ctl_named_node_t;
-typedef struct ctl_indexed_node_s ctl_indexed_node_t;
 typedef struct ctl_arena_stats_s ctl_arena_stats_t;
 typedef struct ctl_stats_s ctl_stats_t;

@@ -13,29 +11,28 @@ typedef struct ctl_stats_s ctl_stats_t;

 struct ctl_node_s {
 	bool			named;
-};
-
-struct ctl_named_node_s {
-	struct ctl_node_s	node;
-	const char		*name;
-	/* If (nchildren == 0), this is a terminal node. */
-	unsigned		nchildren;
-	const			ctl_node_t *children;
-	int			(*ctl)(const size_t *, size_t, void *, size_t *,
-	    void *, size_t);
-};
-
-struct ctl_indexed_node_s {
-	struct ctl_node_s	node;
-	const ctl_named_node_t	*(*index)(const size_t *, size_t, size_t);
+	union {
+		struct {
+			const char	*name;
+			/* If (nchildren == 0), this is a terminal node. */
+			unsigned	nchildren;
+			const	ctl_node_t *children;
+		} named;
+		struct {
+			const ctl_node_t *(*index)(const size_t *, size_t,
+			    size_t);
+		} indexed;
+	} u;
+	int	(*ctl)(const size_t *, size_t, void *, size_t *, void *,
+	    size_t);
 };

 struct ctl_arena_stats_s {
 	bool			initialized;
 	unsigned		nthreads;
-	const char		*dss;
 	size_t			pactive;
 	size_t			pdirty;
+#ifdef JEMALLOC_STATS
 	arena_stats_t		astats;

 	/* Aggregate stats for small size classes, based on bin stats. */
@@ -44,11 +41,13 @@ struct ctl_arena_stats_s {
 	uint64_t		ndalloc_small;
 	uint64_t		nrequests_small;

-	malloc_bin_stats_t	bstats[NBINS];
+	malloc_bin_stats_t	*bstats;	/* nbins elements. */
 	malloc_large_stats_t	*lstats;	/* nlclasses elements. */
+#endif
 };

 struct ctl_stats_s {
+#ifdef JEMALLOC_STATS
 	size_t			allocated;
 	size_t			active;
 	size_t			mapped;
@@ -62,8 +61,11 @@ struct ctl_stats_s {
 		uint64_t	nmalloc;	/* huge_nmalloc */
 		uint64_t	ndalloc;	/* huge_ndalloc */
 	} huge;
-	unsigned		narenas;
+#endif
 	ctl_arena_stats_t	*arenas;	/* (narenas + 1) elements. */
+#ifdef JEMALLOC_SWAP
+	size_t			swap_avail;
+#endif
 };

 #endif /* JEMALLOC_H_STRUCTS */
@@ -77,30 +79,29 @@ int	ctl_nametomib(const char *name, size_t *mibp, size_t *miblenp);
 int	ctl_bymib(const size_t *mib, size_t miblen, void *oldp, size_t *oldlenp,
    void *newp, size_t newlen);
 bool	ctl_boot(void);
-void	ctl_prefork(void);
-void	ctl_postfork_parent(void);
-void	ctl_postfork_child(void);

 #define	xmallctl(name, oldp, oldlenp, newp, newlen) do {		\
-	if (je_mallctl(name, oldp, oldlenp, newp, newlen)		\
+	if (JEMALLOC_P(mallctl)(name, oldp, oldlenp, newp, newlen)	\
 	    != 0) {							\
-		malloc_printf(						\
-		    "<jemalloc>: Failure in xmallctl(\"%s\", ...)\n",	\
-		    name);						\
+		malloc_write("<jemalloc>: Failure in xmallctl(\"");	\
+		malloc_write(name);					\
+		malloc_write("\", ...)\n");				\
 		abort();						\
 	}								\
 } while (0)

 #define	xmallctlnametomib(name, mibp, miblenp) do {			\
-	if (je_mallctlnametomib(name, mibp, miblenp) != 0) {		\
-		malloc_printf("<jemalloc>: Failure in "			\
-		    "xmallctlnametomib(\"%s\", ...)\n", name);		\
+	if (JEMALLOC_P(mallctlnametomib)(name, mibp, miblenp) != 0) {	\
+		malloc_write(						\
+		    "<jemalloc>: Failure in xmallctlnametomib(\"");	\
+		malloc_write(name);					\
+		malloc_write("\", ...)\n");				\
 		abort();						\
 	}								\
 } while (0)

 #define	xmallctlbymib(mib, miblen, oldp, oldlenp, newp, newlen) do {	\
-	if (je_mallctlbymib(mib, miblen, oldp, oldlenp, newp,		\
+	if (JEMALLOC_P(mallctlbymib)(mib, miblen, oldp, oldlenp, newp,	\
 	    newlen) != 0) {						\
 		malloc_write(						\
 		    "<jemalloc>: Failure in xmallctlbymib()\n");	\
@@ -9,23 +9,24 @@ typedef struct extent_node_s extent_node_t;

 /* Tree of extents. */
 struct extent_node_s {
+#if (defined(JEMALLOC_SWAP) || defined(JEMALLOC_DSS))
 	/* Linkage for the size/address-ordered tree. */
 	rb_node(extent_node_t)	link_szad;
+#endif

 	/* Linkage for the address-ordered tree. */
 	rb_node(extent_node_t)	link_ad;

+#ifdef JEMALLOC_PROF
 	/* Profile counters, used for huge objects. */
 	prof_ctx_t		*prof_ctx;
+#endif

 	/* Pointer to the extent that this tree node is responsible for. */
 	void			*addr;

 	/* Total region size. */
 	size_t			size;
-
-	/* True if zero-filled; used by chunk recycling code. */
-	bool			zeroed;
 };
 typedef rb_tree(extent_node_t) extent_tree_t;

@@ -33,7 +34,9 @@ typedef rb_tree(extent_node_t) extent_tree_t;
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

+#if (defined(JEMALLOC_SWAP) || defined(JEMALLOC_DSS))
 rb_proto(, extent_tree_szad_, extent_tree_t, extent_node_t)
+#endif

 rb_proto(, extent_tree_ad_, extent_tree_t, extent_node_t)

@@ -26,7 +26,7 @@ uint64_t	hash(const void *key, size_t len, uint64_t seed);
 JEMALLOC_INLINE uint64_t
 hash(const void *key, size_t len, uint64_t seed)
 {
-	const uint64_t m = UINT64_C(0xc6a4a7935bd1e995);
+	const uint64_t m = 0xc6a4a7935bd1e995;
 	const int r = 47;
 	uint64_t h = seed ^ (len * m);
 	const uint64_t *data = (const uint64_t *)key;
@@ -48,14 +48,14 @@ hash(const void *key, size_t len, uint64_t seed)

 	data2 = (const unsigned char *)data;
 	switch(len & 7) {
-	case 7: h ^= ((uint64_t)(data2[6])) << 48;
-	case 6: h ^= ((uint64_t)(data2[5])) << 40;
-	case 5: h ^= ((uint64_t)(data2[4])) << 32;
-	case 4: h ^= ((uint64_t)(data2[3])) << 24;
-	case 3: h ^= ((uint64_t)(data2[2])) << 16;
-	case 2: h ^= ((uint64_t)(data2[1])) << 8;
-	case 1: h ^= ((uint64_t)(data2[0]));
-		h *= m;
+		case 7: h ^= ((uint64_t)(data2[6])) << 48;
+		case 6: h ^= ((uint64_t)(data2[5])) << 40;
+		case 5: h ^= ((uint64_t)(data2[4])) << 32;
+		case 4: h ^= ((uint64_t)(data2[3])) << 24;
+		case 3: h ^= ((uint64_t)(data2[2])) << 16;
+		case 2: h ^= ((uint64_t)(data2[1])) << 8;
+		case 1: h ^= ((uint64_t)(data2[0]));
+			h *= m;
 	}

 	h ^= h >> r;
@@ -9,10 +9,12 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

+#ifdef JEMALLOC_STATS
 /* Huge allocation statistics. */
 extern uint64_t		huge_nmalloc;
 extern uint64_t		huge_ndalloc;
 extern size_t		huge_allocated;
+#endif

 /* Protects chunk-related data structures. */
 extern malloc_mutex_t	huge_mtx;
@@ -22,15 +24,14 @@ void	*huge_palloc(size_t size, size_t alignment, bool zero);
 void	*huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size,
    size_t extra);
 void	*huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
-    size_t alignment, bool zero, bool try_tcache_dalloc);
+    size_t alignment, bool zero);
 void	huge_dalloc(void *ptr, bool unmap);
 size_t	huge_salloc(const void *ptr);
+#ifdef JEMALLOC_PROF
 prof_ctx_t	*huge_prof_ctx_get(const void *ptr);
 void	huge_prof_ctx_set(const void *ptr, prof_ctx_t *ctx);
+#endif
 bool	huge_boot(void);
-void	huge_prefork(void);
-void	huge_postfork_parent(void);
-void	huge_postfork_child(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -54,7 +54,7 @@ mb_write(void)
 	    );
 #endif
 }
-#elif (defined(__amd64__) || defined(__x86_64__))
+#elif (defined(__amd64_) || defined(__x86_64__))
 JEMALLOC_INLINE void
 mb_write(void)
 {
@@ -87,13 +87,6 @@ mb_write(void)
 	    : "memory" /* Clobbers. */
 	    );
 }
-#elif defined(__tile__)
-JEMALLOC_INLINE void
-mb_write(void)
-{
-
-	__sync_synchronize();
-}
 #else
 /*
 * This is much slower than a simple memory barrier, but the semantics of mutex
@@ -1,42 +1,22 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

-typedef struct malloc_mutex_s malloc_mutex_t;
-
-#ifdef _WIN32
-#  define MALLOC_MUTEX_INITIALIZER
-#elif (defined(JEMALLOC_OSSPIN))
-#  define MALLOC_MUTEX_INITIALIZER {0}
-#elif (defined(JEMALLOC_MUTEX_INIT_CB))
-#  define MALLOC_MUTEX_INITIALIZER {PTHREAD_MUTEX_INITIALIZER, NULL}
+#ifdef JEMALLOC_OSSPIN
+typedef OSSpinLock malloc_mutex_t;
 #else
-#  if (defined(PTHREAD_MUTEX_ADAPTIVE_NP) &&				\
-       defined(PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP))
-#    define MALLOC_MUTEX_TYPE PTHREAD_MUTEX_ADAPTIVE_NP
-#    define MALLOC_MUTEX_INITIALIZER {PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP}
-#  else
-#    define MALLOC_MUTEX_TYPE PTHREAD_MUTEX_DEFAULT
-#    define MALLOC_MUTEX_INITIALIZER {PTHREAD_MUTEX_INITIALIZER}
-#  endif
+typedef pthread_mutex_t malloc_mutex_t;
+#endif
+
+#ifdef PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
+#  define MALLOC_MUTEX_INITIALIZER PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
+#else
+#  define MALLOC_MUTEX_INITIALIZER PTHREAD_MUTEX_INITIALIZER
 #endif

 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
 #ifdef JEMALLOC_H_STRUCTS

-struct malloc_mutex_s {
-#ifdef _WIN32
-	CRITICAL_SECTION	lock;
-#elif (defined(JEMALLOC_OSSPIN))
-	OSSpinLock		lock;
-#elif (defined(JEMALLOC_MUTEX_INIT_CB))
-	pthread_mutex_t		lock;
-	malloc_mutex_t		*postponed_next;
-#else
-	pthread_mutex_t		lock;
-#endif
-};
-
 #endif /* JEMALLOC_H_STRUCTS */
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS
@@ -44,15 +24,11 @@ struct malloc_mutex_s {
 #ifdef JEMALLOC_LAZY_LOCK
 extern bool isthreaded;
 #else
-#  undef isthreaded /* Undo private_namespace.h definition. */
 #  define isthreaded true
 #endif

 bool	malloc_mutex_init(malloc_mutex_t *mutex);
-void	malloc_mutex_prefork(malloc_mutex_t *mutex);
-void	malloc_mutex_postfork_parent(malloc_mutex_t *mutex);
-void	malloc_mutex_postfork_child(malloc_mutex_t *mutex);
-bool	mutex_boot(void);
+void	malloc_mutex_destroy(malloc_mutex_t *mutex);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -60,6 +36,7 @@ bool	mutex_boot(void);

 #ifndef JEMALLOC_ENABLE_INLINE
 void	malloc_mutex_lock(malloc_mutex_t *mutex);
+bool	malloc_mutex_trylock(malloc_mutex_t *mutex);
 void	malloc_mutex_unlock(malloc_mutex_t *mutex);
 #endif

@@ -69,27 +46,37 @@ malloc_mutex_lock(malloc_mutex_t *mutex)
 {

 	if (isthreaded) {
-#ifdef _WIN32
-		EnterCriticalSection(&mutex->lock);
-#elif (defined(JEMALLOC_OSSPIN))
-		OSSpinLockLock(&mutex->lock);
+#ifdef JEMALLOC_OSSPIN
+		OSSpinLockLock(mutex);
 #else
-		pthread_mutex_lock(&mutex->lock);
+		pthread_mutex_lock(mutex);
 #endif
 	}
 }

+JEMALLOC_INLINE bool
+malloc_mutex_trylock(malloc_mutex_t *mutex)
+{
+
+	if (isthreaded) {
+#ifdef JEMALLOC_OSSPIN
+		return (OSSpinLockTry(mutex) == false);
+#else
+		return (pthread_mutex_trylock(mutex) != 0);
+#endif
+	} else
+		return (false);
+}
+
 JEMALLOC_INLINE void
 malloc_mutex_unlock(malloc_mutex_t *mutex)
 {

 	if (isthreaded) {
-#ifdef _WIN32
-		LeaveCriticalSection(&mutex->lock);
-#elif (defined(JEMALLOC_OSSPIN))
-		OSSpinLockUnlock(&mutex->lock);
+#ifdef JEMALLOC_OSSPIN
+		OSSpinLockUnlock(mutex);
 #else
-		pthread_mutex_unlock(&mutex->lock);
+		pthread_mutex_unlock(mutex);
 #endif
 	}
 }
@@ -1,367 +0,0 @@
-#define	a0calloc JEMALLOC_N(a0calloc)
-#define	a0free JEMALLOC_N(a0free)
-#define	a0malloc JEMALLOC_N(a0malloc)
-#define	arena_alloc_junk_small JEMALLOC_N(arena_alloc_junk_small)
-#define	arena_bin_index JEMALLOC_N(arena_bin_index)
-#define	arena_bin_info JEMALLOC_N(arena_bin_info)
-#define	arena_boot JEMALLOC_N(arena_boot)
-#define	arena_dalloc JEMALLOC_N(arena_dalloc)
-#define	arena_dalloc_bin JEMALLOC_N(arena_dalloc_bin)
-#define	arena_dalloc_bin_locked JEMALLOC_N(arena_dalloc_bin_locked)
-#define	arena_dalloc_junk_small JEMALLOC_N(arena_dalloc_junk_small)
-#define	arena_dalloc_large JEMALLOC_N(arena_dalloc_large)
-#define	arena_dalloc_large_locked JEMALLOC_N(arena_dalloc_large_locked)
-#define	arena_dalloc_small JEMALLOC_N(arena_dalloc_small)
-#define	arena_dss_prec_get JEMALLOC_N(arena_dss_prec_get)
-#define	arena_dss_prec_set JEMALLOC_N(arena_dss_prec_set)
-#define	arena_malloc JEMALLOC_N(arena_malloc)
-#define	arena_malloc_large JEMALLOC_N(arena_malloc_large)
-#define	arena_malloc_small JEMALLOC_N(arena_malloc_small)
-#define	arena_mapbits_allocated_get JEMALLOC_N(arena_mapbits_allocated_get)
-#define	arena_mapbits_binind_get JEMALLOC_N(arena_mapbits_binind_get)
-#define	arena_mapbits_dirty_get JEMALLOC_N(arena_mapbits_dirty_get)
-#define	arena_mapbits_get JEMALLOC_N(arena_mapbits_get)
-#define	arena_mapbits_large_binind_set JEMALLOC_N(arena_mapbits_large_binind_set)
-#define	arena_mapbits_large_get JEMALLOC_N(arena_mapbits_large_get)
-#define	arena_mapbits_large_set JEMALLOC_N(arena_mapbits_large_set)
-#define	arena_mapbits_large_size_get JEMALLOC_N(arena_mapbits_large_size_get)
-#define	arena_mapbits_small_runind_get JEMALLOC_N(arena_mapbits_small_runind_get)
-#define	arena_mapbits_small_set JEMALLOC_N(arena_mapbits_small_set)
-#define	arena_mapbits_unallocated_set JEMALLOC_N(arena_mapbits_unallocated_set)
-#define	arena_mapbits_unallocated_size_get JEMALLOC_N(arena_mapbits_unallocated_size_get)
-#define	arena_mapbits_unallocated_size_set JEMALLOC_N(arena_mapbits_unallocated_size_set)
-#define	arena_mapbits_unzeroed_get JEMALLOC_N(arena_mapbits_unzeroed_get)
-#define	arena_mapbits_unzeroed_set JEMALLOC_N(arena_mapbits_unzeroed_set)
-#define	arena_mapbitsp_get JEMALLOC_N(arena_mapbitsp_get)
-#define	arena_mapp_get JEMALLOC_N(arena_mapp_get)
-#define	arena_maxclass JEMALLOC_N(arena_maxclass)
-#define	arena_new JEMALLOC_N(arena_new)
-#define	arena_palloc JEMALLOC_N(arena_palloc)
-#define	arena_postfork_child JEMALLOC_N(arena_postfork_child)
-#define	arena_postfork_parent JEMALLOC_N(arena_postfork_parent)
-#define	arena_prefork JEMALLOC_N(arena_prefork)
-#define	arena_prof_accum JEMALLOC_N(arena_prof_accum)
-#define	arena_prof_ctx_get JEMALLOC_N(arena_prof_ctx_get)
-#define	arena_prof_ctx_set JEMALLOC_N(arena_prof_ctx_set)
-#define	arena_prof_promoted JEMALLOC_N(arena_prof_promoted)
-#define	arena_ptr_small_binind_get JEMALLOC_N(arena_ptr_small_binind_get)
-#define	arena_purge_all JEMALLOC_N(arena_purge_all)
-#define	arena_ralloc JEMALLOC_N(arena_ralloc)
-#define	arena_ralloc_no_move JEMALLOC_N(arena_ralloc_no_move)
-#define	arena_run_regind JEMALLOC_N(arena_run_regind)
-#define	arena_salloc JEMALLOC_N(arena_salloc)
-#define	arena_stats_merge JEMALLOC_N(arena_stats_merge)
-#define	arena_tcache_fill_small JEMALLOC_N(arena_tcache_fill_small)
-#define	arenas JEMALLOC_N(arenas)
-#define	arenas_booted JEMALLOC_N(arenas_booted)
-#define	arenas_cleanup JEMALLOC_N(arenas_cleanup)
-#define	arenas_extend JEMALLOC_N(arenas_extend)
-#define	arenas_initialized JEMALLOC_N(arenas_initialized)
-#define	arenas_lock JEMALLOC_N(arenas_lock)
-#define	arenas_tls JEMALLOC_N(arenas_tls)
-#define	arenas_tsd JEMALLOC_N(arenas_tsd)
-#define	arenas_tsd_boot JEMALLOC_N(arenas_tsd_boot)
-#define	arenas_tsd_cleanup_wrapper JEMALLOC_N(arenas_tsd_cleanup_wrapper)
-#define	arenas_tsd_get JEMALLOC_N(arenas_tsd_get)
-#define	arenas_tsd_set JEMALLOC_N(arenas_tsd_set)
-#define	atomic_add_u JEMALLOC_N(atomic_add_u)
-#define	atomic_add_uint32 JEMALLOC_N(atomic_add_uint32)
-#define	atomic_add_uint64 JEMALLOC_N(atomic_add_uint64)
-#define	atomic_add_z JEMALLOC_N(atomic_add_z)
-#define	atomic_sub_u JEMALLOC_N(atomic_sub_u)
-#define	atomic_sub_uint32 JEMALLOC_N(atomic_sub_uint32)
-#define	atomic_sub_uint64 JEMALLOC_N(atomic_sub_uint64)
-#define	atomic_sub_z JEMALLOC_N(atomic_sub_z)
-#define	base_alloc JEMALLOC_N(base_alloc)
-#define	base_boot JEMALLOC_N(base_boot)
-#define	base_calloc JEMALLOC_N(base_calloc)
-#define	base_node_alloc JEMALLOC_N(base_node_alloc)
-#define	base_node_dealloc JEMALLOC_N(base_node_dealloc)
-#define	base_postfork_child JEMALLOC_N(base_postfork_child)
-#define	base_postfork_parent JEMALLOC_N(base_postfork_parent)
-#define	base_prefork JEMALLOC_N(base_prefork)
-#define	bitmap_full JEMALLOC_N(bitmap_full)
-#define	bitmap_get JEMALLOC_N(bitmap_get)
-#define	bitmap_info_init JEMALLOC_N(bitmap_info_init)
-#define	bitmap_info_ngroups JEMALLOC_N(bitmap_info_ngroups)
-#define	bitmap_init JEMALLOC_N(bitmap_init)
-#define	bitmap_set JEMALLOC_N(bitmap_set)
-#define	bitmap_sfu JEMALLOC_N(bitmap_sfu)
-#define	bitmap_size JEMALLOC_N(bitmap_size)
-#define	bitmap_unset JEMALLOC_N(bitmap_unset)
-#define	bt_init JEMALLOC_N(bt_init)
-#define	buferror JEMALLOC_N(buferror)
-#define	choose_arena JEMALLOC_N(choose_arena)
-#define	choose_arena_hard JEMALLOC_N(choose_arena_hard)
-#define	chunk_alloc JEMALLOC_N(chunk_alloc)
-#define	chunk_alloc_dss JEMALLOC_N(chunk_alloc_dss)
-#define	chunk_alloc_mmap JEMALLOC_N(chunk_alloc_mmap)
-#define	chunk_boot JEMALLOC_N(chunk_boot)
-#define	chunk_dealloc JEMALLOC_N(chunk_dealloc)
-#define	chunk_dealloc_mmap JEMALLOC_N(chunk_dealloc_mmap)
-#define	chunk_dss_boot JEMALLOC_N(chunk_dss_boot)
-#define	chunk_dss_postfork_child JEMALLOC_N(chunk_dss_postfork_child)
-#define	chunk_dss_postfork_parent JEMALLOC_N(chunk_dss_postfork_parent)
-#define	chunk_dss_prec_get JEMALLOC_N(chunk_dss_prec_get)
-#define	chunk_dss_prec_set JEMALLOC_N(chunk_dss_prec_set)
-#define	chunk_dss_prefork JEMALLOC_N(chunk_dss_prefork)
-#define	chunk_in_dss JEMALLOC_N(chunk_in_dss)
-#define	chunk_npages JEMALLOC_N(chunk_npages)
-#define	chunk_postfork_child JEMALLOC_N(chunk_postfork_child)
-#define	chunk_postfork_parent JEMALLOC_N(chunk_postfork_parent)
-#define	chunk_prefork JEMALLOC_N(chunk_prefork)
-#define	chunk_unmap JEMALLOC_N(chunk_unmap)
-#define	chunks_mtx JEMALLOC_N(chunks_mtx)
-#define	chunks_rtree JEMALLOC_N(chunks_rtree)
-#define	chunksize JEMALLOC_N(chunksize)
-#define	chunksize_mask JEMALLOC_N(chunksize_mask)
-#define	ckh_bucket_search JEMALLOC_N(ckh_bucket_search)
-#define	ckh_count JEMALLOC_N(ckh_count)
-#define	ckh_delete JEMALLOC_N(ckh_delete)
-#define	ckh_evict_reloc_insert JEMALLOC_N(ckh_evict_reloc_insert)
-#define	ckh_insert JEMALLOC_N(ckh_insert)
-#define	ckh_isearch JEMALLOC_N(ckh_isearch)
-#define	ckh_iter JEMALLOC_N(ckh_iter)
-#define	ckh_new JEMALLOC_N(ckh_new)
-#define	ckh_pointer_hash JEMALLOC_N(ckh_pointer_hash)
-#define	ckh_pointer_keycomp JEMALLOC_N(ckh_pointer_keycomp)
-#define	ckh_rebuild JEMALLOC_N(ckh_rebuild)
-#define	ckh_remove JEMALLOC_N(ckh_remove)
-#define	ckh_search JEMALLOC_N(ckh_search)
-#define	ckh_string_hash JEMALLOC_N(ckh_string_hash)
-#define	ckh_string_keycomp JEMALLOC_N(ckh_string_keycomp)
-#define	ckh_try_bucket_insert JEMALLOC_N(ckh_try_bucket_insert)
-#define	ckh_try_insert JEMALLOC_N(ckh_try_insert)
-#define	ctl_boot JEMALLOC_N(ctl_boot)
-#define	ctl_bymib JEMALLOC_N(ctl_bymib)
-#define	ctl_byname JEMALLOC_N(ctl_byname)
-#define	ctl_nametomib JEMALLOC_N(ctl_nametomib)
-#define	ctl_postfork_child JEMALLOC_N(ctl_postfork_child)
-#define	ctl_postfork_parent JEMALLOC_N(ctl_postfork_parent)
-#define	ctl_prefork JEMALLOC_N(ctl_prefork)
-#define	dss_prec_names JEMALLOC_N(dss_prec_names)
-#define	extent_tree_ad_first JEMALLOC_N(extent_tree_ad_first)
-#define	extent_tree_ad_insert JEMALLOC_N(extent_tree_ad_insert)
-#define	extent_tree_ad_iter JEMALLOC_N(extent_tree_ad_iter)
-#define	extent_tree_ad_iter_recurse JEMALLOC_N(extent_tree_ad_iter_recurse)
-#define	extent_tree_ad_iter_start JEMALLOC_N(extent_tree_ad_iter_start)
-#define	extent_tree_ad_last JEMALLOC_N(extent_tree_ad_last)
-#define	extent_tree_ad_new JEMALLOC_N(extent_tree_ad_new)
-#define	extent_tree_ad_next JEMALLOC_N(extent_tree_ad_next)
-#define	extent_tree_ad_nsearch JEMALLOC_N(extent_tree_ad_nsearch)
-#define	extent_tree_ad_prev JEMALLOC_N(extent_tree_ad_prev)
-#define	extent_tree_ad_psearch JEMALLOC_N(extent_tree_ad_psearch)
-#define	extent_tree_ad_remove JEMALLOC_N(extent_tree_ad_remove)
-#define	extent_tree_ad_reverse_iter JEMALLOC_N(extent_tree_ad_reverse_iter)
-#define	extent_tree_ad_reverse_iter_recurse JEMALLOC_N(extent_tree_ad_reverse_iter_recurse)
-#define	extent_tree_ad_reverse_iter_start JEMALLOC_N(extent_tree_ad_reverse_iter_start)
-#define	extent_tree_ad_search JEMALLOC_N(extent_tree_ad_search)
-#define	extent_tree_szad_first JEMALLOC_N(extent_tree_szad_first)
-#define	extent_tree_szad_insert JEMALLOC_N(extent_tree_szad_insert)
-#define	extent_tree_szad_iter JEMALLOC_N(extent_tree_szad_iter)
-#define	extent_tree_szad_iter_recurse JEMALLOC_N(extent_tree_szad_iter_recurse)
-#define	extent_tree_szad_iter_start JEMALLOC_N(extent_tree_szad_iter_start)
-#define	extent_tree_szad_last JEMALLOC_N(extent_tree_szad_last)
-#define	extent_tree_szad_new JEMALLOC_N(extent_tree_szad_new)
-#define	extent_tree_szad_next JEMALLOC_N(extent_tree_szad_next)
-#define	extent_tree_szad_nsearch JEMALLOC_N(extent_tree_szad_nsearch)
-#define	extent_tree_szad_prev JEMALLOC_N(extent_tree_szad_prev)
-#define	extent_tree_szad_psearch JEMALLOC_N(extent_tree_szad_psearch)
-#define	extent_tree_szad_remove JEMALLOC_N(extent_tree_szad_remove)
-#define	extent_tree_szad_reverse_iter JEMALLOC_N(extent_tree_szad_reverse_iter)
-#define	extent_tree_szad_reverse_iter_recurse JEMALLOC_N(extent_tree_szad_reverse_iter_recurse)
-#define	extent_tree_szad_reverse_iter_start JEMALLOC_N(extent_tree_szad_reverse_iter_start)
-#define	extent_tree_szad_search JEMALLOC_N(extent_tree_szad_search)
-#define	get_errno JEMALLOC_N(get_errno)
-#define	hash JEMALLOC_N(hash)
-#define	huge_allocated JEMALLOC_N(huge_allocated)
-#define	huge_boot JEMALLOC_N(huge_boot)
-#define	huge_dalloc JEMALLOC_N(huge_dalloc)
-#define	huge_malloc JEMALLOC_N(huge_malloc)
-#define	huge_mtx JEMALLOC_N(huge_mtx)
-#define	huge_ndalloc JEMALLOC_N(huge_ndalloc)
-#define	huge_nmalloc JEMALLOC_N(huge_nmalloc)
-#define	huge_palloc JEMALLOC_N(huge_palloc)
-#define	huge_postfork_child JEMALLOC_N(huge_postfork_child)
-#define	huge_postfork_parent JEMALLOC_N(huge_postfork_parent)
-#define	huge_prefork JEMALLOC_N(huge_prefork)
-#define	huge_prof_ctx_get JEMALLOC_N(huge_prof_ctx_get)
-#define	huge_prof_ctx_set JEMALLOC_N(huge_prof_ctx_set)
-#define	huge_ralloc JEMALLOC_N(huge_ralloc)
-#define	huge_ralloc_no_move JEMALLOC_N(huge_ralloc_no_move)
-#define	huge_salloc JEMALLOC_N(huge_salloc)
-#define	iallocm JEMALLOC_N(iallocm)
-#define	icalloc JEMALLOC_N(icalloc)
-#define	icallocx JEMALLOC_N(icallocx)
-#define	idalloc JEMALLOC_N(idalloc)
-#define	idallocx JEMALLOC_N(idallocx)
-#define	imalloc JEMALLOC_N(imalloc)
-#define	imallocx JEMALLOC_N(imallocx)
-#define	ipalloc JEMALLOC_N(ipalloc)
-#define	ipallocx JEMALLOC_N(ipallocx)
-#define	iqalloc JEMALLOC_N(iqalloc)
-#define	iqallocx JEMALLOC_N(iqallocx)
-#define	iralloc JEMALLOC_N(iralloc)
-#define	irallocx JEMALLOC_N(irallocx)
-#define	isalloc JEMALLOC_N(isalloc)
-#define	isthreaded JEMALLOC_N(isthreaded)
-#define	ivsalloc JEMALLOC_N(ivsalloc)
-#define	jemalloc_postfork_child JEMALLOC_N(jemalloc_postfork_child)
-#define	jemalloc_postfork_parent JEMALLOC_N(jemalloc_postfork_parent)
-#define	jemalloc_prefork JEMALLOC_N(jemalloc_prefork)
-#define	malloc_cprintf JEMALLOC_N(malloc_cprintf)
-#define	malloc_mutex_init JEMALLOC_N(malloc_mutex_init)
-#define	malloc_mutex_lock JEMALLOC_N(malloc_mutex_lock)
-#define	malloc_mutex_postfork_child JEMALLOC_N(malloc_mutex_postfork_child)
-#define	malloc_mutex_postfork_parent JEMALLOC_N(malloc_mutex_postfork_parent)
-#define	malloc_mutex_prefork JEMALLOC_N(malloc_mutex_prefork)
-#define	malloc_mutex_unlock JEMALLOC_N(malloc_mutex_unlock)
-#define	malloc_printf JEMALLOC_N(malloc_printf)
-#define	malloc_snprintf JEMALLOC_N(malloc_snprintf)
-#define	malloc_strtoumax JEMALLOC_N(malloc_strtoumax)
-#define	malloc_tsd_boot JEMALLOC_N(malloc_tsd_boot)
-#define	malloc_tsd_cleanup_register JEMALLOC_N(malloc_tsd_cleanup_register)
-#define	malloc_tsd_dalloc JEMALLOC_N(malloc_tsd_dalloc)
-#define	malloc_tsd_malloc JEMALLOC_N(malloc_tsd_malloc)
-#define	malloc_tsd_no_cleanup JEMALLOC_N(malloc_tsd_no_cleanup)
-#define	malloc_vcprintf JEMALLOC_N(malloc_vcprintf)
-#define	malloc_vsnprintf JEMALLOC_N(malloc_vsnprintf)
-#define	malloc_write JEMALLOC_N(malloc_write)
-#define	map_bias JEMALLOC_N(map_bias)
-#define	mb_write JEMALLOC_N(mb_write)
-#define	mutex_boot JEMALLOC_N(mutex_boot)
-#define	narenas_auto JEMALLOC_N(narenas_auto)
-#define	narenas_total JEMALLOC_N(narenas_total)
-#define	narenas_total_get JEMALLOC_N(narenas_total_get)
-#define	ncpus JEMALLOC_N(ncpus)
-#define	nhbins JEMALLOC_N(nhbins)
-#define	opt_abort JEMALLOC_N(opt_abort)
-#define	opt_junk JEMALLOC_N(opt_junk)
-#define	opt_lg_chunk JEMALLOC_N(opt_lg_chunk)
-#define	opt_lg_dirty_mult JEMALLOC_N(opt_lg_dirty_mult)
-#define	opt_lg_prof_interval JEMALLOC_N(opt_lg_prof_interval)
-#define	opt_lg_prof_sample JEMALLOC_N(opt_lg_prof_sample)
-#define	opt_lg_tcache_max JEMALLOC_N(opt_lg_tcache_max)
-#define	opt_narenas JEMALLOC_N(opt_narenas)
-#define	opt_prof JEMALLOC_N(opt_prof)
-#define	opt_prof_accum JEMALLOC_N(opt_prof_accum)
-#define	opt_prof_active JEMALLOC_N(opt_prof_active)
-#define	opt_prof_final JEMALLOC_N(opt_prof_final)
-#define	opt_prof_gdump JEMALLOC_N(opt_prof_gdump)
-#define	opt_prof_leak JEMALLOC_N(opt_prof_leak)
-#define	opt_prof_prefix JEMALLOC_N(opt_prof_prefix)
-#define	opt_quarantine JEMALLOC_N(opt_quarantine)
-#define	opt_redzone JEMALLOC_N(opt_redzone)
-#define	opt_stats_print JEMALLOC_N(opt_stats_print)
-#define	opt_tcache JEMALLOC_N(opt_tcache)
-#define	opt_utrace JEMALLOC_N(opt_utrace)
-#define	opt_valgrind JEMALLOC_N(opt_valgrind)
-#define	opt_xmalloc JEMALLOC_N(opt_xmalloc)
-#define	opt_zero JEMALLOC_N(opt_zero)
-#define	p2rz JEMALLOC_N(p2rz)
-#define	pages_purge JEMALLOC_N(pages_purge)
-#define	pow2_ceil JEMALLOC_N(pow2_ceil)
-#define	prof_backtrace JEMALLOC_N(prof_backtrace)
-#define	prof_boot0 JEMALLOC_N(prof_boot0)
-#define	prof_boot1 JEMALLOC_N(prof_boot1)
-#define	prof_boot2 JEMALLOC_N(prof_boot2)
-#define	prof_ctx_get JEMALLOC_N(prof_ctx_get)
-#define	prof_ctx_set JEMALLOC_N(prof_ctx_set)
-#define	prof_free JEMALLOC_N(prof_free)
-#define	prof_gdump JEMALLOC_N(prof_gdump)
-#define	prof_idump JEMALLOC_N(prof_idump)
-#define	prof_interval JEMALLOC_N(prof_interval)
-#define	prof_lookup JEMALLOC_N(prof_lookup)
-#define	prof_malloc JEMALLOC_N(prof_malloc)
-#define	prof_mdump JEMALLOC_N(prof_mdump)
-#define	prof_postfork_child JEMALLOC_N(prof_postfork_child)
-#define	prof_postfork_parent JEMALLOC_N(prof_postfork_parent)
-#define	prof_prefork JEMALLOC_N(prof_prefork)
-#define	prof_promote JEMALLOC_N(prof_promote)
-#define	prof_realloc JEMALLOC_N(prof_realloc)
-#define	prof_sample_accum_update JEMALLOC_N(prof_sample_accum_update)
-#define	prof_sample_threshold_update JEMALLOC_N(prof_sample_threshold_update)
-#define	prof_tdata_booted JEMALLOC_N(prof_tdata_booted)
-#define	prof_tdata_cleanup JEMALLOC_N(prof_tdata_cleanup)
-#define	prof_tdata_get JEMALLOC_N(prof_tdata_get)
-#define	prof_tdata_init JEMALLOC_N(prof_tdata_init)
-#define	prof_tdata_initialized JEMALLOC_N(prof_tdata_initialized)
-#define	prof_tdata_tls JEMALLOC_N(prof_tdata_tls)
-#define	prof_tdata_tsd JEMALLOC_N(prof_tdata_tsd)
-#define	prof_tdata_tsd_boot JEMALLOC_N(prof_tdata_tsd_boot)
-#define	prof_tdata_tsd_cleanup_wrapper JEMALLOC_N(prof_tdata_tsd_cleanup_wrapper)
-#define	prof_tdata_tsd_get JEMALLOC_N(prof_tdata_tsd_get)
-#define	prof_tdata_tsd_set JEMALLOC_N(prof_tdata_tsd_set)
-#define	quarantine JEMALLOC_N(quarantine)
-#define	quarantine_boot JEMALLOC_N(quarantine_boot)
-#define	quarantine_tsd_boot JEMALLOC_N(quarantine_tsd_boot)
-#define	quarantine_tsd_cleanup_wrapper JEMALLOC_N(quarantine_tsd_cleanup_wrapper)
-#define	quarantine_tsd_get JEMALLOC_N(quarantine_tsd_get)
-#define	quarantine_tsd_set JEMALLOC_N(quarantine_tsd_set)
-#define	register_zone JEMALLOC_N(register_zone)
-#define	rtree_get JEMALLOC_N(rtree_get)
-#define	rtree_get_locked JEMALLOC_N(rtree_get_locked)
-#define	rtree_new JEMALLOC_N(rtree_new)
-#define	rtree_postfork_child JEMALLOC_N(rtree_postfork_child)
-#define	rtree_postfork_parent JEMALLOC_N(rtree_postfork_parent)
-#define	rtree_prefork JEMALLOC_N(rtree_prefork)
-#define	rtree_set JEMALLOC_N(rtree_set)
-#define	s2u JEMALLOC_N(s2u)
-#define	sa2u JEMALLOC_N(sa2u)
-#define	set_errno JEMALLOC_N(set_errno)
-#define	stats_cactive JEMALLOC_N(stats_cactive)
-#define	stats_cactive_add JEMALLOC_N(stats_cactive_add)
-#define	stats_cactive_get JEMALLOC_N(stats_cactive_get)
-#define	stats_cactive_sub JEMALLOC_N(stats_cactive_sub)
-#define	stats_chunks JEMALLOC_N(stats_chunks)
-#define	stats_print JEMALLOC_N(stats_print)
-#define	tcache_alloc_easy JEMALLOC_N(tcache_alloc_easy)
-#define	tcache_alloc_large JEMALLOC_N(tcache_alloc_large)
-#define	tcache_alloc_small JEMALLOC_N(tcache_alloc_small)
-#define	tcache_alloc_small_hard JEMALLOC_N(tcache_alloc_small_hard)
-#define	tcache_arena_associate JEMALLOC_N(tcache_arena_associate)
-#define	tcache_arena_dissociate JEMALLOC_N(tcache_arena_dissociate)
-#define	tcache_bin_flush_large JEMALLOC_N(tcache_bin_flush_large)
-#define	tcache_bin_flush_small JEMALLOC_N(tcache_bin_flush_small)
-#define	tcache_bin_info JEMALLOC_N(tcache_bin_info)
-#define	tcache_boot0 JEMALLOC_N(tcache_boot0)
-#define	tcache_boot1 JEMALLOC_N(tcache_boot1)
-#define	tcache_booted JEMALLOC_N(tcache_booted)
-#define	tcache_create JEMALLOC_N(tcache_create)
-#define	tcache_dalloc_large JEMALLOC_N(tcache_dalloc_large)
-#define	tcache_dalloc_small JEMALLOC_N(tcache_dalloc_small)
-#define	tcache_destroy JEMALLOC_N(tcache_destroy)
-#define	tcache_enabled_booted JEMALLOC_N(tcache_enabled_booted)
-#define	tcache_enabled_get JEMALLOC_N(tcache_enabled_get)
-#define	tcache_enabled_initialized JEMALLOC_N(tcache_enabled_initialized)
-#define	tcache_enabled_set JEMALLOC_N(tcache_enabled_set)
-#define	tcache_enabled_tls JEMALLOC_N(tcache_enabled_tls)
-#define	tcache_enabled_tsd JEMALLOC_N(tcache_enabled_tsd)
-#define	tcache_enabled_tsd_boot JEMALLOC_N(tcache_enabled_tsd_boot)
-#define	tcache_enabled_tsd_cleanup_wrapper JEMALLOC_N(tcache_enabled_tsd_cleanup_wrapper)
-#define	tcache_enabled_tsd_get JEMALLOC_N(tcache_enabled_tsd_get)
-#define	tcache_enabled_tsd_set JEMALLOC_N(tcache_enabled_tsd_set)
-#define	tcache_event JEMALLOC_N(tcache_event)
-#define	tcache_event_hard JEMALLOC_N(tcache_event_hard)
-#define	tcache_flush JEMALLOC_N(tcache_flush)
-#define	tcache_get JEMALLOC_N(tcache_get)
-#define	tcache_initialized JEMALLOC_N(tcache_initialized)
-#define	tcache_maxclass JEMALLOC_N(tcache_maxclass)
-#define	tcache_salloc JEMALLOC_N(tcache_salloc)
-#define	tcache_stats_merge JEMALLOC_N(tcache_stats_merge)
-#define	tcache_thread_cleanup JEMALLOC_N(tcache_thread_cleanup)
-#define	tcache_tls JEMALLOC_N(tcache_tls)
-#define	tcache_tsd JEMALLOC_N(tcache_tsd)
-#define	tcache_tsd_boot JEMALLOC_N(tcache_tsd_boot)
-#define	tcache_tsd_cleanup_wrapper JEMALLOC_N(tcache_tsd_cleanup_wrapper)
-#define	tcache_tsd_get JEMALLOC_N(tcache_tsd_get)
-#define	tcache_tsd_set JEMALLOC_N(tcache_tsd_set)
-#define	thread_allocated_booted JEMALLOC_N(thread_allocated_booted)
-#define	thread_allocated_initialized JEMALLOC_N(thread_allocated_initialized)
-#define	thread_allocated_tls JEMALLOC_N(thread_allocated_tls)
-#define	thread_allocated_tsd JEMALLOC_N(thread_allocated_tsd)
-#define	thread_allocated_tsd_boot JEMALLOC_N(thread_allocated_tsd_boot)
-#define	thread_allocated_tsd_cleanup_wrapper JEMALLOC_N(thread_allocated_tsd_cleanup_wrapper)
-#define	thread_allocated_tsd_get JEMALLOC_N(thread_allocated_tsd_get)
-#define	thread_allocated_tsd_set JEMALLOC_N(thread_allocated_tsd_set)
-#define	u2rz JEMALLOC_N(u2rz)
@@ -4,7 +4,7 @@
 /*
 * Simple linear congruential pseudo-random number generator:
 *
- *   prng(y) = (a*x + c) % m
+ *   prn(y) = (a*x + c) % m
 *
 * where the following constants ensure maximal period:
 *
@@ -25,7 +25,7 @@
 *   uint32_t state      : Seed value.
 *   const uint32_t a, c : See above discussion.
 */
-#define prng32(r, lg_range, state, a, c) do {				\
+#define prn32(r, lg_range, state, a, c) do {				\
 	assert(lg_range > 0);						\
 	assert(lg_range <= 32);						\
 									\
@@ -34,8 +34,8 @@
 	r >>= (32 - lg_range);						\
 } while (false)

-/* Same as prng32(), but 64 bits of pseudo-randomness, using uint64_t. */
-#define prng64(r, lg_range, state, a, c) do {				\
+/* Same as prn32(), but 64 bits of pseudo-randomness, using uint64_t. */
+#define prn64(r, lg_range, state, a, c) do {				\
 	assert(lg_range > 0);						\
 	assert(lg_range <= 64);						\
 									\
@@ -1,3 +1,4 @@
+#ifdef JEMALLOC_PROF
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

@@ -9,41 +10,28 @@ typedef struct prof_tdata_s prof_tdata_t;

 /* Option defaults. */
 #define	PROF_PREFIX_DEFAULT		"jeprof"
-#define	LG_PROF_SAMPLE_DEFAULT		19
+#define	LG_PROF_BT_MAX_DEFAULT		7
+#define	LG_PROF_SAMPLE_DEFAULT		0
 #define	LG_PROF_INTERVAL_DEFAULT	-1
+#define	LG_PROF_TCMAX_DEFAULT		-1

 /*
- * Hard limit on stack backtrace depth.  The version of prof_backtrace() that
- * is based on __builtin_return_address() necessarily has a hard-coded number
- * of backtrace frame handlers, and should be kept in sync with this setting.
+ * Hard limit on stack backtrace depth.  Note that the version of
+ * prof_backtrace() that is based on __builtin_return_address() necessarily has
+ * a hard-coded number of backtrace frame handlers.
 */
-#define	PROF_BT_MAX			128
-
-/* Maximum number of backtraces to store in each per thread LRU cache. */
-#define	PROF_TCMAX			1024
+#if (defined(JEMALLOC_PROF_LIBGCC) || defined(JEMALLOC_PROF_LIBUNWIND))
+#  define LG_PROF_BT_MAX	((ZU(1) << (LG_SIZEOF_PTR+3)) - 1)
+#else
+#  define LG_PROF_BT_MAX	7 /* >= LG_PROF_BT_MAX_DEFAULT */
+#endif
+#define	PROF_BT_MAX		(1U << LG_PROF_BT_MAX)

 /* Initial hash table size. */
-#define	PROF_CKH_MINITEMS		64
+#define	PROF_CKH_MINITEMS	64

 /* Size of memory buffer to use when writing dump files. */
-#define	PROF_DUMP_BUFSIZE		65536
-
-/* Size of stack-allocated buffer used by prof_printf(). */
-#define	PROF_PRINTF_BUFSIZE		128
-
-/*
- * Number of mutexes shared among all ctx's.  No space is allocated for these
- * unless profiling is enabled, so it's okay to over-provision.
- */
-#define	PROF_NCTX_LOCKS			1024
-
-/*
- * prof_tdata pointers close to NULL are used to encode state information that
- * is used for cleaning up during thread shutdown.
- */
-#define	PROF_TDATA_STATE_REINCARNATED	((prof_tdata_t *)(uintptr_t)1)
-#define	PROF_TDATA_STATE_PURGATORY	((prof_tdata_t *)(uintptr_t)2)
-#define	PROF_TDATA_STATE_MAX		PROF_TDATA_STATE_PURGATORY
+#define	PROF_DUMP_BUF_SIZE	65536

 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
@@ -121,18 +109,8 @@ struct prof_ctx_s {
 	/* Associated backtrace. */
 	prof_bt_t		*bt;

-	/* Protects nlimbo, cnt_merged, and cnts_ql. */
-	malloc_mutex_t		*lock;
-
-	/*
-	 * Number of threads that currently cause this ctx to be in a state of
-	 * limbo due to one of:
-	 *   - Initializing per thread counters associated with this ctx.
-	 *   - Preparing to destroy this ctx.
-	 * nlimbo must be 1 (single destroyer) in order to safely destroy the
-	 * ctx.
-	 */
-	unsigned		nlimbo;
+	/* Protects cnt_merged and cnts_ql. */
+	malloc_mutex_t		lock;

 	/* Temporary storage for summation during dump. */
 	prof_cnt_t		cnt_summed;
@@ -167,14 +145,9 @@ struct prof_tdata_s {
 	void			**vec;

 	/* Sampling state. */
-	uint64_t		prng_state;
+	uint64_t		prn_state;
 	uint64_t		threshold;
 	uint64_t		accum;
-
-	/* State used to avoid dumping while operating on prof internals. */
-	bool			enq;
-	bool			enq_idump;
-	bool			enq_gdump;
 };

 #endif /* JEMALLOC_H_STRUCTS */
@@ -189,12 +162,13 @@ extern bool	opt_prof;
 * to notice state changes.
 */
 extern bool	opt_prof_active;
+extern size_t	opt_lg_prof_bt_max;   /* Maximum backtrace depth. */
 extern size_t	opt_lg_prof_sample;   /* Mean bytes between samples. */
 extern ssize_t	opt_lg_prof_interval; /* lg(prof_interval). */
 extern bool	opt_prof_gdump;       /* High-water memory dumping. */
-extern bool	opt_prof_final;       /* Final profile dumping. */
 extern bool	opt_prof_leak;        /* Dump leak summary at exit. */
 extern bool	opt_prof_accum;       /* Report cumulative bytes. */
+extern ssize_t	opt_lg_prof_tcmax;    /* lg(max per thread bactrace cache) */
 extern char	opt_prof_prefix[PATH_MAX + 1];

 /*
@@ -212,82 +186,50 @@ extern uint64_t	prof_interval;
 */
 extern bool	prof_promote;

+/* (1U << opt_lg_prof_bt_max). */
+extern unsigned	prof_bt_max;
+
+/* Thread-specific backtrace cache, used to reduce bt2ctx contention. */
+#ifndef NO_TLS
+extern __thread prof_tdata_t	*prof_tdata_tls
+    JEMALLOC_ATTR(tls_model("initial-exec"));
+#  define PROF_TCACHE_GET()	prof_tdata_tls
+#  define PROF_TCACHE_SET(v)	do {					\
+	prof_tdata_tls = (v);						\
+	pthread_setspecific(prof_tdata_tsd, (void *)(v));		\
+} while (0)
+#else
+#  define PROF_TCACHE_GET()						\
+	((prof_tdata_t *)pthread_getspecific(prof_tdata_tsd))
+#  define PROF_TCACHE_SET(v)	do {					\
+	pthread_setspecific(prof_tdata_tsd, (void *)(v));		\
+} while (0)
+#endif
+/*
+ * Same contents as b2cnt_tls, but initialized such that the TSD destructor is
+ * called when a thread exits, so that prof_tdata_tls contents can be merged,
+ * unlinked, and deallocated.
+ */
+extern pthread_key_t	prof_tdata_tsd;
+
 void	bt_init(prof_bt_t *bt, void **vec);
-void	prof_backtrace(prof_bt_t *bt, unsigned nignore);
+void	prof_backtrace(prof_bt_t *bt, unsigned nignore, unsigned max);
 prof_thr_cnt_t	*prof_lookup(prof_bt_t *bt);
 void	prof_idump(void);
 bool	prof_mdump(const char *filename);
 void	prof_gdump(void);
 prof_tdata_t	*prof_tdata_init(void);
-void	prof_tdata_cleanup(void *arg);
 void	prof_boot0(void);
 void	prof_boot1(void);
 bool	prof_boot2(void);
-void	prof_prefork(void);
-void	prof_postfork_parent(void);
-void	prof_postfork_child(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
 #ifdef JEMALLOC_H_INLINES

-#define	PROF_ALLOC_PREP(nignore, size, ret) do {			\
-	prof_tdata_t *prof_tdata;					\
-	prof_bt_t bt;							\
-									\
-	assert(size == s2u(size));					\
-									\
-	prof_tdata = prof_tdata_get();					\
-	if ((uintptr_t)prof_tdata <= (uintptr_t)PROF_TDATA_STATE_MAX) {	\
-		if (prof_tdata != NULL)					\
-			ret = (prof_thr_cnt_t *)(uintptr_t)1U;		\
-		else							\
-			ret = NULL;					\
-		break;							\
-	}								\
-									\
-	if (opt_prof_active == false) {					\
-		/* Sampling is currently inactive, so avoid sampling. */\
-		ret = (prof_thr_cnt_t *)(uintptr_t)1U;			\
-	} else if (opt_lg_prof_sample == 0) {				\
-		/* Don't bother with sampling logic, since sampling   */\
-		/* interval is 1.                                     */\
-		bt_init(&bt, prof_tdata->vec);				\
-		prof_backtrace(&bt, nignore);				\
-		ret = prof_lookup(&bt);					\
-	} else {							\
-		if (prof_tdata->threshold == 0) {			\
-			/* Initialize.  Seed the prng differently for */\
-			/* each thread.                               */\
-			prof_tdata->prng_state =			\
-			    (uint64_t)(uintptr_t)&size;			\
-			prof_sample_threshold_update(prof_tdata);	\
-		}							\
-									\
-		/* Determine whether to capture a backtrace based on  */\
-		/* whether size is enough for prof_accum to reach     */\
-		/* prof_tdata->threshold.  However, delay updating    */\
-		/* these variables until prof_{m,re}alloc(), because  */\
-		/* we don't know for sure that the allocation will    */\
-		/* succeed.                                           */\
-		/*                                                    */\
-		/* Use subtraction rather than addition to avoid      */\
-		/* potential integer overflow.                        */\
-		if (size >= prof_tdata->threshold -			\
-		    prof_tdata->accum) {				\
-			bt_init(&bt, prof_tdata->vec);			\
-			prof_backtrace(&bt, nignore);			\
-			ret = prof_lookup(&bt);				\
-		} else							\
-			ret = (prof_thr_cnt_t *)(uintptr_t)1U;		\
-	}								\
-} while (0)
-
 #ifndef JEMALLOC_ENABLE_INLINE
-malloc_tsd_protos(JEMALLOC_ATTR(unused), prof_tdata, prof_tdata_t *)
-
-prof_tdata_t	*prof_tdata_get(void);
 void	prof_sample_threshold_update(prof_tdata_t *prof_tdata);
+prof_thr_cnt_t	*prof_alloc_prep(size_t size);
 prof_ctx_t	*prof_ctx_get(const void *ptr);
 void	prof_ctx_set(const void *ptr, prof_ctx_t *ctx);
 bool	prof_sample_accum_update(size_t size);
@@ -298,35 +240,12 @@ void	prof_free(const void *ptr, size_t size);
 #endif

 #if (defined(JEMALLOC_ENABLE_INLINE) || defined(JEMALLOC_PROF_C_))
-/* Thread-specific backtrace cache, used to reduce bt2ctx contention. */
-malloc_tsd_externs(prof_tdata, prof_tdata_t *)
-malloc_tsd_funcs(JEMALLOC_INLINE, prof_tdata, prof_tdata_t *, NULL,
-    prof_tdata_cleanup)
-
-JEMALLOC_INLINE prof_tdata_t *
-prof_tdata_get(void)
-{
-	prof_tdata_t *prof_tdata;
-
-	cassert(config_prof);
-
-	prof_tdata = *prof_tdata_tsd_get();
-	if ((uintptr_t)prof_tdata <= (uintptr_t)PROF_TDATA_STATE_MAX) {
-		if (prof_tdata == NULL)
-			prof_tdata = prof_tdata_init();
-	}
-
-	return (prof_tdata);
-}
-
 JEMALLOC_INLINE void
 prof_sample_threshold_update(prof_tdata_t *prof_tdata)
 {
 	uint64_t r;
 	double u;

-	cassert(config_prof);
-
 	/*
 	 * Compute sample threshold as a geometrically distributed random
 	 * variable with mean (2^opt_lg_prof_sample).
@@ -345,26 +264,92 @@ prof_sample_threshold_update(prof_tdata_t *prof_tdata)
 	 *   pp 500
 	 *   (http://cg.scs.carleton.ca/~luc/rnbookindex.html)
 	 */
-	prng64(r, 53, prof_tdata->prng_state,
-	    UINT64_C(6364136223846793005), UINT64_C(1442695040888963407));
+	prn64(r, 53, prof_tdata->prn_state,
+	    (uint64_t)6364136223846793005LLU, (uint64_t)1442695040888963407LLU);
 	u = (double)r * (1.0/9007199254740992.0L);
 	prof_tdata->threshold = (uint64_t)(log(u) /
 	    log(1.0 - (1.0 / (double)((uint64_t)1U << opt_lg_prof_sample))))
 	    + (uint64_t)1U;
 }

+JEMALLOC_INLINE prof_thr_cnt_t *
+prof_alloc_prep(size_t size)
+{
+#ifdef JEMALLOC_ENABLE_INLINE
+   /* This function does not have its own stack frame, because it is inlined. */
+#  define NIGNORE 1
+#else
+#  define NIGNORE 2
+#endif
+	prof_thr_cnt_t *ret;
+	prof_tdata_t *prof_tdata;
+	prof_bt_t bt;
+
+	assert(size == s2u(size));
+
+	prof_tdata = PROF_TCACHE_GET();
+	if (prof_tdata == NULL) {
+		prof_tdata = prof_tdata_init();
+		if (prof_tdata == NULL)
+			return (NULL);
+	}
+
+	if (opt_prof_active == false) {
+		/* Sampling is currently inactive, so avoid sampling. */
+		ret = (prof_thr_cnt_t *)(uintptr_t)1U;
+	} else if (opt_lg_prof_sample == 0) {
+		/*
+		 * Don't bother with sampling logic, since sampling interval is
+		 * 1.
+		 */
+		bt_init(&bt, prof_tdata->vec);
+		prof_backtrace(&bt, NIGNORE, prof_bt_max);
+		ret = prof_lookup(&bt);
+	} else {
+		if (prof_tdata->threshold == 0) {
+			/*
+			 * Initialize.  Seed the prng differently for each
+			 * thread.
+			 */
+			prof_tdata->prn_state = (uint64_t)(uintptr_t)&size;
+			prof_sample_threshold_update(prof_tdata);
+		}
+
+		/*
+		 * Determine whether to capture a backtrace based on whether
+		 * size is enough for prof_accum to reach
+		 * prof_tdata->threshold.  However, delay updating these
+		 * variables until prof_{m,re}alloc(), because we don't know
+		 * for sure that the allocation will succeed.
+		 *
+		 * Use subtraction rather than addition to avoid potential
+		 * integer overflow.
+		 */
+		if (size >= prof_tdata->threshold - prof_tdata->accum) {
+			bt_init(&bt, prof_tdata->vec);
+			prof_backtrace(&bt, NIGNORE, prof_bt_max);
+			ret = prof_lookup(&bt);
+		} else
+			ret = (prof_thr_cnt_t *)(uintptr_t)1U;
+	}
+
+	return (ret);
+#undef NIGNORE
+}
+
 JEMALLOC_INLINE prof_ctx_t *
 prof_ctx_get(const void *ptr)
 {
 	prof_ctx_t *ret;
 	arena_chunk_t *chunk;

-	cassert(config_prof);
 	assert(ptr != NULL);

 	chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
 	if (chunk != ptr) {
 		/* Region. */
+		dassert(chunk->arena->magic == ARENA_MAGIC);
+
 		ret = arena_prof_ctx_get(ptr);
 	} else
 		ret = huge_prof_ctx_get(ptr);
@@ -377,12 +362,13 @@ prof_ctx_set(const void *ptr, prof_ctx_t *ctx)
 {
 	arena_chunk_t *chunk;

-	cassert(config_prof);
 	assert(ptr != NULL);

 	chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
 	if (chunk != ptr) {
 		/* Region. */
+		dassert(chunk->arena->magic == ARENA_MAGIC);
+
 		arena_prof_ctx_set(ptr, ctx);
 	} else
 		huge_prof_ctx_set(ptr, ctx);
@@ -393,13 +379,11 @@ prof_sample_accum_update(size_t size)
 {
 	prof_tdata_t *prof_tdata;

-	cassert(config_prof);
 	/* Sampling logic is unnecessary if the interval is 1. */
 	assert(opt_lg_prof_sample != 0);

-	prof_tdata = *prof_tdata_tsd_get();
-	if ((uintptr_t)prof_tdata <= (uintptr_t)PROF_TDATA_STATE_MAX)
-		return (true);
+	prof_tdata = PROF_TCACHE_GET();
+	assert(prof_tdata != NULL);

 	/* Take care to avoid integer overflow. */
 	if (size >= prof_tdata->threshold - prof_tdata->accum) {
@@ -421,9 +405,8 @@ JEMALLOC_INLINE void
 prof_malloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt)
 {

-	cassert(config_prof);
 	assert(ptr != NULL);
-	assert(size == isalloc(ptr, true));
+	assert(size == isalloc(ptr));

 	if (opt_lg_prof_sample != 0) {
 		if (prof_sample_accum_update(size)) {
@@ -432,7 +415,7 @@ prof_malloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt)
 			 * always possible to tell in advance how large an
 			 * object's usable size will be, so there should never
 			 * be a difference between the size passed to
-			 * PROF_ALLOC_PREP() and prof_malloc().
+			 * prof_alloc_prep() and prof_malloc().
 			 */
 			assert((uintptr_t)cnt == (uintptr_t)1U);
 		}
@@ -468,16 +451,15 @@ prof_realloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt,
 {
 	prof_thr_cnt_t *told_cnt;

-	cassert(config_prof);
 	assert(ptr != NULL || (uintptr_t)cnt <= (uintptr_t)1U);

 	if (ptr != NULL) {
-		assert(size == isalloc(ptr, true));
+		assert(size == isalloc(ptr));
 		if (opt_lg_prof_sample != 0) {
 			if (prof_sample_accum_update(size)) {
 				/*
 				 * Don't sample.  The size passed to
-				 * PROF_ALLOC_PREP() was larger than what
+				 * prof_alloc_prep() was larger than what
 				 * actually got allocated, so a backtrace was
 				 * captured for this allocation, even though
 				 * its actual size was insufficient to cross
@@ -495,10 +477,10 @@ prof_realloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt,
 			 * It's too late to propagate OOM for this realloc(),
 			 * so operate directly on old_cnt->ctx->cnt_merged.
 			 */
-			malloc_mutex_lock(old_ctx->lock);
+			malloc_mutex_lock(&old_ctx->lock);
 			old_ctx->cnt_merged.curobjs--;
 			old_ctx->cnt_merged.curbytes -= old_size;
-			malloc_mutex_unlock(old_ctx->lock);
+			malloc_mutex_unlock(&old_ctx->lock);
 			told_cnt = (prof_thr_cnt_t *)(uintptr_t)1U;
 		}
 	} else
@@ -509,7 +491,7 @@ prof_realloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt,
 	if ((uintptr_t)cnt > (uintptr_t)1U) {
 		prof_ctx_set(ptr, cnt->ctx);
 		cnt->epoch++;
-	} else if (ptr != NULL)
+	} else
 		prof_ctx_set(ptr, (prof_ctx_t *)(uintptr_t)1U);
 	/*********/
 	mb_write();
@@ -542,12 +524,9 @@ prof_free(const void *ptr, size_t size)
 {
 	prof_ctx_t *ctx = prof_ctx_get(ptr);

-	cassert(config_prof);
-
 	if ((uintptr_t)ctx > (uintptr_t)1) {
-		prof_thr_cnt_t *tcnt;
-		assert(size == isalloc(ptr, true));
-		tcnt = prof_lookup(ctx->bt);
+		assert(size == isalloc(ptr));
+		prof_thr_cnt_t *tcnt = prof_lookup(ctx->bt);

 		if (tcnt != NULL) {
 			tcnt->epoch++;
@@ -568,10 +547,10 @@ prof_free(const void *ptr, size_t size)
 			 * OOM during free() cannot be propagated, so operate
 			 * directly on cnt->ctx->cnt_merged.
 			 */
-			malloc_mutex_lock(ctx->lock);
+			malloc_mutex_lock(&ctx->lock);
 			ctx->cnt_merged.curobjs--;
 			ctx->cnt_merged.curbytes -= size;
-			malloc_mutex_unlock(ctx->lock);
+			malloc_mutex_unlock(&ctx->lock);
 		}
 	}
 }
@@ -579,3 +558,4 @@ prof_free(const void *ptr, size_t size)

 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/
+#endif /* JEMALLOC_PROF */
@@ -223,88 +223,88 @@ a_prefix##reverse_iter(a_rbt_type *rbtree, a_type *start,		\
 * The following API is generated:
 *
 *   static void
- *   ex_new(ex_t *tree);
+ *   ex_new(ex_t *extree);
 *       Description: Initialize a red-black tree structure.
 *       Args:
- *         tree: Pointer to an uninitialized red-black tree object.
+ *         extree: Pointer to an uninitialized red-black tree object.
 *
 *   static ex_node_t *
- *   ex_first(ex_t *tree);
+ *   ex_first(ex_t *extree);
 *   static ex_node_t *
- *   ex_last(ex_t *tree);
- *       Description: Get the first/last node in tree.
+ *   ex_last(ex_t *extree);
+ *       Description: Get the first/last node in extree.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *       Ret: First/last node in tree, or NULL if tree is empty.
+ *         extree: Pointer to an initialized red-black tree object.
+ *       Ret: First/last node in extree, or NULL if extree is empty.
 *
 *   static ex_node_t *
- *   ex_next(ex_t *tree, ex_node_t *node);
+ *   ex_next(ex_t *extree, ex_node_t *node);
 *   static ex_node_t *
- *   ex_prev(ex_t *tree, ex_node_t *node);
+ *   ex_prev(ex_t *extree, ex_node_t *node);
 *       Description: Get node's successor/predecessor.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *         node: A node in tree.
- *       Ret: node's successor/predecessor in tree, or NULL if node is
+ *         extree: Pointer to an initialized red-black tree object.
+ *         node : A node in extree.
+ *       Ret: node's successor/predecessor in extree, or NULL if node is
 *            last/first.
 *
 *   static ex_node_t *
- *   ex_search(ex_t *tree, ex_node_t *key);
+ *   ex_search(ex_t *extree, ex_node_t *key);
 *       Description: Search for node that matches key.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *         key : Search key.
- *       Ret: Node in tree that matches key, or NULL if no match.
+ *         extree: Pointer to an initialized red-black tree object.
+ *         key  : Search key.
+ *       Ret: Node in extree that matches key, or NULL if no match.
 *
 *   static ex_node_t *
- *   ex_nsearch(ex_t *tree, ex_node_t *key);
+ *   ex_nsearch(ex_t *extree, ex_node_t *key);
 *   static ex_node_t *
- *   ex_psearch(ex_t *tree, ex_node_t *key);
+ *   ex_psearch(ex_t *extree, ex_node_t *key);
 *       Description: Search for node that matches key.  If no match is found,
 *                    return what would be key's successor/predecessor, were
- *                    key in tree.
+ *                    key in extree.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *         key : Search key.
- *       Ret: Node in tree that matches key, or if no match, hypothetical node's
- *            successor/predecessor (NULL if no successor/predecessor).
+ *         extree: Pointer to an initialized red-black tree object.
+ *         key   : Search key.
+ *       Ret: Node in extree that matches key, or if no match, hypothetical
+ *            node's successor/predecessor (NULL if no successor/predecessor).
 *
 *   static void
- *   ex_insert(ex_t *tree, ex_node_t *node);
- *       Description: Insert node into tree.
+ *   ex_insert(ex_t *extree, ex_node_t *node);
+ *       Description: Insert node into extree.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *         node: Node to be inserted into tree.
+ *         extree: Pointer to an initialized red-black tree object.
+ *         node  : Node to be inserted into extree.
 *
 *   static void
- *   ex_remove(ex_t *tree, ex_node_t *node);
- *       Description: Remove node from tree.
+ *   ex_remove(ex_t *extree, ex_node_t *node);
+ *       Description: Remove node from extree.
 *       Args:
- *         tree: Pointer to an initialized red-black tree object.
- *         node: Node in tree to be removed.
+ *         extree: Pointer to an initialized red-black tree object.
+ *         node  : Node in extree to be removed.
 *
 *   static ex_node_t *
- *   ex_iter(ex_t *tree, ex_node_t *start, ex_node_t *(*cb)(ex_t *,
+ *   ex_iter(ex_t *extree, ex_node_t *start, ex_node_t *(*cb)(ex_t *,
 *     ex_node_t *, void *), void *arg);
 *   static ex_node_t *
- *   ex_reverse_iter(ex_t *tree, ex_node_t *start, ex_node *(*cb)(ex_t *,
+ *   ex_reverse_iter(ex_t *extree, ex_node_t *start, ex_node *(*cb)(ex_t *,
 *     ex_node_t *, void *), void *arg);
- *       Description: Iterate forward/backward over tree, starting at node.  If
- *                    tree is modified, iteration must be immediately
+ *       Description: Iterate forward/backward over extree, starting at node.
+ *                    If extree is modified, iteration must be immediately
 *                    terminated by the callback function that causes the
 *                    modification.
 *       Args:
- *         tree : Pointer to an initialized red-black tree object.
- *         start: Node at which to start iteration, or NULL to start at
- *                first/last node.
- *         cb   : Callback function, which is called for each node during
- *                iteration.  Under normal circumstances the callback function
- *                should return NULL, which causes iteration to continue.  If a
- *                callback function returns non-NULL, iteration is immediately
- *                terminated and the non-NULL return value is returned by the
- *                iterator.  This is useful for re-starting iteration after
- *                modifying tree.
- *         arg  : Opaque pointer passed to cb().
+ *         extree: Pointer to an initialized red-black tree object.
+ *         start : Node at which to start iteration, or NULL to start at
+ *                 first/last node.
+ *         cb    : Callback function, which is called for each node during
+ *                 iteration.  Under normal circumstances the callback function
+ *                 should return NULL, which causes iteration to continue.  If a
+ *                 callback function returns non-NULL, iteration is immediately
+ *                 terminated and the non-NULL return value is returned by the
+ *                 iterator.  This is useful for re-starting iteration after
+ *                 modifying extree.
+ *         arg   : Opaque pointer passed to cb().
 *       Ret: NULL if iteration completed, or the non-NULL callback return value
 *            that caused termination of the iteration.
 */
@@ -36,9 +36,6 @@ struct rtree_s {
 #ifdef JEMALLOC_H_EXTERNS

 rtree_t	*rtree_new(unsigned bits);
-void	rtree_prefork(rtree_t *rtree);
-void	rtree_postfork_parent(rtree_t *rtree);
-void	rtree_postfork_child(rtree_t *rtree);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -1,122 +0,0 @@
-#!/bin/sh
-
-# The following limits are chosen such that they cover all supported platforms.
-
-# Range of quanta.
-lg_qmin=3
-lg_qmax=4
-
-# The range of tiny size classes is [2^lg_tmin..2^(lg_q-1)].
-lg_tmin=3
-
-# Range of page sizes.
-lg_pmin=12
-lg_pmax=16
-
-pow2() {
-  e=$1
-  pow2_result=1
-  while [ ${e} -gt 0 ] ; do
-    pow2_result=$((${pow2_result} + ${pow2_result}))
-    e=$((${e} - 1))
-  done
-}
-
-cat <<EOF
-/* This file was automatically generated by size_classes.sh. */
-/******************************************************************************/
-#ifdef JEMALLOC_H_TYPES
-
-EOF
-
-lg_q=${lg_qmin}
-while [ ${lg_q} -le ${lg_qmax} ] ; do
-  lg_t=${lg_tmin}
-  while [ ${lg_t} -le ${lg_q} ] ; do
-    lg_p=${lg_pmin}
-    while [ ${lg_p} -le ${lg_pmax} ] ; do
-      echo "#if (LG_TINY_MIN == ${lg_t} && LG_QUANTUM == ${lg_q} && LG_PAGE == ${lg_p})"
-      echo "#define	SIZE_CLASSES_DEFINED"
-      pow2 ${lg_q}; q=${pow2_result}
-      pow2 ${lg_t}; t=${pow2_result}
-      pow2 ${lg_p}; p=${pow2_result}
-      bin=0
-      psz=0
-      sz=${t}
-      delta=$((${sz} - ${psz}))
-      echo "/*  SIZE_CLASS(bin,	delta,	sz) */"
-      echo "#define	SIZE_CLASSES							\\"
-
-      # Tiny size classes.
-      while [ ${sz} -lt ${q} ] ; do
-        echo "    SIZE_CLASS(${bin},	${delta},	${sz})					\\"
-        bin=$((${bin} + 1))
-        psz=${sz}
-        sz=$((${sz} + ${sz}))
-        delta=$((${sz} - ${psz}))
-      done
-      # Quantum-multiple size classes.  For each doubling of sz, as many as 4
-      # size classes exist.  Their spacing is the greater of:
-      # - q
-      # - sz/4, where sz is a power of 2
-      while [ ${sz} -lt ${p} ] ; do
-        if [ ${sz} -ge $((${q} * 4)) ] ; then
-          i=$((${sz} / 4))
-        else
-          i=${q}
-        fi
-        next_2pow=$((${sz} * 2))
-        while [ ${sz} -lt $next_2pow ] ; do
-          echo "    SIZE_CLASS(${bin},	${delta},	${sz})					\\"
-          bin=$((${bin} + 1))
-          psz=${sz}
-          sz=$((${sz} + ${i}))
-          delta=$((${sz} - ${psz}))
-        done
-      done
-      echo
-      echo "#define	NBINS		${bin}"
-      echo "#define	SMALL_MAXCLASS	${psz}"
-      echo "#endif"
-      echo
-      lg_p=$((${lg_p} + 1))
-    done
-    lg_t=$((${lg_t} + 1))
-  done
-  lg_q=$((${lg_q} + 1))
-done
-
-cat <<EOF
-#ifndef SIZE_CLASSES_DEFINED
-#  error "No size class definitions match configuration"
-#endif
-#undef SIZE_CLASSES_DEFINED
-/*
- * The small_size2bin lookup table uses uint8_t to encode each bin index, so we
- * cannot support more than 256 small size classes.  Further constrain NBINS to
- * 255 to support prof_promote, since all small size classes, plus a "not
- * small" size class must be stored in 8 bits of arena_chunk_map_t's bits
- * field.
- */
-#if (NBINS > 255)
-#  error "Too many small size classes"
-#endif
-
-#endif /* JEMALLOC_H_TYPES */
-/******************************************************************************/
-#ifdef JEMALLOC_H_STRUCTS
-
-
-#endif /* JEMALLOC_H_STRUCTS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_EXTERNS
-
-
-#endif /* JEMALLOC_H_EXTERNS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_INLINES
-
-
-#endif /* JEMALLOC_H_INLINES */
-/******************************************************************************/
-EOF
@@ -1,16 +1,25 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

+#define	UMAX2S_BUFSIZE	65
+
+#ifdef JEMALLOC_STATS
 typedef struct tcache_bin_stats_s tcache_bin_stats_t;
 typedef struct malloc_bin_stats_s malloc_bin_stats_t;
 typedef struct malloc_large_stats_s malloc_large_stats_t;
 typedef struct arena_stats_s arena_stats_t;
+#endif
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
 typedef struct chunk_stats_s chunk_stats_t;
+#endif

 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
 #ifdef JEMALLOC_H_STRUCTS

+#ifdef JEMALLOC_STATS
+
+#ifdef JEMALLOC_TCACHE
 struct tcache_bin_stats_s {
 	/*
 	 * Number of allocation requests that corresponded to the size of this
@@ -18,6 +27,7 @@ struct tcache_bin_stats_s {
 	 */
 	uint64_t	nrequests;
 };
+#endif

 struct malloc_bin_stats_s {
 	/*
@@ -42,11 +52,13 @@ struct malloc_bin_stats_s {
 	 */
 	uint64_t	nrequests;

+#ifdef JEMALLOC_TCACHE
 	/* Number of tcache fills from this bin. */
 	uint64_t	nfills;

 	/* Number of tcache flushes to this bin. */
 	uint64_t	nflushes;
+#endif

 	/* Total number of runs created for this bin's size class. */
 	uint64_t	nruns;
@@ -57,6 +69,9 @@ struct malloc_bin_stats_s {
 	 */
 	uint64_t	reruns;

+	/* High-water mark for this bin. */
+	size_t		highruns;
+
 	/* Current number of runs in this bin. */
 	size_t		curruns;
 };
@@ -78,6 +93,9 @@ struct malloc_large_stats_s {
 	 */
 	uint64_t	nrequests;

+	/* High-water mark for this size class. */
+	size_t		highruns;
+
 	/* Current number of runs of this size class. */
 	size_t		curruns;
 };
@@ -109,10 +127,14 @@ struct arena_stats_s {
 	 */
 	malloc_large_stats_t	*lstats;
 };
+#endif /* JEMALLOC_STATS */

+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
 struct chunk_stats_s {
+#  ifdef JEMALLOC_STATS
 	/* Number of chunks that were allocated. */
 	uint64_t	nchunks;
+#  endif

 	/* High-water mark for number of chunks allocated. */
 	size_t		highchunks;
@@ -124,6 +146,7 @@ struct chunk_stats_s {
 	 */
 	size_t		curchunks;
 };
+#endif /* JEMALLOC_STATS */

 #endif /* JEMALLOC_H_STRUCTS */
 /******************************************************************************/
@@ -131,14 +154,24 @@ struct chunk_stats_s {

 extern bool	opt_stats_print;

+#ifdef JEMALLOC_STATS
 extern size_t	stats_cactive;
+#endif

+char	*u2s(uint64_t x, unsigned base, char *s);
+#ifdef JEMALLOC_STATS
+void malloc_cprintf(void (*write)(void *, const char *), void *cbopaque,
+    const char *format, ...) JEMALLOC_ATTR(format(printf, 3, 4));
+void	malloc_printf(const char *format, ...)
+    JEMALLOC_ATTR(format(printf, 1, 2));
+#endif
 void	stats_print(void (*write)(void *, const char *), void *cbopaque,
    const char *opts);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
 #ifdef JEMALLOC_H_INLINES
+#ifdef JEMALLOC_STATS

 #ifndef JEMALLOC_ENABLE_INLINE
 size_t	stats_cactive_get(void);
@@ -169,5 +202,6 @@ stats_cactive_sub(size_t size)
 }
 #endif

+#endif /* JEMALLOC_STATS */
 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/
@@ -1,3 +1,4 @@
+#ifdef JEMALLOC_TCACHE
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

@@ -5,16 +6,6 @@ typedef struct tcache_bin_info_s tcache_bin_info_t;
 typedef struct tcache_bin_s tcache_bin_t;
 typedef struct tcache_s tcache_t;

-/*
- * tcache pointers close to NULL are used to encode state information that is
- * used for two purposes: preventing thread caching on a per thread basis and
- * cleaning up during thread shutdown.
- */
-#define	TCACHE_STATE_DISABLED		((tcache_t *)(uintptr_t)1)
-#define	TCACHE_STATE_REINCARNATED	((tcache_t *)(uintptr_t)2)
-#define	TCACHE_STATE_PURGATORY		((tcache_t *)(uintptr_t)3)
-#define	TCACHE_STATE_MAX		TCACHE_STATE_PURGATORY
-
 /*
 * Absolute maximum number of cache slots for each small bin in the thread
 * cache.  This is an additional constraint beyond that imposed as: twice the
@@ -31,26 +22,17 @@ typedef struct tcache_s tcache_t;
 #define	LG_TCACHE_MAXCLASS_DEFAULT	15

 /*
- * TCACHE_GC_SWEEP is the approximate number of allocation events between
- * full GC sweeps.  Integer rounding may cause the actual number to be
- * slightly higher, since GC is performed incrementally.
+ * (1U << opt_lg_tcache_gc_sweep) is the approximate number of allocation
+ * events between full GC sweeps (-1: disabled).  Integer rounding may cause
+ * the actual number to be slightly higher, since GC is performed
+ * incrementally.
 */
-#define	TCACHE_GC_SWEEP			8192
-
-/* Number of tcache allocation/deallocation events between incremental GCs. */
-#define	TCACHE_GC_INCR							\
-    ((TCACHE_GC_SWEEP / NBINS) + ((TCACHE_GC_SWEEP / NBINS == 0) ? 0 : 1))
+#define	LG_TCACHE_GC_SWEEP_DEFAULT	13

 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
 #ifdef JEMALLOC_H_STRUCTS

-typedef enum {
-	tcache_enabled_false   = 0, /* Enable cast to/from bool. */
-	tcache_enabled_true    = 1,
-	tcache_enabled_default = 2
-} tcache_enabled_t;
-
 /*
 * Read-only information associated with each element of tcache_t's tbins array
 * is stored separately, mainly to reduce memory usage.
@@ -60,7 +42,9 @@ struct tcache_bin_info_s {
 };

 struct tcache_bin_s {
+#  ifdef JEMALLOC_STATS
 	tcache_bin_stats_t tstats;
+#  endif
 	int		low_water;	/* Min # cached since last GC. */
 	unsigned	lg_fill_div;	/* Fill (ncached_max >> lg_fill_div). */
 	unsigned	ncached;	/* # of cached objects. */
@@ -68,8 +52,12 @@ struct tcache_bin_s {
 };

 struct tcache_s {
+#  ifdef JEMALLOC_STATS
 	ql_elm(tcache_t) link;		/* Used for aggregating stats. */
+#  endif
+#  ifdef JEMALLOC_PROF
 	uint64_t	prof_accumbytes;/* Cleared after arena_prof_accum() */
+#  endif
 	arena_t		*arena;		/* This thread's arena. */
 	unsigned	ev_cnt;		/* Event count since incremental GC. */
 	unsigned	next_gc_bin;	/* Next bin to GC. */
@@ -88,11 +76,29 @@ struct tcache_s {

 extern bool	opt_tcache;
 extern ssize_t	opt_lg_tcache_max;
+extern ssize_t	opt_lg_tcache_gc_sweep;

 extern tcache_bin_info_t	*tcache_bin_info;

+/* Map of thread-specific caches. */
+#ifndef NO_TLS
+extern __thread tcache_t	*tcache_tls
+    JEMALLOC_ATTR(tls_model("initial-exec"));
+#  define TCACHE_GET()	tcache_tls
+#  define TCACHE_SET(v)	do {						\
+	tcache_tls = (tcache_t *)(v);					\
+	pthread_setspecific(tcache_tsd, (void *)(v));			\
+} while (0)
+#else
+#  define TCACHE_GET()	((tcache_t *)pthread_getspecific(tcache_tsd))
+#  define TCACHE_SET(v)	do {						\
+	pthread_setspecific(tcache_tsd, (void *)(v));			\
+} while (0)
+#endif
+extern pthread_key_t		tcache_tsd;
+
 /*
- * Number of tcache bins.  There are NBINS small-object bins, plus 0 or more
+ * Number of tcache bins.  There are nbins small-object bins, plus 0 or more
 * large-object bins.
 */
 extern size_t			nhbins;
@@ -100,159 +106,68 @@ extern size_t			nhbins;
 /* Maximum cached size class. */
 extern size_t			tcache_maxclass;

-size_t	tcache_salloc(const void *ptr);
-void	tcache_event_hard(tcache_t *tcache);
+/* Number of tcache allocation/deallocation events between incremental GCs. */
+extern unsigned			tcache_gc_incr;
+
+void	tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+    , tcache_t *tcache
+#endif
+    );
+void	tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+    , tcache_t *tcache
+#endif
+    );
+tcache_t *tcache_create(arena_t *arena);
 void	*tcache_alloc_small_hard(tcache_t *tcache, tcache_bin_t *tbin,
    size_t binind);
-void	tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
-    tcache_t *tcache);
-void	tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
-    tcache_t *tcache);
-void	tcache_arena_associate(tcache_t *tcache, arena_t *arena);
-void	tcache_arena_dissociate(tcache_t *tcache);
-tcache_t *tcache_create(arena_t *arena);
 void	tcache_destroy(tcache_t *tcache);
-void	tcache_thread_cleanup(void *arg);
+#ifdef JEMALLOC_STATS
 void	tcache_stats_merge(tcache_t *tcache, arena_t *arena);
-bool	tcache_boot0(void);
-bool	tcache_boot1(void);
+#endif
+bool	tcache_boot(void);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
 #ifdef JEMALLOC_H_INLINES

 #ifndef JEMALLOC_ENABLE_INLINE
-malloc_tsd_protos(JEMALLOC_ATTR(unused), tcache, tcache_t *)
-malloc_tsd_protos(JEMALLOC_ATTR(unused), tcache_enabled, tcache_enabled_t)
-
 void	tcache_event(tcache_t *tcache);
-void	tcache_flush(void);
-bool	tcache_enabled_get(void);
-tcache_t *tcache_get(bool create);
-void	tcache_enabled_set(bool enabled);
+tcache_t *tcache_get(void);
 void	*tcache_alloc_easy(tcache_bin_t *tbin);
 void	*tcache_alloc_small(tcache_t *tcache, size_t size, bool zero);
 void	*tcache_alloc_large(tcache_t *tcache, size_t size, bool zero);
-void	tcache_dalloc_small(tcache_t *tcache, void *ptr, size_t binind);
+void	tcache_dalloc_small(tcache_t *tcache, void *ptr);
 void	tcache_dalloc_large(tcache_t *tcache, void *ptr, size_t size);
 #endif

 #if (defined(JEMALLOC_ENABLE_INLINE) || defined(JEMALLOC_TCACHE_C_))
-/* Map of thread-specific caches. */
-malloc_tsd_externs(tcache, tcache_t *)
-malloc_tsd_funcs(JEMALLOC_INLINE, tcache, tcache_t *, NULL,
-    tcache_thread_cleanup)
-/* Per thread flag that allows thread caches to be disabled. */
-malloc_tsd_externs(tcache_enabled, tcache_enabled_t)
-malloc_tsd_funcs(JEMALLOC_INLINE, tcache_enabled, tcache_enabled_t,
-    tcache_enabled_default, malloc_tsd_no_cleanup)
-
-JEMALLOC_INLINE void
-tcache_flush(void)
-{
-	tcache_t *tcache;
-
-	cassert(config_tcache);
-
-	tcache = *tcache_tsd_get();
-	if ((uintptr_t)tcache <= (uintptr_t)TCACHE_STATE_MAX)
-		return;
-	tcache_destroy(tcache);
-	tcache = NULL;
-	tcache_tsd_set(&tcache);
-}
-
-JEMALLOC_INLINE bool
-tcache_enabled_get(void)
-{
-	tcache_enabled_t tcache_enabled;
-
-	cassert(config_tcache);
-
-	tcache_enabled = *tcache_enabled_tsd_get();
-	if (tcache_enabled == tcache_enabled_default) {
-		tcache_enabled = (tcache_enabled_t)opt_tcache;
-		tcache_enabled_tsd_set(&tcache_enabled);
-	}
-
-	return ((bool)tcache_enabled);
-}
-
-JEMALLOC_INLINE void
-tcache_enabled_set(bool enabled)
-{
-	tcache_enabled_t tcache_enabled;
-	tcache_t *tcache;
-
-	cassert(config_tcache);
-
-	tcache_enabled = (tcache_enabled_t)enabled;
-	tcache_enabled_tsd_set(&tcache_enabled);
-	tcache = *tcache_tsd_get();
-	if (enabled) {
-		if (tcache == TCACHE_STATE_DISABLED) {
-			tcache = NULL;
-			tcache_tsd_set(&tcache);
-		}
-	} else /* disabled */ {
-		if (tcache > TCACHE_STATE_MAX) {
-			tcache_destroy(tcache);
-			tcache = NULL;
-		}
-		if (tcache == NULL) {
-			tcache = TCACHE_STATE_DISABLED;
-			tcache_tsd_set(&tcache);
-		}
-	}
-}
-
 JEMALLOC_INLINE tcache_t *
-tcache_get(bool create)
+tcache_get(void)
 {
 	tcache_t *tcache;

-	if (config_tcache == false)
-		return (NULL);
-	if (config_lazy_lock && isthreaded == false)
+	if ((isthreaded & opt_tcache) == false)
 		return (NULL);

-	tcache = *tcache_tsd_get();
-	if ((uintptr_t)tcache <= (uintptr_t)TCACHE_STATE_MAX) {
-		if (tcache == TCACHE_STATE_DISABLED)
-			return (NULL);
+	tcache = TCACHE_GET();
+	if ((uintptr_t)tcache <= (uintptr_t)2) {
 		if (tcache == NULL) {
-			if (create == false) {
+			tcache = tcache_create(choose_arena());
+			if (tcache == NULL)
+				return (NULL);
+		} else {
+			if (tcache == (void *)(uintptr_t)1) {
 				/*
-				 * Creating a tcache here would cause
-				 * allocation as a side effect of free().
-				 * Ordinarily that would be okay since
-				 * tcache_create() failure is a soft failure
-				 * that doesn't propagate.  However, if TLS
-				 * data are freed via free() as in glibc,
-				 * subtle corruption could result from setting
-				 * a TLS variable after its backing memory is
-				 * freed.
+				 * Make a note that an allocator function was
+				 * called after the tcache_thread_cleanup() was
+				 * called.
 				 */
-				return (NULL);
+				TCACHE_SET((uintptr_t)2);
 			}
-			if (tcache_enabled_get() == false) {
-				tcache_enabled_set(false); /* Memoize. */
-				return (NULL);
-			}
-			return (tcache_create(choose_arena(NULL)));
-		}
-		if (tcache == TCACHE_STATE_PURGATORY) {
-			/*
-			 * Make a note that an allocator function was called
-			 * after tcache_thread_cleanup() was called.
-			 */
-			tcache = TCACHE_STATE_REINCARNATED;
-			tcache_tsd_set(&tcache);
 			return (NULL);
 		}
-		if (tcache == TCACHE_STATE_REINCARNATED)
-			return (NULL);
-		not_reached();
 	}

 	return (tcache);
@@ -262,13 +177,60 @@ JEMALLOC_INLINE void
 tcache_event(tcache_t *tcache)
 {

-	if (TCACHE_GC_INCR == 0)
+	if (tcache_gc_incr == 0)
 		return;

 	tcache->ev_cnt++;
-	assert(tcache->ev_cnt <= TCACHE_GC_INCR);
-	if (tcache->ev_cnt == TCACHE_GC_INCR)
-		tcache_event_hard(tcache);
+	assert(tcache->ev_cnt <= tcache_gc_incr);
+	if (tcache->ev_cnt == tcache_gc_incr) {
+		size_t binind = tcache->next_gc_bin;
+		tcache_bin_t *tbin = &tcache->tbins[binind];
+		tcache_bin_info_t *tbin_info = &tcache_bin_info[binind];
+
+		if (tbin->low_water > 0) {
+			/*
+			 * Flush (ceiling) 3/4 of the objects below the low
+			 * water mark.
+			 */
+			if (binind < nbins) {
+				tcache_bin_flush_small(tbin, binind,
+				    tbin->ncached - tbin->low_water +
+				    (tbin->low_water >> 2)
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+				    , tcache
+#endif
+				    );
+			} else {
+				tcache_bin_flush_large(tbin, binind,
+				    tbin->ncached - tbin->low_water +
+				    (tbin->low_water >> 2)
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+				    , tcache
+#endif
+				    );
+			}
+			/*
+			 * Reduce fill count by 2X.  Limit lg_fill_div such that
+			 * the fill count is always at least 1.
+			 */
+			if ((tbin_info->ncached_max >> (tbin->lg_fill_div+1))
+			    >= 1)
+				tbin->lg_fill_div++;
+		} else if (tbin->low_water < 0) {
+			/*
+			 * Increase fill count by 2X.  Make sure lg_fill_div
+			 * stays greater than 0.
+			 */
+			if (tbin->lg_fill_div > 1)
+				tbin->lg_fill_div--;
+		}
+		tbin->low_water = tbin->ncached;
+
+		tcache->next_gc_bin++;
+		if (tcache->next_gc_bin == nhbins)
+			tcache->next_gc_bin = 0;
+		tcache->ev_cnt = 0;
+	}
 }

 JEMALLOC_INLINE void *
@@ -295,7 +257,7 @@ tcache_alloc_small(tcache_t *tcache, size_t size, bool zero)
 	tcache_bin_t *tbin;

 	binind = SMALL_SIZE2BIN(size);
-	assert(binind < NBINS);
+	assert(binind < nbins);
 	tbin = &tcache->tbins[binind];
 	ret = tcache_alloc_easy(tbin);
 	if (ret == NULL) {
@@ -303,29 +265,24 @@ tcache_alloc_small(tcache_t *tcache, size_t size, bool zero)
 		if (ret == NULL)
 			return (NULL);
 	}
-	assert(tcache_salloc(ret) == arena_bin_info[binind].reg_size);
+	assert(arena_salloc(ret) == arena_bin_info[binind].reg_size);

 	if (zero == false) {
-		if (config_fill) {
-			if (opt_junk) {
-				arena_alloc_junk_small(ret,
-				    &arena_bin_info[binind], false);
-			} else if (opt_zero)
-				memset(ret, 0, size);
-		}
-	} else {
-		if (config_fill && opt_junk) {
-			arena_alloc_junk_small(ret, &arena_bin_info[binind],
-			    true);
-		}
-		VALGRIND_MAKE_MEM_UNDEFINED(ret, size);
+#ifdef JEMALLOC_FILL
+		if (opt_junk)
+			memset(ret, 0xa5, size);
+		else if (opt_zero)
+			memset(ret, 0, size);
+#endif
+	} else
 		memset(ret, 0, size);
-	}

-	if (config_stats)
-		tbin->tstats.nrequests++;
-	if (config_prof)
-		tcache->prof_accumbytes += arena_bin_info[binind].reg_size;
+#ifdef JEMALLOC_STATS
+	tbin->tstats.nrequests++;
+#endif
+#ifdef JEMALLOC_PROF
+	tcache->prof_accumbytes += arena_bin_info[binind].reg_size;
+#endif
 	tcache_event(tcache);
 	return (ret);
 }
@@ -339,7 +296,7 @@ tcache_alloc_large(tcache_t *tcache, size_t size, bool zero)

 	size = PAGE_CEILING(size);
 	assert(size <= tcache_maxclass);
-	binind = NBINS + (size >> LG_PAGE) - 1;
+	binind = nbins + (size >> PAGE_SHIFT) - 1;
 	assert(binind < nhbins);
 	tbin = &tcache->tbins[binind];
 	ret = tcache_alloc_easy(tbin);
@@ -352,30 +309,28 @@ tcache_alloc_large(tcache_t *tcache, size_t size, bool zero)
 		if (ret == NULL)
 			return (NULL);
 	} else {
-		if (config_prof && prof_promote && size == PAGE) {
-			arena_chunk_t *chunk =
-			    (arena_chunk_t *)CHUNK_ADDR2BASE(ret);
-			size_t pageind = (((uintptr_t)ret - (uintptr_t)chunk) >>
-			    LG_PAGE);
-			arena_mapbits_large_binind_set(chunk, pageind,
-			    BININD_INVALID);
-		}
+#ifdef JEMALLOC_PROF
+		arena_chunk_t *chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ret);
+		size_t pageind = (((uintptr_t)ret - (uintptr_t)chunk) >>
+		    PAGE_SHIFT);
+		chunk->map[pageind-map_bias].bits &= ~CHUNK_MAP_CLASS_MASK;
+#endif
 		if (zero == false) {
-			if (config_fill) {
-				if (opt_junk)
-					memset(ret, 0xa5, size);
-				else if (opt_zero)
-					memset(ret, 0, size);
-			}
-		} else {
-			VALGRIND_MAKE_MEM_UNDEFINED(ret, size);
+#ifdef JEMALLOC_FILL
+			if (opt_junk)
+				memset(ret, 0xa5, size);
+			else if (opt_zero)
+				memset(ret, 0, size);
+#endif
+		} else
 			memset(ret, 0, size);
-		}

-		if (config_stats)
-			tbin->tstats.nrequests++;
-		if (config_prof)
-			tcache->prof_accumbytes += size;
+#ifdef JEMALLOC_STATS
+		tbin->tstats.nrequests++;
+#endif
+#ifdef JEMALLOC_PROF
+		tcache->prof_accumbytes += size;
+#endif
 	}

 	tcache_event(tcache);
@@ -383,21 +338,45 @@ tcache_alloc_large(tcache_t *tcache, size_t size, bool zero)
 }

 JEMALLOC_INLINE void
-tcache_dalloc_small(tcache_t *tcache, void *ptr, size_t binind)
+tcache_dalloc_small(tcache_t *tcache, void *ptr)
 {
+	arena_t *arena;
+	arena_chunk_t *chunk;
+	arena_run_t *run;
+	arena_bin_t *bin;
 	tcache_bin_t *tbin;
 	tcache_bin_info_t *tbin_info;
+	size_t pageind, binind;
+	arena_chunk_map_t *mapelm;

-	assert(tcache_salloc(ptr) <= SMALL_MAXCLASS);
+	assert(arena_salloc(ptr) <= small_maxclass);

-	if (config_fill && opt_junk)
-		arena_dalloc_junk_small(ptr, &arena_bin_info[binind]);
+	chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
+	arena = chunk->arena;
+	pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> PAGE_SHIFT;
+	mapelm = &chunk->map[pageind-map_bias];
+	run = (arena_run_t *)((uintptr_t)chunk + (uintptr_t)((pageind -
+	    (mapelm->bits >> PAGE_SHIFT)) << PAGE_SHIFT));
+	dassert(run->magic == ARENA_RUN_MAGIC);
+	bin = run->bin;
+	binind = ((uintptr_t)bin - (uintptr_t)&arena->bins) /
+	    sizeof(arena_bin_t);
+	assert(binind < nbins);
+
+#ifdef JEMALLOC_FILL
+	if (opt_junk)
+		memset(ptr, 0x5a, arena_bin_info[binind].reg_size);
+#endif

 	tbin = &tcache->tbins[binind];
 	tbin_info = &tcache_bin_info[binind];
 	if (tbin->ncached == tbin_info->ncached_max) {
 		tcache_bin_flush_small(tbin, binind, (tbin_info->ncached_max >>
-		    1), tcache);
+		    1)
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+		    , tcache
+#endif
+		    );
 	}
 	assert(tbin->ncached < tbin_info->ncached_max);
 	tbin->avail[tbin->ncached] = ptr;
@@ -409,24 +388,35 @@ tcache_dalloc_small(tcache_t *tcache, void *ptr, size_t binind)
 JEMALLOC_INLINE void
 tcache_dalloc_large(tcache_t *tcache, void *ptr, size_t size)
 {
-	size_t binind;
+	arena_t *arena;
+	arena_chunk_t *chunk;
+	size_t pageind, binind;
 	tcache_bin_t *tbin;
 	tcache_bin_info_t *tbin_info;

 	assert((size & PAGE_MASK) == 0);
-	assert(tcache_salloc(ptr) > SMALL_MAXCLASS);
-	assert(tcache_salloc(ptr) <= tcache_maxclass);
+	assert(arena_salloc(ptr) > small_maxclass);
+	assert(arena_salloc(ptr) <= tcache_maxclass);

-	binind = NBINS + (size >> LG_PAGE) - 1;
+	chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
+	arena = chunk->arena;
+	pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> PAGE_SHIFT;
+	binind = nbins + (size >> PAGE_SHIFT) - 1;

-	if (config_fill && opt_junk)
+#ifdef JEMALLOC_FILL
+	if (opt_junk)
 		memset(ptr, 0x5a, size);
+#endif

 	tbin = &tcache->tbins[binind];
 	tbin_info = &tcache_bin_info[binind];
 	if (tbin->ncached == tbin_info->ncached_max) {
 		tcache_bin_flush_large(tbin, binind, (tbin_info->ncached_max >>
-		    1), tcache);
+		    1)
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+		    , tcache
+#endif
+		    );
 	}
 	assert(tbin->ncached < tbin_info->ncached_max);
 	tbin->avail[tbin->ncached] = ptr;
@@ -438,3 +428,4 @@ tcache_dalloc_large(tcache_t *tcache, void *ptr, size_t size)

 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/
+#endif /* JEMALLOC_TCACHE */
@@ -1,397 +0,0 @@
-/******************************************************************************/
-#ifdef JEMALLOC_H_TYPES
-
-/* Maximum number of malloc_tsd users with cleanup functions. */
-#define	MALLOC_TSD_CLEANUPS_MAX	8
-
-typedef bool (*malloc_tsd_cleanup_t)(void);
-
-/*
- * TLS/TSD-agnostic macro-based implementation of thread-specific data.  There
- * are four macros that support (at least) three use cases: file-private,
- * library-private, and library-private inlined.  Following is an example
- * library-private tsd variable:
- *
- * In example.h:
- *   typedef struct {
- *           int x;
- *           int y;
- *   } example_t;
- *   #define EX_INITIALIZER JEMALLOC_CONCAT({0, 0})
- *   malloc_tsd_protos(, example, example_t *)
- *   malloc_tsd_externs(example, example_t *)
- * In example.c:
- *   malloc_tsd_data(, example, example_t *, EX_INITIALIZER)
- *   malloc_tsd_funcs(, example, example_t *, EX_INITIALIZER,
- *       example_tsd_cleanup)
- *
- * The result is a set of generated functions, e.g.:
- *
- *   bool example_tsd_boot(void) {...}
- *   example_t **example_tsd_get() {...}
- *   void example_tsd_set(example_t **val) {...}
- *
- * Note that all of the functions deal in terms of (a_type *) rather than
- * (a_type)  so that it is possible to support non-pointer types (unlike
- * pthreads TSD).  example_tsd_cleanup() is passed an (a_type *) pointer that is
- * cast to (void *).  This means that the cleanup function needs to cast *and*
- * dereference the function argument, e.g.:
- *
- *   void
- *   example_tsd_cleanup(void *arg)
- *   {
- *           example_t *example = *(example_t **)arg;
- *
- *           [...]
- *           if ([want the cleanup function to be called again]) {
- *                   example_tsd_set(&example);
- *           }
- *   }
- *
- * If example_tsd_set() is called within example_tsd_cleanup(), it will be
- * called again.  This is similar to how pthreads TSD destruction works, except
- * that pthreads only calls the cleanup function again if the value was set to
- * non-NULL.
- */
-
-/* malloc_tsd_protos(). */
-#define	malloc_tsd_protos(a_attr, a_name, a_type)			\
-a_attr bool								\
-a_name##_tsd_boot(void);						\
-a_attr a_type *								\
-a_name##_tsd_get(void);							\
-a_attr void								\
-a_name##_tsd_set(a_type *val);
-
-/* malloc_tsd_externs(). */
-#ifdef JEMALLOC_MALLOC_THREAD_CLEANUP
-#define	malloc_tsd_externs(a_name, a_type)				\
-extern __thread a_type	a_name##_tls;					\
-extern __thread bool	a_name##_initialized;				\
-extern bool		a_name##_booted;
-#elif (defined(JEMALLOC_TLS))
-#define	malloc_tsd_externs(a_name, a_type)				\
-extern __thread a_type	a_name##_tls;					\
-extern pthread_key_t	a_name##_tsd;					\
-extern bool		a_name##_booted;
-#elif (defined(_WIN32))
-#define malloc_tsd_externs(a_name, a_type)				\
-extern DWORD		a_name##_tsd;					\
-extern bool		a_name##_booted;
-#else
-#define	malloc_tsd_externs(a_name, a_type)				\
-extern pthread_key_t	a_name##_tsd;					\
-extern bool		a_name##_booted;
-#endif
-
-/* malloc_tsd_data(). */
-#ifdef JEMALLOC_MALLOC_THREAD_CLEANUP
-#define	malloc_tsd_data(a_attr, a_name, a_type, a_initializer)		\
-a_attr __thread a_type JEMALLOC_TLS_MODEL				\
-    a_name##_tls = a_initializer;					\
-a_attr __thread bool JEMALLOC_TLS_MODEL					\
-    a_name##_initialized = false;					\
-a_attr bool		a_name##_booted = false;
-#elif (defined(JEMALLOC_TLS))
-#define	malloc_tsd_data(a_attr, a_name, a_type, a_initializer)		\
-a_attr __thread a_type JEMALLOC_TLS_MODEL				\
-    a_name##_tls = a_initializer;					\
-a_attr pthread_key_t	a_name##_tsd;					\
-a_attr bool		a_name##_booted = false;
-#elif (defined(_WIN32))
-#define	malloc_tsd_data(a_attr, a_name, a_type, a_initializer)		\
-a_attr DWORD		a_name##_tsd;					\
-a_attr bool		a_name##_booted = false;
-#else
-#define	malloc_tsd_data(a_attr, a_name, a_type, a_initializer)		\
-a_attr pthread_key_t	a_name##_tsd;					\
-a_attr bool		a_name##_booted = false;
-#endif
-
-/* malloc_tsd_funcs(). */
-#ifdef JEMALLOC_MALLOC_THREAD_CLEANUP
-#define	malloc_tsd_funcs(a_attr, a_name, a_type, a_initializer,		\
-    a_cleanup)								\
-/* Initialization/cleanup. */						\
-a_attr bool								\
-a_name##_tsd_cleanup_wrapper(void)					\
-{									\
-									\
-	if (a_name##_initialized) {					\
-		a_name##_initialized = false;				\
-		a_cleanup(&a_name##_tls);				\
-	}								\
-	return (a_name##_initialized);					\
-}									\
-a_attr bool								\
-a_name##_tsd_boot(void)							\
-{									\
-									\
-	if (a_cleanup != malloc_tsd_no_cleanup) {			\
-		malloc_tsd_cleanup_register(				\
-		    &a_name##_tsd_cleanup_wrapper);			\
-	}								\
-	a_name##_booted = true;						\
-	return (false);							\
-}									\
-/* Get/set. */								\
-a_attr a_type *								\
-a_name##_tsd_get(void)							\
-{									\
-									\
-	assert(a_name##_booted);					\
-	return (&a_name##_tls);						\
-}									\
-a_attr void								\
-a_name##_tsd_set(a_type *val)						\
-{									\
-									\
-	assert(a_name##_booted);					\
-	a_name##_tls = (*val);						\
-	if (a_cleanup != malloc_tsd_no_cleanup)				\
-		a_name##_initialized = true;				\
-}
-#elif (defined(JEMALLOC_TLS))
-#define	malloc_tsd_funcs(a_attr, a_name, a_type, a_initializer,		\
-    a_cleanup)								\
-/* Initialization/cleanup. */						\
-a_attr bool								\
-a_name##_tsd_boot(void)							\
-{									\
-									\
-	if (a_cleanup != malloc_tsd_no_cleanup) {			\
-		if (pthread_key_create(&a_name##_tsd, a_cleanup) != 0)	\
-			return (true);					\
-	}								\
-	a_name##_booted = true;						\
-	return (false);							\
-}									\
-/* Get/set. */								\
-a_attr a_type *								\
-a_name##_tsd_get(void)							\
-{									\
-									\
-	assert(a_name##_booted);					\
-	return (&a_name##_tls);						\
-}									\
-a_attr void								\
-a_name##_tsd_set(a_type *val)						\
-{									\
-									\
-	assert(a_name##_booted);					\
-	a_name##_tls = (*val);						\
-	if (a_cleanup != malloc_tsd_no_cleanup) {			\
-		if (pthread_setspecific(a_name##_tsd,			\
-		    (void *)(&a_name##_tls))) {				\
-			malloc_write("<jemalloc>: Error"		\
-			    " setting TSD for "#a_name"\n");		\
-			if (opt_abort)					\
-				abort();				\
-		}							\
-	}								\
-}
-#elif (defined(_WIN32))
-#define	malloc_tsd_funcs(a_attr, a_name, a_type, a_initializer,		\
-    a_cleanup)								\
-/* Data structure. */							\
-typedef struct {							\
-	bool	initialized;						\
-	a_type	val;							\
-} a_name##_tsd_wrapper_t;						\
-/* Initialization/cleanup. */						\
-a_attr bool								\
-a_name##_tsd_cleanup_wrapper(void)					\
-{									\
-	a_name##_tsd_wrapper_t *wrapper;				\
-									\
-	wrapper = (a_name##_tsd_wrapper_t *) TlsGetValue(a_name##_tsd);	\
-	if (wrapper == NULL)						\
-		return (false);						\
-	if (a_cleanup != malloc_tsd_no_cleanup &&			\
-	    wrapper->initialized) {					\
-		a_type val = wrapper->val;				\
-		a_type tsd_static_data = a_initializer;			\
-		wrapper->initialized = false;				\
-		wrapper->val = tsd_static_data;				\
-		a_cleanup(&val);					\
-		if (wrapper->initialized) {				\
-			/* Trigger another cleanup round. */		\
-			return (true);					\
-		}							\
-	}								\
-	malloc_tsd_dalloc(wrapper);					\
-	return (false);							\
-}									\
-a_attr bool								\
-a_name##_tsd_boot(void)							\
-{									\
-									\
-	a_name##_tsd = TlsAlloc();					\
-	if (a_name##_tsd == TLS_OUT_OF_INDEXES)				\
-		return (true);						\
-	if (a_cleanup != malloc_tsd_no_cleanup) {			\
-		malloc_tsd_cleanup_register(				\
-		    &a_name##_tsd_cleanup_wrapper);			\
-	}								\
-	a_name##_booted = true;						\
-	return (false);							\
-}									\
-/* Get/set. */								\
-a_attr a_name##_tsd_wrapper_t *						\
-a_name##_tsd_get_wrapper(void)						\
-{									\
-	a_name##_tsd_wrapper_t *wrapper = (a_name##_tsd_wrapper_t *)	\
-	    TlsGetValue(a_name##_tsd);					\
-									\
-	if (wrapper == NULL) {						\
-		wrapper = (a_name##_tsd_wrapper_t *)			\
-		    malloc_tsd_malloc(sizeof(a_name##_tsd_wrapper_t));	\
-		if (wrapper == NULL) {					\
-			malloc_write("<jemalloc>: Error allocating"	\
-			    " TSD for "#a_name"\n");			\
-			abort();					\
-		} else {						\
-			static a_type tsd_static_data = a_initializer;	\
-			wrapper->initialized = false;			\
-			wrapper->val = tsd_static_data;			\
-		}							\
-		if (!TlsSetValue(a_name##_tsd, (void *)wrapper)) {	\
-			malloc_write("<jemalloc>: Error setting"	\
-			    " TSD for "#a_name"\n");			\
-			abort();					\
-		}							\
-	}								\
-	return (wrapper);						\
-}									\
-a_attr a_type *								\
-a_name##_tsd_get(void)							\
-{									\
-	a_name##_tsd_wrapper_t *wrapper;				\
-									\
-	assert(a_name##_booted);					\
-	wrapper = a_name##_tsd_get_wrapper();				\
-	return (&wrapper->val);						\
-}									\
-a_attr void								\
-a_name##_tsd_set(a_type *val)						\
-{									\
-	a_name##_tsd_wrapper_t *wrapper;				\
-									\
-	assert(a_name##_booted);					\
-	wrapper = a_name##_tsd_get_wrapper();				\
-	wrapper->val = *(val);						\
-	if (a_cleanup != malloc_tsd_no_cleanup)				\
-		wrapper->initialized = true;				\
-}
-#else
-#define	malloc_tsd_funcs(a_attr, a_name, a_type, a_initializer,		\
-    a_cleanup)								\
-/* Data structure. */							\
-typedef struct {							\
-	bool	initialized;						\
-	a_type	val;							\
-} a_name##_tsd_wrapper_t;						\
-/* Initialization/cleanup. */						\
-a_attr void								\
-a_name##_tsd_cleanup_wrapper(void *arg)					\
-{									\
-	a_name##_tsd_wrapper_t *wrapper = (a_name##_tsd_wrapper_t *)arg;\
-									\
-	if (a_cleanup != malloc_tsd_no_cleanup &&			\
-	    wrapper->initialized) {					\
-		wrapper->initialized = false;				\
-		a_cleanup(&wrapper->val);				\
-		if (wrapper->initialized) {				\
-			/* Trigger another cleanup round. */		\
-			if (pthread_setspecific(a_name##_tsd,		\
-			    (void *)wrapper)) {				\
-				malloc_write("<jemalloc>: Error"	\
-				    " setting TSD for "#a_name"\n");	\
-				if (opt_abort)				\
-					abort();			\
-			}						\
-			return;						\
-		}							\
-	}								\
-	malloc_tsd_dalloc(wrapper);					\
-}									\
-a_attr bool								\
-a_name##_tsd_boot(void)							\
-{									\
-									\
-	if (pthread_key_create(&a_name##_tsd,				\
-	    a_name##_tsd_cleanup_wrapper) != 0)				\
-		return (true);						\
-	a_name##_booted = true;						\
-	return (false);							\
-}									\
-/* Get/set. */								\
-a_attr a_name##_tsd_wrapper_t *						\
-a_name##_tsd_get_wrapper(void)						\
-{									\
-	a_name##_tsd_wrapper_t *wrapper = (a_name##_tsd_wrapper_t *)	\
-	    pthread_getspecific(a_name##_tsd);				\
-									\
-	if (wrapper == NULL) {						\
-		wrapper = (a_name##_tsd_wrapper_t *)			\
-		    malloc_tsd_malloc(sizeof(a_name##_tsd_wrapper_t));	\
-		if (wrapper == NULL) {					\
-			malloc_write("<jemalloc>: Error allocating"	\
-			    " TSD for "#a_name"\n");			\
-			abort();					\
-		} else {						\
-			static a_type tsd_static_data = a_initializer;	\
-			wrapper->initialized = false;			\
-			wrapper->val = tsd_static_data;			\
-		}							\
-		if (pthread_setspecific(a_name##_tsd,			\
-		    (void *)wrapper)) {					\
-			malloc_write("<jemalloc>: Error setting"	\
-			    " TSD for "#a_name"\n");			\
-			abort();					\
-		}							\
-	}								\
-	return (wrapper);						\
-}									\
-a_attr a_type *								\
-a_name##_tsd_get(void)							\
-{									\
-	a_name##_tsd_wrapper_t *wrapper;				\
-									\
-	assert(a_name##_booted);					\
-	wrapper = a_name##_tsd_get_wrapper();				\
-	return (&wrapper->val);						\
-}									\
-a_attr void								\
-a_name##_tsd_set(a_type *val)						\
-{									\
-	a_name##_tsd_wrapper_t *wrapper;				\
-									\
-	assert(a_name##_booted);					\
-	wrapper = a_name##_tsd_get_wrapper();				\
-	wrapper->val = *(val);						\
-	if (a_cleanup != malloc_tsd_no_cleanup)				\
-		wrapper->initialized = true;				\
-}
-#endif
-
-#endif /* JEMALLOC_H_TYPES */
-/******************************************************************************/
-#ifdef JEMALLOC_H_STRUCTS
-
-#endif /* JEMALLOC_H_STRUCTS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_EXTERNS
-
-void	*malloc_tsd_malloc(size_t size);
-void	malloc_tsd_dalloc(void *wrapper);
-void	malloc_tsd_no_cleanup(void *);
-void	malloc_tsd_cleanup_register(bool (*f)(void));
-void	malloc_tsd_boot(void);
-
-#endif /* JEMALLOC_H_EXTERNS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_INLINES
-
-#endif /* JEMALLOC_H_INLINES */
-/******************************************************************************/
@@ -1,160 +0,0 @@
-/******************************************************************************/
-#ifdef JEMALLOC_H_TYPES
-
-/* Size of stack-allocated buffer passed to buferror(). */
-#define	BUFERROR_BUF		64
-
-/*
- * Size of stack-allocated buffer used by malloc_{,v,vc}printf().  This must be
- * large enough for all possible uses within jemalloc.
- */
-#define	MALLOC_PRINTF_BUFSIZE	4096
-
-/*
- * Wrap a cpp argument that contains commas such that it isn't broken up into
- * multiple arguments.
- */
-#define JEMALLOC_CONCAT(...) __VA_ARGS__
-
-/*
- * Silence compiler warnings due to uninitialized values.  This is used
- * wherever the compiler fails to recognize that the variable is never used
- * uninitialized.
- */
-#ifdef JEMALLOC_CC_SILENCE
-#  define JEMALLOC_CC_SILENCE_INIT(v) = v
-#else
-#  define JEMALLOC_CC_SILENCE_INIT(v)
-#endif
-
-/*
- * Define a custom assert() in order to reduce the chances of deadlock during
- * assertion failure.
- */
-#ifndef assert
-#define	assert(e) do {							\
-	if (config_debug && !(e)) {					\
-		malloc_printf(						\
-		    "<jemalloc>: %s:%d: Failed assertion: \"%s\"\n",	\
-		    __FILE__, __LINE__, #e);				\
-		abort();						\
-	}								\
-} while (0)
-#endif
-
-/* Use to assert a particular configuration, e.g., cassert(config_debug). */
-#define	cassert(c) do {							\
-	if ((c) == false)						\
-		assert(false);						\
-} while (0)
-
-#ifndef not_reached
-#define	not_reached() do {						\
-	if (config_debug) {						\
-		malloc_printf(						\
-		    "<jemalloc>: %s:%d: Unreachable code reached\n",	\
-		    __FILE__, __LINE__);				\
-		abort();						\
-	}								\
-} while (0)
-#endif
-
-#ifndef not_implemented
-#define	not_implemented() do {						\
-	if (config_debug) {						\
-		malloc_printf("<jemalloc>: %s:%d: Not implemented\n",	\
-		    __FILE__, __LINE__);				\
-		abort();						\
-	}								\
-} while (0)
-#endif
-
-#define	assert_not_implemented(e) do {					\
-	if (config_debug && !(e))					\
-		not_implemented();					\
-} while (0)
-
-#endif /* JEMALLOC_H_TYPES */
-/******************************************************************************/
-#ifdef JEMALLOC_H_STRUCTS
-
-#endif /* JEMALLOC_H_STRUCTS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_EXTERNS
-
-int	buferror(char *buf, size_t buflen);
-uintmax_t	malloc_strtoumax(const char *nptr, char **endptr, int base);
-void	malloc_write(const char *s);
-
-/*
- * malloc_vsnprintf() supports a subset of snprintf(3) that avoids floating
- * point math.
- */
-int	malloc_vsnprintf(char *str, size_t size, const char *format,
-    va_list ap);
-int	malloc_snprintf(char *str, size_t size, const char *format, ...)
-    JEMALLOC_ATTR(format(printf, 3, 4));
-void	malloc_vcprintf(void (*write_cb)(void *, const char *), void *cbopaque,
-    const char *format, va_list ap);
-void malloc_cprintf(void (*write)(void *, const char *), void *cbopaque,
-    const char *format, ...) JEMALLOC_ATTR(format(printf, 3, 4));
-void	malloc_printf(const char *format, ...)
-    JEMALLOC_ATTR(format(printf, 1, 2));
-
-#endif /* JEMALLOC_H_EXTERNS */
-/******************************************************************************/
-#ifdef JEMALLOC_H_INLINES
-
-#ifndef JEMALLOC_ENABLE_INLINE
-size_t	pow2_ceil(size_t x);
-void	malloc_write(const char *s);
-void	set_errno(int errnum);
-int	get_errno(void);
-#endif
-
-#if (defined(JEMALLOC_ENABLE_INLINE) || defined(JEMALLOC_UTIL_C_))
-/* Compute the smallest power of 2 that is >= x. */
-JEMALLOC_INLINE size_t
-pow2_ceil(size_t x)
-{
-
-	x--;
-	x |= x >> 1;
-	x |= x >> 2;
-	x |= x >> 4;
-	x |= x >> 8;
-	x |= x >> 16;
-#if (LG_SIZEOF_PTR == 3)
-	x |= x >> 32;
-#endif
-	x++;
-	return (x);
-}
-
-/* Sets error code */
-JEMALLOC_INLINE void
-set_errno(int errnum)
-{
-
-#ifdef _WIN32
-	SetLastError(errnum);
-#else
-	errno = errnum;
-#endif
-}
-
-/* Get last error code */
-JEMALLOC_INLINE int
-get_errno(void)
-{
-
-#ifdef _WIN32
-	return (GetLastError());
-#else
-	return (errno);
-#endif
-}
-#endif
-
-#endif /* JEMALLOC_H_INLINES */
-/******************************************************************************/
@@ -1,9 +1,9 @@
+#ifndef JEMALLOC_ZONE
+#  error "This source file is for zones on Darwin (OS X)."
+#endif
 /******************************************************************************/
 #ifdef JEMALLOC_H_TYPES

-/* Default per thread quarantine size if valgrind is enabled. */
-#define	JEMALLOC_VALGRIND_QUARANTINE_DEFAULT	(ZU(1) << 24)
-
 #endif /* JEMALLOC_H_TYPES */
 /******************************************************************************/
 #ifdef JEMALLOC_H_STRUCTS
@@ -12,8 +12,8 @@
 /******************************************************************************/
 #ifdef JEMALLOC_H_EXTERNS

-void	quarantine(void *ptr);
-bool	quarantine_boot(void);
+malloc_zone_t *create_zone(void);
+void	szone2ozone(malloc_zone_t *zone);

 #endif /* JEMALLOC_H_EXTERNS */
 /******************************************************************************/
@@ -21,4 +21,3 @@ bool	quarantine_boot(void);

 #endif /* JEMALLOC_H_INLINES */
 /******************************************************************************/
-
@@ -15,8 +15,10 @@ extern "C" {
 #define	JEMALLOC_VERSION_GID "@jemalloc_version_gid@"

 #include "jemalloc_defs@install_suffix@.h"
+#ifndef JEMALLOC_P
+#  define JEMALLOC_P(s) s
+#endif

-#ifdef JEMALLOC_EXPERIMENTAL
 #define	ALLOCM_LG_ALIGN(la)	(la)
 #if LG_SIZEOF_PTR == 2
 #define	ALLOCM_ALIGN(a)	(ffs(a)-1)
@@ -25,131 +27,38 @@ extern "C" {
 #endif
 #define	ALLOCM_ZERO	((int)0x40)
 #define	ALLOCM_NO_MOVE	((int)0x80)
-/* Bias arena index bits so that 0 encodes "ALLOCM_ARENA() unspecified". */
-#define	ALLOCM_ARENA(a)	((int)(((a)+1) << 8))

 #define	ALLOCM_SUCCESS		0
 #define	ALLOCM_ERR_OOM		1
 #define	ALLOCM_ERR_NOT_MOVED	2
-#endif

-/*
- * The je_ prefix on the following public symbol declarations is an artifact of
- * namespace management, and should be omitted in application code unless
- * JEMALLOC_NO_DEMANGLE is defined (see below).
- */
-extern JEMALLOC_EXPORT const char	*je_malloc_conf;
-extern JEMALLOC_EXPORT void		(*je_malloc_message)(void *cbopaque,
-    const char *s);
+extern const char	*JEMALLOC_P(malloc_conf);
+extern void		(*JEMALLOC_P(malloc_message))(void *, const char *);

-JEMALLOC_EXPORT void	*je_malloc(size_t size) JEMALLOC_ATTR(malloc);
-JEMALLOC_EXPORT void	*je_calloc(size_t num, size_t size)
-    JEMALLOC_ATTR(malloc);
-JEMALLOC_EXPORT int	je_posix_memalign(void **memptr, size_t alignment,
-    size_t size) JEMALLOC_ATTR(nonnull(1));
-JEMALLOC_EXPORT void	*je_aligned_alloc(size_t alignment, size_t size)
-    JEMALLOC_ATTR(malloc);
-JEMALLOC_EXPORT void	*je_realloc(void *ptr, size_t size);
-JEMALLOC_EXPORT void	je_free(void *ptr);
+void	*JEMALLOC_P(malloc)(size_t size) JEMALLOC_ATTR(malloc);
+void	*JEMALLOC_P(calloc)(size_t num, size_t size) JEMALLOC_ATTR(malloc);
+int	JEMALLOC_P(posix_memalign)(void **memptr, size_t alignment, size_t size)
+    JEMALLOC_ATTR(nonnull(1));
+void	*JEMALLOC_P(realloc)(void *ptr, size_t size);
+void	JEMALLOC_P(free)(void *ptr);

-#ifdef JEMALLOC_OVERRIDE_MEMALIGN
-JEMALLOC_EXPORT void *	je_memalign(size_t alignment, size_t size)
-    JEMALLOC_ATTR(malloc);
-#endif
-
-#ifdef JEMALLOC_OVERRIDE_VALLOC
-JEMALLOC_EXPORT void *	je_valloc(size_t size) JEMALLOC_ATTR(malloc);
-#endif
-
-JEMALLOC_EXPORT size_t	je_malloc_usable_size(
-    JEMALLOC_USABLE_SIZE_CONST void *ptr);
-JEMALLOC_EXPORT void	je_malloc_stats_print(void (*write_cb)(void *,
-    const char *), void *je_cbopaque, const char *opts);
-JEMALLOC_EXPORT int	je_mallctl(const char *name, void *oldp,
-    size_t *oldlenp, void *newp, size_t newlen);
-JEMALLOC_EXPORT int	je_mallctlnametomib(const char *name, size_t *mibp,
+size_t	JEMALLOC_P(malloc_usable_size)(const void *ptr);
+void	JEMALLOC_P(malloc_stats_print)(void (*write_cb)(void *, const char *),
+    void *cbopaque, const char *opts);
+int	JEMALLOC_P(mallctl)(const char *name, void *oldp, size_t *oldlenp,
+    void *newp, size_t newlen);
+int	JEMALLOC_P(mallctlnametomib)(const char *name, size_t *mibp,
    size_t *miblenp);
-JEMALLOC_EXPORT int	je_mallctlbymib(const size_t *mib, size_t miblen,
-    void *oldp, size_t *oldlenp, void *newp, size_t newlen);
+int	JEMALLOC_P(mallctlbymib)(const size_t *mib, size_t miblen, void *oldp,
+    size_t *oldlenp, void *newp, size_t newlen);

-#ifdef JEMALLOC_EXPERIMENTAL
-JEMALLOC_EXPORT int	je_allocm(void **ptr, size_t *rsize, size_t size,
-    int flags) JEMALLOC_ATTR(nonnull(1));
-JEMALLOC_EXPORT int	je_rallocm(void **ptr, size_t *rsize, size_t size,
+int	JEMALLOC_P(allocm)(void **ptr, size_t *rsize, size_t size, int flags)
+    JEMALLOC_ATTR(nonnull(1));
+int	JEMALLOC_P(rallocm)(void **ptr, size_t *rsize, size_t size,
    size_t extra, int flags) JEMALLOC_ATTR(nonnull(1));
-JEMALLOC_EXPORT int	je_sallocm(const void *ptr, size_t *rsize, int flags)
+int	JEMALLOC_P(sallocm)(const void *ptr, size_t *rsize, int flags)
    JEMALLOC_ATTR(nonnull(1));
-JEMALLOC_EXPORT int	je_dallocm(void *ptr, int flags)
-    JEMALLOC_ATTR(nonnull(1));
-JEMALLOC_EXPORT int	je_nallocm(size_t *rsize, size_t size, int flags);
-#endif
-
-/*
- * By default application code must explicitly refer to mangled symbol names,
- * so that it is possible to use jemalloc in conjunction with another allocator
- * in the same application.  Define JEMALLOC_MANGLE in order to cause automatic
- * name mangling that matches the API prefixing that happened as a result of
- * --with-mangling and/or --with-jemalloc-prefix configuration settings.
- */
-#ifdef JEMALLOC_MANGLE
-#ifndef JEMALLOC_NO_DEMANGLE
-#define	JEMALLOC_NO_DEMANGLE
-#endif
-#define	malloc_conf je_malloc_conf
-#define	malloc_message je_malloc_message
-#define	malloc je_malloc
-#define	calloc je_calloc
-#define	posix_memalign je_posix_memalign
-#define	aligned_alloc je_aligned_alloc
-#define	realloc je_realloc
-#define	free je_free
-#define	malloc_usable_size je_malloc_usable_size
-#define	malloc_stats_print je_malloc_stats_print
-#define	mallctl je_mallctl
-#define	mallctlnametomib je_mallctlnametomib
-#define	mallctlbymib je_mallctlbymib
-#define	memalign je_memalign
-#define	valloc je_valloc
-#ifdef JEMALLOC_EXPERIMENTAL
-#define	allocm je_allocm
-#define	rallocm je_rallocm
-#define	sallocm je_sallocm
-#define	dallocm je_dallocm
-#define	nallocm je_nallocm
-#endif
-#endif
-
-/*
- * The je_* macros can be used as stable alternative names for the public
- * jemalloc API if JEMALLOC_NO_DEMANGLE is defined.  This is primarily meant
- * for use in jemalloc itself, but it can be used by application code to
- * provide isolation from the name mangling specified via --with-mangling
- * and/or --with-jemalloc-prefix.
- */
-#ifndef JEMALLOC_NO_DEMANGLE
-#undef je_malloc_conf
-#undef je_malloc_message
-#undef je_malloc
-#undef je_calloc
-#undef je_posix_memalign
-#undef je_aligned_alloc
-#undef je_realloc
-#undef je_free
-#undef je_malloc_usable_size
-#undef je_malloc_stats_print
-#undef je_mallctl
-#undef je_mallctlnametomib
-#undef je_mallctlbymib
-#undef je_memalign
-#undef je_valloc
-#ifdef JEMALLOC_EXPERIMENTAL
-#undef je_allocm
-#undef je_rallocm
-#undef je_sallocm
-#undef je_dallocm
-#undef je_nallocm
-#endif
-#endif
+int	JEMALLOC_P(dallocm)(void *ptr, int flags) JEMALLOC_ATTR(nonnull(1));

 #ifdef __cplusplus
 };
@@ -1,45 +1,22 @@
+#ifndef JEMALLOC_DEFS_H_
+#define	JEMALLOC_DEFS_H_
+
 /*
- * If JEMALLOC_PREFIX is defined via --with-jemalloc-prefix, it will cause all
- * public APIs to be prefixed.  This makes it possible, with some care, to use
- * multiple allocators simultaneously.
+ * If JEMALLOC_PREFIX is defined, it will cause all public APIs to be prefixed.
+ * This makes it possible, with some care, to use multiple allocators
+ * simultaneously.
+ *
+ * In many cases it is more convenient to manually prefix allocator function
+ * calls than to let macros do it automatically, particularly when using
+ * multiple allocators simultaneously.  Define JEMALLOC_MANGLE before
+ * #include'ing jemalloc.h in order to cause name mangling that corresponds to
+ * the API prefixing.
 */
 #undef JEMALLOC_PREFIX
 #undef JEMALLOC_CPREFIX
-
-/*
- * Name mangling for public symbols is controlled by --with-mangling and
- * --with-jemalloc-prefix.  With default settings the je_ prefix is stripped by
- * these macro definitions.
- */
-#undef je_malloc_conf
-#undef je_malloc_message
-#undef je_malloc
-#undef je_calloc
-#undef je_posix_memalign
-#undef je_aligned_alloc
-#undef je_realloc
-#undef je_free
-#undef je_malloc_usable_size
-#undef je_malloc_stats_print
-#undef je_mallctl
-#undef je_mallctlnametomib
-#undef je_mallctlbymib
-#undef je_memalign
-#undef je_valloc
-#undef je_allocm
-#undef je_rallocm
-#undef je_sallocm
-#undef je_dallocm
-#undef je_nallocm
-
-/*
- * JEMALLOC_PRIVATE_NAMESPACE is used as a prefix for all library-private APIs.
- * For shared libraries, symbol visibility mechanisms prevent these symbols
- * from being exported, but for static libraries, naming collisions are a real
- * possibility.
- */
-#undef JEMALLOC_PRIVATE_NAMESPACE
-#undef JEMALLOC_N
+#if (defined(JEMALLOC_PREFIX) && defined(JEMALLOC_MANGLE))
+#undef JEMALLOC_P
+#endif

 /*
 * Hyper-threaded CPUs may need a special instruction inside spin loops in
@@ -47,92 +24,26 @@
 */
 #undef CPU_SPINWAIT

-/* Defined if the equivalent of FreeBSD's atomic(9) functions are available. */
-#undef JEMALLOC_ATOMIC9
-
 /*
 * Defined if OSAtomic*() functions are available, as provided by Darwin, and
 * documented in the atomic(3) manual page.
 */
 #undef JEMALLOC_OSATOMIC

-/*
- * Defined if __sync_add_and_fetch(uint32_t *, uint32_t) and
- * __sync_sub_and_fetch(uint32_t *, uint32_t) are available, despite
- * __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 not being defined (which means the
- * functions are defined in libgcc instead of being inlines)
- */
-#undef JE_FORCE_SYNC_COMPARE_AND_SWAP_4
-
-/*
- * Defined if __sync_add_and_fetch(uint64_t *, uint64_t) and
- * __sync_sub_and_fetch(uint64_t *, uint64_t) are available, despite
- * __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 not being defined (which means the
- * functions are defined in libgcc instead of being inlines)
- */
-#undef JE_FORCE_SYNC_COMPARE_AND_SWAP_8
-
 /*
 * Defined if OSSpin*() functions are available, as provided by Darwin, and
 * documented in the spinlock(3) manual page.
 */
 #undef JEMALLOC_OSSPIN

-/*
- * Defined if _malloc_thread_cleanup() exists.  At least in the case of
- * FreeBSD, pthread_key_create() allocates, which if used during malloc
- * bootstrapping will cause recursion into the pthreads library.  Therefore, if
- * _malloc_thread_cleanup() exists, use it as the basis for thread cleanup in
- * malloc_tsd.
- */
-#undef JEMALLOC_MALLOC_THREAD_CLEANUP
-
-/*
- * Defined if threaded initialization is known to be safe on this platform.
- * Among other things, it must be possible to initialize a mutex without
- * triggering allocation in order for threaded allocation to be safe.
- */
-#undef JEMALLOC_THREADED_INIT
-
-/*
- * Defined if the pthreads implementation defines
- * _pthread_mutex_init_calloc_cb(), in which case the function is used in order
- * to avoid recursive allocation during mutex initialization.
- */
-#undef JEMALLOC_MUTEX_INIT_CB
-
 /* Defined if __attribute__((...)) syntax is supported. */
 #undef JEMALLOC_HAVE_ATTR
 #ifdef JEMALLOC_HAVE_ATTR
 #  define JEMALLOC_ATTR(s) __attribute__((s))
-#  define JEMALLOC_EXPORT JEMALLOC_ATTR(visibility("default"))
-#  define JEMALLOC_ALIGNED(s) JEMALLOC_ATTR(aligned(s))
-#  define JEMALLOC_SECTION(s) JEMALLOC_ATTR(section(s))
-#  define JEMALLOC_NOINLINE JEMALLOC_ATTR(noinline)
-#elif _MSC_VER
-#  define JEMALLOC_ATTR(s)
-#  ifdef DLLEXPORT
-#    define JEMALLOC_EXPORT __declspec(dllexport)
-#  else
-#    define JEMALLOC_EXPORT __declspec(dllimport)
-#  endif
-#  define JEMALLOC_ALIGNED(s) __declspec(align(s))
-#  define JEMALLOC_SECTION(s) __declspec(allocate(s))
-#  define JEMALLOC_NOINLINE __declspec(noinline)
 #else
 #  define JEMALLOC_ATTR(s)
-#  define JEMALLOC_EXPORT
-#  define JEMALLOC_ALIGNED(s)
-#  define JEMALLOC_SECTION(s)
-#  define JEMALLOC_NOINLINE
 #endif

-/* Defined if sbrk() is supported. */
-#undef JEMALLOC_HAVE_SBRK
-
-/* Non-empty if the tls_model attribute is supported. */
-#undef JEMALLOC_TLS_MODEL
-
 /* JEMALLOC_CC_SILENCE enables code that silences unuseful compiler warnings. */
 #undef JEMALLOC_CC_SILENCE

@@ -157,6 +68,12 @@
 /* Use gcc intrinsics for profile backtracing if defined. */
 #undef JEMALLOC_PROF_GCC

+/*
+ * JEMALLOC_TINY enables support for tiny objects, which are smaller than one
+ * quantum.
+ */
+#undef JEMALLOC_TINY
+
 /*
 * JEMALLOC_TCACHE enables a thread-specific caching layer for small objects.
 * This makes it possible to allocate/deallocate objects without any locking
@@ -170,43 +87,29 @@
 */
 #undef JEMALLOC_DSS

-/* Support memory filling (junk/zero/quarantine/redzone). */
+/* JEMALLOC_SWAP enables mmap()ed swap file support. */
+#undef JEMALLOC_SWAP
+
+/* Support memory filling (junk/zero). */
 #undef JEMALLOC_FILL

-/* Support the experimental API. */
-#undef JEMALLOC_EXPERIMENTAL
-
-/* Support utrace(2)-based tracing. */
-#undef JEMALLOC_UTRACE
-
-/* Support Valgrind. */
-#undef JEMALLOC_VALGRIND
-
 /* Support optional abort() on OOM. */
 #undef JEMALLOC_XMALLOC

+/* Support SYSV semantics. */
+#undef JEMALLOC_SYSV
+
 /* Support lazy locking (avoid locking unless a second thread is launched). */
 #undef JEMALLOC_LAZY_LOCK

+/* Determine page size at run time if defined. */
+#undef DYNAMIC_PAGE_SHIFT
+
 /* One page is 2^STATIC_PAGE_SHIFT bytes. */
 #undef STATIC_PAGE_SHIFT

-/*
- * If defined, use munmap() to unmap freed chunks, rather than storing them for
- * later reuse.  This is disabled by default on Linux because common sequences
- * of mmap()/munmap() calls will cause virtual memory map holes.
- */
-#undef JEMALLOC_MUNMAP
-
-/*
- * If defined, use mremap(...MREMAP_FIXED...) for huge realloc().  This is
- * disabled by default because it is Linux-specific and it will cause virtual
- * memory map holes, much like munmap(2) does.
- */
-#undef JEMALLOC_MREMAP
-
 /* TLS is used to map arenas and magazine caches to threads. */
-#undef JEMALLOC_TLS
+#undef NO_TLS

 /*
 * JEMALLOC_IVSALLOC enables ivsalloc(), which verifies that pointers reside
@@ -221,21 +124,15 @@
 #undef JEMALLOC_OVERRIDE_MEMALIGN
 #undef JEMALLOC_OVERRIDE_VALLOC

-/*
- * At least Linux omits the "const" in:
- *
- *   size_t malloc_usable_size(const void *ptr);
- *
- * Match the operating system's prototype.
- */
-#undef JEMALLOC_USABLE_SIZE_CONST
-
 /*
 * Darwin (OS X) uses zones to work around Mach-O symbol override shortcomings.
 */
 #undef JEMALLOC_ZONE
 #undef JEMALLOC_ZONE_VERSION

+/* If defined, use mremap(...MREMAP_FIXED...) for huge realloc(). */
+#undef JEMALLOC_MREMAP_FIXED
+
 /*
 * Methods for purging unused pages differ between operating systems.
 *
@@ -258,5 +155,4 @@
 /* sizeof(long) == 2^LG_SIZEOF_LONG. */
 #undef LG_SIZEOF_LONG

-/* sizeof(intmax_t) == 2^LG_SIZEOF_INTMAX_T. */
-#undef LG_SIZEOF_INTMAX_T
+#endif /* JEMALLOC_DEFS_H_ */
@@ -1,313 +0,0 @@
-// ISO C9x  compliant inttypes.h for Microsoft Visual Studio
-// Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124 
-// 
-//  Copyright (c) 2006 Alexander Chemeris
-// 
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are met:
-// 
-//   1. Redistributions of source code must retain the above copyright notice,
-//      this list of conditions and the following disclaimer.
-// 
-//   2. Redistributions in binary form must reproduce the above copyright
-//      notice, this list of conditions and the following disclaimer in the
-//      documentation and/or other materials provided with the distribution.
-// 
-//   3. The name of the author may be used to endorse or promote products
-//      derived from this software without specific prior written permission.
-// 
-// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
-// WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
-// EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-// OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 
-// WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-// OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-// ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-// 
-///////////////////////////////////////////////////////////////////////////////
-
-#ifndef _MSC_VER // [
-#error "Use this header only with Microsoft Visual C++ compilers!"
-#endif // _MSC_VER ]
-
-#ifndef _MSC_INTTYPES_H_ // [
-#define _MSC_INTTYPES_H_
-
-#if _MSC_VER > 1000
-#pragma once
-#endif
-
-#include "stdint.h"
-
-// 7.8 Format conversion of integer types
-
-typedef struct {
-   intmax_t quot;
-   intmax_t rem;
-} imaxdiv_t;
-
-// 7.8.1 Macros for format specifiers
-
-#if !defined(__cplusplus) || defined(__STDC_FORMAT_MACROS) // [   See footnote 185 at page 198
-
-#ifdef _WIN64
-#  define __PRI64_PREFIX        "l"
-#  define __PRIPTR_PREFIX       "l"
-#else
-#  define __PRI64_PREFIX        "ll"
-#  define __PRIPTR_PREFIX
-#endif
-
-// The fprintf macros for signed integers are:
-#define PRId8       "d"
-#define PRIi8       "i"
-#define PRIdLEAST8  "d"
-#define PRIiLEAST8  "i"
-#define PRIdFAST8   "d"
-#define PRIiFAST8   "i"
-
-#define PRId16       "hd"
-#define PRIi16       "hi"
-#define PRIdLEAST16  "hd"
-#define PRIiLEAST16  "hi"
-#define PRIdFAST16   "hd"
-#define PRIiFAST16   "hi"
-
-#define PRId32       "d"
-#define PRIi32       "i"
-#define PRIdLEAST32  "d"
-#define PRIiLEAST32  "i"
-#define PRIdFAST32   "d"
-#define PRIiFAST32   "i"
-
-#define PRId64       __PRI64_PREFIX "d"
-#define PRIi64       __PRI64_PREFIX "i"
-#define PRIdLEAST64  __PRI64_PREFIX "d"
-#define PRIiLEAST64  __PRI64_PREFIX "i"
-#define PRIdFAST64   __PRI64_PREFIX "d"
-#define PRIiFAST64   __PRI64_PREFIX "i"
-
-#define PRIdMAX     __PRI64_PREFIX "d"
-#define PRIiMAX     __PRI64_PREFIX "i"
-
-#define PRIdPTR     __PRIPTR_PREFIX "d"
-#define PRIiPTR     __PRIPTR_PREFIX "i"
-
-// The fprintf macros for unsigned integers are:
-#define PRIo8       "o"
-#define PRIu8       "u"
-#define PRIx8       "x"
-#define PRIX8       "X"
-#define PRIoLEAST8  "o"
-#define PRIuLEAST8  "u"
-#define PRIxLEAST8  "x"
-#define PRIXLEAST8  "X"
-#define PRIoFAST8   "o"
-#define PRIuFAST8   "u"
-#define PRIxFAST8   "x"
-#define PRIXFAST8   "X"
-
-#define PRIo16       "ho"
-#define PRIu16       "hu"
-#define PRIx16       "hx"
-#define PRIX16       "hX"
-#define PRIoLEAST16  "ho"
-#define PRIuLEAST16  "hu"
-#define PRIxLEAST16  "hx"
-#define PRIXLEAST16  "hX"
-#define PRIoFAST16   "ho"
-#define PRIuFAST16   "hu"
-#define PRIxFAST16   "hx"
-#define PRIXFAST16   "hX"
-
-#define PRIo32       "o"
-#define PRIu32       "u"
-#define PRIx32       "x"
-#define PRIX32       "X"
-#define PRIoLEAST32  "o"
-#define PRIuLEAST32  "u"
-#define PRIxLEAST32  "x"
-#define PRIXLEAST32  "X"
-#define PRIoFAST32   "o"
-#define PRIuFAST32   "u"
-#define PRIxFAST32   "x"
-#define PRIXFAST32   "X"
-
-#define PRIo64       __PRI64_PREFIX "o"
-#define PRIu64       __PRI64_PREFIX "u"
-#define PRIx64       __PRI64_PREFIX "x"
-#define PRIX64       __PRI64_PREFIX "X"
-#define PRIoLEAST64  __PRI64_PREFIX "o"
-#define PRIuLEAST64  __PRI64_PREFIX "u"
-#define PRIxLEAST64  __PRI64_PREFIX "x"
-#define PRIXLEAST64  __PRI64_PREFIX "X"
-#define PRIoFAST64   __PRI64_PREFIX "o"
-#define PRIuFAST64   __PRI64_PREFIX "u"
-#define PRIxFAST64   __PRI64_PREFIX "x"
-#define PRIXFAST64   __PRI64_PREFIX "X"
-
-#define PRIoMAX     __PRI64_PREFIX "o"
-#define PRIuMAX     __PRI64_PREFIX "u"
-#define PRIxMAX     __PRI64_PREFIX "x"
-#define PRIXMAX     __PRI64_PREFIX "X"
-
-#define PRIoPTR     __PRIPTR_PREFIX "o"
-#define PRIuPTR     __PRIPTR_PREFIX "u"
-#define PRIxPTR     __PRIPTR_PREFIX "x"
-#define PRIXPTR     __PRIPTR_PREFIX "X"
-
-// The fscanf macros for signed integers are:
-#define SCNd8       "d"
-#define SCNi8       "i"
-#define SCNdLEAST8  "d"
-#define SCNiLEAST8  "i"
-#define SCNdFAST8   "d"
-#define SCNiFAST8   "i"
-
-#define SCNd16       "hd"
-#define SCNi16       "hi"
-#define SCNdLEAST16  "hd"
-#define SCNiLEAST16  "hi"
-#define SCNdFAST16   "hd"
-#define SCNiFAST16   "hi"
-
-#define SCNd32       "ld"
-#define SCNi32       "li"
-#define SCNdLEAST32  "ld"
-#define SCNiLEAST32  "li"
-#define SCNdFAST32   "ld"
-#define SCNiFAST32   "li"
-
-#define SCNd64       "I64d"
-#define SCNi64       "I64i"
-#define SCNdLEAST64  "I64d"
-#define SCNiLEAST64  "I64i"
-#define SCNdFAST64   "I64d"
-#define SCNiFAST64   "I64i"
-
-#define SCNdMAX     "I64d"
-#define SCNiMAX     "I64i"
-
-#ifdef _WIN64 // [
-#  define SCNdPTR     "I64d"
-#  define SCNiPTR     "I64i"
-#else  // _WIN64 ][
-#  define SCNdPTR     "ld"
-#  define SCNiPTR     "li"
-#endif  // _WIN64 ]
-
-// The fscanf macros for unsigned integers are:
-#define SCNo8       "o"
-#define SCNu8       "u"
-#define SCNx8       "x"
-#define SCNX8       "X"
-#define SCNoLEAST8  "o"
-#define SCNuLEAST8  "u"
-#define SCNxLEAST8  "x"
-#define SCNXLEAST8  "X"
-#define SCNoFAST8   "o"
-#define SCNuFAST8   "u"
-#define SCNxFAST8   "x"
-#define SCNXFAST8   "X"
-
-#define SCNo16       "ho"
-#define SCNu16       "hu"
-#define SCNx16       "hx"
-#define SCNX16       "hX"
-#define SCNoLEAST16  "ho"
-#define SCNuLEAST16  "hu"
-#define SCNxLEAST16  "hx"
-#define SCNXLEAST16  "hX"
-#define SCNoFAST16   "ho"
-#define SCNuFAST16   "hu"
-#define SCNxFAST16   "hx"
-#define SCNXFAST16   "hX"
-
-#define SCNo32       "lo"
-#define SCNu32       "lu"
-#define SCNx32       "lx"
-#define SCNX32       "lX"
-#define SCNoLEAST32  "lo"
-#define SCNuLEAST32  "lu"
-#define SCNxLEAST32  "lx"
-#define SCNXLEAST32  "lX"
-#define SCNoFAST32   "lo"
-#define SCNuFAST32   "lu"
-#define SCNxFAST32   "lx"
-#define SCNXFAST32   "lX"
-
-#define SCNo64       "I64o"
-#define SCNu64       "I64u"
-#define SCNx64       "I64x"
-#define SCNX64       "I64X"
-#define SCNoLEAST64  "I64o"
-#define SCNuLEAST64  "I64u"
-#define SCNxLEAST64  "I64x"
-#define SCNXLEAST64  "I64X"
-#define SCNoFAST64   "I64o"
-#define SCNuFAST64   "I64u"
-#define SCNxFAST64   "I64x"
-#define SCNXFAST64   "I64X"
-
-#define SCNoMAX     "I64o"
-#define SCNuMAX     "I64u"
-#define SCNxMAX     "I64x"
-#define SCNXMAX     "I64X"
-
-#ifdef _WIN64 // [
-#  define SCNoPTR     "I64o"
-#  define SCNuPTR     "I64u"
-#  define SCNxPTR     "I64x"
-#  define SCNXPTR     "I64X"
-#else  // _WIN64 ][
-#  define SCNoPTR     "lo"
-#  define SCNuPTR     "lu"
-#  define SCNxPTR     "lx"
-#  define SCNXPTR     "lX"
-#endif  // _WIN64 ]
-
-#endif // __STDC_FORMAT_MACROS ]
-
-// 7.8.2 Functions for greatest-width integer types
-
-// 7.8.2.1 The imaxabs function
-#define imaxabs _abs64
-
-// 7.8.2.2 The imaxdiv function
-
-// This is modified version of div() function from Microsoft's div.c found
-// in %MSVC.NET%\crt\src\div.c
-#ifdef STATIC_IMAXDIV // [
-static
-#else // STATIC_IMAXDIV ][
-_inline
-#endif // STATIC_IMAXDIV ]
-imaxdiv_t __cdecl imaxdiv(intmax_t numer, intmax_t denom)
-{
-   imaxdiv_t result;
-
-   result.quot = numer / denom;
-   result.rem = numer % denom;
-
-   if (numer < 0 && result.rem > 0) {
-      // did division wrong; must fix up
-      ++result.quot;
-      result.rem -= denom;
-   }
-
-   return result;
-}
-
-// 7.8.2.3 The strtoimax and strtoumax functions
-#define strtoimax _strtoi64
-#define strtoumax _strtoui64
-
-// 7.8.2.4 The wcstoimax and wcstoumax functions
-#define wcstoimax _wcstoi64
-#define wcstoumax _wcstoui64
-
-
-#endif // _MSC_INTTYPES_H_ ]
@@ -1,16 +0,0 @@
-#ifndef stdbool_h
-#define stdbool_h
-
-#include <wtypes.h>
-
-/* MSVC doesn't define _Bool or bool in C, but does have BOOL */
-/* Note this doesn't pass autoconf's test because (bool) 0.5 != true */
-typedef BOOL _Bool;
-
-#define bool _Bool
-#define true 1
-#define false 0
-
-#define __bool_true_false_are_defined 1
-
-#endif /* stdbool_h */
@@ -1,247 +0,0 @@
-// ISO C9x  compliant stdint.h for Microsoft Visual Studio
-// Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124 
-// 
-//  Copyright (c) 2006-2008 Alexander Chemeris
-// 
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are met:
-// 
-//   1. Redistributions of source code must retain the above copyright notice,
-//      this list of conditions and the following disclaimer.
-// 
-//   2. Redistributions in binary form must reproduce the above copyright
-//      notice, this list of conditions and the following disclaimer in the
-//      documentation and/or other materials provided with the distribution.
-// 
-//   3. The name of the author may be used to endorse or promote products
-//      derived from this software without specific prior written permission.
-// 
-// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
-// WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
-// EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-// OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 
-// WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-// OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-// ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-// 
-///////////////////////////////////////////////////////////////////////////////
-
-#ifndef _MSC_VER // [
-#error "Use this header only with Microsoft Visual C++ compilers!"
-#endif // _MSC_VER ]
-
-#ifndef _MSC_STDINT_H_ // [
-#define _MSC_STDINT_H_
-
-#if _MSC_VER > 1000
-#pragma once
-#endif
-
-#include <limits.h>
-
-// For Visual Studio 6 in C++ mode and for many Visual Studio versions when
-// compiling for ARM we should wrap <wchar.h> include with 'extern "C++" {}'
-// or compiler give many errors like this:
-//   error C2733: second C linkage of overloaded function 'wmemchr' not allowed
-#ifdef __cplusplus
-extern "C" {
-#endif
-#  include <wchar.h>
-#ifdef __cplusplus
-}
-#endif
-
-// Define _W64 macros to mark types changing their size, like intptr_t.
-#ifndef _W64
-#  if !defined(__midl) && (defined(_X86_) || defined(_M_IX86)) && _MSC_VER >= 1300
-#     define _W64 __w64
-#  else
-#     define _W64
-#  endif
-#endif
-
-
-// 7.18.1 Integer types
-
-// 7.18.1.1 Exact-width integer types
-
-// Visual Studio 6 and Embedded Visual C++ 4 doesn't
-// realize that, e.g. char has the same size as __int8
-// so we give up on __intX for them.
-#if (_MSC_VER < 1300)
-   typedef signed char       int8_t;
-   typedef signed short      int16_t;
-   typedef signed int        int32_t;
-   typedef unsigned char     uint8_t;
-   typedef unsigned short    uint16_t;
-   typedef unsigned int      uint32_t;
-#else
-   typedef signed __int8     int8_t;
-   typedef signed __int16    int16_t;
-   typedef signed __int32    int32_t;
-   typedef unsigned __int8   uint8_t;
-   typedef unsigned __int16  uint16_t;
-   typedef unsigned __int32  uint32_t;
-#endif
-typedef signed __int64       int64_t;
-typedef unsigned __int64     uint64_t;
-
-
-// 7.18.1.2 Minimum-width integer types
-typedef int8_t    int_least8_t;
-typedef int16_t   int_least16_t;
-typedef int32_t   int_least32_t;
-typedef int64_t   int_least64_t;
-typedef uint8_t   uint_least8_t;
-typedef uint16_t  uint_least16_t;
-typedef uint32_t  uint_least32_t;
-typedef uint64_t  uint_least64_t;
-
-// 7.18.1.3 Fastest minimum-width integer types
-typedef int8_t    int_fast8_t;
-typedef int16_t   int_fast16_t;
-typedef int32_t   int_fast32_t;
-typedef int64_t   int_fast64_t;
-typedef uint8_t   uint_fast8_t;
-typedef uint16_t  uint_fast16_t;
-typedef uint32_t  uint_fast32_t;
-typedef uint64_t  uint_fast64_t;
-
-// 7.18.1.4 Integer types capable of holding object pointers
-#ifdef _WIN64 // [
-   typedef signed __int64    intptr_t;
-   typedef unsigned __int64  uintptr_t;
-#else // _WIN64 ][
-   typedef _W64 signed int   intptr_t;
-   typedef _W64 unsigned int uintptr_t;
-#endif // _WIN64 ]
-
-// 7.18.1.5 Greatest-width integer types
-typedef int64_t   intmax_t;
-typedef uint64_t  uintmax_t;
-
-
-// 7.18.2 Limits of specified-width integer types
-
-#if !defined(__cplusplus) || defined(__STDC_LIMIT_MACROS) // [   See footnote 220 at page 257 and footnote 221 at page 259
-
-// 7.18.2.1 Limits of exact-width integer types
-#define INT8_MIN     ((int8_t)_I8_MIN)
-#define INT8_MAX     _I8_MAX
-#define INT16_MIN    ((int16_t)_I16_MIN)
-#define INT16_MAX    _I16_MAX
-#define INT32_MIN    ((int32_t)_I32_MIN)
-#define INT32_MAX    _I32_MAX
-#define INT64_MIN    ((int64_t)_I64_MIN)
-#define INT64_MAX    _I64_MAX
-#define UINT8_MAX    _UI8_MAX
-#define UINT16_MAX   _UI16_MAX
-#define UINT32_MAX   _UI32_MAX
-#define UINT64_MAX   _UI64_MAX
-
-// 7.18.2.2 Limits of minimum-width integer types
-#define INT_LEAST8_MIN    INT8_MIN
-#define INT_LEAST8_MAX    INT8_MAX
-#define INT_LEAST16_MIN   INT16_MIN
-#define INT_LEAST16_MAX   INT16_MAX
-#define INT_LEAST32_MIN   INT32_MIN
-#define INT_LEAST32_MAX   INT32_MAX
-#define INT_LEAST64_MIN   INT64_MIN
-#define INT_LEAST64_MAX   INT64_MAX
-#define UINT_LEAST8_MAX   UINT8_MAX
-#define UINT_LEAST16_MAX  UINT16_MAX
-#define UINT_LEAST32_MAX  UINT32_MAX
-#define UINT_LEAST64_MAX  UINT64_MAX
-
-// 7.18.2.3 Limits of fastest minimum-width integer types
-#define INT_FAST8_MIN    INT8_MIN
-#define INT_FAST8_MAX    INT8_MAX
-#define INT_FAST16_MIN   INT16_MIN
-#define INT_FAST16_MAX   INT16_MAX
-#define INT_FAST32_MIN   INT32_MIN
-#define INT_FAST32_MAX   INT32_MAX
-#define INT_FAST64_MIN   INT64_MIN
-#define INT_FAST64_MAX   INT64_MAX
-#define UINT_FAST8_MAX   UINT8_MAX
-#define UINT_FAST16_MAX  UINT16_MAX
-#define UINT_FAST32_MAX  UINT32_MAX
-#define UINT_FAST64_MAX  UINT64_MAX
-
-// 7.18.2.4 Limits of integer types capable of holding object pointers
-#ifdef _WIN64 // [
-#  define INTPTR_MIN   INT64_MIN
-#  define INTPTR_MAX   INT64_MAX
-#  define UINTPTR_MAX  UINT64_MAX
-#else // _WIN64 ][
-#  define INTPTR_MIN   INT32_MIN
-#  define INTPTR_MAX   INT32_MAX
-#  define UINTPTR_MAX  UINT32_MAX
-#endif // _WIN64 ]
-
-// 7.18.2.5 Limits of greatest-width integer types
-#define INTMAX_MIN   INT64_MIN
-#define INTMAX_MAX   INT64_MAX
-#define UINTMAX_MAX  UINT64_MAX
-
-// 7.18.3 Limits of other integer types
-
-#ifdef _WIN64 // [
-#  define PTRDIFF_MIN  _I64_MIN
-#  define PTRDIFF_MAX  _I64_MAX
-#else  // _WIN64 ][
-#  define PTRDIFF_MIN  _I32_MIN
-#  define PTRDIFF_MAX  _I32_MAX
-#endif  // _WIN64 ]
-
-#define SIG_ATOMIC_MIN  INT_MIN
-#define SIG_ATOMIC_MAX  INT_MAX
-
-#ifndef SIZE_MAX // [
-#  ifdef _WIN64 // [
-#     define SIZE_MAX  _UI64_MAX
-#  else // _WIN64 ][
-#     define SIZE_MAX  _UI32_MAX
-#  endif // _WIN64 ]
-#endif // SIZE_MAX ]
-
-// WCHAR_MIN and WCHAR_MAX are also defined in <wchar.h>
-#ifndef WCHAR_MIN // [
-#  define WCHAR_MIN  0
-#endif  // WCHAR_MIN ]
-#ifndef WCHAR_MAX // [
-#  define WCHAR_MAX  _UI16_MAX
-#endif  // WCHAR_MAX ]
-
-#define WINT_MIN  0
-#define WINT_MAX  _UI16_MAX
-
-#endif // __STDC_LIMIT_MACROS ]
-
-
-// 7.18.4 Limits of other integer types
-
-#if !defined(__cplusplus) || defined(__STDC_CONSTANT_MACROS) // [   See footnote 224 at page 260
-
-// 7.18.4.1 Macros for minimum-width integer constants
-
-#define INT8_C(val)  val##i8
-#define INT16_C(val) val##i16
-#define INT32_C(val) val##i32
-#define INT64_C(val) val##i64
-
-#define UINT8_C(val)  val##ui8
-#define UINT16_C(val) val##ui16
-#define UINT32_C(val) val##ui32
-#define UINT64_C(val) val##ui64
-
-// 7.18.4.2 Macros for greatest-width integer constants
-#define INTMAX_C   INT64_C
-#define UINTMAX_C  UINT64_C
-
-#endif // __STDC_CONSTANT_MACROS ]
-
-
-#endif // _MSC_STDINT_H_ ]
@@ -1,23 +0,0 @@
-#ifndef strings_h
-#define strings_h
-
-/* MSVC doesn't define ffs/ffsl. This dummy strings.h header is provided
- * for both */
-#include <intrin.h>
-#pragma intrinsic(_BitScanForward)
-static __forceinline int ffsl(long x)
-{
-	unsigned long i;
-
-	if (_BitScanForward(&i, x))
-		return (i + 1);
-	return (0);
-}
-
-static __forceinline int ffs(int x)
-{
-
-	return (ffsl(x));
-}
-
-#endif
@@ -4,7 +4,7 @@
 /******************************************************************************/
 /* Data. */

-static malloc_mutex_t	base_mtx;
+malloc_mutex_t	base_mtx;

 /*
 * Current pages that are being used for internal memory allocations.  These
@@ -32,8 +32,7 @@ base_pages_alloc(size_t minsize)
 	assert(minsize != 0);
 	csize = CHUNK_CEILING(minsize);
 	zero = false;
-	base_pages = chunk_alloc(csize, chunksize, true, &zero,
-	    chunk_dss_prec_get());
+	base_pages = chunk_alloc(csize, true, &zero);
 	if (base_pages == NULL)
 		return (true);
 	base_next_addr = base_pages;
@@ -67,17 +66,6 @@ base_alloc(size_t size)
 	return (ret);
 }

-void *
-base_calloc(size_t number, size_t size)
-{
-	void *ret = base_alloc(number * size);
-
-	if (ret != NULL)
-		memset(ret, 0, number * size);
-
-	return (ret);
-}
-
 extent_node_t *
 base_node_alloc(void)
 {
@@ -116,24 +104,3 @@ base_boot(void)

 	return (false);
 }
-
-void
-base_prefork(void)
-{
-
-	malloc_mutex_prefork(&base_mtx);
-}
-
-void
-base_postfork_parent(void)
-{
-
-	malloc_mutex_postfork_parent(&base_mtx);
-}
-
-void
-base_postfork_child(void)
-{
-
-	malloc_mutex_postfork_child(&base_mtx);
-}
@@ -4,24 +4,19 @@
 /******************************************************************************/
 /* Data. */

-const char	*opt_dss = DSS_DEFAULT;
-size_t		opt_lg_chunk = LG_CHUNK_DEFAULT;
+size_t	opt_lg_chunk = LG_CHUNK_DEFAULT;
+#ifdef JEMALLOC_SWAP
+bool	opt_overcommit = true;
+#endif

+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
 malloc_mutex_t	chunks_mtx;
 chunk_stats_t	stats_chunks;
+#endif

-/*
- * Trees of chunks that were previously allocated (trees differ only in node
- * ordering).  These are used when allocating chunks, in an attempt to re-use
- * address space.  Depending on function, different tree orderings are needed,
- * which is why there are two trees with the same contents.
- */
-static extent_tree_t	chunks_szad_mmap;
-static extent_tree_t	chunks_ad_mmap;
-static extent_tree_t	chunks_szad_dss;
-static extent_tree_t	chunks_ad_dss;
-
+#ifdef JEMALLOC_IVSALLOC
 rtree_t		*chunks_rtree;
+#endif

 /* Various chunk-related settings. */
 size_t		chunksize;
@@ -31,103 +26,6 @@ size_t		map_bias;
 size_t		arena_maxclass; /* Max size class for arenas. */

 /******************************************************************************/
-/* Function prototypes for non-inline static functions. */
-
-static void	*chunk_recycle(extent_tree_t *chunks_szad,
-    extent_tree_t *chunks_ad, size_t size, size_t alignment, bool base,
-    bool *zero);
-static void	chunk_record(extent_tree_t *chunks_szad,
-    extent_tree_t *chunks_ad, void *chunk, size_t size);
-
-/******************************************************************************/
-
-static void *
-chunk_recycle(extent_tree_t *chunks_szad, extent_tree_t *chunks_ad, size_t size,
-    size_t alignment, bool base, bool *zero)
-{
-	void *ret;
-	extent_node_t *node;
-	extent_node_t key;
-	size_t alloc_size, leadsize, trailsize;
-	bool zeroed;
-
-	if (base) {
-		/*
-		 * This function may need to call base_node_{,de}alloc(), but
-		 * the current chunk allocation request is on behalf of the
-		 * base allocator.  Avoid deadlock (and if that weren't an
-		 * issue, potential for infinite recursion) by returning NULL.
-		 */
-		return (NULL);
-	}
-
-	alloc_size = size + alignment - chunksize;
-	/* Beware size_t wrap-around. */
-	if (alloc_size < size)
-		return (NULL);
-	key.addr = NULL;
-	key.size = alloc_size;
-	malloc_mutex_lock(&chunks_mtx);
-	node = extent_tree_szad_nsearch(chunks_szad, &key);
-	if (node == NULL) {
-		malloc_mutex_unlock(&chunks_mtx);
-		return (NULL);
-	}
-	leadsize = ALIGNMENT_CEILING((uintptr_t)node->addr, alignment) -
-	    (uintptr_t)node->addr;
-	assert(node->size >= leadsize + size);
-	trailsize = node->size - leadsize - size;
-	ret = (void *)((uintptr_t)node->addr + leadsize);
-	/* Remove node from the tree. */
-	extent_tree_szad_remove(chunks_szad, node);
-	extent_tree_ad_remove(chunks_ad, node);
-	if (leadsize != 0) {
-		/* Insert the leading space as a smaller chunk. */
-		node->size = leadsize;
-		extent_tree_szad_insert(chunks_szad, node);
-		extent_tree_ad_insert(chunks_ad, node);
-		node = NULL;
-	}
-	if (trailsize != 0) {
-		/* Insert the trailing space as a smaller chunk. */
-		if (node == NULL) {
-			/*
-			 * An additional node is required, but
-			 * base_node_alloc() can cause a new base chunk to be
-			 * allocated.  Drop chunks_mtx in order to avoid
-			 * deadlock, and if node allocation fails, deallocate
-			 * the result before returning an error.
-			 */
-			malloc_mutex_unlock(&chunks_mtx);
-			node = base_node_alloc();
-			if (node == NULL) {
-				chunk_dealloc(ret, size, true);
-				return (NULL);
-			}
-			malloc_mutex_lock(&chunks_mtx);
-		}
-		node->addr = (void *)((uintptr_t)(ret) + size);
-		node->size = trailsize;
-		extent_tree_szad_insert(chunks_szad, node);
-		extent_tree_ad_insert(chunks_ad, node);
-		node = NULL;
-	}
-	malloc_mutex_unlock(&chunks_mtx);
-
-	zeroed = false;
-	if (node != NULL) {
-		if (node->zeroed) {
-			zeroed = true;
-			*zero = true;
-		}
-		base_node_dealloc(node);
-	}
-	if (zeroed == false && *zero) {
-		VALGRIND_MAKE_MEM_UNDEFINED(ret, size);
-		memset(ret, 0, size);
-	}
-	return (ret);
-}

 /*
 * If the caller specifies (*zero == false), it is still possible to receive
@@ -136,187 +34,106 @@ chunk_recycle(extent_tree_t *chunks_szad, extent_tree_t *chunks_ad, size_t size,
 * advantage of them if they are returned.
 */
 void *
-chunk_alloc(size_t size, size_t alignment, bool base, bool *zero,
-    dss_prec_t dss_prec)
+chunk_alloc(size_t size, bool base, bool *zero)
 {
 	void *ret;

 	assert(size != 0);
 	assert((size & chunksize_mask) == 0);
-	assert(alignment != 0);
-	assert((alignment & chunksize_mask) == 0);

-	/* "primary" dss. */
-	if (config_dss && dss_prec == dss_prec_primary) {
-		if ((ret = chunk_recycle(&chunks_szad_dss, &chunks_ad_dss, size,
-		    alignment, base, zero)) != NULL)
-			goto label_return;
-		if ((ret = chunk_alloc_dss(size, alignment, zero)) != NULL)
-			goto label_return;
+#ifdef JEMALLOC_SWAP
+	if (swap_enabled) {
+		ret = chunk_alloc_swap(size, zero);
+		if (ret != NULL)
+			goto RETURN;
 	}
-	/* mmap. */
-	if ((ret = chunk_recycle(&chunks_szad_mmap, &chunks_ad_mmap, size,
-	    alignment, base, zero)) != NULL)
-		goto label_return;
-	if ((ret = chunk_alloc_mmap(size, alignment, zero)) != NULL)
-		goto label_return;
-	/* "secondary" dss. */
-	if (config_dss && dss_prec == dss_prec_secondary) {
-		if ((ret = chunk_recycle(&chunks_szad_dss, &chunks_ad_dss, size,
-		    alignment, base, zero)) != NULL)
-			goto label_return;
-		if ((ret = chunk_alloc_dss(size, alignment, zero)) != NULL)
-			goto label_return;
+
+	if (swap_enabled == false || opt_overcommit) {
+#endif
+#ifdef JEMALLOC_DSS
+		ret = chunk_alloc_dss(size, zero);
+		if (ret != NULL)
+			goto RETURN;
+#endif
+		ret = chunk_alloc_mmap(size);
+		if (ret != NULL) {
+			*zero = true;
+			goto RETURN;
+		}
+#ifdef JEMALLOC_SWAP
 	}
+#endif

 	/* All strategies for allocation failed. */
 	ret = NULL;
-label_return:
-	if (config_ivsalloc && base == false && ret != NULL) {
+RETURN:
+#ifdef JEMALLOC_IVSALLOC
+	if (base == false && ret != NULL) {
 		if (rtree_set(chunks_rtree, (uintptr_t)ret, ret)) {
-			chunk_dealloc(ret, size, true);
+			chunk_dealloc(ret, size);
 			return (NULL);
 		}
 	}
-	if ((config_stats || config_prof) && ret != NULL) {
+#endif
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+	if (ret != NULL) {
+#  ifdef JEMALLOC_PROF
 		bool gdump;
+#  endif
 		malloc_mutex_lock(&chunks_mtx);
-		if (config_stats)
-			stats_chunks.nchunks += (size / chunksize);
+#  ifdef JEMALLOC_STATS
+		stats_chunks.nchunks += (size / chunksize);
+#  endif
 		stats_chunks.curchunks += (size / chunksize);
 		if (stats_chunks.curchunks > stats_chunks.highchunks) {
 			stats_chunks.highchunks = stats_chunks.curchunks;
-			if (config_prof)
-				gdump = true;
-		} else if (config_prof)
+#  ifdef JEMALLOC_PROF
+			gdump = true;
+#  endif
+		}
+#  ifdef JEMALLOC_PROF
+		else
 			gdump = false;
+#  endif
 		malloc_mutex_unlock(&chunks_mtx);
-		if (config_prof && opt_prof && opt_prof_gdump && gdump)
+#  ifdef JEMALLOC_PROF
+		if (opt_prof && opt_prof_gdump && gdump)
 			prof_gdump();
+#  endif
 	}
-	if (config_debug && *zero && ret != NULL) {
-		size_t i;
-		size_t *p = (size_t *)(uintptr_t)ret;
+#endif

-		VALGRIND_MAKE_MEM_DEFINED(ret, size);
-		for (i = 0; i < size / sizeof(size_t); i++)
-			assert(p[i] == 0);
-	}
 	assert(CHUNK_ADDR2BASE(ret) == ret);
 	return (ret);
 }

-static void
-chunk_record(extent_tree_t *chunks_szad, extent_tree_t *chunks_ad, void *chunk,
-    size_t size)
+void
+chunk_dealloc(void *chunk, size_t size)
 {
-	bool unzeroed;
-	extent_node_t *xnode, *node, *prev, key;

-	unzeroed = pages_purge(chunk, size);
-
-	/*
-	 * Allocate a node before acquiring chunks_mtx even though it might not
-	 * be needed, because base_node_alloc() may cause a new base chunk to
-	 * be allocated, which could cause deadlock if chunks_mtx were already
-	 * held.
-	 */
-	xnode = base_node_alloc();
+	assert(chunk != NULL);
+	assert(CHUNK_ADDR2BASE(chunk) == chunk);
+	assert(size != 0);
+	assert((size & chunksize_mask) == 0);

+#ifdef JEMALLOC_IVSALLOC
+	rtree_set(chunks_rtree, (uintptr_t)chunk, NULL);
+#endif
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
 	malloc_mutex_lock(&chunks_mtx);
-	key.addr = (void *)((uintptr_t)chunk + size);
-	node = extent_tree_ad_nsearch(chunks_ad, &key);
-	/* Try to coalesce forward. */
-	if (node != NULL && node->addr == key.addr) {
-		/*
-		 * Coalesce chunk with the following address range.  This does
-		 * not change the position within chunks_ad, so only
-		 * remove/insert from/into chunks_szad.
-		 */
-		extent_tree_szad_remove(chunks_szad, node);
-		node->addr = chunk;
-		node->size += size;
-		node->zeroed = (node->zeroed && (unzeroed == false));
-		extent_tree_szad_insert(chunks_szad, node);
-		if (xnode != NULL)
-			base_node_dealloc(xnode);
-	} else {
-		/* Coalescing forward failed, so insert a new node. */
-		if (xnode == NULL) {
-			/*
-			 * base_node_alloc() failed, which is an exceedingly
-			 * unlikely failure.  Leak chunk; its pages have
-			 * already been purged, so this is only a virtual
-			 * memory leak.
-			 */
-			malloc_mutex_unlock(&chunks_mtx);
-			return;
-		}
-		node = xnode;
-		node->addr = chunk;
-		node->size = size;
-		node->zeroed = (unzeroed == false);
-		extent_tree_ad_insert(chunks_ad, node);
-		extent_tree_szad_insert(chunks_szad, node);
-	}
-
-	/* Try to coalesce backward. */
-	prev = extent_tree_ad_prev(chunks_ad, node);
-	if (prev != NULL && (void *)((uintptr_t)prev->addr + prev->size) ==
-	    chunk) {
-		/*
-		 * Coalesce chunk with the previous address range.  This does
-		 * not change the position within chunks_ad, so only
-		 * remove/insert node from/into chunks_szad.
-		 */
-		extent_tree_szad_remove(chunks_szad, prev);
-		extent_tree_ad_remove(chunks_ad, prev);
-
-		extent_tree_szad_remove(chunks_szad, node);
-		node->addr = prev->addr;
-		node->size += prev->size;
-		node->zeroed = (node->zeroed && prev->zeroed);
-		extent_tree_szad_insert(chunks_szad, node);
-
-		base_node_dealloc(prev);
-	}
+	stats_chunks.curchunks -= (size / chunksize);
 	malloc_mutex_unlock(&chunks_mtx);
-}
+#endif

-void
-chunk_unmap(void *chunk, size_t size)
-{
-	assert(chunk != NULL);
-	assert(CHUNK_ADDR2BASE(chunk) == chunk);
-	assert(size != 0);
-	assert((size & chunksize_mask) == 0);
-
-	if (config_dss && chunk_in_dss(chunk))
-		chunk_record(&chunks_szad_dss, &chunks_ad_dss, chunk, size);
-	else if (chunk_dealloc_mmap(chunk, size))
-		chunk_record(&chunks_szad_mmap, &chunks_ad_mmap, chunk, size);
-}
-
-void
-chunk_dealloc(void *chunk, size_t size, bool unmap)
-{
-
-	assert(chunk != NULL);
-	assert(CHUNK_ADDR2BASE(chunk) == chunk);
-	assert(size != 0);
-	assert((size & chunksize_mask) == 0);
-
-	if (config_ivsalloc)
-		rtree_set(chunks_rtree, (uintptr_t)chunk, NULL);
-	if (config_stats || config_prof) {
-		malloc_mutex_lock(&chunks_mtx);
-		assert(stats_chunks.curchunks >= (size / chunksize));
-		stats_chunks.curchunks -= (size / chunksize);
-		malloc_mutex_unlock(&chunks_mtx);
-	}
-
-	if (unmap)
-		chunk_unmap(chunk, size);
+#ifdef JEMALLOC_SWAP
+	if (swap_enabled && chunk_dealloc_swap(chunk, size) == false)
+		return;
+#endif
+#ifdef JEMALLOC_DSS
+	if (chunk_dealloc_dss(chunk, size) == false)
+		return;
+#endif
+	chunk_dealloc_mmap(chunk, size);
 }

 bool
@@ -325,57 +142,30 @@ chunk_boot(void)

 	/* Set variables according to the value of opt_lg_chunk. */
 	chunksize = (ZU(1) << opt_lg_chunk);
-	assert(chunksize >= PAGE);
+	assert(chunksize >= PAGE_SIZE);
 	chunksize_mask = chunksize - 1;
-	chunk_npages = (chunksize >> LG_PAGE);
+	chunk_npages = (chunksize >> PAGE_SHIFT);

-	if (config_stats || config_prof) {
-		if (malloc_mutex_init(&chunks_mtx))
-			return (true);
-		memset(&stats_chunks, 0, sizeof(chunk_stats_t));
-	}
-	if (config_dss && chunk_dss_boot())
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+	if (malloc_mutex_init(&chunks_mtx))
 		return (true);
-	extent_tree_szad_new(&chunks_szad_mmap);
-	extent_tree_ad_new(&chunks_ad_mmap);
-	extent_tree_szad_new(&chunks_szad_dss);
-	extent_tree_ad_new(&chunks_ad_dss);
-	if (config_ivsalloc) {
-		chunks_rtree = rtree_new((ZU(1) << (LG_SIZEOF_PTR+3)) -
-		    opt_lg_chunk);
-		if (chunks_rtree == NULL)
-			return (true);
-	}
+	memset(&stats_chunks, 0, sizeof(chunk_stats_t));
+#endif
+#ifdef JEMALLOC_SWAP
+	if (chunk_swap_boot())
+		return (true);
+#endif
+	if (chunk_mmap_boot())
+		return (true);
+#ifdef JEMALLOC_DSS
+	if (chunk_dss_boot())
+		return (true);
+#endif
+#ifdef JEMALLOC_IVSALLOC
+	chunks_rtree = rtree_new((ZU(1) << (LG_SIZEOF_PTR+3)) - opt_lg_chunk);
+	if (chunks_rtree == NULL)
+		return (true);
+#endif

 	return (false);
 }
-
-void
-chunk_prefork(void)
-{
-
-	malloc_mutex_lock(&chunks_mtx);
-	if (config_ivsalloc)
-		rtree_prefork(chunks_rtree);
-	chunk_dss_prefork();
-}
-
-void
-chunk_postfork_parent(void)
-{
-
-	chunk_dss_postfork_parent();
-	if (config_ivsalloc)
-		rtree_postfork_parent(chunks_rtree);
-	malloc_mutex_postfork_parent(&chunks_mtx);
-}
-
-void
-chunk_postfork_child(void)
-{
-
-	chunk_dss_postfork_child();
-	if (config_ivsalloc)
-		rtree_postfork_child(chunks_rtree);
-	malloc_mutex_postfork_child(&chunks_mtx);
-}
@@ -1,77 +1,82 @@
 #define	JEMALLOC_CHUNK_DSS_C_
 #include "jemalloc/internal/jemalloc_internal.h"
+#ifdef JEMALLOC_DSS
 /******************************************************************************/
 /* Data. */

-const char	*dss_prec_names[] = {
-	"disabled",
-	"primary",
-	"secondary",
-	"N/A"
-};
-
-/* Current dss precedence default, used when creating new arenas. */
-static dss_prec_t	dss_prec_default = DSS_PREC_DEFAULT;
-
-/*
- * Protects sbrk() calls.  This avoids malloc races among threads, though it
- * does not protect against races with threads that call sbrk() directly.
- */
-static malloc_mutex_t	dss_mtx;
+malloc_mutex_t	dss_mtx;

 /* Base address of the DSS. */
-static void		*dss_base;
+static void	*dss_base;
 /* Current end of the DSS, or ((void *)-1) if the DSS is exhausted. */
-static void		*dss_prev;
+static void	*dss_prev;
 /* Current upper limit on DSS addresses. */
-static void		*dss_max;
+static void	*dss_max;
+
+/*
+ * Trees of chunks that were previously allocated (trees differ only in node
+ * ordering).  These are used when allocating chunks, in an attempt to re-use
+ * address space.  Depending on function, different tree orderings are needed,
+ * which is why there are two trees with the same contents.
+ */
+static extent_tree_t	dss_chunks_szad;
+static extent_tree_t	dss_chunks_ad;
+
+/******************************************************************************/
+/* Function prototypes for non-inline static functions. */
+
+static void	*chunk_recycle_dss(size_t size, bool *zero);
+static extent_node_t *chunk_dealloc_dss_record(void *chunk, size_t size);

 /******************************************************************************/

-#ifndef JEMALLOC_HAVE_SBRK
 static void *
-sbrk(intptr_t increment)
+chunk_recycle_dss(size_t size, bool *zero)
 {
+	extent_node_t *node, key;

-	not_implemented();
+	key.addr = NULL;
+	key.size = size;
+	malloc_mutex_lock(&dss_mtx);
+	node = extent_tree_szad_nsearch(&dss_chunks_szad, &key);
+	if (node != NULL) {
+		void *ret = node->addr;
+
+		/* Remove node from the tree. */
+		extent_tree_szad_remove(&dss_chunks_szad, node);
+		if (node->size == size) {
+			extent_tree_ad_remove(&dss_chunks_ad, node);
+			base_node_dealloc(node);
+		} else {
+			/*
+			 * Insert the remainder of node's address range as a
+			 * smaller chunk.  Its position within dss_chunks_ad
+			 * does not change.
+			 */
+			assert(node->size > size);
+			node->addr = (void *)((uintptr_t)node->addr + size);
+			node->size -= size;
+			extent_tree_szad_insert(&dss_chunks_szad, node);
+		}
+		malloc_mutex_unlock(&dss_mtx);
+
+		if (*zero)
+			memset(ret, 0, size);
+		return (ret);
+	}
+	malloc_mutex_unlock(&dss_mtx);

 	return (NULL);
 }
-#endif
-
-dss_prec_t
-chunk_dss_prec_get(void)
-{
-	dss_prec_t ret;
-
-	if (config_dss == false)
-		return (dss_prec_disabled);
-	malloc_mutex_lock(&dss_mtx);
-	ret = dss_prec_default;
-	malloc_mutex_unlock(&dss_mtx);
-	return (ret);
-}
-
-bool
-chunk_dss_prec_set(dss_prec_t dss_prec)
-{
-
-	if (config_dss == false)
-		return (true);
-	malloc_mutex_lock(&dss_mtx);
-	dss_prec_default = dss_prec;
-	malloc_mutex_unlock(&dss_mtx);
-	return (false);
-}

 void *
-chunk_alloc_dss(size_t size, size_t alignment, bool *zero)
+chunk_alloc_dss(size_t size, bool *zero)
 {
 	void *ret;

-	cassert(config_dss);
-	assert(size > 0 && (size & chunksize_mask) == 0);
-	assert(alignment > 0 && (alignment & chunksize_mask) == 0);
+	ret = chunk_recycle_dss(size, zero);
+	if (ret != NULL)
+		return (ret);

 	/*
 	 * sbrk() uses a signed increment argument, so take care not to
@@ -82,8 +87,6 @@ chunk_alloc_dss(size_t size, size_t alignment, bool *zero)

 	malloc_mutex_lock(&dss_mtx);
 	if (dss_prev != (void *)-1) {
-		size_t gap_size, cpad_size;
-		void *cpad, *dss_next;
 		intptr_t incr;

 		/*
@@ -94,40 +97,26 @@ chunk_alloc_dss(size_t size, size_t alignment, bool *zero)
 		do {
 			/* Get the current end of the DSS. */
 			dss_max = sbrk(0);
+
 			/*
 			 * Calculate how much padding is necessary to
 			 * chunk-align the end of the DSS.
 			 */
-			gap_size = (chunksize - CHUNK_ADDR2OFFSET(dss_max)) &
-			    chunksize_mask;
-			/*
-			 * Compute how much chunk-aligned pad space (if any) is
-			 * necessary to satisfy alignment.  This space can be
-			 * recycled for later use.
-			 */
-			cpad = (void *)((uintptr_t)dss_max + gap_size);
-			ret = (void *)ALIGNMENT_CEILING((uintptr_t)dss_max,
-			    alignment);
-			cpad_size = (uintptr_t)ret - (uintptr_t)cpad;
-			dss_next = (void *)((uintptr_t)ret + size);
-			if ((uintptr_t)ret < (uintptr_t)dss_max ||
-			    (uintptr_t)dss_next < (uintptr_t)dss_max) {
-				/* Wrap-around. */
-				malloc_mutex_unlock(&dss_mtx);
-				return (NULL);
+			incr = (intptr_t)size
+			    - (intptr_t)CHUNK_ADDR2OFFSET(dss_max);
+			if (incr == (intptr_t)size)
+				ret = dss_max;
+			else {
+				ret = (void *)((intptr_t)dss_max + incr);
+				incr += size;
 			}
-			incr = gap_size + cpad_size + size;
+
 			dss_prev = sbrk(incr);
 			if (dss_prev == dss_max) {
 				/* Success. */
-				dss_max = dss_next;
+				dss_max = (void *)((intptr_t)dss_prev + incr);
 				malloc_mutex_unlock(&dss_mtx);
-				if (cpad_size != 0)
-					chunk_unmap(cpad, cpad_size);
-				if (*zero) {
-					VALGRIND_MAKE_MEM_UNDEFINED(ret, size);
-					memset(ret, 0, size);
-				}
+				*zero = true;
 				return (ret);
 			}
 		} while (dss_prev != (void *)-1);
@@ -137,13 +126,84 @@ chunk_alloc_dss(size_t size, size_t alignment, bool *zero)
 	return (NULL);
 }

+static extent_node_t *
+chunk_dealloc_dss_record(void *chunk, size_t size)
+{
+	extent_node_t *xnode, *node, *prev, key;
+
+	xnode = NULL;
+	while (true) {
+		key.addr = (void *)((uintptr_t)chunk + size);
+		node = extent_tree_ad_nsearch(&dss_chunks_ad, &key);
+		/* Try to coalesce forward. */
+		if (node != NULL && node->addr == key.addr) {
+			/*
+			 * Coalesce chunk with the following address range.
+			 * This does not change the position within
+			 * dss_chunks_ad, so only remove/insert from/into
+			 * dss_chunks_szad.
+			 */
+			extent_tree_szad_remove(&dss_chunks_szad, node);
+			node->addr = chunk;
+			node->size += size;
+			extent_tree_szad_insert(&dss_chunks_szad, node);
+			break;
+		} else if (xnode == NULL) {
+			/*
+			 * It is possible that base_node_alloc() will cause a
+			 * new base chunk to be allocated, so take care not to
+			 * deadlock on dss_mtx, and recover if another thread
+			 * deallocates an adjacent chunk while this one is busy
+			 * allocating xnode.
+			 */
+			malloc_mutex_unlock(&dss_mtx);
+			xnode = base_node_alloc();
+			malloc_mutex_lock(&dss_mtx);
+			if (xnode == NULL)
+				return (NULL);
+		} else {
+			/* Coalescing forward failed, so insert a new node. */
+			node = xnode;
+			xnode = NULL;
+			node->addr = chunk;
+			node->size = size;
+			extent_tree_ad_insert(&dss_chunks_ad, node);
+			extent_tree_szad_insert(&dss_chunks_szad, node);
+			break;
+		}
+	}
+	/* Discard xnode if it ended up unused do to a race. */
+	if (xnode != NULL)
+		base_node_dealloc(xnode);
+
+	/* Try to coalesce backward. */
+	prev = extent_tree_ad_prev(&dss_chunks_ad, node);
+	if (prev != NULL && (void *)((uintptr_t)prev->addr + prev->size) ==
+	    chunk) {
+		/*
+		 * Coalesce chunk with the previous address range.  This does
+		 * not change the position within dss_chunks_ad, so only
+		 * remove/insert node from/into dss_chunks_szad.
+		 */
+		extent_tree_szad_remove(&dss_chunks_szad, prev);
+		extent_tree_ad_remove(&dss_chunks_ad, prev);
+
+		extent_tree_szad_remove(&dss_chunks_szad, node);
+		node->addr = prev->addr;
+		node->size += prev->size;
+		extent_tree_szad_insert(&dss_chunks_szad, node);
+
+		base_node_dealloc(prev);
+	}
+
+	return (node);
+}
+
 bool
 chunk_in_dss(void *chunk)
 {
 	bool ret;

-	cassert(config_dss);
-
 	malloc_mutex_lock(&dss_mtx);
 	if ((uintptr_t)chunk >= (uintptr_t)dss_base
 	    && (uintptr_t)chunk < (uintptr_t)dss_max)
@@ -155,43 +215,70 @@ chunk_in_dss(void *chunk)
 	return (ret);
 }

+bool
+chunk_dealloc_dss(void *chunk, size_t size)
+{
+	bool ret;
+
+	malloc_mutex_lock(&dss_mtx);
+	if ((uintptr_t)chunk >= (uintptr_t)dss_base
+	    && (uintptr_t)chunk < (uintptr_t)dss_max) {
+		extent_node_t *node;
+
+		/* Try to coalesce with other unused chunks. */
+		node = chunk_dealloc_dss_record(chunk, size);
+		if (node != NULL) {
+			chunk = node->addr;
+			size = node->size;
+		}
+
+		/* Get the current end of the DSS. */
+		dss_max = sbrk(0);
+
+		/*
+		 * Try to shrink the DSS if this chunk is at the end of the
+		 * DSS.  The sbrk() call here is subject to a race condition
+		 * with threads that use brk(2) or sbrk(2) directly, but the
+		 * alternative would be to leak memory for the sake of poorly
+		 * designed multi-threaded programs.
+		 */
+		if ((void *)((uintptr_t)chunk + size) == dss_max
+		    && (dss_prev = sbrk(-(intptr_t)size)) == dss_max) {
+			/* Success. */
+			dss_max = (void *)((intptr_t)dss_prev - (intptr_t)size);
+
+			if (node != NULL) {
+				extent_tree_szad_remove(&dss_chunks_szad, node);
+				extent_tree_ad_remove(&dss_chunks_ad, node);
+				base_node_dealloc(node);
+			}
+		} else
+			madvise(chunk, size, MADV_DONTNEED);
+
+		ret = false;
+		goto RETURN;
+	}
+
+	ret = true;
+RETURN:
+	malloc_mutex_unlock(&dss_mtx);
+	return (ret);
+}
+
 bool
 chunk_dss_boot(void)
 {

-	cassert(config_dss);
-
 	if (malloc_mutex_init(&dss_mtx))
 		return (true);
 	dss_base = sbrk(0);
 	dss_prev = dss_base;
 	dss_max = dss_base;
+	extent_tree_szad_new(&dss_chunks_szad);
+	extent_tree_ad_new(&dss_chunks_ad);

 	return (false);
 }

-void
-chunk_dss_prefork(void)
-{
-
-	if (config_dss)
-		malloc_mutex_prefork(&dss_mtx);
-}
-
-void
-chunk_dss_postfork_parent(void)
-{
-
-	if (config_dss)
-		malloc_mutex_postfork_parent(&dss_mtx);
-}
-
-void
-chunk_dss_postfork_child(void)
-{
-
-	if (config_dss)
-		malloc_mutex_postfork_child(&dss_mtx);
-}
-
 /******************************************************************************/
+#endif /* JEMALLOC_DSS */
@@ -1,37 +1,54 @@
 #define	JEMALLOC_CHUNK_MMAP_C_
 #include "jemalloc/internal/jemalloc_internal.h"

+/******************************************************************************/
+/* Data. */
+
+/*
+ * Used by chunk_alloc_mmap() to decide whether to attempt the fast path and
+ * potentially avoid some system calls.
+ */
+#ifndef NO_TLS
+static __thread bool	mmap_unaligned_tls
+    JEMALLOC_ATTR(tls_model("initial-exec"));
+#define	MMAP_UNALIGNED_GET()	mmap_unaligned_tls
+#define	MMAP_UNALIGNED_SET(v)	do {					\
+	mmap_unaligned_tls = (v);					\
+} while (0)
+#else
+static pthread_key_t	mmap_unaligned_tsd;
+#define	MMAP_UNALIGNED_GET()	((bool)pthread_getspecific(mmap_unaligned_tsd))
+#define	MMAP_UNALIGNED_SET(v)	do {					\
+	pthread_setspecific(mmap_unaligned_tsd, (void *)(v));		\
+} while (0)
+#endif
+
 /******************************************************************************/
 /* Function prototypes for non-inline static functions. */

-static void	*pages_map(void *addr, size_t size);
+static void	*pages_map(void *addr, size_t size, bool noreserve);
 static void	pages_unmap(void *addr, size_t size);
-static void	*chunk_alloc_mmap_slow(size_t size, size_t alignment,
-    bool *zero);
+static void	*chunk_alloc_mmap_slow(size_t size, bool unaligned,
+    bool noreserve);
+static void	*chunk_alloc_mmap_internal(size_t size, bool noreserve);

 /******************************************************************************/

 static void *
-pages_map(void *addr, size_t size)
+pages_map(void *addr, size_t size, bool noreserve)
 {
 	void *ret;

-	assert(size != 0);
-
-#ifdef _WIN32
-	/*
-	 * If VirtualAlloc can't allocate at the given address when one is
-	 * given, it fails and returns NULL.
-	 */
-	ret = VirtualAlloc(addr, size, MEM_COMMIT | MEM_RESERVE,
-	    PAGE_READWRITE);
-#else
 	/*
 	 * We don't use MAP_FIXED here, because it can cause the *replacement*
 	 * of existing mappings, and we only want to create new mappings.
 	 */
-	ret = mmap(addr, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON,
-	    -1, 0);
+	int flags = MAP_PRIVATE | MAP_ANON;
+#ifdef MAP_NORESERVE
+	if (noreserve)
+		flags |= MAP_NORESERVE;
+#endif
+	ret = mmap(addr, size, PROT_READ | PROT_WRITE, flags, -1, 0);
 	assert(ret != NULL);

 	if (ret == MAP_FAILED)
@@ -43,15 +60,16 @@ pages_map(void *addr, size_t size)
 		if (munmap(ret, size) == -1) {
 			char buf[BUFERROR_BUF];

-			buferror(buf, sizeof(buf));
-			malloc_printf("<jemalloc: Error in munmap(): %s\n",
-			    buf);
+			buferror(errno, buf, sizeof(buf));
+			malloc_write("<jemalloc>: Error in munmap(): ");
+			malloc_write(buf);
+			malloc_write("\n");
 			if (opt_abort)
 				abort();
 		}
 		ret = NULL;
 	}
-#endif
+
 	assert(ret == NULL || (addr == NULL && ret != addr)
 	    || (addr != NULL && ret == addr));
 	return (ret);
@@ -61,150 +79,161 @@ static void
 pages_unmap(void *addr, size_t size)
 {

-#ifdef _WIN32
-	if (VirtualFree(addr, 0, MEM_RELEASE) == 0)
-#else
-	if (munmap(addr, size) == -1)
-#endif
-	{
+	if (munmap(addr, size) == -1) {
 		char buf[BUFERROR_BUF];

-		buferror(buf, sizeof(buf));
-		malloc_printf("<jemalloc>: Error in "
-#ifdef _WIN32
-		              "VirtualFree"
-#else
-		              "munmap"
-#endif
-		              "(): %s\n", buf);
+		buferror(errno, buf, sizeof(buf));
+		malloc_write("<jemalloc>: Error in munmap(): ");
+		malloc_write(buf);
+		malloc_write("\n");
 		if (opt_abort)
 			abort();
 	}
 }

 static void *
-pages_trim(void *addr, size_t alloc_size, size_t leadsize, size_t size)
-{
-	void *ret = (void *)((uintptr_t)addr + leadsize);
-
-	assert(alloc_size >= leadsize + size);
-#ifdef _WIN32
-	{
-		void *new_addr;
-
-		pages_unmap(addr, alloc_size);
-		new_addr = pages_map(ret, size);
-		if (new_addr == ret)
-			return (ret);
-		if (new_addr)
-			pages_unmap(new_addr, size);
-		return (NULL);
-	}
-#else
-	{
-		size_t trailsize = alloc_size - leadsize - size;
-
-		if (leadsize != 0)
-			pages_unmap(addr, leadsize);
-		if (trailsize != 0)
-			pages_unmap((void *)((uintptr_t)ret + size), trailsize);
-		return (ret);
-	}
-#endif
-}
-
-bool
-pages_purge(void *addr, size_t length)
-{
-	bool unzeroed;
-
-#ifdef _WIN32
-	VirtualAlloc(addr, length, MEM_RESET, PAGE_READWRITE);
-	unzeroed = true;
-#else
-#  ifdef JEMALLOC_PURGE_MADVISE_DONTNEED
-#    define JEMALLOC_MADV_PURGE MADV_DONTNEED
-#    define JEMALLOC_MADV_ZEROS true
-#  elif defined(JEMALLOC_PURGE_MADVISE_FREE)
-#    define JEMALLOC_MADV_PURGE MADV_FREE
-#    define JEMALLOC_MADV_ZEROS false
-#  else
-#    error "No method defined for purging unused dirty pages."
-#  endif
-	int err = madvise(addr, length, JEMALLOC_MADV_PURGE);
-	unzeroed = (JEMALLOC_MADV_ZEROS == false || err != 0);
-#  undef JEMALLOC_MADV_PURGE
-#  undef JEMALLOC_MADV_ZEROS
-#endif
-	return (unzeroed);
-}
-
-static void *
-chunk_alloc_mmap_slow(size_t size, size_t alignment, bool *zero)
-{
-	void *ret, *pages;
-	size_t alloc_size, leadsize;
-
-	alloc_size = size + alignment - PAGE;
-	/* Beware size_t wrap-around. */
-	if (alloc_size < size)
-		return (NULL);
-	do {
-		pages = pages_map(NULL, alloc_size);
-		if (pages == NULL)
-			return (NULL);
-		leadsize = ALIGNMENT_CEILING((uintptr_t)pages, alignment) -
-		    (uintptr_t)pages;
-		ret = pages_trim(pages, alloc_size, leadsize, size);
-	} while (ret == NULL);
-
-	assert(ret != NULL);
-	*zero = true;
-	return (ret);
-}
-
-void *
-chunk_alloc_mmap(size_t size, size_t alignment, bool *zero)
+chunk_alloc_mmap_slow(size_t size, bool unaligned, bool noreserve)
 {
 	void *ret;
 	size_t offset;

+	/* Beware size_t wrap-around. */
+	if (size + chunksize <= size)
+		return (NULL);
+
+	ret = pages_map(NULL, size + chunksize, noreserve);
+	if (ret == NULL)
+		return (NULL);
+
+	/* Clean up unneeded leading/trailing space. */
+	offset = CHUNK_ADDR2OFFSET(ret);
+	if (offset != 0) {
+		/* Note that mmap() returned an unaligned mapping. */
+		unaligned = true;
+
+		/* Leading space. */
+		pages_unmap(ret, chunksize - offset);
+
+		ret = (void *)((uintptr_t)ret +
+		    (chunksize - offset));
+
+		/* Trailing space. */
+		pages_unmap((void *)((uintptr_t)ret + size),
+		    offset);
+	} else {
+		/* Trailing space only. */
+		pages_unmap((void *)((uintptr_t)ret + size),
+		    chunksize);
+	}
+
+	/*
+	 * If mmap() returned an aligned mapping, reset mmap_unaligned so that
+	 * the next chunk_alloc_mmap() execution tries the fast allocation
+	 * method.
+	 */
+	if (unaligned == false)
+		MMAP_UNALIGNED_SET(false);
+
+	return (ret);
+}
+
+static void *
+chunk_alloc_mmap_internal(size_t size, bool noreserve)
+{
+	void *ret;
+
 	/*
 	 * Ideally, there would be a way to specify alignment to mmap() (like
 	 * NetBSD has), but in the absence of such a feature, we have to work
 	 * hard to efficiently create aligned mappings.  The reliable, but
 	 * slow method is to create a mapping that is over-sized, then trim the
-	 * excess.  However, that always results in one or two calls to
+	 * excess.  However, that always results in at least one call to
 	 * pages_unmap().
 	 *
-	 * Optimistically try mapping precisely the right amount before falling
-	 * back to the slow method, with the expectation that the optimistic
-	 * approach works most of the time.
+	 * A more optimistic approach is to try mapping precisely the right
+	 * amount, then try to append another mapping if alignment is off.  In
+	 * practice, this works out well as long as the application is not
+	 * interleaving mappings via direct mmap() calls.  If we do run into a
+	 * situation where there is an interleaved mapping and we are unable to
+	 * extend an unaligned mapping, our best option is to switch to the
+	 * slow method until mmap() returns another aligned mapping.  This will
+	 * tend to leave a gap in the memory map that is too small to cause
+	 * later problems for the optimistic method.
+	 *
+	 * Another possible confounding factor is address space layout
+	 * randomization (ASLR), which causes mmap(2) to disregard the
+	 * requested address.  mmap_unaligned tracks whether the previous
+	 * chunk_alloc_mmap() execution received any unaligned or relocated
+	 * mappings, and if so, the current execution will immediately fall
+	 * back to the slow method.  However, we keep track of whether the fast
+	 * method would have succeeded, and if so, we make a note to try the
+	 * fast method next time.
 	 */

-	assert(alignment != 0);
-	assert((alignment & chunksize_mask) == 0);
+	if (MMAP_UNALIGNED_GET() == false) {
+		size_t offset;

-	ret = pages_map(NULL, size);
-	if (ret == NULL)
-		return (NULL);
-	offset = ALIGNMENT_ADDR2OFFSET(ret, alignment);
-	if (offset != 0) {
-		pages_unmap(ret, size);
-		return (chunk_alloc_mmap_slow(size, alignment, zero));
-	}
+		ret = pages_map(NULL, size, noreserve);
+		if (ret == NULL)
+			return (NULL);
+
+		offset = CHUNK_ADDR2OFFSET(ret);
+		if (offset != 0) {
+			MMAP_UNALIGNED_SET(true);
+			/* Try to extend chunk boundary. */
+			if (pages_map((void *)((uintptr_t)ret + size),
+			    chunksize - offset, noreserve) == NULL) {
+				/*
+				 * Extension failed.  Clean up, then revert to
+				 * the reliable-but-expensive method.
+				 */
+				pages_unmap(ret, size);
+				ret = chunk_alloc_mmap_slow(size, true,
+				    noreserve);
+			} else {
+				/* Clean up unneeded leading space. */
+				pages_unmap(ret, chunksize - offset);
+				ret = (void *)((uintptr_t)ret + (chunksize -
+				    offset));
+			}
+		}
+	} else
+		ret = chunk_alloc_mmap_slow(size, false, noreserve);

-	assert(ret != NULL);
-	*zero = true;
 	return (ret);
 }

-bool
+void *
+chunk_alloc_mmap(size_t size)
+{
+
+	return (chunk_alloc_mmap_internal(size, false));
+}
+
+void *
+chunk_alloc_mmap_noreserve(size_t size)
+{
+
+	return (chunk_alloc_mmap_internal(size, true));
+}
+
+void
 chunk_dealloc_mmap(void *chunk, size_t size)
 {

-	if (config_munmap)
-		pages_unmap(chunk, size);
+	pages_unmap(chunk, size);
+}

-	return (config_munmap == false);
+bool
+chunk_mmap_boot(void)
+{
+
+#ifdef NO_TLS
+	if (pthread_key_create(&mmap_unaligned_tsd, NULL) != 0) {
+		malloc_write("<jemalloc>: Error in pthread_key_create()\n");
+		return (true);
+	}
+#endif
+
+	return (false);
 }
@@ -0,0 +1,402 @@
+#define	JEMALLOC_CHUNK_SWAP_C_
+#include "jemalloc/internal/jemalloc_internal.h"
+#ifdef JEMALLOC_SWAP
+/******************************************************************************/
+/* Data. */
+
+malloc_mutex_t	swap_mtx;
+bool		swap_enabled;
+bool		swap_prezeroed;
+size_t		swap_nfds;
+int		*swap_fds;
+#ifdef JEMALLOC_STATS
+size_t		swap_avail;
+#endif
+
+/* Base address of the mmap()ed file(s). */
+static void	*swap_base;
+/* Current end of the space in use (<= swap_max). */
+static void	*swap_end;
+/* Absolute upper limit on file-backed addresses. */
+static void	*swap_max;
+
+/*
+ * Trees of chunks that were previously allocated (trees differ only in node
+ * ordering).  These are used when allocating chunks, in an attempt to re-use
+ * address space.  Depending on function, different tree orderings are needed,
+ * which is why there are two trees with the same contents.
+ */
+static extent_tree_t	swap_chunks_szad;
+static extent_tree_t	swap_chunks_ad;
+
+/******************************************************************************/
+/* Function prototypes for non-inline static functions. */
+
+static void	*chunk_recycle_swap(size_t size, bool *zero);
+static extent_node_t *chunk_dealloc_swap_record(void *chunk, size_t size);
+
+/******************************************************************************/
+
+static void *
+chunk_recycle_swap(size_t size, bool *zero)
+{
+	extent_node_t *node, key;
+
+	key.addr = NULL;
+	key.size = size;
+	malloc_mutex_lock(&swap_mtx);
+	node = extent_tree_szad_nsearch(&swap_chunks_szad, &key);
+	if (node != NULL) {
+		void *ret = node->addr;
+
+		/* Remove node from the tree. */
+		extent_tree_szad_remove(&swap_chunks_szad, node);
+		if (node->size == size) {
+			extent_tree_ad_remove(&swap_chunks_ad, node);
+			base_node_dealloc(node);
+		} else {
+			/*
+			 * Insert the remainder of node's address range as a
+			 * smaller chunk.  Its position within swap_chunks_ad
+			 * does not change.
+			 */
+			assert(node->size > size);
+			node->addr = (void *)((uintptr_t)node->addr + size);
+			node->size -= size;
+			extent_tree_szad_insert(&swap_chunks_szad, node);
+		}
+#ifdef JEMALLOC_STATS
+		swap_avail -= size;
+#endif
+		malloc_mutex_unlock(&swap_mtx);
+
+		if (*zero)
+			memset(ret, 0, size);
+		return (ret);
+	}
+	malloc_mutex_unlock(&swap_mtx);
+
+	return (NULL);
+}
+
+void *
+chunk_alloc_swap(size_t size, bool *zero)
+{
+	void *ret;
+
+	assert(swap_enabled);
+
+	ret = chunk_recycle_swap(size, zero);
+	if (ret != NULL)
+		return (ret);
+
+	malloc_mutex_lock(&swap_mtx);
+	if ((uintptr_t)swap_end + size <= (uintptr_t)swap_max) {
+		ret = swap_end;
+		swap_end = (void *)((uintptr_t)swap_end + size);
+#ifdef JEMALLOC_STATS
+		swap_avail -= size;
+#endif
+		malloc_mutex_unlock(&swap_mtx);
+
+		if (swap_prezeroed)
+			*zero = true;
+		else if (*zero)
+			memset(ret, 0, size);
+	} else {
+		malloc_mutex_unlock(&swap_mtx);
+		return (NULL);
+	}
+
+	return (ret);
+}
+
+static extent_node_t *
+chunk_dealloc_swap_record(void *chunk, size_t size)
+{
+	extent_node_t *xnode, *node, *prev, key;
+
+	xnode = NULL;
+	while (true) {
+		key.addr = (void *)((uintptr_t)chunk + size);
+		node = extent_tree_ad_nsearch(&swap_chunks_ad, &key);
+		/* Try to coalesce forward. */
+		if (node != NULL && node->addr == key.addr) {
+			/*
+			 * Coalesce chunk with the following address range.
+			 * This does not change the position within
+			 * swap_chunks_ad, so only remove/insert from/into
+			 * swap_chunks_szad.
+			 */
+			extent_tree_szad_remove(&swap_chunks_szad, node);
+			node->addr = chunk;
+			node->size += size;
+			extent_tree_szad_insert(&swap_chunks_szad, node);
+			break;
+		} else if (xnode == NULL) {
+			/*
+			 * It is possible that base_node_alloc() will cause a
+			 * new base chunk to be allocated, so take care not to
+			 * deadlock on swap_mtx, and recover if another thread
+			 * deallocates an adjacent chunk while this one is busy
+			 * allocating xnode.
+			 */
+			malloc_mutex_unlock(&swap_mtx);
+			xnode = base_node_alloc();
+			malloc_mutex_lock(&swap_mtx);
+			if (xnode == NULL)
+				return (NULL);
+		} else {
+			/* Coalescing forward failed, so insert a new node. */
+			node = xnode;
+			xnode = NULL;
+			node->addr = chunk;
+			node->size = size;
+			extent_tree_ad_insert(&swap_chunks_ad, node);
+			extent_tree_szad_insert(&swap_chunks_szad, node);
+			break;
+		}
+	}
+	/* Discard xnode if it ended up unused do to a race. */
+	if (xnode != NULL)
+		base_node_dealloc(xnode);
+
+	/* Try to coalesce backward. */
+	prev = extent_tree_ad_prev(&swap_chunks_ad, node);
+	if (prev != NULL && (void *)((uintptr_t)prev->addr + prev->size) ==
+	    chunk) {
+		/*
+		 * Coalesce chunk with the previous address range.  This does
+		 * not change the position within swap_chunks_ad, so only
+		 * remove/insert node from/into swap_chunks_szad.
+		 */
+		extent_tree_szad_remove(&swap_chunks_szad, prev);
+		extent_tree_ad_remove(&swap_chunks_ad, prev);
+
+		extent_tree_szad_remove(&swap_chunks_szad, node);
+		node->addr = prev->addr;
+		node->size += prev->size;
+		extent_tree_szad_insert(&swap_chunks_szad, node);
+
+		base_node_dealloc(prev);
+	}
+
+	return (node);
+}
+
+bool
+chunk_in_swap(void *chunk)
+{
+	bool ret;
+
+	assert(swap_enabled);
+
+	malloc_mutex_lock(&swap_mtx);
+	if ((uintptr_t)chunk >= (uintptr_t)swap_base
+	    && (uintptr_t)chunk < (uintptr_t)swap_max)
+		ret = true;
+	else
+		ret = false;
+	malloc_mutex_unlock(&swap_mtx);
+
+	return (ret);
+}
+
+bool
+chunk_dealloc_swap(void *chunk, size_t size)
+{
+	bool ret;
+
+	assert(swap_enabled);
+
+	malloc_mutex_lock(&swap_mtx);
+	if ((uintptr_t)chunk >= (uintptr_t)swap_base
+	    && (uintptr_t)chunk < (uintptr_t)swap_max) {
+		extent_node_t *node;
+
+		/* Try to coalesce with other unused chunks. */
+		node = chunk_dealloc_swap_record(chunk, size);
+		if (node != NULL) {
+			chunk = node->addr;
+			size = node->size;
+		}
+
+		/*
+		 * Try to shrink the in-use memory if this chunk is at the end
+		 * of the in-use memory.
+		 */
+		if ((void *)((uintptr_t)chunk + size) == swap_end) {
+			swap_end = (void *)((uintptr_t)swap_end - size);
+
+			if (node != NULL) {
+				extent_tree_szad_remove(&swap_chunks_szad,
+				    node);
+				extent_tree_ad_remove(&swap_chunks_ad, node);
+				base_node_dealloc(node);
+			}
+		} else
+			madvise(chunk, size, MADV_DONTNEED);
+
+#ifdef JEMALLOC_STATS
+		swap_avail += size;
+#endif
+		ret = false;
+		goto RETURN;
+	}
+
+	ret = true;
+RETURN:
+	malloc_mutex_unlock(&swap_mtx);
+	return (ret);
+}
+
+bool
+chunk_swap_enable(const int *fds, unsigned nfds, bool prezeroed)
+{
+	bool ret;
+	unsigned i;
+	off_t off;
+	void *vaddr;
+	size_t cumsize, voff;
+	size_t sizes[nfds];
+
+	malloc_mutex_lock(&swap_mtx);
+
+	/* Get file sizes. */
+	for (i = 0, cumsize = 0; i < nfds; i++) {
+		off = lseek(fds[i], 0, SEEK_END);
+		if (off == ((off_t)-1)) {
+			ret = true;
+			goto RETURN;
+		}
+		if (PAGE_CEILING(off) != off) {
+			/* Truncate to a multiple of the page size. */
+			off &= ~PAGE_MASK;
+			if (ftruncate(fds[i], off) != 0) {
+				ret = true;
+				goto RETURN;
+			}
+		}
+		sizes[i] = off;
+		if (cumsize + off < cumsize) {
+			/*
+			 * Cumulative file size is greater than the total
+			 * address space.  Bail out while it's still obvious
+			 * what the problem is.
+			 */
+			ret = true;
+			goto RETURN;
+		}
+		cumsize += off;
+	}
+
+	/* Round down to a multiple of the chunk size. */
+	cumsize &= ~chunksize_mask;
+	if (cumsize == 0) {
+		ret = true;
+		goto RETURN;
+	}
+
+	/*
+	 * Allocate a chunk-aligned region of anonymous memory, which will
+	 * be the final location for the memory-mapped files.
+	 */
+	vaddr = chunk_alloc_mmap_noreserve(cumsize);
+	if (vaddr == NULL) {
+		ret = true;
+		goto RETURN;
+	}
+
+	/* Overlay the files onto the anonymous mapping. */
+	for (i = 0, voff = 0; i < nfds; i++) {
+		void *addr = mmap((void *)((uintptr_t)vaddr + voff), sizes[i],
+		    PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, fds[i], 0);
+		if (addr == MAP_FAILED) {
+			char buf[BUFERROR_BUF];
+
+
+			buferror(errno, buf, sizeof(buf));
+			malloc_write(
+			    "<jemalloc>: Error in mmap(..., MAP_FIXED, ...): ");
+			malloc_write(buf);
+			malloc_write("\n");
+			if (opt_abort)
+				abort();
+			if (munmap(vaddr, voff) == -1) {
+				buferror(errno, buf, sizeof(buf));
+				malloc_write("<jemalloc>: Error in munmap(): ");
+				malloc_write(buf);
+				malloc_write("\n");
+			}
+			ret = true;
+			goto RETURN;
+		}
+		assert(addr == (void *)((uintptr_t)vaddr + voff));
+
+		/*
+		 * Tell the kernel that the mapping will be accessed randomly,
+		 * and that it should not gratuitously sync pages to the
+		 * filesystem.
+		 */
+#ifdef MADV_RANDOM
+		madvise(addr, sizes[i], MADV_RANDOM);
+#endif
+#ifdef MADV_NOSYNC
+		madvise(addr, sizes[i], MADV_NOSYNC);
+#endif
+
+		voff += sizes[i];
+	}
+
+	swap_prezeroed = prezeroed;
+	swap_base = vaddr;
+	swap_end = swap_base;
+	swap_max = (void *)((uintptr_t)vaddr + cumsize);
+
+	/* Copy the fds array for mallctl purposes. */
+	swap_fds = (int *)base_alloc(nfds * sizeof(int));
+	if (swap_fds == NULL) {
+		ret = true;
+		goto RETURN;
+	}
+	memcpy(swap_fds, fds, nfds * sizeof(int));
+	swap_nfds = nfds;
+
+#ifdef JEMALLOC_STATS
+	swap_avail = cumsize;
+#endif
+
+	swap_enabled = true;
+
+	ret = false;
+RETURN:
+	malloc_mutex_unlock(&swap_mtx);
+	return (ret);
+}
+
+bool
+chunk_swap_boot(void)
+{
+
+	if (malloc_mutex_init(&swap_mtx))
+		return (true);
+
+	swap_enabled = false;
+	swap_prezeroed = false; /* swap.* mallctl's depend on this. */
+	swap_nfds = 0;
+	swap_fds = NULL;
+#ifdef JEMALLOC_STATS
+	swap_avail = 0;
+#endif
+	swap_base = NULL;
+	swap_end = NULL;
+	swap_max = NULL;
+
+	extent_tree_szad_new(&swap_chunks_szad);
+	extent_tree_ad_new(&swap_chunks_ad);
+
+	return (false);
+}
+
+/******************************************************************************/
+#endif /* JEMALLOC_SWAP */
@@ -73,6 +73,7 @@ ckh_isearch(ckh_t *ckh, const void *key)
 	size_t hash1, hash2, bucket, cell;

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);

 	ckh->hash(key, ckh->lg_curbuckets, &hash1, &hash2);

@@ -99,7 +100,7 @@ ckh_try_bucket_insert(ckh_t *ckh, size_t bucket, const void *key,
 	 * Cycle through the cells in the bucket, starting at a random position.
 	 * The randomness avoids worst-case search overhead as buckets fill up.
 	 */
-	prng32(offset, LG_CKH_BUCKET_CELLS, ckh->prng_state, CKH_A, CKH_C);
+	prn32(offset, LG_CKH_BUCKET_CELLS, ckh->prn_state, CKH_A, CKH_C);
 	for (i = 0; i < (ZU(1) << LG_CKH_BUCKET_CELLS); i++) {
 		cell = &ckh->tab[(bucket << LG_CKH_BUCKET_CELLS) +
 		    ((i + offset) & ((ZU(1) << LG_CKH_BUCKET_CELLS) - 1))];
@@ -141,7 +142,7 @@ ckh_evict_reloc_insert(ckh_t *ckh, size_t argbucket, void const **argkey,
 		 * were an item for which both hashes indicated the same
 		 * bucket.
 		 */
-		prng32(i, LG_CKH_BUCKET_CELLS, ckh->prng_state, CKH_A, CKH_C);
+		prn32(i, LG_CKH_BUCKET_CELLS, ckh->prn_state, CKH_A, CKH_C);
 		cell = &ckh->tab[(bucket << LG_CKH_BUCKET_CELLS) + i];
 		assert(cell->key != NULL);

@@ -264,15 +265,15 @@ ckh_grow(ckh_t *ckh)
 		size_t usize;

 		lg_curcells++;
-		usize = sa2u(sizeof(ckhc_t) << lg_curcells, CACHELINE);
+		usize = sa2u(sizeof(ckhc_t) << lg_curcells, CACHELINE, NULL);
 		if (usize == 0) {
 			ret = true;
-			goto label_return;
+			goto RETURN;
 		}
 		tab = (ckhc_t *)ipalloc(usize, CACHELINE, true);
 		if (tab == NULL) {
 			ret = true;
-			goto label_return;
+			goto RETURN;
 		}
 		/* Swap in new table. */
 		ttab = ckh->tab;
@@ -292,7 +293,7 @@ ckh_grow(ckh_t *ckh)
 	}

 	ret = false;
-label_return:
+RETURN:
 	return (ret);
 }

@@ -309,7 +310,7 @@ ckh_shrink(ckh_t *ckh)
 	 */
 	lg_prevbuckets = ckh->lg_curbuckets;
 	lg_curcells = ckh->lg_curbuckets + LG_CKH_BUCKET_CELLS - 1;
-	usize = sa2u(sizeof(ckhc_t) << lg_curcells, CACHELINE);
+	usize = sa2u(sizeof(ckhc_t) << lg_curcells, CACHELINE, NULL);
 	if (usize == 0)
 		return;
 	tab = (ckhc_t *)ipalloc(usize, CACHELINE, true);
@@ -361,7 +362,7 @@ ckh_new(ckh_t *ckh, size_t minitems, ckh_hash_t *hash, ckh_keycomp_t *keycomp)
 	ckh->ninserts = 0;
 	ckh->nrelocs = 0;
 #endif
-	ckh->prng_state = 42; /* Value doesn't really matter. */
+	ckh->prn_state = 42; /* Value doesn't really matter. */
 	ckh->count = 0;

 	/*
@@ -382,19 +383,23 @@ ckh_new(ckh_t *ckh, size_t minitems, ckh_hash_t *hash, ckh_keycomp_t *keycomp)
 	ckh->hash = hash;
 	ckh->keycomp = keycomp;

-	usize = sa2u(sizeof(ckhc_t) << lg_mincells, CACHELINE);
+	usize = sa2u(sizeof(ckhc_t) << lg_mincells, CACHELINE, NULL);
 	if (usize == 0) {
 		ret = true;
-		goto label_return;
+		goto RETURN;
 	}
 	ckh->tab = (ckhc_t *)ipalloc(usize, CACHELINE, true);
 	if (ckh->tab == NULL) {
 		ret = true;
-		goto label_return;
+		goto RETURN;
 	}

+#ifdef JEMALLOC_DEBUG
+	ckh->magic = CKH_MAGIC;
+#endif
+
 	ret = false;
-label_return:
+RETURN:
 	return (ret);
 }

@@ -403,6 +408,7 @@ ckh_delete(ckh_t *ckh)
 {

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);

 #ifdef CKH_VERBOSE
 	malloc_printf(
@@ -427,6 +433,7 @@ ckh_count(ckh_t *ckh)
 {

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);

 	return (ckh->count);
 }
@@ -457,6 +464,7 @@ ckh_insert(ckh_t *ckh, const void *key, const void *data)
 	bool ret;

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);
 	assert(ckh_search(ckh, key, NULL, NULL));

 #ifdef CKH_COUNT
@@ -466,12 +474,12 @@ ckh_insert(ckh_t *ckh, const void *key, const void *data)
 	while (ckh_try_insert(ckh, &key, &data)) {
 		if (ckh_grow(ckh)) {
 			ret = true;
-			goto label_return;
+			goto RETURN;
 		}
 	}

 	ret = false;
-label_return:
+RETURN:
 	return (ret);
 }

@@ -481,6 +489,7 @@ ckh_remove(ckh_t *ckh, const void *searchkey, void **key, void **data)
 	size_t cell;

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);

 	cell = ckh_isearch(ckh, searchkey);
 	if (cell != SIZE_T_MAX) {
@@ -512,6 +521,7 @@ ckh_search(ckh_t *ckh, const void *searchkey, void **key, void **data)
 	size_t cell;

 	assert(ckh != NULL);
+	dassert(ckh->magic == CKH_MAGIC);

 	cell = ckh_isearch(ckh, searchkey);
 	if (cell != SIZE_T_MAX) {
@@ -535,7 +545,7 @@ ckh_string_hash(const void *key, unsigned minbits, size_t *hash1, size_t *hash2)
 	assert(hash1 != NULL);
 	assert(hash2 != NULL);

-	h = hash(key, strlen((const char *)key), UINT64_C(0x94122f335b332aea));
+	h = hash(key, strlen((const char *)key), 0x94122f335b332aeaLLU);
 	if (minbits <= 32) {
 		/*
 		 * Avoid doing multiple hashes, since a single hash provides
@@ -546,7 +556,7 @@ ckh_string_hash(const void *key, unsigned minbits, size_t *hash1, size_t *hash2)
 	} else {
 		ret1 = h;
 		ret2 = hash(key, strlen((const char *)key),
-		    UINT64_C(0x8432a476666bbc13));
+		    0x8432a476666bbc13U);
 	}

 	*hash1 = ret1;
@@ -583,7 +593,7 @@ ckh_pointer_hash(const void *key, unsigned minbits, size_t *hash1,
 	u.i = 0;
 #endif
 	u.v = key;
-	h = hash(&u.i, sizeof(u.i), UINT64_C(0xd983396e68886082));
+	h = hash(&u.i, sizeof(u.i), 0xd983396e68886082LLU);
 	if (minbits <= 32) {
 		/*
 		 * Avoid doing multiple hashes, since a single hash provides
@@ -594,7 +604,7 @@ ckh_pointer_hash(const void *key, unsigned minbits, size_t *hash1,
 	} else {
 		assert(SIZEOF_PTR == 8);
 		ret1 = h;
-		ret2 = hash(&u.i, sizeof(u.i), UINT64_C(0x5e2be9aff8709a5d));
+		ret2 = hash(&u.i, sizeof(u.i), 0x5e2be9aff8709a5dLLU);
 	}

 	*hash1 = ret1;
@@ -3,6 +3,7 @@

 /******************************************************************************/

+#if (defined(JEMALLOC_SWAP) || defined(JEMALLOC_DSS))
 static inline int
 extent_szad_comp(extent_node_t *a, extent_node_t *b)
 {
@@ -24,6 +25,7 @@ extent_szad_comp(extent_node_t *a, extent_node_t *b)
 /* Generate red-black tree functions. */
 rb_gen(, extent_tree_szad_, extent_tree_t, extent_node_t, link_szad,
    extent_szad_comp)
+#endif

 static inline int
 extent_ad_comp(extent_node_t *a, extent_node_t *b)
@@ -4,9 +4,11 @@
 /******************************************************************************/
 /* Data. */

+#ifdef JEMALLOC_STATS
 uint64_t	huge_nmalloc;
 uint64_t	huge_ndalloc;
 size_t		huge_allocated;
+#endif

 malloc_mutex_t	huge_mtx;

@@ -17,18 +19,10 @@ static extent_tree_t	huge;

 void *
 huge_malloc(size_t size, bool zero)
-{
-
-	return (huge_palloc(size, chunksize, zero));
-}
-
-void *
-huge_palloc(size_t size, size_t alignment, bool zero)
 {
 	void *ret;
 	size_t csize;
 	extent_node_t *node;
-	bool is_zeroed;

 	/* Allocate one or more contiguous chunks for this request. */

@@ -43,13 +37,7 @@ huge_palloc(size_t size, size_t alignment, bool zero)
 	if (node == NULL)
 		return (NULL);

-	/*
-	 * Copy zero into is_zeroed and pass the copy to chunk_alloc(), so that
-	 * it is possible to make correct junk/zero fill decisions below.
-	 */
-	is_zeroed = zero;
-	ret = chunk_alloc(csize, alignment, false, &is_zeroed,
-	    chunk_dss_prec_get());
+	ret = chunk_alloc(csize, false, &zero);
 	if (ret == NULL) {
 		base_node_dealloc(node);
 		return (NULL);
@@ -61,19 +49,106 @@ huge_palloc(size_t size, size_t alignment, bool zero)

 	malloc_mutex_lock(&huge_mtx);
 	extent_tree_ad_insert(&huge, node);
-	if (config_stats) {
-		stats_cactive_add(csize);
-		huge_nmalloc++;
-		huge_allocated += csize;
-	}
+#ifdef JEMALLOC_STATS
+	stats_cactive_add(csize);
+	huge_nmalloc++;
+	huge_allocated += csize;
+#endif
 	malloc_mutex_unlock(&huge_mtx);

-	if (config_fill && zero == false) {
+#ifdef JEMALLOC_FILL
+	if (zero == false) {
 		if (opt_junk)
 			memset(ret, 0xa5, csize);
-		else if (opt_zero && is_zeroed == false)
+		else if (opt_zero)
 			memset(ret, 0, csize);
 	}
+#endif
+
+	return (ret);
+}
+
+/* Only handles large allocations that require more than chunk alignment. */
+void *
+huge_palloc(size_t size, size_t alignment, bool zero)
+{
+	void *ret;
+	size_t alloc_size, chunk_size, offset;
+	extent_node_t *node;
+
+	/*
+	 * This allocation requires alignment that is even larger than chunk
+	 * alignment.  This means that huge_malloc() isn't good enough.
+	 *
+	 * Allocate almost twice as many chunks as are demanded by the size or
+	 * alignment, in order to assure the alignment can be achieved, then
+	 * unmap leading and trailing chunks.
+	 */
+	assert(alignment > chunksize);
+
+	chunk_size = CHUNK_CEILING(size);
+
+	if (size >= alignment)
+		alloc_size = chunk_size + alignment - chunksize;
+	else
+		alloc_size = (alignment << 1) - chunksize;
+
+	/* Allocate an extent node with which to track the chunk. */
+	node = base_node_alloc();
+	if (node == NULL)
+		return (NULL);
+
+	ret = chunk_alloc(alloc_size, false, &zero);
+	if (ret == NULL) {
+		base_node_dealloc(node);
+		return (NULL);
+	}
+
+	offset = (uintptr_t)ret & (alignment - 1);
+	assert((offset & chunksize_mask) == 0);
+	assert(offset < alloc_size);
+	if (offset == 0) {
+		/* Trim trailing space. */
+		chunk_dealloc((void *)((uintptr_t)ret + chunk_size), alloc_size
+		    - chunk_size);
+	} else {
+		size_t trailsize;
+
+		/* Trim leading space. */
+		chunk_dealloc(ret, alignment - offset);
+
+		ret = (void *)((uintptr_t)ret + (alignment - offset));
+
+		trailsize = alloc_size - (alignment - offset) - chunk_size;
+		if (trailsize != 0) {
+		    /* Trim trailing space. */
+		    assert(trailsize < alloc_size);
+		    chunk_dealloc((void *)((uintptr_t)ret + chunk_size),
+			trailsize);
+		}
+	}
+
+	/* Insert node into huge. */
+	node->addr = ret;
+	node->size = chunk_size;
+
+	malloc_mutex_lock(&huge_mtx);
+	extent_tree_ad_insert(&huge, node);
+#ifdef JEMALLOC_STATS
+	stats_cactive_add(chunk_size);
+	huge_nmalloc++;
+	huge_allocated += chunk_size;
+#endif
+	malloc_mutex_unlock(&huge_mtx);
+
+#ifdef JEMALLOC_FILL
+	if (zero == false) {
+		if (opt_junk)
+			memset(ret, 0xa5, chunk_size);
+		else if (opt_zero)
+			memset(ret, 0, chunk_size);
+	}
+#endif

 	return (ret);
 }
@@ -89,10 +164,12 @@ huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size, size_t extra)
 	    && CHUNK_CEILING(oldsize) >= CHUNK_CEILING(size)
 	    && CHUNK_CEILING(oldsize) <= CHUNK_CEILING(size+extra)) {
 		assert(CHUNK_CEILING(oldsize) == oldsize);
-		if (config_fill && opt_junk && size < oldsize) {
+#ifdef JEMALLOC_FILL
+		if (opt_junk && size < oldsize) {
 			memset((void *)((uintptr_t)ptr + size), 0x5a,
 			    oldsize - size);
 		}
+#endif
 		return (ptr);
 	}

@@ -102,7 +179,7 @@ huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size, size_t extra)

 void *
 huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
-    size_t alignment, bool zero, bool try_tcache_dalloc)
+    size_t alignment, bool zero)
 {
 	void *ret;
 	size_t copysize;
@@ -141,22 +218,22 @@ huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
 	 */
 	copysize = (size < oldsize) ? size : oldsize;

-#ifdef JEMALLOC_MREMAP
 	/*
 	 * Use mremap(2) if this is a huge-->huge reallocation, and neither the
-	 * source nor the destination are in dss.
+	 * source nor the destination are in swap or dss.
 	 */
-	if (oldsize >= chunksize && (config_dss == false || (chunk_in_dss(ptr)
-	    == false && chunk_in_dss(ret) == false))) {
+#ifdef JEMALLOC_MREMAP_FIXED
+	if (oldsize >= chunksize
+#  ifdef JEMALLOC_SWAP
+	    && (swap_enabled == false || (chunk_in_swap(ptr) == false &&
+	    chunk_in_swap(ret) == false))
+#  endif
+#  ifdef JEMALLOC_DSS
+	    && chunk_in_dss(ptr) == false && chunk_in_dss(ret) == false
+#  endif
+	    ) {
 		size_t newsize = huge_salloc(ret);

-		/*
-		 * Remove ptr from the tree of huge allocations before
-		 * performing the remap operation, in order to avoid the
-		 * possibility of another thread acquiring that mapping before
-		 * this one removes it from the tree.
-		 */
-		huge_dalloc(ptr, false);
 		if (mremap(ptr, oldsize, newsize, MREMAP_MAYMOVE|MREMAP_FIXED,
 		    ret) == MAP_FAILED) {
 			/*
@@ -169,19 +246,21 @@ huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
 			 */
 			char buf[BUFERROR_BUF];

-			buferror(buf, sizeof(buf));
-			malloc_printf("<jemalloc>: Error in mremap(): %s\n",
-			    buf);
+			buferror(errno, buf, sizeof(buf));
+			malloc_write("<jemalloc>: Error in mremap(): ");
+			malloc_write(buf);
+			malloc_write("\n");
 			if (opt_abort)
 				abort();
 			memcpy(ret, ptr, copysize);
-			chunk_dealloc_mmap(ptr, oldsize);
-		}
+			idalloc(ptr);
+		} else
+			huge_dalloc(ptr, false);
 	} else
 #endif
 	{
 		memcpy(ret, ptr, copysize);
-		iqallocx(ptr, try_tcache_dalloc);
+		idalloc(ptr);
 	}
 	return (ret);
 }
@@ -200,18 +279,24 @@ huge_dalloc(void *ptr, bool unmap)
 	assert(node->addr == ptr);
 	extent_tree_ad_remove(&huge, node);

-	if (config_stats) {
-		stats_cactive_sub(node->size);
-		huge_ndalloc++;
-		huge_allocated -= node->size;
-	}
+#ifdef JEMALLOC_STATS
+	stats_cactive_sub(node->size);
+	huge_ndalloc++;
+	huge_allocated -= node->size;
+#endif

 	malloc_mutex_unlock(&huge_mtx);

-	if (unmap && config_fill && config_dss && opt_junk)
-		memset(node->addr, 0x5a, node->size);
-
-	chunk_dealloc(node->addr, node->size, unmap);
+	if (unmap) {
+	/* Unmap chunk. */
+#ifdef JEMALLOC_FILL
+#if (defined(JEMALLOC_SWAP) || defined(JEMALLOC_DSS))
+		if (opt_junk)
+			memset(node->addr, 0x5a, node->size);
+#endif
+#endif
+		chunk_dealloc(node->addr, node->size);
+	}

 	base_node_dealloc(node);
 }
@@ -236,6 +321,7 @@ huge_salloc(const void *ptr)
 	return (ret);
 }

+#ifdef JEMALLOC_PROF
 prof_ctx_t *
 huge_prof_ctx_get(const void *ptr)
 {
@@ -272,6 +358,7 @@ huge_prof_ctx_set(const void *ptr, prof_ctx_t *ctx)

 	malloc_mutex_unlock(&huge_mtx);
 }
+#endif

 bool
 huge_boot(void)
@@ -282,32 +369,11 @@ huge_boot(void)
 		return (true);
 	extent_tree_ad_new(&huge);

-	if (config_stats) {
-		huge_nmalloc = 0;
-		huge_ndalloc = 0;
-		huge_allocated = 0;
-	}
+#ifdef JEMALLOC_STATS
+	huge_nmalloc = 0;
+	huge_ndalloc = 0;
+	huge_allocated = 0;
+#endif

 	return (false);
 }
-
-void
-huge_prefork(void)
-{
-
-	malloc_mutex_prefork(&huge_mtx);
-}
-
-void
-huge_postfork_parent(void)
-{
-
-	malloc_mutex_postfork_parent(&huge_mtx);
-}
-
-void
-huge_postfork_child(void)
-{
-
-	malloc_mutex_postfork_child(&huge_mtx);
-}
@@ -1,26 +1,14 @@
 #define	JEMALLOC_MUTEX_C_
 #include "jemalloc/internal/jemalloc_internal.h"

-#if defined(JEMALLOC_LAZY_LOCK) && !defined(_WIN32)
-#include <dlfcn.h>
-#endif
-
-#ifndef _CRT_SPINCOUNT
-#define _CRT_SPINCOUNT 4000
-#endif
-
 /******************************************************************************/
 /* Data. */

 #ifdef JEMALLOC_LAZY_LOCK
 bool isthreaded = false;
 #endif
-#ifdef JEMALLOC_MUTEX_INIT_CB
-static bool		postpone_init = true;
-static malloc_mutex_t	*postponed_mutexes = NULL;
-#endif

-#if defined(JEMALLOC_LAZY_LOCK) && !defined(_WIN32)
+#ifdef JEMALLOC_LAZY_LOCK
 static void	pthread_create_once(void);
 #endif

@@ -30,7 +18,7 @@ static void	pthread_create_once(void);
 * process goes multi-threaded.
 */

-#if defined(JEMALLOC_LAZY_LOCK) && !defined(_WIN32)
+#ifdef JEMALLOC_LAZY_LOCK
 static int (*pthread_create_fptr)(pthread_t *__restrict, const pthread_attr_t *,
    void *(*)(void *), void *__restrict);

@@ -48,7 +36,8 @@ pthread_create_once(void)
 	isthreaded = true;
 }

-JEMALLOC_EXPORT int
+JEMALLOC_ATTR(visibility("default"))
+int
 pthread_create(pthread_t *__restrict thread,
    const pthread_attr_t *__restrict attr, void *(*start_routine)(void *),
    void *__restrict arg)
@@ -63,87 +52,39 @@ pthread_create(pthread_t *__restrict thread,

 /******************************************************************************/

-#ifdef JEMALLOC_MUTEX_INIT_CB
-JEMALLOC_EXPORT int	_pthread_mutex_init_calloc_cb(pthread_mutex_t *mutex,
-    void *(calloc_cb)(size_t, size_t));
-#endif
-
 bool
 malloc_mutex_init(malloc_mutex_t *mutex)
 {
-
-#ifdef _WIN32
-	if (!InitializeCriticalSectionAndSpinCount(&mutex->lock,
-	    _CRT_SPINCOUNT))
-		return (true);
-#elif (defined(JEMALLOC_OSSPIN))
-	mutex->lock = 0;
-#elif (defined(JEMALLOC_MUTEX_INIT_CB))
-	if (postpone_init) {
-		mutex->postponed_next = postponed_mutexes;
-		postponed_mutexes = mutex;
-	} else {
-		if (_pthread_mutex_init_calloc_cb(&mutex->lock, base_calloc) !=
-		    0)
-			return (true);
-	}
+#ifdef JEMALLOC_OSSPIN
+	*mutex = 0;
 #else
 	pthread_mutexattr_t attr;

 	if (pthread_mutexattr_init(&attr) != 0)
 		return (true);
-	pthread_mutexattr_settype(&attr, MALLOC_MUTEX_TYPE);
-	if (pthread_mutex_init(&mutex->lock, &attr) != 0) {
+#ifdef PTHREAD_MUTEX_ADAPTIVE_NP
+	pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ADAPTIVE_NP);
+#else
+	pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_DEFAULT);
+#endif
+	if (pthread_mutex_init(mutex, &attr) != 0) {
 		pthread_mutexattr_destroy(&attr);
 		return (true);
 	}
 	pthread_mutexattr_destroy(&attr);
+
 #endif
 	return (false);
 }

 void
-malloc_mutex_prefork(malloc_mutex_t *mutex)
+malloc_mutex_destroy(malloc_mutex_t *mutex)
 {

-	malloc_mutex_lock(mutex);
-}
-
-void
-malloc_mutex_postfork_parent(malloc_mutex_t *mutex)
-{
-
-	malloc_mutex_unlock(mutex);
-}
-
-void
-malloc_mutex_postfork_child(malloc_mutex_t *mutex)
-{
-
-#ifdef JEMALLOC_MUTEX_INIT_CB
-	malloc_mutex_unlock(mutex);
-#else
-	if (malloc_mutex_init(mutex)) {
-		malloc_printf("<jemalloc>: Error re-initializing mutex in "
-		    "child\n");
-		if (opt_abort)
-			abort();
+#ifndef JEMALLOC_OSSPIN
+	if (pthread_mutex_destroy(mutex) != 0) {
+		malloc_write("<jemalloc>: Error in pthread_mutex_destroy()\n");
+		abort();
 	}
 #endif
 }
-
-bool
-mutex_boot(void)
-{
-
-#ifdef JEMALLOC_MUTEX_INIT_CB
-	postpone_init = false;
-	while (postponed_mutexes != NULL) {
-		if (_pthread_mutex_init_calloc_cb(&postponed_mutexes->lock,
-		    base_calloc) != 0)
-			return (true);
-		postponed_mutexes = postponed_mutexes->postponed_next;
-	}
-#endif
-	return (false);
-}
@@ -1,210 +0,0 @@
-#include "jemalloc/internal/jemalloc_internal.h"
-
-/*
- * quarantine pointers close to NULL are used to encode state information that
- * is used for cleaning up during thread shutdown.
- */
-#define	QUARANTINE_STATE_REINCARNATED	((quarantine_t *)(uintptr_t)1)
-#define	QUARANTINE_STATE_PURGATORY	((quarantine_t *)(uintptr_t)2)
-#define	QUARANTINE_STATE_MAX		QUARANTINE_STATE_PURGATORY
-
-/******************************************************************************/
-/* Data. */
-
-typedef struct quarantine_obj_s quarantine_obj_t;
-typedef struct quarantine_s quarantine_t;
-
-struct quarantine_obj_s {
-	void	*ptr;
-	size_t	usize;
-};
-
-struct quarantine_s {
-	size_t			curbytes;
-	size_t			curobjs;
-	size_t			first;
-#define	LG_MAXOBJS_INIT 10
-	size_t			lg_maxobjs;
-	quarantine_obj_t	objs[1]; /* Dynamically sized ring buffer. */
-};
-
-static void	quarantine_cleanup(void *arg);
-
-malloc_tsd_data(static, quarantine, quarantine_t *, NULL)
-malloc_tsd_funcs(JEMALLOC_INLINE, quarantine, quarantine_t *, NULL,
-    quarantine_cleanup)
-
-/******************************************************************************/
-/* Function prototypes for non-inline static functions. */
-
-static quarantine_t	*quarantine_init(size_t lg_maxobjs);
-static quarantine_t	*quarantine_grow(quarantine_t *quarantine);
-static void	quarantine_drain(quarantine_t *quarantine, size_t upper_bound);
-
-/******************************************************************************/
-
-static quarantine_t *
-quarantine_init(size_t lg_maxobjs)
-{
-	quarantine_t *quarantine;
-
-	quarantine = (quarantine_t *)imalloc(offsetof(quarantine_t, objs) +
-	    ((ZU(1) << lg_maxobjs) * sizeof(quarantine_obj_t)));
-	if (quarantine == NULL)
-		return (NULL);
-	quarantine->curbytes = 0;
-	quarantine->curobjs = 0;
-	quarantine->first = 0;
-	quarantine->lg_maxobjs = lg_maxobjs;
-
-	quarantine_tsd_set(&quarantine);
-
-	return (quarantine);
-}
-
-static quarantine_t *
-quarantine_grow(quarantine_t *quarantine)
-{
-	quarantine_t *ret;
-
-	ret = quarantine_init(quarantine->lg_maxobjs + 1);
-	if (ret == NULL)
-		return (quarantine);
-
-	ret->curbytes = quarantine->curbytes;
-	ret->curobjs = quarantine->curobjs;
-	if (quarantine->first + quarantine->curobjs <= (ZU(1) <<
-	    quarantine->lg_maxobjs)) {
-		/* objs ring buffer data are contiguous. */
-		memcpy(ret->objs, &quarantine->objs[quarantine->first],
-		    quarantine->curobjs * sizeof(quarantine_obj_t));
-	} else {
-		/* objs ring buffer data wrap around. */
-		size_t ncopy_a = (ZU(1) << quarantine->lg_maxobjs) -
-		    quarantine->first;
-		size_t ncopy_b = quarantine->curobjs - ncopy_a;
-
-		memcpy(ret->objs, &quarantine->objs[quarantine->first], ncopy_a
-		    * sizeof(quarantine_obj_t));
-		memcpy(&ret->objs[ncopy_a], quarantine->objs, ncopy_b *
-		    sizeof(quarantine_obj_t));
-	}
-
-	return (ret);
-}
-
-static void
-quarantine_drain(quarantine_t *quarantine, size_t upper_bound)
-{
-
-	while (quarantine->curbytes > upper_bound && quarantine->curobjs > 0) {
-		quarantine_obj_t *obj = &quarantine->objs[quarantine->first];
-		assert(obj->usize == isalloc(obj->ptr, config_prof));
-		idalloc(obj->ptr);
-		quarantine->curbytes -= obj->usize;
-		quarantine->curobjs--;
-		quarantine->first = (quarantine->first + 1) & ((ZU(1) <<
-		    quarantine->lg_maxobjs) - 1);
-	}
-}
-
-void
-quarantine(void *ptr)
-{
-	quarantine_t *quarantine;
-	size_t usize = isalloc(ptr, config_prof);
-
-	cassert(config_fill);
-	assert(opt_quarantine);
-
-	quarantine = *quarantine_tsd_get();
-	if ((uintptr_t)quarantine <= (uintptr_t)QUARANTINE_STATE_MAX) {
-		if (quarantine == NULL) {
-			if ((quarantine = quarantine_init(LG_MAXOBJS_INIT)) ==
-			    NULL) {
-				idalloc(ptr);
-				return;
-			}
-		} else {
-			if (quarantine == QUARANTINE_STATE_PURGATORY) {
-				/*
-				 * Make a note that quarantine() was called
-				 * after quarantine_cleanup() was called.
-				 */
-				quarantine = QUARANTINE_STATE_REINCARNATED;
-				quarantine_tsd_set(&quarantine);
-			}
-			idalloc(ptr);
-			return;
-		}
-	}
-	/*
-	 * Drain one or more objects if the quarantine size limit would be
-	 * exceeded by appending ptr.
-	 */
-	if (quarantine->curbytes + usize > opt_quarantine) {
-		size_t upper_bound = (opt_quarantine >= usize) ? opt_quarantine
-		    - usize : 0;
-		quarantine_drain(quarantine, upper_bound);
-	}
-	/* Grow the quarantine ring buffer if it's full. */
-	if (quarantine->curobjs == (ZU(1) << quarantine->lg_maxobjs))
-		quarantine = quarantine_grow(quarantine);
-	/* quarantine_grow() must free a slot if it fails to grow. */
-	assert(quarantine->curobjs < (ZU(1) << quarantine->lg_maxobjs));
-	/* Append ptr if its size doesn't exceed the quarantine size. */
-	if (quarantine->curbytes + usize <= opt_quarantine) {
-		size_t offset = (quarantine->first + quarantine->curobjs) &
-		    ((ZU(1) << quarantine->lg_maxobjs) - 1);
-		quarantine_obj_t *obj = &quarantine->objs[offset];
-		obj->ptr = ptr;
-		obj->usize = usize;
-		quarantine->curbytes += usize;
-		quarantine->curobjs++;
-		if (opt_junk)
-			memset(ptr, 0x5a, usize);
-	} else {
-		assert(quarantine->curbytes == 0);
-		idalloc(ptr);
-	}
-}
-
-static void
-quarantine_cleanup(void *arg)
-{
-	quarantine_t *quarantine = *(quarantine_t **)arg;
-
-	if (quarantine == QUARANTINE_STATE_REINCARNATED) {
-		/*
-		 * Another destructor deallocated memory after this destructor
-		 * was called.  Reset quarantine to QUARANTINE_STATE_PURGATORY
-		 * in order to receive another callback.
-		 */
-		quarantine = QUARANTINE_STATE_PURGATORY;
-		quarantine_tsd_set(&quarantine);
-	} else if (quarantine == QUARANTINE_STATE_PURGATORY) {
-		/*
-		 * The previous time this destructor was called, we set the key
-		 * to QUARANTINE_STATE_PURGATORY so that other destructors
-		 * wouldn't cause re-creation of the quarantine.  This time, do
-		 * nothing, so that the destructor will not be called again.
-		 */
-	} else if (quarantine != NULL) {
-		quarantine_drain(quarantine, 0);
-		idalloc(quarantine);
-		quarantine = QUARANTINE_STATE_PURGATORY;
-		quarantine_tsd_set(&quarantine);
-	}
-}
-
-bool
-quarantine_boot(void)
-{
-
-	cassert(config_fill);
-
-	if (quarantine_tsd_boot())
-		return (true);
-
-	return (false);
-}
@@ -44,24 +44,3 @@ rtree_new(unsigned bits)

 	return (ret);
 }
-
-void
-rtree_prefork(rtree_t *rtree)
-{
-
-	malloc_mutex_prefork(&rtree->mutex);
-}
-
-void
-rtree_postfork_parent(rtree_t *rtree)
-{
-
-	malloc_mutex_postfork_parent(&rtree->mutex);
-}
-
-void
-rtree_postfork_child(rtree_t *rtree)
-{
-
-	malloc_mutex_postfork_child(&rtree->mutex);
-}
@@ -39,40 +39,140 @@

 bool	opt_stats_print = false;

+#ifdef JEMALLOC_STATS
 size_t	stats_cactive = 0;
+#endif

 /******************************************************************************/
 /* Function prototypes for non-inline static functions. */

+#ifdef JEMALLOC_STATS
+static void	malloc_vcprintf(void (*write_cb)(void *, const char *),
+    void *cbopaque, const char *format, va_list ap);
 static void	stats_arena_bins_print(void (*write_cb)(void *, const char *),
    void *cbopaque, unsigned i);
 static void	stats_arena_lruns_print(void (*write_cb)(void *, const char *),
    void *cbopaque, unsigned i);
 static void	stats_arena_print(void (*write_cb)(void *, const char *),
-    void *cbopaque, unsigned i, bool bins, bool large);
+    void *cbopaque, unsigned i);
+#endif

 /******************************************************************************/

+/*
+ * We don't want to depend on vsnprintf() for production builds, since that can
+ * cause unnecessary bloat for static binaries.  u2s() provides minimal integer
+ * printing functionality, so that malloc_printf() use can be limited to
+ * JEMALLOC_STATS code.
+ */
+char *
+u2s(uint64_t x, unsigned base, char *s)
+{
+	unsigned i;
+
+	i = UMAX2S_BUFSIZE - 1;
+	s[i] = '\0';
+	switch (base) {
+	case 10:
+		do {
+			i--;
+			s[i] = "0123456789"[x % (uint64_t)10];
+			x /= (uint64_t)10;
+		} while (x > 0);
+		break;
+	case 16:
+		do {
+			i--;
+			s[i] = "0123456789abcdef"[x & 0xf];
+			x >>= 4;
+		} while (x > 0);
+		break;
+	default:
+		do {
+			i--;
+			s[i] = "0123456789abcdefghijklmnopqrstuvwxyz"[x %
+			    (uint64_t)base];
+			x /= (uint64_t)base;
+		} while (x > 0);
+	}
+
+	return (&s[i]);
+}
+
+#ifdef JEMALLOC_STATS
+static void
+malloc_vcprintf(void (*write_cb)(void *, const char *), void *cbopaque,
+    const char *format, va_list ap)
+{
+	char buf[4096];
+
+	if (write_cb == NULL) {
+		/*
+		 * The caller did not provide an alternate write_cb callback
+		 * function, so use the default one.  malloc_write() is an
+		 * inline function, so use malloc_message() directly here.
+		 */
+		write_cb = JEMALLOC_P(malloc_message);
+		cbopaque = NULL;
+	}
+
+	vsnprintf(buf, sizeof(buf), format, ap);
+	write_cb(cbopaque, buf);
+}
+
+/*
+ * Print to a callback function in such a way as to (hopefully) avoid memory
+ * allocation.
+ */
+JEMALLOC_ATTR(format(printf, 3, 4))
+void
+malloc_cprintf(void (*write_cb)(void *, const char *), void *cbopaque,
+    const char *format, ...)
+{
+	va_list ap;
+
+	va_start(ap, format);
+	malloc_vcprintf(write_cb, cbopaque, format, ap);
+	va_end(ap);
+}
+
+/*
+ * Print to stderr in such a way as to (hopefully) avoid memory allocation.
+ */
+JEMALLOC_ATTR(format(printf, 1, 2))
+void
+malloc_printf(const char *format, ...)
+{
+	va_list ap;
+
+	va_start(ap, format);
+	malloc_vcprintf(NULL, NULL, format, ap);
+	va_end(ap);
+}
+#endif
+
+#ifdef JEMALLOC_STATS
 static void
 stats_arena_bins_print(void (*write_cb)(void *, const char *), void *cbopaque,
    unsigned i)
 {
-	size_t page;
+	size_t pagesize;
 	bool config_tcache;
 	unsigned nbins, j, gap_start;

-	CTL_GET("arenas.page", &page, size_t);
+	CTL_GET("arenas.pagesize", &pagesize, size_t);

 	CTL_GET("config.tcache", &config_tcache, bool);
 	if (config_tcache) {
 		malloc_cprintf(write_cb, cbopaque,
-		    "bins:     bin  size regs pgs    allocated      nmalloc"
+		    "bins:     bin    size regs pgs    allocated      nmalloc"
 		    "      ndalloc    nrequests       nfills     nflushes"
-		    "      newruns       reruns      curruns\n");
+		    "      newruns       reruns      maxruns      curruns\n");
 	} else {
 		malloc_cprintf(write_cb, cbopaque,
-		    "bins:     bin  size regs pgs    allocated      nmalloc"
-		    "      ndalloc      newruns       reruns      curruns\n");
+		    "bins:     bin    size regs pgs    allocated      nmalloc"
+		    "      ndalloc      newruns       reruns      maxruns"
+		    "      curruns\n");
 	}
 	CTL_GET("arenas.nbins", &nbins, unsigned);
 	for (j = 0, gap_start = UINT_MAX; j < nbins; j++) {
@@ -83,11 +183,12 @@ stats_arena_bins_print(void (*write_cb)(void *, const char *), void *cbopaque,
 			if (gap_start == UINT_MAX)
 				gap_start = j;
 		} else {
+			unsigned ntbins_, nqbins, ncbins, nsbins;
 			size_t reg_size, run_size, allocated;
 			uint32_t nregs;
 			uint64_t nmalloc, ndalloc, nrequests, nfills, nflushes;
 			uint64_t reruns;
-			size_t curruns;
+			size_t highruns, curruns;

 			if (gap_start != UINT_MAX) {
 				if (j > gap_start + 1) {
@@ -102,6 +203,10 @@ stats_arena_bins_print(void (*write_cb)(void *, const char *), void *cbopaque,
 				}
 				gap_start = UINT_MAX;
 			}
+			CTL_GET("arenas.ntbins", &ntbins_, unsigned);
+			CTL_GET("arenas.nqbins", &nqbins, unsigned);
+			CTL_GET("arenas.ncbins", &ncbins, unsigned);
+			CTL_GET("arenas.nsbins", &nsbins, unsigned);
 			CTL_J_GET("arenas.bin.0.size", &reg_size, size_t);
 			CTL_J_GET("arenas.bin.0.nregs", &nregs, uint32_t);
 			CTL_J_GET("arenas.bin.0.run_size", &run_size, size_t);
@@ -121,25 +226,36 @@ stats_arena_bins_print(void (*write_cb)(void *, const char *), void *cbopaque,
 			}
 			CTL_IJ_GET("stats.arenas.0.bins.0.nreruns", &reruns,
 			    uint64_t);
+			CTL_IJ_GET("stats.arenas.0.bins.0.highruns", &highruns,
+			    size_t);
 			CTL_IJ_GET("stats.arenas.0.bins.0.curruns", &curruns,
 			    size_t);
 			if (config_tcache) {
 				malloc_cprintf(write_cb, cbopaque,
-				    "%13u %5zu %4u %3zu %12zu %12"PRIu64
+				    "%13u %1s %5zu %4u %3zu %12zu %12"PRIu64
 				    " %12"PRIu64" %12"PRIu64" %12"PRIu64
 				    " %12"PRIu64" %12"PRIu64" %12"PRIu64
-				    " %12zu\n",
-				    j, reg_size, nregs, run_size / page,
+				    " %12zu %12zu\n",
+				    j,
+				    j < ntbins_ ? "T" : j < ntbins_ + nqbins ?
+				    "Q" : j < ntbins_ + nqbins + ncbins ? "C" :
+				    "S",
+				    reg_size, nregs, run_size / pagesize,
 				    allocated, nmalloc, ndalloc, nrequests,
-				    nfills, nflushes, nruns, reruns, curruns);
+				    nfills, nflushes, nruns, reruns, highruns,
+				    curruns);
 			} else {
 				malloc_cprintf(write_cb, cbopaque,
-				    "%13u %5zu %4u %3zu %12zu %12"PRIu64
+				    "%13u %1s %5zu %4u %3zu %12zu %12"PRIu64
 				    " %12"PRIu64" %12"PRIu64" %12"PRIu64
-				    " %12zu\n",
-				    j, reg_size, nregs, run_size / page,
+				    " %12zu %12zu\n",
+				    j,
+				    j < ntbins_ ? "T" : j < ntbins_ + nqbins ?
+				    "Q" : j < ntbins_ + nqbins + ncbins ? "C" :
+				    "S",
+				    reg_size, nregs, run_size / pagesize,
 				    allocated, nmalloc, ndalloc, nruns, reruns,
-				    curruns);
+				    highruns, curruns);
 			}
 		}
 	}
@@ -159,18 +275,18 @@ static void
 stats_arena_lruns_print(void (*write_cb)(void *, const char *), void *cbopaque,
    unsigned i)
 {
-	size_t page, nlruns, j;
+	size_t pagesize, nlruns, j;
 	ssize_t gap_start;

-	CTL_GET("arenas.page", &page, size_t);
+	CTL_GET("arenas.pagesize", &pagesize, size_t);

 	malloc_cprintf(write_cb, cbopaque,
 	    "large:   size pages      nmalloc      ndalloc    nrequests"
-	    "      curruns\n");
+	    "      maxruns      curruns\n");
 	CTL_GET("arenas.nlruns", &nlruns, size_t);
 	for (j = 0, gap_start = -1; j < nlruns; j++) {
 		uint64_t nmalloc, ndalloc, nrequests;
-		size_t run_size, curruns;
+		size_t run_size, highruns, curruns;

 		CTL_IJ_GET("stats.arenas.0.lruns.0.nmalloc", &nmalloc,
 		    uint64_t);
@@ -183,6 +299,8 @@ stats_arena_lruns_print(void (*write_cb)(void *, const char *), void *cbopaque,
 				gap_start = j;
 		} else {
 			CTL_J_GET("arenas.lrun.0.size", &run_size, size_t);
+			CTL_IJ_GET("stats.arenas.0.lruns.0.highruns", &highruns,
+			    size_t);
 			CTL_IJ_GET("stats.arenas.0.lruns.0.curruns", &curruns,
 			    size_t);
 			if (gap_start != -1) {
@@ -192,9 +310,9 @@ stats_arena_lruns_print(void (*write_cb)(void *, const char *), void *cbopaque,
 			}
 			malloc_cprintf(write_cb, cbopaque,
 			    "%13zu %5zu %12"PRIu64" %12"PRIu64" %12"PRIu64
-			    " %12zu\n",
-			    run_size, run_size / page, nmalloc, ndalloc,
-			    nrequests, curruns);
+			    " %12zu %12zu\n",
+			    run_size, run_size / pagesize, nmalloc, ndalloc,
+			    nrequests, highruns, curruns);
 		}
 	}
 	if (gap_start != -1)
@@ -203,25 +321,21 @@ stats_arena_lruns_print(void (*write_cb)(void *, const char *), void *cbopaque,

 static void
 stats_arena_print(void (*write_cb)(void *, const char *), void *cbopaque,
-    unsigned i, bool bins, bool large)
+    unsigned i)
 {
 	unsigned nthreads;
-	const char *dss;
-	size_t page, pactive, pdirty, mapped;
+	size_t pagesize, pactive, pdirty, mapped;
 	uint64_t npurge, nmadvise, purged;
 	size_t small_allocated;
 	uint64_t small_nmalloc, small_ndalloc, small_nrequests;
 	size_t large_allocated;
 	uint64_t large_nmalloc, large_ndalloc, large_nrequests;

-	CTL_GET("arenas.page", &page, size_t);
+	CTL_GET("arenas.pagesize", &pagesize, size_t);

 	CTL_I_GET("stats.arenas.0.nthreads", &nthreads, unsigned);
 	malloc_cprintf(write_cb, cbopaque,
 	    "assigned threads: %u\n", nthreads);
-	CTL_I_GET("stats.arenas.0.dss", &dss, const char *);
-	malloc_cprintf(write_cb, cbopaque, "dss allocation precedence: %s\n",
-	    dss);
 	CTL_I_GET("stats.arenas.0.pactive", &pactive, size_t);
 	CTL_I_GET("stats.arenas.0.pdirty", &pdirty, size_t);
 	CTL_I_GET("stats.arenas.0.npurge", &npurge, uint64_t);
@@ -255,15 +369,15 @@ stats_arena_print(void (*write_cb)(void *, const char *), void *cbopaque,
 	    small_nmalloc + large_nmalloc,
 	    small_ndalloc + large_ndalloc,
 	    small_nrequests + large_nrequests);
-	malloc_cprintf(write_cb, cbopaque, "active:  %12zu\n", pactive * page);
+	malloc_cprintf(write_cb, cbopaque, "active:  %12zu\n",
+	    pactive * pagesize );
 	CTL_I_GET("stats.arenas.0.mapped", &mapped, size_t);
 	malloc_cprintf(write_cb, cbopaque, "mapped:  %12zu\n", mapped);

-	if (bins)
-		stats_arena_bins_print(write_cb, cbopaque, i);
-	if (large)
-		stats_arena_lruns_print(write_cb, cbopaque, i);
+	stats_arena_bins_print(write_cb, cbopaque, i);
+	stats_arena_lruns_print(write_cb, cbopaque, i);
 }
+#endif

 void
 stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
@@ -272,6 +386,7 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 	int err;
 	uint64_t epoch;
 	size_t u64sz;
+	char s[UMAX2S_BUFSIZE];
 	bool general = true;
 	bool merged = true;
 	bool unmerged = true;
@@ -287,7 +402,8 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 	 * */
 	epoch = 1;
 	u64sz = sizeof(uint64_t);
-	err = je_mallctl("epoch", &epoch, &u64sz, &epoch, sizeof(uint64_t));
+	err = JEMALLOC_P(mallctl)("epoch", &epoch, &u64sz, &epoch,
+	    sizeof(uint64_t));
 	if (err != 0) {
 		if (err == EAGAIN) {
 			malloc_write("<jemalloc>: Memory allocation failure in "
@@ -299,33 +415,42 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 		abort();
 	}

+	if (write_cb == NULL) {
+		/*
+		 * The caller did not provide an alternate write_cb callback
+		 * function, so use the default one.  malloc_write() is an
+		 * inline function, so use malloc_message() directly here.
+		 */
+		write_cb = JEMALLOC_P(malloc_message);
+		cbopaque = NULL;
+	}
+
 	if (opts != NULL) {
 		unsigned i;

 		for (i = 0; opts[i] != '\0'; i++) {
 			switch (opts[i]) {
-			case 'g':
-				general = false;
-				break;
-			case 'm':
-				merged = false;
-				break;
-			case 'a':
-				unmerged = false;
-				break;
-			case 'b':
-				bins = false;
-				break;
-			case 'l':
-				large = false;
-				break;
-			default:;
+				case 'g':
+					general = false;
+					break;
+				case 'm':
+					merged = false;
+					break;
+				case 'a':
+					unmerged = false;
+					break;
+				case 'b':
+					bins = false;
+					break;
+				case 'l':
+					large = false;
+					break;
+				default:;
 			}
 		}
 	}

-	malloc_cprintf(write_cb, cbopaque,
-	    "___ Begin jemalloc statistics ___\n");
+	write_cb(cbopaque, "___ Begin jemalloc statistics ___\n");
 	if (general) {
 		int err;
 		const char *cpv;
@@ -340,127 +465,229 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 		cpsz = sizeof(const char *);

 		CTL_GET("version", &cpv, const char *);
-		malloc_cprintf(write_cb, cbopaque, "Version: %s\n", cpv);
+		write_cb(cbopaque, "Version: ");
+		write_cb(cbopaque, cpv);
+		write_cb(cbopaque, "\n");
 		CTL_GET("config.debug", &bv, bool);
-		malloc_cprintf(write_cb, cbopaque, "Assertions %s\n",
-		    bv ? "enabled" : "disabled");
+		write_cb(cbopaque, "Assertions ");
+		write_cb(cbopaque, bv ? "enabled" : "disabled");
+		write_cb(cbopaque, "\n");

 #define OPT_WRITE_BOOL(n)						\
-		if ((err = je_mallctl("opt."#n, &bv, &bsz, NULL, 0))	\
-		    == 0) {						\
-			malloc_cprintf(write_cb, cbopaque,		\
-			    "  opt."#n": %s\n", bv ? "true" : "false");	\
+		if ((err = JEMALLOC_P(mallctl)("opt."#n, &bv, &bsz,	\
+		    NULL, 0)) == 0) {					\
+			write_cb(cbopaque, "  opt."#n": ");		\
+			write_cb(cbopaque, bv ? "true" : "false");	\
+			write_cb(cbopaque, "\n");			\
 		}
 #define OPT_WRITE_SIZE_T(n)						\
-		if ((err = je_mallctl("opt."#n, &sv, &ssz, NULL, 0))	\
-		    == 0) {						\
-			malloc_cprintf(write_cb, cbopaque,		\
-			"  opt."#n": %zu\n", sv);			\
+		if ((err = JEMALLOC_P(mallctl)("opt."#n, &sv, &ssz,	\
+		    NULL, 0)) == 0) {					\
+			write_cb(cbopaque, "  opt."#n": ");		\
+			write_cb(cbopaque, u2s(sv, 10, s));		\
+			write_cb(cbopaque, "\n");			\
 		}
 #define OPT_WRITE_SSIZE_T(n)						\
-		if ((err = je_mallctl("opt."#n, &ssv, &sssz, NULL, 0))	\
-		    == 0) {						\
-			malloc_cprintf(write_cb, cbopaque,		\
-			    "  opt."#n": %zd\n", ssv);			\
+		if ((err = JEMALLOC_P(mallctl)("opt."#n, &ssv, &sssz,	\
+		    NULL, 0)) == 0) {					\
+			if (ssv >= 0) {					\
+				write_cb(cbopaque, "  opt."#n": ");	\
+				write_cb(cbopaque, u2s(ssv, 10, s));	\
+			} else {					\
+				write_cb(cbopaque, "  opt."#n": -");	\
+				write_cb(cbopaque, u2s(-ssv, 10, s));	\
+			}						\
+			write_cb(cbopaque, "\n");			\
 		}
 #define OPT_WRITE_CHAR_P(n)						\
-		if ((err = je_mallctl("opt."#n, &cpv, &cpsz, NULL, 0))	\
-		    == 0) {						\
-			malloc_cprintf(write_cb, cbopaque,		\
-			    "  opt."#n": \"%s\"\n", cpv);		\
+		if ((err = JEMALLOC_P(mallctl)("opt."#n, &cpv, &cpsz,	\
+		    NULL, 0)) == 0) {					\
+			write_cb(cbopaque, "  opt."#n": \"");		\
+			write_cb(cbopaque, cpv);			\
+			write_cb(cbopaque, "\"\n");			\
 		}

-		malloc_cprintf(write_cb, cbopaque,
-		    "Run-time option settings:\n");
+		write_cb(cbopaque, "Run-time option settings:\n");
 		OPT_WRITE_BOOL(abort)
+		OPT_WRITE_SIZE_T(lg_qspace_max)
+		OPT_WRITE_SIZE_T(lg_cspace_max)
 		OPT_WRITE_SIZE_T(lg_chunk)
-		OPT_WRITE_CHAR_P(dss)
 		OPT_WRITE_SIZE_T(narenas)
 		OPT_WRITE_SSIZE_T(lg_dirty_mult)
 		OPT_WRITE_BOOL(stats_print)
 		OPT_WRITE_BOOL(junk)
-		OPT_WRITE_SIZE_T(quarantine)
-		OPT_WRITE_BOOL(redzone)
 		OPT_WRITE_BOOL(zero)
-		OPT_WRITE_BOOL(utrace)
-		OPT_WRITE_BOOL(valgrind)
+		OPT_WRITE_BOOL(sysv)
 		OPT_WRITE_BOOL(xmalloc)
 		OPT_WRITE_BOOL(tcache)
+		OPT_WRITE_SSIZE_T(lg_tcache_gc_sweep)
 		OPT_WRITE_SSIZE_T(lg_tcache_max)
 		OPT_WRITE_BOOL(prof)
 		OPT_WRITE_CHAR_P(prof_prefix)
+		OPT_WRITE_SIZE_T(lg_prof_bt_max)
 		OPT_WRITE_BOOL(prof_active)
 		OPT_WRITE_SSIZE_T(lg_prof_sample)
 		OPT_WRITE_BOOL(prof_accum)
+		OPT_WRITE_SSIZE_T(lg_prof_tcmax)
 		OPT_WRITE_SSIZE_T(lg_prof_interval)
 		OPT_WRITE_BOOL(prof_gdump)
-		OPT_WRITE_BOOL(prof_final)
 		OPT_WRITE_BOOL(prof_leak)
+		OPT_WRITE_BOOL(overcommit)

 #undef OPT_WRITE_BOOL
 #undef OPT_WRITE_SIZE_T
 #undef OPT_WRITE_SSIZE_T
 #undef OPT_WRITE_CHAR_P

-		malloc_cprintf(write_cb, cbopaque, "CPUs: %u\n", ncpus);
+		write_cb(cbopaque, "CPUs: ");
+		write_cb(cbopaque, u2s(ncpus, 10, s));
+		write_cb(cbopaque, "\n");

 		CTL_GET("arenas.narenas", &uv, unsigned);
-		malloc_cprintf(write_cb, cbopaque, "Arenas: %u\n", uv);
+		write_cb(cbopaque, "Max arenas: ");
+		write_cb(cbopaque, u2s(uv, 10, s));
+		write_cb(cbopaque, "\n");

-		malloc_cprintf(write_cb, cbopaque, "Pointer size: %zu\n",
-		    sizeof(void *));
+		write_cb(cbopaque, "Pointer size: ");
+		write_cb(cbopaque, u2s(sizeof(void *), 10, s));
+		write_cb(cbopaque, "\n");

 		CTL_GET("arenas.quantum", &sv, size_t);
-		malloc_cprintf(write_cb, cbopaque, "Quantum size: %zu\n", sv);
+		write_cb(cbopaque, "Quantum size: ");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "\n");

-		CTL_GET("arenas.page", &sv, size_t);
-		malloc_cprintf(write_cb, cbopaque, "Page size: %zu\n", sv);
+		CTL_GET("arenas.cacheline", &sv, size_t);
+		write_cb(cbopaque, "Cacheline size (assumed): ");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "\n");
+
+		CTL_GET("arenas.subpage", &sv, size_t);
+		write_cb(cbopaque, "Subpage spacing: ");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "\n");
+
+		if ((err = JEMALLOC_P(mallctl)("arenas.tspace_min", &sv, &ssz,
+		    NULL, 0)) == 0) {
+			write_cb(cbopaque, "Tiny 2^n-spaced sizes: [");
+			write_cb(cbopaque, u2s(sv, 10, s));
+			write_cb(cbopaque, "..");
+
+			CTL_GET("arenas.tspace_max", &sv, size_t);
+			write_cb(cbopaque, u2s(sv, 10, s));
+			write_cb(cbopaque, "]\n");
+		}
+
+		CTL_GET("arenas.qspace_min", &sv, size_t);
+		write_cb(cbopaque, "Quantum-spaced sizes: [");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "..");
+		CTL_GET("arenas.qspace_max", &sv, size_t);
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "]\n");
+
+		CTL_GET("arenas.cspace_min", &sv, size_t);
+		write_cb(cbopaque, "Cacheline-spaced sizes: [");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "..");
+		CTL_GET("arenas.cspace_max", &sv, size_t);
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "]\n");
+
+		CTL_GET("arenas.sspace_min", &sv, size_t);
+		write_cb(cbopaque, "Subpage-spaced sizes: [");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "..");
+		CTL_GET("arenas.sspace_max", &sv, size_t);
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, "]\n");

 		CTL_GET("opt.lg_dirty_mult", &ssv, ssize_t);
 		if (ssv >= 0) {
-			malloc_cprintf(write_cb, cbopaque,
-			    "Min active:dirty page ratio per arena: %u:1\n",
-			    (1U << ssv));
+			write_cb(cbopaque,
+			    "Min active:dirty page ratio per arena: ");
+			write_cb(cbopaque, u2s((1U << ssv), 10, s));
+			write_cb(cbopaque, ":1\n");
 		} else {
-			malloc_cprintf(write_cb, cbopaque,
+			write_cb(cbopaque,
 			    "Min active:dirty page ratio per arena: N/A\n");
 		}
-		if ((err = je_mallctl("arenas.tcache_max", &sv, &ssz, NULL, 0))
-		    == 0) {
-			malloc_cprintf(write_cb, cbopaque,
-			    "Maximum thread-cached size class: %zu\n", sv);
+		if ((err = JEMALLOC_P(mallctl)("arenas.tcache_max", &sv,
+		    &ssz, NULL, 0)) == 0) {
+			write_cb(cbopaque,
+			    "Maximum thread-cached size class: ");
+			write_cb(cbopaque, u2s(sv, 10, s));
+			write_cb(cbopaque, "\n");
 		}
-		if ((err = je_mallctl("opt.prof", &bv, &bsz, NULL, 0)) == 0 &&
-		    bv) {
+		if ((err = JEMALLOC_P(mallctl)("opt.lg_tcache_gc_sweep", &ssv,
+		    &ssz, NULL, 0)) == 0) {
+			size_t tcache_gc_sweep = (1U << ssv);
+			bool tcache_enabled;
+			CTL_GET("opt.tcache", &tcache_enabled, bool);
+			write_cb(cbopaque, "Thread cache GC sweep interval: ");
+			write_cb(cbopaque, tcache_enabled && ssv >= 0 ?
+			    u2s(tcache_gc_sweep, 10, s) : "N/A");
+			write_cb(cbopaque, "\n");
+		}
+		if ((err = JEMALLOC_P(mallctl)("opt.prof", &bv, &bsz, NULL, 0))
+		   == 0 && bv) {
+			CTL_GET("opt.lg_prof_bt_max", &sv, size_t);
+			write_cb(cbopaque, "Maximum profile backtrace depth: ");
+			write_cb(cbopaque, u2s((1U << sv), 10, s));
+			write_cb(cbopaque, "\n");
+
+			CTL_GET("opt.lg_prof_tcmax", &ssv, ssize_t);
+			write_cb(cbopaque,
+			    "Maximum per thread backtrace cache: ");
+			if (ssv >= 0) {
+				write_cb(cbopaque, u2s((1U << ssv), 10, s));
+				write_cb(cbopaque, " (2^");
+				write_cb(cbopaque, u2s(ssv, 10, s));
+				write_cb(cbopaque, ")\n");
+			} else
+				write_cb(cbopaque, "N/A\n");
+
 			CTL_GET("opt.lg_prof_sample", &sv, size_t);
-			malloc_cprintf(write_cb, cbopaque,
-			    "Average profile sample interval: %"PRIu64
-			    " (2^%zu)\n", (((uint64_t)1U) << sv), sv);
+			write_cb(cbopaque, "Average profile sample interval: ");
+			write_cb(cbopaque, u2s((((uint64_t)1U) << sv), 10, s));
+			write_cb(cbopaque, " (2^");
+			write_cb(cbopaque, u2s(sv, 10, s));
+			write_cb(cbopaque, ")\n");

 			CTL_GET("opt.lg_prof_interval", &ssv, ssize_t);
+			write_cb(cbopaque, "Average profile dump interval: ");
 			if (ssv >= 0) {
-				malloc_cprintf(write_cb, cbopaque,
-				    "Average profile dump interval: %"PRIu64
-				    " (2^%zd)\n",
-				    (((uint64_t)1U) << ssv), ssv);
-			} else {
-				malloc_cprintf(write_cb, cbopaque,
-				    "Average profile dump interval: N/A\n");
-			}
+				write_cb(cbopaque, u2s((((uint64_t)1U) << ssv),
+				    10, s));
+				write_cb(cbopaque, " (2^");
+				write_cb(cbopaque, u2s(ssv, 10, s));
+				write_cb(cbopaque, ")\n");
+			} else
+				write_cb(cbopaque, "N/A\n");
 		}
+		CTL_GET("arenas.chunksize", &sv, size_t);
+		write_cb(cbopaque, "Chunk size: ");
+		write_cb(cbopaque, u2s(sv, 10, s));
 		CTL_GET("opt.lg_chunk", &sv, size_t);
-		malloc_cprintf(write_cb, cbopaque, "Chunk size: %zu (2^%zu)\n",
-		    (ZU(1) << sv), sv);
+		write_cb(cbopaque, " (2^");
+		write_cb(cbopaque, u2s(sv, 10, s));
+		write_cb(cbopaque, ")\n");
 	}

-	if (config_stats) {
+#ifdef JEMALLOC_STATS
+	{
+		int err;
+		size_t sszp, ssz;
 		size_t *cactive;
 		size_t allocated, active, mapped;
-		size_t chunks_current, chunks_high;
+		size_t chunks_current, chunks_high, swap_avail;
 		uint64_t chunks_total;
 		size_t huge_allocated;
 		uint64_t huge_nmalloc, huge_ndalloc;

+		sszp = sizeof(size_t *);
+		ssz = sizeof(size_t);
+
 		CTL_GET("stats.cactive", &cactive, size_t *);
 		CTL_GET("stats.allocated", &allocated, size_t);
 		CTL_GET("stats.active", &active, size_t);
@@ -475,11 +702,24 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 		CTL_GET("stats.chunks.total", &chunks_total, uint64_t);
 		CTL_GET("stats.chunks.high", &chunks_high, size_t);
 		CTL_GET("stats.chunks.current", &chunks_current, size_t);
-		malloc_cprintf(write_cb, cbopaque, "chunks: nchunks   "
-		    "highchunks    curchunks\n");
-		malloc_cprintf(write_cb, cbopaque,
-		    "  %13"PRIu64" %12zu %12zu\n",
-		    chunks_total, chunks_high, chunks_current);
+		if ((err = JEMALLOC_P(mallctl)("swap.avail", &swap_avail, &ssz,
+		    NULL, 0)) == 0) {
+			size_t lg_chunk;
+
+			malloc_cprintf(write_cb, cbopaque, "chunks: nchunks   "
+			    "highchunks    curchunks   swap_avail\n");
+			CTL_GET("opt.lg_chunk", &lg_chunk, size_t);
+			malloc_cprintf(write_cb, cbopaque,
+			    "  %13"PRIu64"%13zu%13zu%13zu\n",
+			    chunks_total, chunks_high, chunks_current,
+			    swap_avail << lg_chunk);
+		} else {
+			malloc_cprintf(write_cb, cbopaque, "chunks: nchunks   "
+			    "highchunks    curchunks\n");
+			malloc_cprintf(write_cb, cbopaque,
+			    "  %13"PRIu64"%13zu%13zu\n",
+			    chunks_total, chunks_high, chunks_current);
+		}

 		/* Print huge stats. */
 		CTL_GET("stats.huge.nmalloc", &huge_nmalloc, uint64_t);
@@ -496,11 +736,11 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,

 			CTL_GET("arenas.narenas", &narenas, unsigned);
 			{
-				VARIABLE_ARRAY(bool, initialized, narenas);
+				bool initialized[narenas];
 				size_t isz;
 				unsigned i, ninitialized;

-				isz = sizeof(bool) * narenas;
+				isz = sizeof(initialized);
 				xmallctl("arenas.initialized", initialized,
 				    &isz, NULL, 0);
 				for (i = ninitialized = 0; i < narenas; i++) {
@@ -508,12 +748,12 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 						ninitialized++;
 				}

-				if (ninitialized > 1 || unmerged == false) {
+				if (ninitialized > 1) {
 					/* Print merged arena stats. */
 					malloc_cprintf(write_cb, cbopaque,
 					    "\nMerged arenas stats:\n");
 					stats_arena_print(write_cb, cbopaque,
-					    narenas, bins, large);
+					    narenas);
 				}
 			}
 		}
@@ -525,11 +765,11 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,

 			CTL_GET("arenas.narenas", &narenas, unsigned);
 			{
-				VARIABLE_ARRAY(bool, initialized, narenas);
+				bool initialized[narenas];
 				size_t isz;
 				unsigned i;

-				isz = sizeof(bool) * narenas;
+				isz = sizeof(initialized);
 				xmallctl("arenas.initialized", initialized,
 				    &isz, NULL, 0);

@@ -539,11 +779,12 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
 						    cbopaque,
 						    "\narenas[%u]:\n", i);
 						stats_arena_print(write_cb,
-						    cbopaque, i, bins, large);
+						    cbopaque, i);
 					}
 				}
 			}
 		}
 	}
-	malloc_cprintf(write_cb, cbopaque, "--- End jemalloc statistics ---\n");
+#endif /* #ifdef JEMALLOC_STATS */
+	write_cb(cbopaque, "--- End jemalloc statistics ---\n");
 }
@@ -1,92 +1,70 @@
 #define	JEMALLOC_TCACHE_C_
 #include "jemalloc/internal/jemalloc_internal.h"
-
+#ifdef JEMALLOC_TCACHE
 /******************************************************************************/
 /* Data. */

-malloc_tsd_data(, tcache, tcache_t *, NULL)
-malloc_tsd_data(, tcache_enabled, tcache_enabled_t, tcache_enabled_default)
-
 bool	opt_tcache = true;
 ssize_t	opt_lg_tcache_max = LG_TCACHE_MAXCLASS_DEFAULT;
+ssize_t	opt_lg_tcache_gc_sweep = LG_TCACHE_GC_SWEEP_DEFAULT;

 tcache_bin_info_t	*tcache_bin_info;
 static unsigned		stack_nelms; /* Total stack elms per tcache. */

-size_t			nhbins;
-size_t			tcache_maxclass;
+/* Map of thread-specific caches. */
+#ifndef NO_TLS
+__thread tcache_t	*tcache_tls JEMALLOC_ATTR(tls_model("initial-exec"));
+#endif
+
+/*
+ * Same contents as tcache, but initialized such that the TSD destructor is
+ * called when a thread exits, so that the cache can be cleaned up.
+ */
+pthread_key_t		tcache_tsd;
+
+size_t				nhbins;
+size_t				tcache_maxclass;
+unsigned			tcache_gc_incr;

 /******************************************************************************/
+/* Function prototypes for non-inline static functions. */

-size_t	tcache_salloc(const void *ptr)
-{
+static void	tcache_thread_cleanup(void *arg);

-	return (arena_salloc(ptr, false));
-}
-
-void
-tcache_event_hard(tcache_t *tcache)
-{
-	size_t binind = tcache->next_gc_bin;
-	tcache_bin_t *tbin = &tcache->tbins[binind];
-	tcache_bin_info_t *tbin_info = &tcache_bin_info[binind];
-
-	if (tbin->low_water > 0) {
-		/*
-		 * Flush (ceiling) 3/4 of the objects below the low water mark.
-		 */
-		if (binind < NBINS) {
-			tcache_bin_flush_small(tbin, binind, tbin->ncached -
-			    tbin->low_water + (tbin->low_water >> 2), tcache);
-		} else {
-			tcache_bin_flush_large(tbin, binind, tbin->ncached -
-			    tbin->low_water + (tbin->low_water >> 2), tcache);
-		}
-		/*
-		 * Reduce fill count by 2X.  Limit lg_fill_div such that the
-		 * fill count is always at least 1.
-		 */
-		if ((tbin_info->ncached_max >> (tbin->lg_fill_div+1)) >= 1)
-			tbin->lg_fill_div++;
-	} else if (tbin->low_water < 0) {
-		/*
-		 * Increase fill count by 2X.  Make sure lg_fill_div stays
-		 * greater than 0.
-		 */
-		if (tbin->lg_fill_div > 1)
-			tbin->lg_fill_div--;
-	}
-	tbin->low_water = tbin->ncached;
-
-	tcache->next_gc_bin++;
-	if (tcache->next_gc_bin == nhbins)
-		tcache->next_gc_bin = 0;
-	tcache->ev_cnt = 0;
-}
+/******************************************************************************/

 void *
 tcache_alloc_small_hard(tcache_t *tcache, tcache_bin_t *tbin, size_t binind)
 {
 	void *ret;

-	arena_tcache_fill_small(tcache->arena, tbin, binind,
-	    config_prof ? tcache->prof_accumbytes : 0);
-	if (config_prof)
-		tcache->prof_accumbytes = 0;
+	arena_tcache_fill_small(tcache->arena, tbin, binind
+#ifdef JEMALLOC_PROF
+	    , tcache->prof_accumbytes
+#endif
+	    );
+#ifdef JEMALLOC_PROF
+	tcache->prof_accumbytes = 0;
+#endif
 	ret = tcache_alloc_easy(tbin);

 	return (ret);
 }

 void
-tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
-    tcache_t *tcache)
+tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+    , tcache_t *tcache
+#endif
+    )
 {
 	void *ptr;
 	unsigned i, nflush, ndeferred;
+#ifdef JEMALLOC_STATS
 	bool merged_stats = false;
+#endif

-	assert(binind < NBINS);
+	assert(binind < nbins);
 	assert(rem <= tbin->ncached);

 	for (nflush = tbin->ncached - rem; nflush > 0; nflush = ndeferred) {
@@ -96,21 +74,25 @@ tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		arena_t *arena = chunk->arena;
 		arena_bin_t *bin = &arena->bins[binind];

-		if (config_prof && arena == tcache->arena) {
+#ifdef JEMALLOC_PROF
+		if (arena == tcache->arena) {
 			malloc_mutex_lock(&arena->lock);
 			arena_prof_accum(arena, tcache->prof_accumbytes);
 			malloc_mutex_unlock(&arena->lock);
 			tcache->prof_accumbytes = 0;
 		}
+#endif

 		malloc_mutex_lock(&bin->lock);
-		if (config_stats && arena == tcache->arena) {
+#ifdef JEMALLOC_STATS
+		if (arena == tcache->arena) {
 			assert(merged_stats == false);
 			merged_stats = true;
 			bin->stats.nflushes++;
 			bin->stats.nrequests += tbin->tstats.nrequests;
 			tbin->tstats.nrequests = 0;
 		}
+#endif
 		ndeferred = 0;
 		for (i = 0; i < nflush; i++) {
 			ptr = tbin->avail[i];
@@ -118,15 +100,10 @@ tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
 			chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
 			if (chunk->arena == arena) {
 				size_t pageind = ((uintptr_t)ptr -
-				    (uintptr_t)chunk) >> LG_PAGE;
+				    (uintptr_t)chunk) >> PAGE_SHIFT;
 				arena_chunk_map_t *mapelm =
-				    arena_mapp_get(chunk, pageind);
-				if (config_fill && opt_junk) {
-					arena_alloc_junk_small(ptr,
-					    &arena_bin_info[binind], true);
-				}
-				arena_dalloc_bin_locked(arena, chunk, ptr,
-				    mapelm);
+				    &chunk->map[pageind-map_bias];
+				arena_dalloc_bin(arena, chunk, ptr, mapelm);
 			} else {
 				/*
 				 * This object was allocated via a different
@@ -140,7 +117,8 @@ tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		}
 		malloc_mutex_unlock(&bin->lock);
 	}
-	if (config_stats && merged_stats == false) {
+#ifdef JEMALLOC_STATS
+	if (merged_stats == false) {
 		/*
 		 * The flush loop didn't happen to flush to this thread's
 		 * arena, so the stats didn't get merged.  Manually do so now.
@@ -152,6 +130,7 @@ tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		tbin->tstats.nrequests = 0;
 		malloc_mutex_unlock(&bin->lock);
 	}
+#endif

 	memmove(tbin->avail, &tbin->avail[tbin->ncached - rem],
 	    rem * sizeof(void *));
@@ -161,12 +140,17 @@ tcache_bin_flush_small(tcache_bin_t *tbin, size_t binind, unsigned rem,
 }

 void
-tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
-    tcache_t *tcache)
+tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+    , tcache_t *tcache
+#endif
+    )
 {
 	void *ptr;
 	unsigned i, nflush, ndeferred;
+#ifdef JEMALLOC_STATS
 	bool merged_stats = false;
+#endif

 	assert(binind < nhbins);
 	assert(rem <= tbin->ncached);
@@ -178,28 +162,30 @@ tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		arena_t *arena = chunk->arena;

 		malloc_mutex_lock(&arena->lock);
-		if ((config_prof || config_stats) && arena == tcache->arena) {
-			if (config_prof) {
-				arena_prof_accum(arena,
-				    tcache->prof_accumbytes);
-				tcache->prof_accumbytes = 0;
-			}
-			if (config_stats) {
-				merged_stats = true;
-				arena->stats.nrequests_large +=
-				    tbin->tstats.nrequests;
-				arena->stats.lstats[binind - NBINS].nrequests +=
-				    tbin->tstats.nrequests;
-				tbin->tstats.nrequests = 0;
-			}
+#if (defined(JEMALLOC_PROF) || defined(JEMALLOC_STATS))
+		if (arena == tcache->arena) {
+#endif
+#ifdef JEMALLOC_PROF
+			arena_prof_accum(arena, tcache->prof_accumbytes);
+			tcache->prof_accumbytes = 0;
+#endif
+#ifdef JEMALLOC_STATS
+			merged_stats = true;
+			arena->stats.nrequests_large += tbin->tstats.nrequests;
+			arena->stats.lstats[binind - nbins].nrequests +=
+			    tbin->tstats.nrequests;
+			tbin->tstats.nrequests = 0;
+#endif
+#if (defined(JEMALLOC_PROF) || defined(JEMALLOC_STATS))
 		}
+#endif
 		ndeferred = 0;
 		for (i = 0; i < nflush; i++) {
 			ptr = tbin->avail[i];
 			assert(ptr != NULL);
 			chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
 			if (chunk->arena == arena)
-				arena_dalloc_large_locked(arena, chunk, ptr);
+				arena_dalloc_large(arena, chunk, ptr);
 			else {
 				/*
 				 * This object was allocated via a different
@@ -213,7 +199,8 @@ tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		}
 		malloc_mutex_unlock(&arena->lock);
 	}
-	if (config_stats && merged_stats == false) {
+#ifdef JEMALLOC_STATS
+	if (merged_stats == false) {
 		/*
 		 * The flush loop didn't happen to flush to this thread's
 		 * arena, so the stats didn't get merged.  Manually do so now.
@@ -221,11 +208,12 @@ tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		arena_t *arena = tcache->arena;
 		malloc_mutex_lock(&arena->lock);
 		arena->stats.nrequests_large += tbin->tstats.nrequests;
-		arena->stats.lstats[binind - NBINS].nrequests +=
+		arena->stats.lstats[binind - nbins].nrequests +=
 		    tbin->tstats.nrequests;
 		tbin->tstats.nrequests = 0;
 		malloc_mutex_unlock(&arena->lock);
 	}
+#endif

 	memmove(tbin->avail, &tbin->avail[tbin->ncached - rem],
 	    rem * sizeof(void *));
@@ -234,33 +222,6 @@ tcache_bin_flush_large(tcache_bin_t *tbin, size_t binind, unsigned rem,
 		tbin->low_water = tbin->ncached;
 }

-void
-tcache_arena_associate(tcache_t *tcache, arena_t *arena)
-{
-
-	if (config_stats) {
-		/* Link into list of extant tcaches. */
-		malloc_mutex_lock(&arena->lock);
-		ql_elm_new(tcache, link);
-		ql_tail_insert(&arena->tcache_ql, tcache, link);
-		malloc_mutex_unlock(&arena->lock);
-	}
-	tcache->arena = arena;
-}
-
-void
-tcache_arena_dissociate(tcache_t *tcache)
-{
-
-	if (config_stats) {
-		/* Unlink from list of extant tcaches. */
-		malloc_mutex_lock(&tcache->arena->lock);
-		ql_remove(&tcache->arena->tcache_ql, tcache, link);
-		malloc_mutex_unlock(&tcache->arena->lock);
-		tcache_stats_merge(tcache, tcache->arena);
-	}
-}
-
 tcache_t *
 tcache_create(arena_t *arena)
 {
@@ -283,18 +244,25 @@ tcache_create(arena_t *arena)
 	 */
 	size = (size + CACHELINE_MASK) & (-CACHELINE);

-	if (size <= SMALL_MAXCLASS)
+	if (size <= small_maxclass)
 		tcache = (tcache_t *)arena_malloc_small(arena, size, true);
 	else if (size <= tcache_maxclass)
 		tcache = (tcache_t *)arena_malloc_large(arena, size, true);
 	else
-		tcache = (tcache_t *)icallocx(size, false, arena);
+		tcache = (tcache_t *)icalloc(size);

 	if (tcache == NULL)
 		return (NULL);

-	tcache_arena_associate(tcache, arena);
+#ifdef JEMALLOC_STATS
+	/* Link into list of extant tcaches. */
+	malloc_mutex_lock(&arena->lock);
+	ql_elm_new(tcache, link);
+	ql_tail_insert(&arena->tcache_ql, tcache, link);
+	malloc_mutex_unlock(&arena->lock);
+#endif

+	tcache->arena = arena;
 	assert((TCACHE_NSLOTS_SMALL_MAX & 1U) == 0);
 	for (i = 0; i < nhbins; i++) {
 		tcache->tbins[i].lg_fill_div = 1;
@@ -303,7 +271,7 @@ tcache_create(arena_t *arena)
 		stack_offset += tcache_bin_info[i].ncached_max * sizeof(void *);
 	}

-	tcache_tsd_set(&tcache);
+	TCACHE_SET(tcache);

 	return (tcache);
 }
@@ -314,96 +282,121 @@ tcache_destroy(tcache_t *tcache)
 	unsigned i;
 	size_t tcache_size;

-	tcache_arena_dissociate(tcache);
+#ifdef JEMALLOC_STATS
+	/* Unlink from list of extant tcaches. */
+	malloc_mutex_lock(&tcache->arena->lock);
+	ql_remove(&tcache->arena->tcache_ql, tcache, link);
+	malloc_mutex_unlock(&tcache->arena->lock);
+	tcache_stats_merge(tcache, tcache->arena);
+#endif

-	for (i = 0; i < NBINS; i++) {
+	for (i = 0; i < nbins; i++) {
 		tcache_bin_t *tbin = &tcache->tbins[i];
-		tcache_bin_flush_small(tbin, i, 0, tcache);
+		tcache_bin_flush_small(tbin, i, 0
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+		    , tcache
+#endif
+		    );

-		if (config_stats && tbin->tstats.nrequests != 0) {
+#ifdef JEMALLOC_STATS
+		if (tbin->tstats.nrequests != 0) {
 			arena_t *arena = tcache->arena;
 			arena_bin_t *bin = &arena->bins[i];
 			malloc_mutex_lock(&bin->lock);
 			bin->stats.nrequests += tbin->tstats.nrequests;
 			malloc_mutex_unlock(&bin->lock);
 		}
+#endif
 	}

 	for (; i < nhbins; i++) {
 		tcache_bin_t *tbin = &tcache->tbins[i];
-		tcache_bin_flush_large(tbin, i, 0, tcache);
+		tcache_bin_flush_large(tbin, i, 0
+#if (defined(JEMALLOC_STATS) || defined(JEMALLOC_PROF))
+		    , tcache
+#endif
+		    );

-		if (config_stats && tbin->tstats.nrequests != 0) {
+#ifdef JEMALLOC_STATS
+		if (tbin->tstats.nrequests != 0) {
 			arena_t *arena = tcache->arena;
 			malloc_mutex_lock(&arena->lock);
 			arena->stats.nrequests_large += tbin->tstats.nrequests;
-			arena->stats.lstats[i - NBINS].nrequests +=
+			arena->stats.lstats[i - nbins].nrequests +=
 			    tbin->tstats.nrequests;
 			malloc_mutex_unlock(&arena->lock);
 		}
+#endif
 	}

-	if (config_prof && tcache->prof_accumbytes > 0) {
+#ifdef JEMALLOC_PROF
+	if (tcache->prof_accumbytes > 0) {
 		malloc_mutex_lock(&tcache->arena->lock);
 		arena_prof_accum(tcache->arena, tcache->prof_accumbytes);
 		malloc_mutex_unlock(&tcache->arena->lock);
 	}
+#endif

-	tcache_size = arena_salloc(tcache, false);
-	if (tcache_size <= SMALL_MAXCLASS) {
+	tcache_size = arena_salloc(tcache);
+	if (tcache_size <= small_maxclass) {
 		arena_chunk_t *chunk = CHUNK_ADDR2BASE(tcache);
 		arena_t *arena = chunk->arena;
 		size_t pageind = ((uintptr_t)tcache - (uintptr_t)chunk) >>
-		    LG_PAGE;
-		arena_chunk_map_t *mapelm = arena_mapp_get(chunk, pageind);
+		    PAGE_SHIFT;
+		arena_chunk_map_t *mapelm = &chunk->map[pageind-map_bias];
+		arena_run_t *run = (arena_run_t *)((uintptr_t)chunk +
+		    (uintptr_t)((pageind - (mapelm->bits >> PAGE_SHIFT)) <<
+		    PAGE_SHIFT));
+		arena_bin_t *bin = run->bin;

-		arena_dalloc_bin(arena, chunk, tcache, pageind, mapelm);
+		malloc_mutex_lock(&bin->lock);
+		arena_dalloc_bin(arena, chunk, tcache, mapelm);
+		malloc_mutex_unlock(&bin->lock);
 	} else if (tcache_size <= tcache_maxclass) {
 		arena_chunk_t *chunk = CHUNK_ADDR2BASE(tcache);
 		arena_t *arena = chunk->arena;

+		malloc_mutex_lock(&arena->lock);
 		arena_dalloc_large(arena, chunk, tcache);
+		malloc_mutex_unlock(&arena->lock);
 	} else
-		idallocx(tcache, false);
+		idalloc(tcache);
 }

-void
+static void
 tcache_thread_cleanup(void *arg)
 {
-	tcache_t *tcache = *(tcache_t **)arg;
+	tcache_t *tcache = (tcache_t *)arg;

-	if (tcache == TCACHE_STATE_DISABLED) {
-		/* Do nothing. */
-	} else if (tcache == TCACHE_STATE_REINCARNATED) {
-		/*
-		 * Another destructor called an allocator function after this
-		 * destructor was called.  Reset tcache to
-		 * TCACHE_STATE_PURGATORY in order to receive another callback.
-		 */
-		tcache = TCACHE_STATE_PURGATORY;
-		tcache_tsd_set(&tcache);
-	} else if (tcache == TCACHE_STATE_PURGATORY) {
+	if (tcache == (void *)(uintptr_t)1) {
 		/*
 		 * The previous time this destructor was called, we set the key
-		 * to TCACHE_STATE_PURGATORY so that other destructors wouldn't
-		 * cause re-creation of the tcache.  This time, do nothing, so
-		 * that the destructor will not be called again.
+		 * to 1 so that other destructors wouldn't cause re-creation of
+		 * the tcache.  This time, do nothing, so that the destructor
+		 * will not be called again.
 		 */
+	} else if (tcache == (void *)(uintptr_t)2) {
+		/*
+		 * Another destructor called an allocator function after this
+		 * destructor was called.  Reset tcache to 1 in order to
+		 * receive another callback.
+		 */
+		TCACHE_SET((uintptr_t)1);
 	} else if (tcache != NULL) {
-		assert(tcache != TCACHE_STATE_PURGATORY);
+		assert(tcache != (void *)(uintptr_t)1);
 		tcache_destroy(tcache);
-		tcache = TCACHE_STATE_PURGATORY;
-		tcache_tsd_set(&tcache);
+		TCACHE_SET((uintptr_t)1);
 	}
 }

+#ifdef JEMALLOC_STATS
 void
 tcache_stats_merge(tcache_t *tcache, arena_t *arena)
 {
 	unsigned i;

 	/* Merge and reset tcache stats. */
-	for (i = 0; i < NBINS; i++) {
+	for (i = 0; i < nbins; i++) {
 		arena_bin_t *bin = &arena->bins[i];
 		tcache_bin_t *tbin = &tcache->tbins[i];
 		malloc_mutex_lock(&bin->lock);
@@ -413,62 +406,75 @@ tcache_stats_merge(tcache_t *tcache, arena_t *arena)
 	}

 	for (; i < nhbins; i++) {
-		malloc_large_stats_t *lstats = &arena->stats.lstats[i - NBINS];
+		malloc_large_stats_t *lstats = &arena->stats.lstats[i - nbins];
 		tcache_bin_t *tbin = &tcache->tbins[i];
 		arena->stats.nrequests_large += tbin->tstats.nrequests;
 		lstats->nrequests += tbin->tstats.nrequests;
 		tbin->tstats.nrequests = 0;
 	}
 }
+#endif

 bool
-tcache_boot0(void)
+tcache_boot(void)
 {
-	unsigned i;

-	/*
-	 * If necessary, clamp opt_lg_tcache_max, now that arena_maxclass is
-	 * known.
-	 */
-	if (opt_lg_tcache_max < 0 || (1U << opt_lg_tcache_max) < SMALL_MAXCLASS)
-		tcache_maxclass = SMALL_MAXCLASS;
-	else if ((1U << opt_lg_tcache_max) > arena_maxclass)
-		tcache_maxclass = arena_maxclass;
-	else
-		tcache_maxclass = (1U << opt_lg_tcache_max);
+	if (opt_tcache) {
+		unsigned i;

-	nhbins = NBINS + (tcache_maxclass >> LG_PAGE);
+		/*
+		 * If necessary, clamp opt_lg_tcache_max, now that
+		 * small_maxclass and arena_maxclass are known.
+		 */
+		if (opt_lg_tcache_max < 0 || (1U <<
+		    opt_lg_tcache_max) < small_maxclass)
+			tcache_maxclass = small_maxclass;
+		else if ((1U << opt_lg_tcache_max) > arena_maxclass)
+			tcache_maxclass = arena_maxclass;
+		else
+			tcache_maxclass = (1U << opt_lg_tcache_max);

-	/* Initialize tcache_bin_info. */
-	tcache_bin_info = (tcache_bin_info_t *)base_alloc(nhbins *
-	    sizeof(tcache_bin_info_t));
-	if (tcache_bin_info == NULL)
-		return (true);
-	stack_nelms = 0;
-	for (i = 0; i < NBINS; i++) {
-		if ((arena_bin_info[i].nregs << 1) <= TCACHE_NSLOTS_SMALL_MAX) {
-			tcache_bin_info[i].ncached_max =
-			    (arena_bin_info[i].nregs << 1);
-		} else {
-			tcache_bin_info[i].ncached_max =
-			    TCACHE_NSLOTS_SMALL_MAX;
+		nhbins = nbins + (tcache_maxclass >> PAGE_SHIFT);
+
+		/* Initialize tcache_bin_info. */
+		tcache_bin_info = (tcache_bin_info_t *)base_alloc(nhbins *
+		    sizeof(tcache_bin_info_t));
+		if (tcache_bin_info == NULL)
+			return (true);
+		stack_nelms = 0;
+		for (i = 0; i < nbins; i++) {
+			if ((arena_bin_info[i].nregs << 1) <=
+			    TCACHE_NSLOTS_SMALL_MAX) {
+				tcache_bin_info[i].ncached_max =
+				    (arena_bin_info[i].nregs << 1);
+			} else {
+				tcache_bin_info[i].ncached_max =
+				    TCACHE_NSLOTS_SMALL_MAX;
+			}
+			stack_nelms += tcache_bin_info[i].ncached_max;
+		}
+		for (; i < nhbins; i++) {
+			tcache_bin_info[i].ncached_max = TCACHE_NSLOTS_LARGE;
+			stack_nelms += tcache_bin_info[i].ncached_max;
+		}
+
+		/* Compute incremental GC event threshold. */
+		if (opt_lg_tcache_gc_sweep >= 0) {
+			tcache_gc_incr = ((1U << opt_lg_tcache_gc_sweep) /
+			    nbins) + (((1U << opt_lg_tcache_gc_sweep) % nbins ==
+			    0) ? 0 : 1);
+		} else
+			tcache_gc_incr = 0;
+
+		if (pthread_key_create(&tcache_tsd, tcache_thread_cleanup) !=
+		    0) {
+			malloc_write(
+			    "<jemalloc>: Error in pthread_key_create()\n");
+			abort();
 		}
-		stack_nelms += tcache_bin_info[i].ncached_max;
-	}
-	for (; i < nhbins; i++) {
-		tcache_bin_info[i].ncached_max = TCACHE_NSLOTS_LARGE;
-		stack_nelms += tcache_bin_info[i].ncached_max;
 	}

 	return (false);
 }
-
-bool
-tcache_boot1(void)
-{
-
-	if (tcache_tsd_boot() || tcache_enabled_tsd_boot())
-		return (true);
-
-	return (false);
-}
+/******************************************************************************/
+#endif /* JEMALLOC_TCACHE */
@@ -1,107 +0,0 @@
-#define	JEMALLOC_TSD_C_
-#include "jemalloc/internal/jemalloc_internal.h"
-
-/******************************************************************************/
-/* Data. */
-
-static unsigned ncleanups;
-static malloc_tsd_cleanup_t cleanups[MALLOC_TSD_CLEANUPS_MAX];
-
-/******************************************************************************/
-
-void *
-malloc_tsd_malloc(size_t size)
-{
-
-	/* Avoid choose_arena() in order to dodge bootstrapping issues. */
-	return (arena_malloc(arenas[0], size, false, false));
-}
-
-void
-malloc_tsd_dalloc(void *wrapper)
-{
-
-	idalloc(wrapper);
-}
-
-void
-malloc_tsd_no_cleanup(void *arg)
-{
-
-	not_reached();
-}
-
-#if defined(JEMALLOC_MALLOC_THREAD_CLEANUP) || defined(_WIN32)
-#ifndef _WIN32
-JEMALLOC_EXPORT
-#endif
-void
-_malloc_thread_cleanup(void)
-{
-	bool pending[MALLOC_TSD_CLEANUPS_MAX], again;
-	unsigned i;
-
-	for (i = 0; i < ncleanups; i++)
-		pending[i] = true;
-
-	do {
-		again = false;
-		for (i = 0; i < ncleanups; i++) {
-			if (pending[i]) {
-				pending[i] = cleanups[i]();
-				if (pending[i])
-					again = true;
-			}
-		}
-	} while (again);
-}
-#endif
-
-void
-malloc_tsd_cleanup_register(bool (*f)(void))
-{
-
-	assert(ncleanups < MALLOC_TSD_CLEANUPS_MAX);
-	cleanups[ncleanups] = f;
-	ncleanups++;
-}
-
-void
-malloc_tsd_boot(void)
-{
-
-	ncleanups = 0;
-}
-
-#ifdef _WIN32
-static BOOL WINAPI
-_tls_callback(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
-{
-
-	switch (fdwReason) {
-#ifdef JEMALLOC_LAZY_LOCK
-	case DLL_THREAD_ATTACH:
-		isthreaded = true;
-		break;
-#endif
-	case DLL_THREAD_DETACH:
-		_malloc_thread_cleanup();
-		break;
-	default:
-		break;
-	}
-	return (true);
-}
-
-#ifdef _MSC_VER
-#  ifdef _M_IX86
-#    pragma comment(linker, "/INCLUDE:__tls_used")
-#  else
-#    pragma comment(linker, "/INCLUDE:_tls_used")
-#  endif
-#  pragma section(".CRT$XLY",long,read)
-#endif
-JEMALLOC_SECTION(".CRT$XLY") JEMALLOC_ATTR(used)
-static const BOOL	(WINAPI *tls_callback)(HINSTANCE hinstDLL,
-    DWORD fdwReason, LPVOID lpvReserved) = _tls_callback;
-#endif
@@ -1,641 +0,0 @@
-#define	assert(e) do {							\
-	if (config_debug && !(e)) {					\
-		malloc_write("<jemalloc>: Failed assertion\n");		\
-		abort();						\
-	}								\
-} while (0)
-
-#define	not_reached() do {						\
-	if (config_debug) {						\
-		malloc_write("<jemalloc>: Unreachable code reached\n");	\
-		abort();						\
-	}								\
-} while (0)
-
-#define	not_implemented() do {						\
-	if (config_debug) {						\
-		malloc_write("<jemalloc>: Not implemented\n");		\
-		abort();						\
-	}								\
-} while (0)
-
-#define	JEMALLOC_UTIL_C_
-#include "jemalloc/internal/jemalloc_internal.h"
-
-/******************************************************************************/
-/* Function prototypes for non-inline static functions. */
-
-static void	wrtmessage(void *cbopaque, const char *s);
-#define	U2S_BUFSIZE	((1U << (LG_SIZEOF_INTMAX_T + 3)) + 1)
-static char	*u2s(uintmax_t x, unsigned base, bool uppercase, char *s,
-    size_t *slen_p);
-#define	D2S_BUFSIZE	(1 + U2S_BUFSIZE)
-static char	*d2s(intmax_t x, char sign, char *s, size_t *slen_p);
-#define	O2S_BUFSIZE	(1 + U2S_BUFSIZE)
-static char	*o2s(uintmax_t x, bool alt_form, char *s, size_t *slen_p);
-#define	X2S_BUFSIZE	(2 + U2S_BUFSIZE)
-static char	*x2s(uintmax_t x, bool alt_form, bool uppercase, char *s,
-    size_t *slen_p);
-
-/******************************************************************************/
-
-/* malloc_message() setup. */
-static void
-wrtmessage(void *cbopaque, const char *s)
-{
-
-#ifdef SYS_write
-	/*
-	 * Use syscall(2) rather than write(2) when possible in order to avoid
-	 * the possibility of memory allocation within libc.  This is necessary
-	 * on FreeBSD; most operating systems do not have this problem though.
-	 */
-	UNUSED int result = syscall(SYS_write, STDERR_FILENO, s, strlen(s));
-#else
-	UNUSED int result = write(STDERR_FILENO, s, strlen(s));
-#endif
-}
-
-JEMALLOC_EXPORT void	(*je_malloc_message)(void *, const char *s);
-
-/*
- * Wrapper around malloc_message() that avoids the need for
- * je_malloc_message(...) throughout the code.
- */
-void
-malloc_write(const char *s)
-{
-
-	if (je_malloc_message != NULL)
-		je_malloc_message(NULL, s);
-	else
-		wrtmessage(NULL, s);
-}
-
-/*
- * glibc provides a non-standard strerror_r() when _GNU_SOURCE is defined, so
- * provide a wrapper.
- */
-int
-buferror(char *buf, size_t buflen)
-{
-
-#ifdef _WIN32
-	FormatMessageA(FORMAT_MESSAGE_FROM_SYSTEM, NULL, GetLastError(), 0,
-	    (LPSTR)buf, buflen, NULL);
-	return (0);
-#elif defined(_GNU_SOURCE)
-	char *b = strerror_r(errno, buf, buflen);
-	if (b != buf) {
-		strncpy(buf, b, buflen);
-		buf[buflen-1] = '\0';
-	}
-	return (0);
-#else
-	return (strerror_r(errno, buf, buflen));
-#endif
-}
-
-uintmax_t
-malloc_strtoumax(const char *nptr, char **endptr, int base)
-{
-	uintmax_t ret, digit;
-	int b;
-	bool neg;
-	const char *p, *ns;
-
-	if (base < 0 || base == 1 || base > 36) {
-		set_errno(EINVAL);
-		return (UINTMAX_MAX);
-	}
-	b = base;
-
-	/* Swallow leading whitespace and get sign, if any. */
-	neg = false;
-	p = nptr;
-	while (true) {
-		switch (*p) {
-		case '\t': case '\n': case '\v': case '\f': case '\r': case ' ':
-			p++;
-			break;
-		case '-':
-			neg = true;
-			/* Fall through. */
-		case '+':
-			p++;
-			/* Fall through. */
-		default:
-			goto label_prefix;
-		}
-	}
-
-	/* Get prefix, if any. */
-	label_prefix:
-	/*
-	 * Note where the first non-whitespace/sign character is so that it is
-	 * possible to tell whether any digits are consumed (e.g., "  0" vs.
-	 * "  -x").
-	 */
-	ns = p;
-	if (*p == '0') {
-		switch (p[1]) {
-		case '0': case '1': case '2': case '3': case '4': case '5':
-		case '6': case '7':
-			if (b == 0)
-				b = 8;
-			if (b == 8)
-				p++;
-			break;
-		case 'x':
-			switch (p[2]) {
-			case '0': case '1': case '2': case '3': case '4':
-			case '5': case '6': case '7': case '8': case '9':
-			case 'A': case 'B': case 'C': case 'D': case 'E':
-			case 'F':
-			case 'a': case 'b': case 'c': case 'd': case 'e':
-			case 'f':
-				if (b == 0)
-					b = 16;
-				if (b == 16)
-					p += 2;
-				break;
-			default:
-				break;
-			}
-			break;
-		default:
-			break;
-		}
-	}
-	if (b == 0)
-		b = 10;
-
-	/* Convert. */
-	ret = 0;
-	while ((*p >= '0' && *p <= '9' && (digit = *p - '0') < b)
-	    || (*p >= 'A' && *p <= 'Z' && (digit = 10 + *p - 'A') < b)
-	    || (*p >= 'a' && *p <= 'z' && (digit = 10 + *p - 'a') < b)) {
-		uintmax_t pret = ret;
-		ret *= b;
-		ret += digit;
-		if (ret < pret) {
-			/* Overflow. */
-			set_errno(ERANGE);
-			return (UINTMAX_MAX);
-		}
-		p++;
-	}
-	if (neg)
-		ret = -ret;
-
-	if (endptr != NULL) {
-		if (p == ns) {
-			/* No characters were converted. */
-			*endptr = (char *)nptr;
-		} else
-			*endptr = (char *)p;
-	}
-
-	return (ret);
-}
-
-static char *
-u2s(uintmax_t x, unsigned base, bool uppercase, char *s, size_t *slen_p)
-{
-	unsigned i;
-
-	i = U2S_BUFSIZE - 1;
-	s[i] = '\0';
-	switch (base) {
-	case 10:
-		do {
-			i--;
-			s[i] = "0123456789"[x % (uint64_t)10];
-			x /= (uint64_t)10;
-		} while (x > 0);
-		break;
-	case 16: {
-		const char *digits = (uppercase)
-		    ? "0123456789ABCDEF"
-		    : "0123456789abcdef";
-
-		do {
-			i--;
-			s[i] = digits[x & 0xf];
-			x >>= 4;
-		} while (x > 0);
-		break;
-	} default: {
-		const char *digits = (uppercase)
-		    ? "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
-		    : "0123456789abcdefghijklmnopqrstuvwxyz";
-
-		assert(base >= 2 && base <= 36);
-		do {
-			i--;
-			s[i] = digits[x % (uint64_t)base];
-			x /= (uint64_t)base;
-		} while (x > 0);
-	}}
-
-	*slen_p = U2S_BUFSIZE - 1 - i;
-	return (&s[i]);
-}
-
-static char *
-d2s(intmax_t x, char sign, char *s, size_t *slen_p)
-{
-	bool neg;
-
-	if ((neg = (x < 0)))
-		x = -x;
-	s = u2s(x, 10, false, s, slen_p);
-	if (neg)
-		sign = '-';
-	switch (sign) {
-	case '-':
-		if (neg == false)
-			break;
-		/* Fall through. */
-	case ' ':
-	case '+':
-		s--;
-		(*slen_p)++;
-		*s = sign;
-		break;
-	default: not_reached();
-	}
-	return (s);
-}
-
-static char *
-o2s(uintmax_t x, bool alt_form, char *s, size_t *slen_p)
-{
-
-	s = u2s(x, 8, false, s, slen_p);
-	if (alt_form && *s != '0') {
-		s--;
-		(*slen_p)++;
-		*s = '0';
-	}
-	return (s);
-}
-
-static char *
-x2s(uintmax_t x, bool alt_form, bool uppercase, char *s, size_t *slen_p)
-{
-
-	s = u2s(x, 16, uppercase, s, slen_p);
-	if (alt_form) {
-		s -= 2;
-		(*slen_p) += 2;
-		memcpy(s, uppercase ? "0X" : "0x", 2);
-	}
-	return (s);
-}
-
-int
-malloc_vsnprintf(char *str, size_t size, const char *format, va_list ap)
-{
-	int ret;
-	size_t i;
-	const char *f;
-
-#define	APPEND_C(c) do {						\
-	if (i < size)							\
-		str[i] = (c);						\
-	i++;								\
-} while (0)
-#define	APPEND_S(s, slen) do {						\
-	if (i < size) {							\
-		size_t cpylen = (slen <= size - i) ? slen : size - i;	\
-		memcpy(&str[i], s, cpylen);				\
-	}								\
-	i += slen;							\
-} while (0)
-#define	APPEND_PADDED_S(s, slen, width, left_justify) do {		\
-	/* Left padding. */						\
-	size_t pad_len = (width == -1) ? 0 : ((slen < (size_t)width) ?	\
-	    (size_t)width - slen : 0);					\
-	if (left_justify == false && pad_len != 0) {			\
-		size_t j;						\
-		for (j = 0; j < pad_len; j++)				\
-			APPEND_C(' ');					\
-	}								\
-	/* Value. */							\
-	APPEND_S(s, slen);						\
-	/* Right padding. */						\
-	if (left_justify && pad_len != 0) {				\
-		size_t j;						\
-		for (j = 0; j < pad_len; j++)				\
-			APPEND_C(' ');					\
-	}								\
-} while (0)
-#define GET_ARG_NUMERIC(val, len) do {					\
-	switch (len) {							\
-	case '?':							\
-		val = va_arg(ap, int);					\
-		break;							\
-	case '?' | 0x80:						\
-		val = va_arg(ap, unsigned int);				\
-		break;							\
-	case 'l':							\
-		val = va_arg(ap, long);					\
-		break;							\
-	case 'l' | 0x80:						\
-		val = va_arg(ap, unsigned long);			\
-		break;							\
-	case 'q':							\
-		val = va_arg(ap, long long);				\
-		break;							\
-	case 'q' | 0x80:						\
-		val = va_arg(ap, unsigned long long);			\
-		break;							\
-	case 'j':							\
-		val = va_arg(ap, intmax_t);				\
-		break;							\
-	case 't':							\
-		val = va_arg(ap, ptrdiff_t);				\
-		break;							\
-	case 'z':							\
-		val = va_arg(ap, ssize_t);				\
-		break;							\
-	case 'z' | 0x80:						\
-		val = va_arg(ap, size_t);				\
-		break;							\
-	case 'p': /* Synthetic; used for %p. */				\
-		val = va_arg(ap, uintptr_t);				\
-		break;							\
-	default: not_reached();						\
-	}								\
-} while (0)
-
-	i = 0;
-	f = format;
-	while (true) {
-		switch (*f) {
-		case '\0': goto label_out;
-		case '%': {
-			bool alt_form = false;
-			bool left_justify = false;
-			bool plus_space = false;
-			bool plus_plus = false;
-			int prec = -1;
-			int width = -1;
-			unsigned char len = '?';
-
-			f++;
-			if (*f == '%') {
-				/* %% */
-				APPEND_C(*f);
-				break;
-			}
-			/* Flags. */
-			while (true) {
-				switch (*f) {
-				case '#':
-					assert(alt_form == false);
-					alt_form = true;
-					break;
-				case '-':
-					assert(left_justify == false);
-					left_justify = true;
-					break;
-				case ' ':
-					assert(plus_space == false);
-					plus_space = true;
-					break;
-				case '+':
-					assert(plus_plus == false);
-					plus_plus = true;
-					break;
-				default: goto label_width;
-				}
-				f++;
-			}
-			/* Width. */
-			label_width:
-			switch (*f) {
-			case '*':
-				width = va_arg(ap, int);
-				f++;
-				break;
-			case '0': case '1': case '2': case '3': case '4':
-			case '5': case '6': case '7': case '8': case '9': {
-				uintmax_t uwidth;
-				set_errno(0);
-				uwidth = malloc_strtoumax(f, (char **)&f, 10);
-				assert(uwidth != UINTMAX_MAX || get_errno() !=
-				    ERANGE);
-				width = (int)uwidth;
-				if (*f == '.') {
-					f++;
-					goto label_precision;
-				} else
-					goto label_length;
-				break;
-			} case '.':
-				f++;
-				goto label_precision;
-			default: goto label_length;
-			}
-			/* Precision. */
-			label_precision:
-			switch (*f) {
-			case '*':
-				prec = va_arg(ap, int);
-				f++;
-				break;
-			case '0': case '1': case '2': case '3': case '4':
-			case '5': case '6': case '7': case '8': case '9': {
-				uintmax_t uprec;
-				set_errno(0);
-				uprec = malloc_strtoumax(f, (char **)&f, 10);
-				assert(uprec != UINTMAX_MAX || get_errno() !=
-				    ERANGE);
-				prec = (int)uprec;
-				break;
-			}
-			default: break;
-			}
-			/* Length. */
-			label_length:
-			switch (*f) {
-			case 'l':
-				f++;
-				if (*f == 'l') {
-					len = 'q';
-					f++;
-				} else
-					len = 'l';
-				break;
-			case 'j':
-				len = 'j';
-				f++;
-				break;
-			case 't':
-				len = 't';
-				f++;
-				break;
-			case 'z':
-				len = 'z';
-				f++;
-				break;
-			default: break;
-			}
-			/* Conversion specifier. */
-			switch (*f) {
-				char *s;
-				size_t slen;
-			case 'd': case 'i': {
-				intmax_t val JEMALLOC_CC_SILENCE_INIT(0);
-				char buf[D2S_BUFSIZE];
-
-				GET_ARG_NUMERIC(val, len);
-				s = d2s(val, (plus_plus ? '+' : (plus_space ?
-				    ' ' : '-')), buf, &slen);
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			} case 'o': {
-				uintmax_t val JEMALLOC_CC_SILENCE_INIT(0);
-				char buf[O2S_BUFSIZE];
-
-				GET_ARG_NUMERIC(val, len | 0x80);
-				s = o2s(val, alt_form, buf, &slen);
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			} case 'u': {
-				uintmax_t val JEMALLOC_CC_SILENCE_INIT(0);
-				char buf[U2S_BUFSIZE];
-
-				GET_ARG_NUMERIC(val, len | 0x80);
-				s = u2s(val, 10, false, buf, &slen);
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			} case 'x': case 'X': {
-				uintmax_t val JEMALLOC_CC_SILENCE_INIT(0);
-				char buf[X2S_BUFSIZE];
-
-				GET_ARG_NUMERIC(val, len | 0x80);
-				s = x2s(val, alt_form, *f == 'X', buf, &slen);
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			} case 'c': {
-				unsigned char val;
-				char buf[2];
-
-				assert(len == '?' || len == 'l');
-				assert_not_implemented(len != 'l');
-				val = va_arg(ap, int);
-				buf[0] = val;
-				buf[1] = '\0';
-				APPEND_PADDED_S(buf, 1, width, left_justify);
-				f++;
-				break;
-			} case 's':
-				assert(len == '?' || len == 'l');
-				assert_not_implemented(len != 'l');
-				s = va_arg(ap, char *);
-				slen = (prec == -1) ? strlen(s) : prec;
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			case 'p': {
-				uintmax_t val;
-				char buf[X2S_BUFSIZE];
-
-				GET_ARG_NUMERIC(val, 'p');
-				s = x2s(val, true, false, buf, &slen);
-				APPEND_PADDED_S(s, slen, width, left_justify);
-				f++;
-				break;
-			}
-			default: not_implemented();
-			}
-			break;
-		} default: {
-			APPEND_C(*f);
-			f++;
-			break;
-		}}
-	}
-	label_out:
-	if (i < size)
-		str[i] = '\0';
-	else
-		str[size - 1] = '\0';
-	ret = i;
-
-#undef APPEND_C
-#undef APPEND_S
-#undef APPEND_PADDED_S
-#undef GET_ARG_NUMERIC
-	return (ret);
-}
-
-JEMALLOC_ATTR(format(printf, 3, 4))
-int
-malloc_snprintf(char *str, size_t size, const char *format, ...)
-{
-	int ret;
-	va_list ap;
-
-	va_start(ap, format);
-	ret = malloc_vsnprintf(str, size, format, ap);
-	va_end(ap);
-
-	return (ret);
-}
-
-void
-malloc_vcprintf(void (*write_cb)(void *, const char *), void *cbopaque,
-    const char *format, va_list ap)
-{
-	char buf[MALLOC_PRINTF_BUFSIZE];
-
-	if (write_cb == NULL) {
-		/*
-		 * The caller did not provide an alternate write_cb callback
-		 * function, so use the default one.  malloc_write() is an
-		 * inline function, so use malloc_message() directly here.
-		 */
-		write_cb = (je_malloc_message != NULL) ? je_malloc_message :
-		    wrtmessage;
-		cbopaque = NULL;
-	}
-
-	malloc_vsnprintf(buf, sizeof(buf), format, ap);
-	write_cb(cbopaque, buf);
-}
-
-/*
- * Print to a callback function in such a way as to (hopefully) avoid memory
- * allocation.
- */
-JEMALLOC_ATTR(format(printf, 3, 4))
-void
-malloc_cprintf(void (*write_cb)(void *, const char *), void *cbopaque,
-    const char *format, ...)
-{
-	va_list ap;
-
-	va_start(ap, format);
-	malloc_vcprintf(write_cb, cbopaque, format, ap);
-	va_end(ap);
-}
-
-/* Print to stderr in such a way as to avoid memory allocation. */
-JEMALLOC_ATTR(format(printf, 1, 2))
-void
-malloc_printf(const char *format, ...)
-{
-	va_list ap;
-
-	va_start(ap, format);
-	malloc_vcprintf(NULL, NULL, format, ap);
-	va_end(ap);
-}
@@ -3,18 +3,11 @@
 #  error "This source file is for zones on Darwin (OS X)."
 #endif

-/*
- * The malloc_default_purgeable_zone function is only available on >= 10.6.
- * We need to check whether it is present at runtime, thus the weak_import.
- */
-extern malloc_zone_t *malloc_default_purgeable_zone(void)
-JEMALLOC_ATTR(weak_import);
-
 /******************************************************************************/
 /* Data. */

-static malloc_zone_t zone;
-static struct malloc_introspection_t zone_introspect;
+static malloc_zone_t zone, szone;
+static struct malloc_introspection_t zone_introspect, ozone_introspect;

 /******************************************************************************/
 /* Function prototypes for non-inline static functions. */
@@ -25,10 +18,8 @@ static void	*zone_calloc(malloc_zone_t *zone, size_t num, size_t size);
 static void	*zone_valloc(malloc_zone_t *zone, size_t size);
 static void	zone_free(malloc_zone_t *zone, void *ptr);
 static void	*zone_realloc(malloc_zone_t *zone, void *ptr, size_t size);
-#if (JEMALLOC_ZONE_VERSION >= 5)
-static void	*zone_memalign(malloc_zone_t *zone, size_t alignment,
-#endif
 #if (JEMALLOC_ZONE_VERSION >= 6)
+static void	*zone_memalign(malloc_zone_t *zone, size_t alignment,
    size_t size);
 static void	zone_free_definite_size(malloc_zone_t *zone, void *ptr,
    size_t size);
@@ -37,6 +28,19 @@ static void	*zone_destroy(malloc_zone_t *zone);
 static size_t	zone_good_size(malloc_zone_t *zone, size_t size);
 static void	zone_force_lock(malloc_zone_t *zone);
 static void	zone_force_unlock(malloc_zone_t *zone);
+static size_t	ozone_size(malloc_zone_t *zone, void *ptr);
+static void	ozone_free(malloc_zone_t *zone, void *ptr);
+static void	*ozone_realloc(malloc_zone_t *zone, void *ptr, size_t size);
+static unsigned	ozone_batch_malloc(malloc_zone_t *zone, size_t size,
+    void **results, unsigned num_requested);
+static void	ozone_batch_free(malloc_zone_t *zone, void **to_be_freed,
+    unsigned num);
+#if (JEMALLOC_ZONE_VERSION >= 6)
+static void	ozone_free_definite_size(malloc_zone_t *zone, void *ptr,
+    size_t size);
+#endif
+static void	ozone_force_lock(malloc_zone_t *zone);
+static void	ozone_force_unlock(malloc_zone_t *zone);

 /******************************************************************************/
 /*
@@ -56,21 +60,21 @@ zone_size(malloc_zone_t *zone, void *ptr)
 	 * not work in practice, we must check all pointers to assure that they
 	 * reside within a mapped chunk before determining size.
 	 */
-	return (ivsalloc(ptr, config_prof));
+	return (ivsalloc(ptr));
 }

 static void *
 zone_malloc(malloc_zone_t *zone, size_t size)
 {

-	return (je_malloc(size));
+	return (JEMALLOC_P(malloc)(size));
 }

 static void *
 zone_calloc(malloc_zone_t *zone, size_t num, size_t size)
 {

-	return (je_calloc(num, size));
+	return (JEMALLOC_P(calloc)(num, size));
 }

 static void *
@@ -78,7 +82,7 @@ zone_valloc(malloc_zone_t *zone, size_t size)
 {
 	void *ret = NULL; /* Assignment avoids useless compiler warning. */

-	je_posix_memalign(&ret, PAGE, size);
+	JEMALLOC_P(posix_memalign)(&ret, PAGE_SIZE, size);

 	return (ret);
 }
@@ -87,48 +91,33 @@ static void
 zone_free(malloc_zone_t *zone, void *ptr)
 {

-	if (ivsalloc(ptr, config_prof) != 0) {
-		je_free(ptr);
-		return;
-	}
-
-	free(ptr);
+	JEMALLOC_P(free)(ptr);
 }

 static void *
 zone_realloc(malloc_zone_t *zone, void *ptr, size_t size)
 {

-	if (ivsalloc(ptr, config_prof) != 0)
-		return (je_realloc(ptr, size));
-
-	return (realloc(ptr, size));
+	return (JEMALLOC_P(realloc)(ptr, size));
 }

-#if (JEMALLOC_ZONE_VERSION >= 5)
+#if (JEMALLOC_ZONE_VERSION >= 6)
 static void *
 zone_memalign(malloc_zone_t *zone, size_t alignment, size_t size)
 {
 	void *ret = NULL; /* Assignment avoids useless compiler warning. */

-	je_posix_memalign(&ret, alignment, size);
+	JEMALLOC_P(posix_memalign)(&ret, alignment, size);

 	return (ret);
 }
-#endif

-#if (JEMALLOC_ZONE_VERSION >= 6)
 static void
 zone_free_definite_size(malloc_zone_t *zone, void *ptr, size_t size)
 {

-	if (ivsalloc(ptr, config_prof) != 0) {
-		assert(ivsalloc(ptr, config_prof) == size);
-		je_free(ptr);
-		return;
-	}
-
-	free(ptr);
+	assert(ivsalloc(ptr) == size);
+	JEMALLOC_P(free)(ptr);
 }
 #endif

@@ -144,10 +133,22 @@ zone_destroy(malloc_zone_t *zone)
 static size_t
 zone_good_size(malloc_zone_t *zone, size_t size)
 {
+	size_t ret;
+	void *p;

-	if (size == 0)
-		size = 1;
-	return (s2u(size));
+	/*
+	 * Actually create an object of the appropriate size, then find out
+	 * how large it could have been without moving up to the next size
+	 * class.
+	 */
+	p = JEMALLOC_P(malloc)(size);
+	if (p != NULL) {
+		ret = isalloc(p);
+		JEMALLOC_P(free)(p);
+	} else
+		ret = size;
+
+	return (ret);
 }

 static void
@@ -163,24 +164,13 @@ zone_force_unlock(malloc_zone_t *zone)
 {

 	if (isthreaded)
-		jemalloc_postfork_parent();
+		jemalloc_postfork();
 }

-JEMALLOC_ATTR(constructor)
-void
-register_zone(void)
+malloc_zone_t *
+create_zone(void)
 {

-	/*
-	 * If something else replaced the system default zone allocator, don't
-	 * register jemalloc's.
-	 */
-	malloc_zone_t *default_zone = malloc_default_zone();
-	if (!default_zone->zone_name ||
-	    strcmp(default_zone->zone_name, "DefaultMallocZone") != 0) {
-		return;
-	}
-
 	zone.size = (void *)zone_size;
 	zone.malloc = (void *)zone_malloc;
 	zone.calloc = (void *)zone_calloc;
@@ -193,15 +183,10 @@ register_zone(void)
 	zone.batch_free = NULL;
 	zone.introspect = &zone_introspect;
 	zone.version = JEMALLOC_ZONE_VERSION;
-#if (JEMALLOC_ZONE_VERSION >= 5)
-	zone.memalign = zone_memalign;
-#endif
 #if (JEMALLOC_ZONE_VERSION >= 6)
+	zone.memalign = zone_memalign;
 	zone.free_definite_size = zone_free_definite_size;
 #endif
-#if (JEMALLOC_ZONE_VERSION >= 8)
-	zone.pressure_relief = NULL;
-#endif

 	zone_introspect.enumerator = NULL;
 	zone_introspect.good_size = (void *)zone_good_size;
@@ -213,46 +198,157 @@ register_zone(void)
 	zone_introspect.statistics = NULL;
 #if (JEMALLOC_ZONE_VERSION >= 6)
 	zone_introspect.zone_locked = NULL;
-#endif
-#if (JEMALLOC_ZONE_VERSION >= 7)
-	zone_introspect.enable_discharge_checking = NULL;
-	zone_introspect.disable_discharge_checking = NULL;
-	zone_introspect.discharge = NULL;
-#ifdef __BLOCKS__
-	zone_introspect.enumerate_discharged_pointers = NULL;
-#else
-	zone_introspect.enumerate_unavailable_without_blocks = NULL;
-#endif
 #endif

+	return (&zone);
+}
+
+static size_t
+ozone_size(malloc_zone_t *zone, void *ptr)
+{
+	size_t ret;
+
+	ret = ivsalloc(ptr);
+	if (ret == 0)
+		ret = szone.size(zone, ptr);
+
+	return (ret);
+}
+
+static void
+ozone_free(malloc_zone_t *zone, void *ptr)
+{
+
+	if (ivsalloc(ptr) != 0)
+		JEMALLOC_P(free)(ptr);
+	else {
+		size_t size = szone.size(zone, ptr);
+		if (size != 0)
+			(szone.free)(zone, ptr);
+	}
+}
+
+static void *
+ozone_realloc(malloc_zone_t *zone, void *ptr, size_t size)
+{
+	size_t oldsize;
+
+	if (ptr == NULL)
+		return (JEMALLOC_P(malloc)(size));
+
+	oldsize = ivsalloc(ptr);
+	if (oldsize != 0)
+		return (JEMALLOC_P(realloc)(ptr, size));
+	else {
+		oldsize = szone.size(zone, ptr);
+		if (oldsize == 0)
+			return (JEMALLOC_P(malloc)(size));
+		else {
+			void *ret = JEMALLOC_P(malloc)(size);
+			if (ret != NULL) {
+				memcpy(ret, ptr, (oldsize < size) ? oldsize :
+				    size);
+				(szone.free)(zone, ptr);
+			}
+			return (ret);
+		}
+	}
+}
+
+static unsigned
+ozone_batch_malloc(malloc_zone_t *zone, size_t size, void **results,
+    unsigned num_requested)
+{
+
+	/* Don't bother implementing this interface, since it isn't required. */
+	return (0);
+}
+
+static void
+ozone_batch_free(malloc_zone_t *zone, void **to_be_freed, unsigned num)
+{
+	unsigned i;
+
+	for (i = 0; i < num; i++)
+		ozone_free(zone, to_be_freed[i]);
+}
+
+#if (JEMALLOC_ZONE_VERSION >= 6)
+static void
+ozone_free_definite_size(malloc_zone_t *zone, void *ptr, size_t size)
+{
+
+	if (ivsalloc(ptr) != 0) {
+		assert(ivsalloc(ptr) == size);
+		JEMALLOC_P(free)(ptr);
+	} else {
+		assert(size == szone.size(zone, ptr));
+		szone.free_definite_size(zone, ptr, size);
+	}
+}
+#endif
+
+static void
+ozone_force_lock(malloc_zone_t *zone)
+{
+
+	/* jemalloc locking is taken care of by the normal jemalloc zone. */
+	szone.introspect->force_lock(zone);
+}
+
+static void
+ozone_force_unlock(malloc_zone_t *zone)
+{
+
+	/* jemalloc locking is taken care of by the normal jemalloc zone. */
+	szone.introspect->force_unlock(zone);
+}
+
+/*
+ * Overlay the default scalable zone (szone) such that existing allocations are
+ * drained, and further allocations come from jemalloc.  This is necessary
+ * because Core Foundation directly accesses and uses the szone before the
+ * jemalloc library is even loaded.
+ */
+void
+szone2ozone(malloc_zone_t *zone)
+{
+
 	/*
-	 * The default purgeable zone is created lazily by OSX's libc.  It uses
-	 * the default zone when it is created for "small" allocations
-	 * (< 15 KiB), but assumes the default zone is a scalable_zone.  This
-	 * obviously fails when the default zone is the jemalloc zone, so
-	 * malloc_default_purgeable_zone is called beforehand so that the
-	 * default purgeable zone is created when the default zone is still
-	 * a scalable_zone.  As purgeable zones only exist on >= 10.6, we need
-	 * to check for the existence of malloc_default_purgeable_zone() at
-	 * run time.
+	 * Stash a copy of the original szone so that we can call its
+	 * functions as needed.  Note that the internally, the szone stores its
+	 * bookkeeping data structures immediately following the malloc_zone_t
+	 * header, so when calling szone functions, we need to pass a pointer
+	 * to the original zone structure.
 	 */
-	if (malloc_default_purgeable_zone != NULL)
-		malloc_default_purgeable_zone();
+	memcpy(&szone, zone, sizeof(malloc_zone_t));

-	/* Register the custom zone.  At this point it won't be the default. */
-	malloc_zone_register(&zone);
+	zone->size = (void *)ozone_size;
+	zone->malloc = (void *)zone_malloc;
+	zone->calloc = (void *)zone_calloc;
+	zone->valloc = (void *)zone_valloc;
+	zone->free = (void *)ozone_free;
+	zone->realloc = (void *)ozone_realloc;
+	zone->destroy = (void *)zone_destroy;
+	zone->zone_name = "jemalloc_ozone";
+	zone->batch_malloc = ozone_batch_malloc;
+	zone->batch_free = ozone_batch_free;
+	zone->introspect = &ozone_introspect;
+	zone->version = JEMALLOC_ZONE_VERSION;
+#if (JEMALLOC_ZONE_VERSION >= 6)
+	zone->memalign = zone_memalign;
+	zone->free_definite_size = ozone_free_definite_size;
+#endif

-	/*
-	 * Unregister and reregister the default zone.  On OSX >= 10.6,
-	 * unregistering takes the last registered zone and places it at the
-	 * location of the specified zone.  Unregistering the default zone thus
-	 * makes the last registered one the default.  On OSX < 10.6,
-	 * unregistering shifts all registered zones.  The first registered zone
-	 * then becomes the default.
-	 */
-	do {
-		default_zone = malloc_default_zone();
-		malloc_zone_unregister(default_zone);
-		malloc_zone_register(default_zone);
-	} while (malloc_default_zone() != &zone);
+	ozone_introspect.enumerator = NULL;
+	ozone_introspect.good_size = (void *)zone_good_size;
+	ozone_introspect.check = NULL;
+	ozone_introspect.print = NULL;
+	ozone_introspect.log = NULL;
+	ozone_introspect.force_lock = (void *)ozone_force_lock;
+	ozone_introspect.force_unlock = (void *)ozone_force_unlock;
+	ozone_introspect.statistics = NULL;
+#if (JEMALLOC_ZONE_VERSION >= 6)
+	ozone_introspect.zone_locked = NULL;
+#endif
 }
--- a/Show More
+++ b/Show More