Upgrade to jemalloc 4.4.0

jemalloc 4.4.0 includes some optimizations and bug fixes.

Optimizations that we care about:
 - "Add extent serial numbers and use them where appropriate as a sort key that
   is higher priority than address, so that the allocation policy prefers older
   extents. This tends to improve locality (decrease fragmentation) when memory
   grows downward."

This actually causes a slight slowdown (4-5%) for binary-trees, but this is
likely an artifact of more purging occurring (leading to less fragmentation) so
it's actually a good thing. See jemalloc/jemalloc#147
for more info.

For the full changelog see:
 - https://github.com/jemalloc/jemalloc/releases/4.4.0

runtime/src/mem/jemalloc/mem-jemalloc.c

@@ -66,11 +66,11 @@ static void* chunk_alloc(void *chunk, size_t size, size_t alignment, bool *zero,
 
   // compute our current aligned pointer into the shared heap
   //
-  //   jemalloc 4.3.1 man: "The alignment parameter is always a power of two at
+  //   jemalloc 4.4.0 man: "The alignment parameter is always a power of two at
   //   least as large as the chunk size."
   cur_chunk_base = alignHelper(heap.base, heap.cur_offset, alignment);
 
-  // jemalloc 4.3.1 man: "If chunk is not NULL, the returned pointer must be
+  // jemalloc 4.4.0 man: "If chunk is not NULL, the returned pointer must be
   // chunk on success or NULL on error"
   if (chunk && chunk != cur_chunk_base) {
     pthread_mutex_unlock(&heap.alloc_lock);
@@ -91,7 +91,7 @@ static void* chunk_alloc(void *chunk, size_t size, size_t alignment, bool *zero,
   // now that cur_heap_offset is updated, we can unlock
   pthread_mutex_unlock(&heap.alloc_lock);
 
-  // jemalloc 4.3.1 man: "Zeroing is mandatory if *zero is true upon entry."
+  // jemalloc 4.4.0 man: "Zeroing is mandatory if *zero is true upon entry."
   if (*zero) {
      memset(cur_chunk_base, 0, size);
   }
@@ -159,7 +159,7 @@ static void initialize_arenas(void) {
   // for each non-zero arena, set the current thread to use it (this
   // initializes each arena). arena 0 is automatically initialized.
   //
-  //   jemalloc 4.3.1 man: "If the specified arena was not initialized
+  //   jemalloc 4.4.0 man: "If the specified arena was not initialized
   //   beforehand, it will be automatically initialized as a side effect of
   //   calling this interface."
   narenas = get_num_arenas();
@@ -249,7 +249,7 @@ static bool addressNotInHeap(void* ptr) {
 // grab (and leak) whatever memory jemalloc got on it's own, that's not in
 // our shared heap
 //
-//   jemalloc 4.3.1 man: "arenas may have already created chunks prior to the
+//   jemalloc 4.4.0 man: "arenas may have already created chunks prior to the
 //   application having an opportunity to take over chunk allocation."
 //
 // jemalloc grabs "chunks" from the system in order to store metadata and some
@@ -307,7 +307,7 @@ void chpl_mem_layerInit(void) {
   // of initializing jemalloc. If we're not using a shared heap, do a first
   // allocation to allow jemalloc to set up:
   //
-  //   jemalloc 4.3.1 man: "Once, when the first call is made to one of the
+  //   jemalloc 4.4.0 man: "Once, when the first call is made to one of the
   //   memory allocation routines, the allocator initializes its internals"
   if (heap_base != NULL) {
     heap.base = heap_base;

third-party/jemalloc/README

@@ -2,7 +2,7 @@
 jemalloc README for Chapel
 ==========================
 
-This copy of jemalloc 4.3.1 is being released with Chapel for
+This copy of jemalloc 4.4.0 is being released with Chapel for
 convenience and was obtained from:
 
   https://github.com/jemalloc/jemalloc
@@ -18,9 +18,9 @@ The directory $CHPL_HOME/third-party/jemalloc/jemalloc-src contains the
 un-tarballed jemalloc package contents. Version updates should be done as
 follows, assuming the CWD is $CHPL_HOME/third-party/jemalloc/:
 
-1. download and untar the latest jemalloc version: e.g. jemalloc-4.3.1
+1. download and untar the latest jemalloc version: e.g. jemalloc-4.4.0
 2. `rm -rf jemalloc-src`
-3. `mv jemalloc-4.3.1 jemalloc-src`
+3. `mv jemalloc-4.4.0 jemalloc-src`
 4. `git add --force jemalloc-src` (--force to ignore our .gitignore)
 5. update the version number mentioned above
 6. verify the references to jemalloc's man page in the runtime shim are

third-party/jemalloc/jemalloc-src/ChangeLog

@@ -4,6 +4,33 @@ brevity.  Much more detail can be found in the git revision history:
 
     https://github.com/jemalloc/jemalloc
 
+* 4.4.0 (December 3, 2016)
+
+  New features:
+  - Add configure support for *-*-linux-android.  (@cferris1000, @jasone)
+  - Add the --disable-syscall configure option, for use on systems that place
+    security-motivated limitations on syscall(2).  (@jasone)
+  - Add support for Debian GNU/kFreeBSD.  (@thesam)
+
+  Optimizations:
+  - Add extent serial numbers and use them where appropriate as a sort key that
+    is higher priority than address, so that the allocation policy prefers older
+    extents.  This tends to improve locality (decrease fragmentation) when
+    memory grows downward.  (@jasone)
+  - Refactor madvise(2) configuration so that MADV_FREE is detected and utilized
+    on Linux 4.5 and newer.  (@jasone)
+  - Mark partially purged arena chunks as non-huge-page.  This improves
+    interaction with Linux's transparent huge page functionality.  (@jasone)
+
+  Bug fixes:
+  - Fix size class computations for edge conditions involving extremely large
+    allocations.  This regression was first released in 4.0.0.  (@jasone,
+    @ingvarha)
+  - Remove overly restrictive assertions related to the cactive statistic.  This
+    regression was first released in 4.1.0.  (@jasone)
+  - Implement a more reliable detection scheme for os_unfair_lock on macOS.
+    (@jszakmeister)
+
 * 4.3.1 (November 7, 2016)
 
   Bug fixes:

third-party/jemalloc/jemalloc-src/INSTALL

@@ -206,6 +206,11 @@ any of the following arguments (not a definitive list) to 'configure':
     most extreme case increases physical memory usage for the 16 KiB size class
     to 20 KiB.
 
+--disable-syscall
+    Disable use of syscall(2) rather than {open,read,write,close}(2).  This is
+    intended as a workaround for systems that place security limitations on
+    syscall(2).
+
 --with-xslroot=<path>
     Specify where to find DocBook XSL stylesheets when building the
     documentation.
@@ -327,6 +332,15 @@ LDFLAGS="?"
 PATH="?"
     'configure' uses this to find programs.
 
+In some cases it may be necessary to work around configuration results that do
+not match reality.  For example, Linux 4.5 added support for the MADV_FREE flag
+to madvise(2), which can cause problems if building on a host with MADV_FREE
+support and deploying to a target without.  To work around this, use a cache
+file to override the relevant configuration variable defined in configure.ac,
+e.g.:
+
+    echo "je_cv_madv_free=no" > config.cache && ./configure -C
+
 === Advanced compilation =======================================================
 
 To build only parts of jemalloc, use the following targets:

third-party/jemalloc/jemalloc-src/Makefile.in

@@ -166,6 +166,8 @@ TESTS_UNIT := \
 	$(srcroot)test/unit/math.c \
 	$(srcroot)test/unit/mq.c \
 	$(srcroot)test/unit/mtx.c \
+	$(srcroot)test/unit/pack.c \
+	$(srcroot)test/unit/pages.c \
 	$(srcroot)test/unit/ph.c \
 	$(srcroot)test/unit/prng.c \
 	$(srcroot)test/unit/prof_accum.c \

third-party/jemalloc/jemalloc-src/VERSION

@@ -1 +1 @@
-4.3.1-0-g0110fa8451af905affd77c3bea0d545fee2251b2
+4.4.0-0-gf1f76357313e7dcad7262f17a48ff0a2e005fcdc