Initial commit

Author: Meghana Gupta

test/benchmarks/ARES-6/ARES-6.svg

@@ -0,0 +1,152 @@
+# All about Air.js
+
+Air.js is an ES6 benchmark. It tries to faithfully use new features like arrow
+functions, classes, for-of, and Map/Set, among others. Air.js doesn't avoid any
+features out of fear that they might be slow, in the hope that we might learn
+how to make those features fast by looking at how Air.js and other benchmarks
+use them.
+
+This documents the motivation, design, and license of Air.js.
+
+To run Air.js, simply open "[Air.js/test.html](test.html)" in your browser. It
+will only run correctly if your browser supports ES6.
+
+## Motivation
+
+At the time that Air.js was written, most JavaScript benchmarks used ES5 or
+older versions of the language. ES6 testing mostly relied on microbenchmarks or
+conversions of existing tests to ES6. We try to use larger benchmarks to avoid
+over-optimizing for small pieces of code, and we avoid making changes to
+existing benchmarks because that approach has no limiting principle: if it's OK
+to change a benchmark to use a feature, does that mean we can also change it to
+remove the use of a feature we don't like? We feel that the best way to avoid
+falling into the trap of creating benchmarks that reinforce what some JS engine
+is already good at is to create a new benchmark from first principles.
+
+We only recently completed our new JavaScript compiler, called
+[B3](https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/). B3's
+backend, called
+[Air](https://webkit.org/docs/b3/assembly-intermediate-representation.html), is
+very CPU-intensive and uses a combination of object-oriented and functional
+idioms in C++. Additionally, it relies heavily on high speed maps and sets. It
+goes so far as to use customized map/set implementations - even more so than
+the rest of WebKit. This makes Air a great candidate for ES6 benchmarking.
+Air.js is a faithful ES6 implementation of Air. It pulls no punches: just as
+the original C++ Air was written with expressiveness as a top priority, Air.js
+is liberal in its use of modern ES6 idioms whenever this helps make the code
+more readable. Unlike the original C++ Air, Air.js doesn't exploit a deep
+understanding of compilers to make the code easy to compile.
+
+## Design
+
+Air.js runs one of the more expensive Air phases, Air::allocateStack(). This
+turns abstract stack references into concrete stack references, by selecting
+how to lay out stack slots in the stack frame. This requires liveness analysis
+and an interference graph.
+
+Air.js relies on three major ES6 features more so than most of the others:
+
+- Arrow functions. Like the C++ Air, Air.js uses a functional style of
+  iterating most non-trivial data-structures:
+
+        inst.forEachArg((arg, role, type, width) => ...)
+  
+  This is because the functional style allows the callbacks to mutate the data
+  being iterated: if the callback returns a non-null value, forEachArg() will
+  replace the argument with that value. This would not have been possible with
+  for-of.
+
+- For-of. Many Air data structures are amenable to for-of iteration. While the
+  innermost loops tend to use functional iteration, pretty much all of the
+  outer logic uses for-of heavily. For example:
+
+        for (let block of code) // Iterate over the basic blocks
+            for (let inst of block) // Iterate over the instructions in a block
+                ...
+
+- Map/Set. The liveness analysis and Air::allocateStack() rely on maps and
+  sets. For example, we use a liveAtHead map that is keyed by basic block. Its
+  values are sets of live stack slots. This is a relatively crude way of doing
+  liveness, but it is exactly how the original Air::LivenessAnalysis worked, so
+  we view it as being quite faithful to how a sensible programmer might use Map
+  and Set.
+
+Air.js also uses some other ES6 features. For example, it uses a Proxy
+in one place, though we doubt that it's on a critical path. Air.js uses classes
+and let/const extensively, as well a symbols. Symbols are used as enumeration
+elements, and so they frequently show up as cases in switch statements.
+
+The workflow of an Air.js run is pretty simple: we do 150 runs of allocateStack
+on four IR payloads.
+
+Each IR payload is a large piece of ES6 code that constructs an Air.js Code
+object, complete with blocks, temporaries, stack slots, and instructions. These
+payloads are generated by running Air::dumpAsJS() phase just prior to the
+native allocateStack phase on the largest hot function in four major JS
+benchmarks according to JavaScriptCore's internal profiling:
+
+- Octane/GBEmu, the executeIteration function.
+- Kraken/imaging-gaussian-blur, the gaussianBlur function.
+- Octane/Typescript, the scanIdentifier function,
+- Air.js, an anonymous closure identified by our profiler as ACLj8C.
+
+These payloads allow Air.js to precisely replay allocateStack on those actual
+functions.
+
+It was an a priori goal of Air.js to spend most of the time in the
+allocateStack phase. This is a faithful reproduction of the C++ allocateStack
+phase, including its use of an abstract liveness analysis. It's abstract in the
+sense that the same liveness algorithm can be reused for temporaries,
+registers, or stack slots. In C++ this meant using templates, while in ES6 it
+means more run-time dynamic dispatch.
+
+Each IR payload is executable code that allocates the IR, and about 15% of
+benchmark execution time is spent in that code. This is significant, but having
+learned this, we don't feel that it would be honest to try to change the
+efficiency of payload initialization. What if the payload initialization was
+more expensive on our engine than others? If it was, then such a change would
+not be fair.
+
+Air.js validates its results. We added a Code hashing capability to both the
+C++ Air and Air.js, and we assert each payload looks identical after
+allocateStack to what it would have looked like after the original C++
+allocateStack. We also validate that payloads hash properly before
+allcoateStack, to help catch bugs during payload initialization. We have not
+measured how long hashing takes, but it's a O(N) operation, while allocateStack
+is closer to O(N^2). We suspect that barring some engine pathologies, hashing
+should be much faster than allocateStack, and allocateStack should be where the
+bulk of time is spent.
+
+## License
+
+Copyright (C) 2016 Apple Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+
+## Summary
+
+At the time that Air.js was written, we weren't happy with the ES6 benchmarks
+that were available to us. Air.js uses some ES6 features in anger, in the hope
+that we can learn about possible optimization strategies by looking at this and
+other benchmarks.

test/benchmarks/ARES-6/Air/README.md

@@ -0,0 +1,28 @@
+# Copyright (C) 2016 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1.  Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer. 
+# 2.  Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the distribution. 
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+- path: .
+  tests:
+    - stress-test.js
+  cmd: defaultRunNoisyTest unless parseRunCommands
+

test/benchmarks/ARES-6/Air/airjs-tests.yaml

@@ -0,0 +1,18 @@
+"use strict";
+
+load("symbols.js");
+load("tmp_base.js");
+load("arg.js");
+load("basic_block.js");
+load("code.js");
+load("frequented_block.js");
+load("inst.js");
+load("opcode.js");
+load("reg.js");
+load("stack_slot.js");
+load("tmp.js");
+load("util.js");
+load("custom.js");
+load("liveness.js");
+load("insertion_set.js");
+load("allocate_stack.js");