forked from clutchio/clutchio.github.com
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 440ec39
Showing
126 changed files
with
9,584 additions
and
0 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
Android SDK | ||
=========== | ||
|
||
If you don't have an account on your Clutch.io instance yet, you'll need to | ||
`do that first`_. Otherwise, follow the steps below. | ||
|
||
|
||
Getting Started | ||
--------------- | ||
|
||
First, download the `Clutch library`_ JAR file, and add it to your project. | ||
Now, in your project's main Activity, you should add three calls: one in | ||
``onCreate``, another in ``onPause``, and finally one in ``onResume``. Here's | ||
how it should look: | ||
|
||
In your ``onCreate``, call the ``ClutchAB.setup(Context ctx, String key)`` | ||
method: | ||
|
||
.. code-block:: java | ||
|
||
public void onCreate(Bundle savedInstanceState) { | ||
super.onCreate(savedInstanceState); | ||
|
||
ClutchAB.setup(this.getApplicationContext() | ||
"YOUR_APPLICATION_KEY", | ||
"YOUR_RPC_URL" | ||
); | ||
|
||
// The rest of your code here, maybe something like: | ||
// setContentView(R.layout.main); | ||
} | ||
|
||
In your ``onPause``, call the ``ClutchAB.onPause()`` method: | ||
|
||
.. code-block:: java | ||
|
||
protected void onPause() { | ||
super.onPause(); | ||
ClutchAB.onPause(); | ||
} | ||
|
||
In your ``onResume``, call the ``ClutchAB.onResume()`` method: | ||
|
||
.. code-block:: java | ||
|
||
protected void onResume() { | ||
super.onResume(); | ||
ClutchAB.onResume(); | ||
} | ||
|
||
Those three integration points will set everything up, and prepare your code to | ||
begin A/B testing. | ||
|
||
|
||
Normal Tests | ||
------------ | ||
|
||
The function for running a test is | ||
``ClutchAB.test(String name, ClutchABTest test)``. The ``ClutchABTest`` object | ||
should be an anonymous subclass and you can implement ``public void A()`` all | ||
the way up to ``public void J()`` for up to ten different tests per experiment. | ||
|
||
Example: | ||
|
||
.. code-block:: java | ||
|
||
// Find our login button | ||
final Button loginButton = (Button)findViewById(R.id.login_button); | ||
|
||
// Test which color performs better | ||
ClutchAB.test("loginButtonColor", new ClutchABTest() { | ||
public void A() { | ||
// Red? | ||
loginButton.setBackgroundColor(Color.RED); | ||
} | ||
public void B() { | ||
// Or green? | ||
loginButton.setBackgroundColor(Color.GREEN); | ||
} | ||
}); | ||
|
||
|
||
Data-driven Tests | ||
----------------- | ||
|
||
The function for running a data-driven test is | ||
``ClutchAB.test(String name, ClutchABDataTest test)``. The | ||
``ClutchABDataTest`` object should be an anonymous subclass that must implement | ||
``public void action(JSONObject testData)``. | ||
|
||
Example: | ||
|
||
.. code-block:: java | ||
|
||
// Find our login button | ||
final Button loginButton = (Button)findViewById(R.id.login_button); | ||
|
||
ClutchAB.test("loginButtonTitle", new ClutchABDataTest() { | ||
public void action(JSONObject testData) { | ||
loginButton.setText(testData.optString("title")); | ||
} | ||
}); | ||
|
||
|
||
Goal Reached | ||
------------ | ||
|
||
The function for noting that a goal was reached is | ||
``ClutchAB.goalReached(String name)``, where the argument is the test's short | ||
name. | ||
|
||
Example: | ||
|
||
.. code-block:: java | ||
|
||
public void onNewAccountCreated() { | ||
// A new account was created, so whatever button color was chosen, worked! | ||
ClutchAB.goalReached("loginButtonColor"); | ||
} | ||
|
||
|
||
.. _`do that first`: http://127.0.0.1:8000/register/ | ||
.. _`Clutch library`: https://github.com/downloads/clutchio/clutchandroid/Clutch-Android-Latest.jar |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
iOS SDK | ||
======= | ||
|
||
If you don't have an account on your Clutch.io instance yet, you'll need to | ||
`do that first`_. Otherwise, follow the steps below. | ||
|
||
|
||
Getting Started | ||
--------------- | ||
|
||
First make sure you have the `Clutch library`_ added to your project. To do | ||
so, download the library, extract the zip file, and drag the Clutch.framework | ||
folder into your project. Make sure that the library is in your project's | ||
linked frameworks section. The `Clutch library`_ depends on libsqlite3.dylib, | ||
so make sure to link that as well. | ||
|
||
Now import the Clutch A/B Testing library in your project's ``AppDelegate.h``, | ||
like so: | ||
|
||
.. code-block:: obj-c | ||
|
||
#import <Clutch/ClutchAB.h> | ||
|
||
Now in your project's ``AppDelegate.m``, under | ||
``application:didFinishLaunchingWithOptions:``, add the following code: | ||
|
||
.. code-block:: obj-c | ||
|
||
[ClutchAB setupForKey:@"YOUR_APPLICATION_KEY" rpcURL:@"YOUR_RPC_URL"]; | ||
|
||
That's it--you're now set up to run tests! | ||
|
||
|
||
Normal Tests | ||
------------ | ||
|
||
The function for running a test is ``testWithName:A:B:``. This can be given up | ||
to 10 different code blocks, by adding the next letter in the alphabet. A test | ||
with four code blocks would look like this: ``testWithName:A:B:C:D:``. | ||
|
||
Example: | ||
|
||
.. code-block:: obj-c | ||
|
||
// Create a button and add it to the navigation bar | ||
UIBarButtonItem *loginButton = [ | ||
[UIBarButtonItem alloc] initWithTitle:@"Log In" | ||
style:UIBarButtonItemStyleBordered | ||
target:self | ||
action:@selector(ensureUser)]; | ||
|
||
self.navigationItem.rightBarButtonItem = loginButton; | ||
|
||
// Test which color of tint performs better | ||
[ClutchAB testWithName:@"loginButtonColor" A:^{ | ||
// Red? | ||
loginButton.tintColor = [UIColor redColor]; | ||
} B:^{ | ||
// Or green? | ||
loginButton.tintColor = [UIColor greenColor]; | ||
}]; | ||
|
||
[loginButton release]; | ||
|
||
|
||
Data-driven Tests | ||
----------------- | ||
|
||
The function for running a data-driven test is ``testWithName:data``. The code | ||
block that's passed in must accept one NSDictionary argument. | ||
|
||
Example: | ||
|
||
.. code-block:: obj-c | ||
|
||
// Create a button and add it to the navigation bar | ||
UIBarButtonItem *loginButton = [ | ||
[UIBarButtonItem alloc] initWithTitle:@"Log In" | ||
style:UIBarButtonItemStyleBordered | ||
target:self | ||
action:@selector(ensureUser)]; | ||
|
||
self.navigationItem.rightBarButtonItem = loginButton; | ||
|
||
[ClutchAB testWithName:@"loginButtonTitle" data:^(NSDictionary *testData) { | ||
// Extract the title from the testData dictionary, and assign it to the button. | ||
loginButton.title = [testData objectForKey:@"title"]; | ||
}]; | ||
|
||
[loginButton release]; | ||
|
||
|
||
Goal Reached | ||
------------ | ||
|
||
The function for noting that a goal was reached is ``goalReached:``, where the | ||
argument is the test's short name. | ||
|
||
Example: | ||
|
||
.. code-block:: obj-c | ||
|
||
- (void)newAccountCreated { | ||
// A new account was created, so whatever button color was chosen, worked! | ||
[ClutchAB goalReached:@"loginButtonColor"]; | ||
} | ||
|
||
|
||
Extras | ||
------ | ||
|
||
It's very common for data-driven tests to be color-related. To aid in this, we | ||
have provided a simple function for getting a UIColor out of a hex-string. | ||
|
||
Example: | ||
|
||
.. code-block:: obj-c | ||
|
||
UIColor *buttonColor = [ClutchAB colorFromHex:@"FF0044"]; | ||
|
||
Here's how it might be used with a data-driven test: | ||
|
||
.. code-block:: obj-c | ||
|
||
[ClutchAB testWithName:@"loginButtonVariableColor" data:^(NSDictionary *testData) { | ||
// Extract the color from the testData dictionary, and assign it to the button. | ||
loginButton.tintColor = [ClutchAB colorFromHex:[testData objectForKey:@"color"]]; | ||
}]; | ||
|
||
|
||
.. _`do that first`: http://127.0.0.1:8000/register/ | ||
.. _`Clutch library`: https://github.com/downloads/clutchio/clutchios/Clutch-iOS-Latest.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
Overview | ||
======== | ||
|
||
A/B testing is a way of showing users different variations of your app, and | ||
then measuring how each variation affects your goals. For example, if you're | ||
interested in having your users make more in-app purchases, you might test the | ||
phrase "buy more coins now!" versus "save time by buying coins". Clutch's A/B | ||
testing tools can tell you which one will make you more money. | ||
|
||
Types of Tests (Normal vs. Data-Driven) | ||
--------------------------------------- | ||
|
||
With Clutch's A/B testing framework, you have the option of two different types | ||
of tests: normal, and data-driven. | ||
|
||
**Normal** tests are simpler: you give the framework two (or more) different | ||
pieces of code to run. We then choose one of those pieces of code, and run it, | ||
and report back to you which version improves whichever metrics you care about. | ||
This is best for qualitative things like whether a label should go on the left | ||
or right side of an image. | ||
|
||
**Data-driven** tests only have one code path, but that code is passed data from | ||
the server, which can be changed after the app is live in the app store. Your | ||
code should extract some variable from that data, and use it in it's test. | ||
This is good for quantitative things like button color, font size, or text | ||
copy. | ||
|
||
|
||
Tracking Goals | ||
-------------- | ||
|
||
Tests are only good when they can measure some metric and show you which path | ||
is performing better. To help out our system, that metric should be a "goal". | ||
Good examples of goals are things like an in-app purchase, a new account | ||
created, or even a tap on an advertisement. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
Screencasts | ||
=========== | ||
|
||
Everyone learns in different ways. For some, the best way to learn a new tool | ||
is to pore over documentation for hours. For others, jumping in on the deep | ||
end and just starting to build something is the best. For others still, visual | ||
and/or audio walkthroughs are best. Because of that, we want to make sure | ||
there are ample examples of each! Here's our collection of screencasts: | ||
|
||
Getting Started - iOS | ||
--------------------- | ||
|
||
In this screencast we will start a new Clutch A/B testing project for an | ||
existing iOS application, explaining in detail each step that's required and | ||
why. | ||
|
||
.. raw:: html | ||
|
||
<iframe src="http://player.vimeo.com/video/36469397?title=0&byline=0&portrait=0" width="670" height="380" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen></iframe> | ||
|
||
|
||
Getting Started - Android | ||
------------------------- | ||
|
||
In this screencast we will start a new Clutch A/B testing project for an | ||
existing Android application, explaining in detail each step that's required | ||
and why. | ||
|
||
.. raw:: html | ||
|
||
<iframe src="http://player.vimeo.com/video/36468715?byline=0&portrait=0" width="670" height="380" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowFullScreen></iframe> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
Clutch A/B Testing | ||
================== | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
ab-testing-overview | ||
ab-testing-ios | ||
ab-testing-android | ||
ab-testing-screencasts |
Oops, something went wrong.