Prequential Evaluation Task

Prequential Evaluation task is a scheme in evaluating performance of online classifiers which uses each instance for testing online classifiers model and then it further uses the same instance for training the model(Test-then-train). This task currently only supports basic classification performance evaluator which measures the accuracy of the classifier model since the start of the evaluation.

Examples of Prequential Evaluation task in SAMOA command line when deploying into Storm

cd src/main/resources
./samoa storm ../../../target/SAMOA-Storm-0.0.1.jar "PrequentialEvaluation -d /tmp/dump.csv -i 1000000 -f 100000 -l (com.yahoo.labs.samoa.learners.classifiers.trees.VerticalHoeffdingTree -p 4) -s (com.yahoo.labs.samoa.moa.streams.generators.RandomTreeGenerator -c 2 -o 10 -u 10)"

Parameters:

• -l: classifier to train
• -s: stream to learn from
• -e: classification performance evaluation method
• -i: maximum number of instances to test/train on (-1 = no limit)
• -f: how many instances between samples of the learning performance
• -n: evaluation name (default: PrequentialEvaluation_TimeStamp)
• -d: file to append intermediate csv results to

In term of SAMOA API, Prequential Evaluation consists of source PI, a classifier, and evaluator PI as shown below. Source PI sends the instances to the classifier using source stream. The classifier sends the classification results to evaluator PI via result stream. The source PI corresponds to the -s option of Prequential Evaluation, the classifier corresponds to the -l option, and the evaluator PI corresponds to the -e option.