add documentation

Signed-off-by: Andres Taylor <[email protected]>

Author: systay

README.md

@@ -29,9 +29,12 @@ This method allows us to thoroughly test queries within a sharded environment, e
 
 ## How to use
 
-Build the `vitess-tester` binary:
+After installing the `vitess-tester` binary, you need to have Vitess installed and in your path. 
+To run vitess, you'll need to set VTDATAROOT and VTROOT environment variables.
+You can do this, and set up the Vitess environment by running the following command:
+
 ```sh
-make
+source build.env
 ```
 
 Basic usage:
@@ -68,32 +71,35 @@ Usage of ./vitess-tester:
   -perf-test
         include end-to-end performance tests
   -sharded
-        run all tests on a sharded keyspace
+        Run all tests on a sharded keyspace and using auto-vschema. This cannot be used with either -vschema or -vtexplain-vschema.
   -stderrthreshold value
         logs at or above this threshold go to stderr (default 2)
-  -test-dir string
-        Directory for the test files (default "./t/")
   -topo-flavor string
         choose a topo server from etcd2, zk2 or consul (default "etcd2")
   -v value
         log level for V logs
   -vmodule value
         comma-separated list of pattern=N settings for file-filtered logging
   -vschema string
-        Disable auto-vschema by providing your own vschema file
+        Disable auto-vschema by providing your own vschema file. This cannot be used with either -vtexplain-vschema or -sharded.
+  -vtexplain-vschema string
+        Disable auto-vschema by providing your own vtexplain vschema file. This cannot be used with either -vschema or -sharded.
   -xunit
         Get output in an xml file instead of errors directory
 ```
 
-It will bring up an entire Vitess cluster on 127.0.0.1, unsharded or sharded depending on the `-sharded` flag. MySQL and VTGate both start with root and no password configured.
+It will bring up an entire Vitess cluster on 127.0.0.1, unsharded or sharded depending on the configuration. MySQL and VTGate both start with root and no password configured.
 
 ```sh
-./vitess-tester t/example.test # run a specified test
-./vitess-tester t/example1.test t/example2.test  t/example3.test # separate different tests with one or more spaces
-./vitess-tester t/*.test   # wildcards can be used
-./vitess-tester https://raw.githubusercontent.com/vitessio/vitess-tester/main/t/basic.test # can also be run against an URL
+vitess-tester t/example.test # run a specified test
+vitess-tester t/example1.test t/example2.test  t/example3.test # separate different tests with one or more spaces
+vitess-tester t/*.test   # wildcards can be used
+vitess-tester https://raw.githubusercontent.com/vitessio/vitess-tester/main/t/basic.test # can also be run against an URL
+vitess-tester -vtexplain-vschema t/vtexplain-vschema.json t/vtexplain.test # run a test with a custom vschema
 ```
 
+The test files can be amended with directives to control the testing process. Check out `directives.test` to see examples of what directives are available. 
+
 ## Contributing
 
 Contributions are welcomed and greatly appreciated. You can help by:

build.env

@@ -0,0 +1,27 @@
+# No shebang line as this script is sourced from an external shell.
+
+# Copyright 2019 The Vitess Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+source ./tools/shell_functions.inc
+
+export VTROOT="$PWD"
+export VTDATAROOT="${VTDATAROOT:-${VTROOT}/vtdataroot}"
+export PROTOC_VER=21.3
+export ZK_VER=${ZK_VERSION:-3.8.0}
+export ETCD_VER=v3.5.6
+export CONSUL_VER=1.11.4
+export TOXIPROXY_VER=v2.7.0
+
+mkdir -p "$VTDATAROOT"

t/directives.test

@@ -0,0 +1,20 @@
+# vitess-tester, just like mysql-tester, uses `--<directive>` in the .test files to do things outside the regular script
+# --skip <comment> allows you to skip a test with a comment
+--skip "This test is not ready yet"
+select * from table_doesnt_exist;
+
+# skip_if_below_version <binary> <version> allows you to skip a query if the binary version is below the specified version
+--skip_if_below_version vtgate 999.0
+select * from table_doesnt_exist;
+
+# --error allows to assert that a query fails with an error
+--error the following query should fail
+select * from table_doesnt_exist;
+
+# --vexplain <mode> allows to run a query with vexplain to see the plan or which queries are sent to mysql. Mode can be `plan` or `queries`
+--vexplain plan
+select 1;
+
+# --wait_authoritative <table> <keyspace> allows to wait until the vtgate has authoritative information about the table
+# This is useful when you provide your own vschema and want it amended by the schema tracker
+--wait_authoritative table_doesnt_exist keyspace_doesnt_exist