Merge pull request #80 from data-apis/document-adding-new-model

Add instructions on adding new builds

Author: saulshanabrook

README.md

@@ -28,11 +28,6 @@ def argmax(
 
 Contributions are very welcome! Please feel free to open an issue, or reach out directly, if there is anything you would like to discuss or explore! You can also check out the [issue tracker](https://github.com/data-apis/python-record-api/issues) for some possible next steps that we could use help on.
 
-## Hosted Usage
-
-We have this repository set up with Kubernetes and Github Actions to automatically analyze a number of libraries. The ones we have added are in [`k8/images`](./k8/images). Do you have a library that you would also like to see analyzed? Please open a PR adding that image to the folder there. Make sure to test it locally first, to see that it runs.
-
-Once it's added to the repo, it will be run and the data will be added to `data/api/<library_name>.json` and from there, it will be used to generate the NumPy and Pandas APIs. Those are present in [`data/api.json`](./data/api.json), in machine readable form, as well as in [`data/typing`](data/typing) in human readable form.
 
 ## Usage
 
@@ -84,6 +79,57 @@ env PYTHON_RECORD_API_OUTPUT=typing/ \
 ```
 
 
+## Hosted Usage
+
+We have this repository set up with Kubernetes and Github Actions to automatically analyze a number of libraries. The ones we have added are in [`k8/images`](./k8/images).
+
+Once it's added to the repo, it will be run and the data will be added to `data/api/<library_name>.json` and from there, it will be used to generate the NumPy and Pandas APIs. Those are present in [`data/api.json`](./data/api.json), in machine readable form, as well as in [`data/typing`](data/typing) in human readable form.
+
+### Adding more libraries
+
+Do you have a library that you would also like to see analyzed? We welcome PRs!
+
+You can add a new backend just from the Github editor UX, but you probably want to test it locally first.
+
+
+First install Docker >= v18.06 and Python.
+
+Then [fork the library](https://github.com/data-apis/python-record-api/fork), clone your fork, and check out a new branch:
+
+```bash
+git clone [email protected]:<FORK_NAME>/python-record-api.git
+cd python-record-api
+
+git checout -b <CHANGES_NAME>
+```
+
+And install the python package locally:
+
+```bash
+pip install flit
+flit install
+```
+
+Now either:
+
+1. Edit an existing library, by editing a `Dockerfile` in any of the directories in `./k8/images`. If you make any changes, also increment number in the corresponding `version` file.
+2. Add a new library, by creating a new directory with the name of the module in the `./k8/images` directory. The name cannot contain any underscores. It should have at least a `Dockerfile` and a `versions` file, with the contents of `0`. You should copy an existing `Dockerfile` and modify it to download some downstream library and run some command as the entrypoint which produces a trace.
+3. Add a new module to trace calls against, by modifying the `TO_MODULES` constant in `./k8/Makefile` to add another Python module and also incrementing the `./k8/argo/version` file.
+
+Then you can test an image locally, to make sure the Dockerfile builds correctly and the tracing works:
+
+```bash
+cd k8
+make test-local-<LIBRARY NAME>
+```
+
+This should output a number of lines of traces.
+
+Now, commit your changes, push your branch, and open a PR. This will then build the image in CI and attempt to kick off build of it.
+
+Once the PR is merged, every time we update the core library, this tracing will be re-run and the updated data will be added to the repo.
+
+
 ## Development
 
 First install the local package:

k8/Makefile

@@ -1,4 +1,5 @@
 
+TO_MODULES := pandas,numpy
 PYTHON_PACKAGE_VERSION := $(shell python -c 'import record_api; print(record_api.__version__)')
 
 $(info Python package version = ${PYTHON_PACKAGE_VERSION})
@@ -86,22 +87,34 @@ argo-submit-%:
 		-p label="$(*F)" \
 		-p base-image="$(BASE_IMAGE)" \
 		-p image="$(call sub_image,$(*F))" \
+		-p to_modules="$(TO_MODULES)" \
 		--name "$(call workflow_name,$(*F))"
 
 
 push-images: docker-bake.json
 	docker buildx bake base --push
 	docker buildx bake --push
 
+test-local-%:
+	env DOCKER_BUILDKIT=1 docker build \
+		--tag $(call sub_image,$(*F)) \
+		--build-arg "FROM=${BASE_IMAGE}" \
+		images/$(*F)
+	docker run \
+		--rm \
+		-it \
+		-e PYTHON_RECORD_API_TO_MODULES=$(TO_MODULES) \
+		-e PYTHON_RECORD_API_OUTPUT_FILE=/dev/stdout \
+		$(call sub_image,$(*F))
 
-test-%: docker-bake.json
-	# docker buildx bake --push $(*F)
+test-remote-%: docker-bake.json
+	docker buildx bake --push $(*F)
 	kubectl run \
 		$(*F) \
 		--rm \
 		-it \
 		--restart='Never' \
-		--env=PYTHON_RECORD_API_TO_MODULES=numpy,pandas \
+		--env=PYTHON_RECORD_API_TO_MODULES=$(TO_MODULES) \
 		--env=PYTHON_RECORD_API_OUTPUT_FILE=/dev/stdout \
 		--image=$(call sub_image,$(*F))
 

k8/argo/workflow.yml

@@ -26,6 +26,8 @@ spec:
         value: "..."
       - name: image
         value: "..."
+      - name: to_modules
+        value: "..."
   templates:
     - name: all
       steps:
@@ -48,7 +50,7 @@ spec:
           - name: PYTHON_RECORD_API_OUTPUT_FILE
             value: /tmp/vol/raw.jsonl
           - name: PYTHON_RECORD_API_TO_MODULES
-            value: numpy,pandas
+            value: "{{workflow.parameters.to_modules}}"
         # https://github.com/argoproj/argo/blob/master/docs/resource-duration.md#request-defaults
         # Must override or get defaults which are too low
         resources: