Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs updates #170

Merged
merged 23 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/publish-docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Publish docs

on: # yamllint disable-line rule:truthy
release:
types: [published]

permissions:
contents: write

jobs:
deploy:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.9
- run: pip install -r ./docs/requirements.txt
- run: mkdocs gh-deploy --force
211 changes: 175 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,73 +61,212 @@ pip install noteable-origami --pre

<!-- --8<-- [end:install] -->

<!-- --8<-- [start:start] -->

## Getting Started

> **Warning**
> Developer note: this documentation is written for the 1.0 alpha release. For stable release, see [pre-1.0 README](https://github.com/noteable-io/origami/blob/release/0.0.35/README.md)
> **Note**
> Developer note: For pre-1.0 release information, see the [pre-1.0 README](https://github.com/noteable-io/origami/blob/release/0.0.35/README.md)

### API Tokens

<!-- --8<-- [start:api-tokens] -->
The Noteable API requires an authentication token. You can manage tokens at the Noteable user settings page.

1. Log in to Noteable (sign up is free)
2. In the User Settings tab, navigate to `API Tokens` and generate a new token
1. Log in to [Noteable](https://app.noteable.io) (sign up is free).
2. In the User Settings tab, navigate to `API Tokens` and generate a new token.
![](./screenshots/user_settings__api_tokens.png)
3. Copy the generated token to the clipboard and save in a secure location, to be read into your Python environment later.
![](./screenshots/user_settings__api_tokens2.png)

The token can be passed directly in to `APIClient` on initialization, or set it as env var `NOTEABLE_TOKEN`.

### Usage
<!-- --8<-- [end:api-tokens] -->

The example below shows how to create a Notebook, launch a Kernel, add new cells, and execute code.
The example below will guide you through the basics of creating a notebook, adding content, executing code, and seeing the output. For more examples, see our [Use Cases](../usage) section.

```python
# Get your API token from the User Settings page by clicking your account name in the upper right
api_token = os.environ['NOTEABLE_TOKEN']
### Setting up the `APIClient`
<!-- --8<-- [start:api-client] -->
Using the API token you created previously, load it into your notebook environment so it can be passed into the `APIClient` directly. (If you're in [Noteable](https://app.noteable.io), you can create a [Secret](https://docs.noteable.io/product-docs/collaborate/access-and-visibility/secrets-permissions) that can be read in as an environment variable.)

# Client for interacting with Noteables REST API
```python
import os
from origami.clients.api import APIClient
api_client = APIClient(api_token)

# Sanity check your user information
# if we have the `NOTEABLE_TOKEN` environment variable set,
# we don't need to pass it in to the APIClient directly
api_client = APIClient()
```
*The `APIClient` is what we'll use to make HTTP requests to Noteable's REST API.*
<!-- --8<-- [end:api-client] -->

### Checking your user information
<!-- --8<-- [start:user-info] -->
```python
user = await api_client.user_info()
user
```
``` {.python .no-copy }
User(
id=UUID('f1a2b3c4-5678-4d90-ef01-23456789abcd'),
created_at=datetime.datetime(2023, 1, 1, 0, 0, 0, 0, tzinfo=datetime.timezone.utc),
updated_at=datetime.datetime(2023, 1, 1, 0, 0, 0, 0, tzinfo=datetime.timezone.utc),
deleted_at=None,
handle='ori.gami',
email='[email protected]',
first_name='Ori',
last_name='Gami',
origamist_default_project_id=UUID('a1b2c3d4-e5f6-4a7b-8123-abcdef123456'),
principal_sub='pat:0a1b2c3d4e5f6g7h8i9j10k11l',
auth_type='pat:0a1b2c3d4e5f6g7h8i9j10k11l'
)
```
(The information returned should match your user account information associated with the previously-generated API token.)
<!-- --8<-- [end:user-info] -->

# Choose a project to create the notebook in, here using the ChatGPT plugin default project
### Creating a new Notebook

> **Note**
> For this example, we're using the `origamist_default_project_id`, which is the default project designed to be used by the ChatGPT plugin. Feel free to replace it with projects you have access to in [Noteable](https://app.noteable.io/)!

<!-- --8<-- [start:create-notebook] -->
Provide a file `path` as well as a `project_id` (UUID) where the Notebook will exist.
```python
project_id = user.origamist_default_project_id

# Create a new Notebook
file = await api_client.create_notebook(project_id=project_id, path="Demo.ipynb")
file = await api_client.create_notebook(
project_id=project_id,
path="Origami Demo.ipynb"
)
file
```
``` {.python .no-copy }
File(
id=UUID('bcd12345-6789-4abc-d012-3456abcdef90'),
created_at=datetime.datetime(2023, 2, 2, 0, 0, 0, 0, tzinfo=datetime.timezone.utc),
updated_at=datetime.datetime(2023, 2, 2, 0, 0, 0, 0, tzinfo=datetime.timezone.utc),
deleted_at=None,
filename='Origami Demo.ipynb',
path=PosixPath('Origami Demo.ipynb'),
project_id=UUID('a1b2c3d4-e5f6-4a7b-8123-abcdef123456'),
space_id=UUID('7890ab12-3412-4cde-8901-2345abcdef67'),
size=0,
mimetype=None,
type='notebook',
current_version_id=None,
presigned_download_url=None,
url='https://app.noteable.io/f/abc12312-3412-4abc-8123-abc12312abc1/Origami Demo.ipynb'
)
```
<!-- --8<-- [end:create-notebook] -->

### Launching a Kernel
<!-- --8<-- [start:launch-kernel] -->
At a minimum, the `file_id` from the Notebook is required. Additionally, you can specify:

+ `kernel_name` (default `python3`, see more about [available kernels](https://docs.noteable.io/product-docs/work-with-notebooks/manage-kernels/noteable-provided-kernels))
+ `hardware_size` (default `small`, see more about [hardware options](https://docs.noteable.io/product-docs/work-with-notebooks/manage-hardware)).

# Start a Kernel
await api_client.launch_kernel(file.id)
```python
kernel_session = await api_client.launch_kernel(file_id=file.id)
kernel_session
```
```{.python .no-copy}
KernelSession(
id=UUID('e1f2a345-6789-4b01-cdef-1234567890ab'),
kernel=KernelDetails(
name='python3',
last_activity=datetime.datetime(2023, 2, 2, 1, 0, 0, 0, tzinfo=datetime.timezone.utc),
execution_state='idle'
)
)
```
<!-- --8<-- [end:launch-kernel] -->

# Client for Real-time Updates (RTU), used with Notebooks
### Adding Cells
<!-- --8<-- [start:connect-rtu] -->
Content updates and code execution is handled through the Noteable Real-Time Update (RTU) websocket connection.
```python
realtime_notebook = await api_client.connect_realtime(file)
```
<!-- --8<-- [end:connect-rtu] -->

# Add a new cell
> **Warning**
> You may see messages like `Received un-modeled RTU message msg.channel= ...`. This is expected as we update the Noteable backend services' messaging.

<!-- --8<-- [start:add-cells] -->
Once the RTU client is connected, we can begin adding cells, executing code, and more! First, let's add a code cell with a basic Python `print` statement.
```python
from origami.models.notebook import CodeCell

cell = CodeCell(source="print('Hello World')")
await realtime_notebook.add_cell(cell)
await realtime_notebook.add_cell(cell=cell)
```
(You can also pass code source directly into `.add_cell(source='CODE HERE')` as a shortcut.)
<!-- --8<-- [end:add-cells] -->

# Execute the cell. The returned value is a dictionary of Futures. Awaiting those futures will
# block until the cells have completed execution
queued_execution = await realtime_notebook.queue_execution(cell.id)
### Running a Code Cell
<!-- --8<-- [start:run-code-cell] -->
The returned value is a dictionary of `asyncio.Future`s. Awaiting those futures will block until the cells have completed execution.
The return value of the Futures is the up-to-date cell. If there's output, an output collection id will be set on the cell metadata.
```python
import asyncio

# The return value of the Futures is the up-to-date cell. If there's output, an output collection id
# will be set on the cell metadata
queued_execution = await realtime_notebook.queue_execution(cell.id)
cells = await asyncio.gather(*queued_execution)

# Grab the output
cell = cells[0]
output_collection = await api_client.get_output_collection(cell.output_collection_id)
print(output_collection.outputs[0].content.raw) # 'Hello World\n'
cell
```
```{.python .no-copy}
CodeCell(
id='2345ab6c-de78-4901-bcde-f1234567890a',
source="print('Hello World')",
metadata={
'noteable': {'output_collection_id': UUID('d1234e5f-6789-4a0b-c123-4567890abcdef')},
'ExecuteTime': {
'start_time': '2023-02-02T01:00:00.000000+00:00',
'end_time': '2023-02-02T01:00:00.050000+00:00'
}
},
cell_type='code',
execution_count=None,
outputs=[]
)
```
<!-- --8<-- [end:run-code-cell] -->

<!-- --8<-- [end:start] -->

## 1.0 Roadmap

Origami 1.0 implies that we have a stable architecture such as a split of `APIClient` and `RTUClient` and the layout of the RTU modeling. Some syntax may change as Origami is integrated into production components. Any breaking changes will be a minor version bump.
### Getting Cell Output
<!-- --8<-- [start:get-cell-output] -->
We can call the `.output_collection_id` property on cells directly, rather than having to parse the cell metadata.
```python
output_collection = await api_client.get_output_collection(cell.output_collection_id)
output_collection
```
```{.python .no-copy}
KernelOutputCollection(
id=UUID('d1234e5f-6789-4a0b-c123-4567890abcdef'),
created_at=datetime.datetime(2023, 2, 2, 1, 0, 1, 000000, tzinfo=datetime.timezone.utc),
updated_at=datetime.datetime(2023, 2, 2, 1, 0, 1, 000000, tzinfo=datetime.timezone.utc),
deleted_at=None,
cell_id='2345ab6c-de78-4901-bcde-f1234567890a',
widget_model_id=None,
file_id=UUID('bcd12345-6789-4abc-d012-3456abcdef90'),
outputs=[
KernelOutput(
id=UUID('abcdef90-1234-4a56-7890-abcdef123456'),
created_at=datetime.datetime(2023, 2, 2, 1, 0, 1, 000000, tzinfo=datetime.timezone.utc),
updated_at=datetime.datetime(2023, 2, 2, 1, 0, 1, 000000, tzinfo=datetime.timezone.utc),
deleted_at=None,
type='stream',
display_id=None,
available_mimetypes=['text/plain'],
content_metadata=KernelOutputContent(raw='{"name":"stdout"}', url=None, mimetype='application/json'),
content=KernelOutputContent(raw='Hello World\n', url=None, mimetype='text/plain'),
content_for_llm=KernelOutputContent(raw='Hello World\n', url=None, mimetype='text/plain'),
parent_collection_id=UUID('d1234e5f-6789-4a0b-c123-4567890abcdef')
)
]
)
```
<!-- --8<-- [end:get-cell-output] -->

## CLI

Expand Down
6 changes: 3 additions & 3 deletions docs/gen_doc_stubs.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@
full_doc_path = Path("reference", doc_path)
parts = list(module_path.parts)

if parts[-1].startswith("_"):
if parts[-1].startswith("_"):
continue

nav[parts] = doc_path.as_posix()

with mkdocs_gen_files.open(full_doc_path, "w") as fd:
Expand All @@ -26,5 +26,5 @@
# nav["mkdocs_autorefs", "references"] = "autorefs/references.md"
# nav["mkdocs_autorefs", "plugin"] = "autorefs/plugin.md"

with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: #
with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file:
nav_file.writelines(nav.build_literate_nav())
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Origami

<img src="./papersnake.svg" width="250px" />

--8<-- "README.md:intro"

--8<-- "README.md:requirements"

--8<-- "README.md:install"

--8<-- "README.md:start"

## Contributing

Expand Down
3 changes: 3 additions & 0 deletions docs/papersnake-dark.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 41 additions & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
@@ -1 +1,41 @@
--8<-- "README.md:start"
# Quick Start
The example below will guide you through the basics of creating a notebook, adding content, executing code, and seeing the output. For more examples, see our [Use Cases](../usage) section.

!!! note "Developer note: For pre-1.0 release information, see the [pre-1.0 README](https://github.com/noteable-io/origami/blob/release/0.0.35/README.md)"

--8<-- "README.md:install"

## API Tokens
--8<-- "README.md:api-tokens"

## Setting up the `APIClient`
--8<-- "README.md:api-client"

## Checking your user information
--8<-- "README.md:user-info"

## Creating a new Notebook

!!! note "For this example, we're using the `origamist_default_project_id`, which is the default project designed to be used by the ChatGPT plugin. Feel free to replace it with projects you have access to in [Noteable](https://app.noteable.io/)!"

--8<-- "README.md:create-notebook"

## Launching a Kernel

--8<-- "README.md:launch-kernel"

## Adding Cells

--8<-- "README.md:connect-rtu"

!!! warning "You may see messages like `Received un-modeled RTU message msg.channel= ...`. This is expected as we update the Noteable backend services' messaging."

--8<-- "README.md:add-cells"

## Running a Code Cell

--8<-- "README.md:run-code-cell"

## Getting Cell Output

--8<-- "README.md:get-cell-output"
1 change: 1 addition & 0 deletions docs/reference/clients/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: clients.api
1 change: 1 addition & 0 deletions docs/reference/clients/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: clients.cache
1 change: 1 addition & 0 deletions docs/reference/clients/rtu.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: clients.rtu
1 change: 1 addition & 0 deletions docs/reference/models/api/base.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.base
1 change: 1 addition & 0 deletions docs/reference/models/api/datasources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.datasources
1 change: 1 addition & 0 deletions docs/reference/models/api/files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.files
1 change: 1 addition & 0 deletions docs/reference/models/api/outputs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.outputs
1 change: 1 addition & 0 deletions docs/reference/models/api/projects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.projects
1 change: 1 addition & 0 deletions docs/reference/models/api/spaces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.spaces
1 change: 1 addition & 0 deletions docs/reference/models/api/users.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.api.users
1 change: 1 addition & 0 deletions docs/reference/models/deltas/base.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.deltas.base
1 change: 1 addition & 0 deletions docs/reference/models/deltas/delta_types/cell_contents.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.deltas.delta_types.cell_contents
1 change: 1 addition & 0 deletions docs/reference/models/deltas/delta_types/cell_execute.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.deltas.delta_types.cell_execute
1 change: 1 addition & 0 deletions docs/reference/models/deltas/delta_types/cell_metadata.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.deltas.delta_types.cell_metadata
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: models.deltas.delta_types.cell_output_collection
Loading