Skip to content

Commit d140494

Browse files
blythedblythed
committed
Repair templates for 0.6 release
1 parent 81ea0af commit d140494

File tree

112 files changed

+1233
-2947
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

112 files changed

+1233
-2947
lines changed

‎docs/ai_plugins/anthropic.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_anthropic
2+
# `superduper_anthropic`
33

44
Superduper allows users to work with anthropic API models. The key integration is the integration of high-quality API-hosted LLM services.
55

‎docs/ai_plugins/cohere.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_cohere
2+
# `superduper_cohere`
33

44
Superduper allows users to work with cohere API models.
55

‎docs/ai_plugins/custom.md

-12
This file was deleted.

‎docs/ai_plugins/jina.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_jina
2+
# `superduper_jina`
33

44
Superduper allows users to work with Jina Embeddings models through the Jina Embedding API.
55

‎docs/ai_plugins/llamacpp.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_llamacpp
2+
# `superduper_llamacpp`
33

44
Superduper allows users to work with self-hosted LLM models via [Llama.cpp](https://github.com/ggerganov/llama.cpp).
55

‎docs/ai_plugins/openai.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_openai
2+
# `superduper_openai`
33

44
Superduper allows users to work with openai API models.
55

‎docs/ai_plugins/pillow.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_pillow
2+
# `superduper_pillow`
33

44
`superduper_pillow` is a plugin for SuperDuper that provides support for Pillow.
55

‎docs/ai_plugins/sentence_transformers.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_sentence_transformers
2+
# `superduper_sentence_transformers`
33

44
superduper allows users to work with self-hosted embedding models via [Sentence-Transformers](https://sbert.net).
55

‎docs/ai_plugins/sklearn.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_sklearn
2+
# `superduper_sklearn`
33

44
superduper allows users to work with arbitrary sklearn estimators, with additional support for pre-, post-processing and input/ output data-types.
55

‎docs/ai_plugins/supported_ai_frameworks.md

-32
This file was deleted.

‎docs/ai_plugins/torch.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_torch
2+
# `superduper_torch`
33

44
Superduper allows users to work with arbitrary `torch` models, with custom pre-, post-processing and input/ output data-types, as well as offering training with superduper
55

‎docs/ai_plugins/transformers.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_transformers
2+
# `superduper_transformers`
33

44
Transformers is a popular AI framework, and we have incorporated native support for Transformers to provide essential Large Language Model (LLM) capabilities.
55

‎docs/ai_plugins/vllm.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!-- Auto-generated content start -->
2-
# superduper_vllm
2+
# `superduper_vllm`
33

44
Superduper allows users to work with self-hosted LLM models via [vLLM](https://github.com/vllm-project/vllm).
55

‎docs/api/base/apply.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,16 @@ apply(db: 'Datalayer',
1010
Sequence[Any],
1111
Any],
1212
force: bool | None = None,
13-
wait: bool = False) -> 'Component'
13+
wait: bool = False,
14+
jobs: bool = True) -> 'Component'
1415
```
1516
| Parameter | Description |
1617
|-----------|-------------|
1718
| db | Datalayer instance |
1819
| object | Object to be stored. |
1920
| force | List of jobs which should execute before component initialization begins. |
2021
| wait | Blocks execution till create events finish. |
22+
| jobs | Whether to execute jobs or not. |
2123

2224
Add functionality in the form of components.
2325

‎docs/api/base/datatype.md

+27-27
Original file line numberDiff line numberDiff line change
@@ -50,19 +50,19 @@ Blob(self,
5050

5151
Placeholder for a blob of bytes.
5252

53-
## `ComponentType`
53+
## `BaseType`
5454

5555
```python
56-
ComponentType(self) -> None
56+
BaseType(self) -> None
5757
```
58-
Datatype for encoding leafs.
58+
Datatype for encoding base instances.
5959

60-
## `LeafType`
60+
## `ComponentType`
6161

6262
```python
63-
LeafType(self) -> None
63+
ComponentType(self) -> None
6464
```
65-
Datatype for encoding leafs.
65+
Datatype for encoding `Component` instances.
6666

6767
## `FileItem`
6868

@@ -112,6 +112,20 @@ BaseVector(self,
112112

113113
Base class for vector.
114114

115+
## `ComponentDict`
116+
117+
```python
118+
ComponentDict(self) -> None
119+
```
120+
Datatype for encoding dictionaries which are supported as dict by databackend.
121+
122+
## `ComponentList`
123+
124+
```python
125+
ComponentList(self) -> None
126+
```
127+
Datatype for encoding lists which are supported as list by databackend.
128+
115129
## `Dill`
116130

117131
```python
@@ -131,13 +145,6 @@ DillEncoder(self) -> None
131145
```
132146
Encoder with dill.
133147

134-
## `FDict`
135-
136-
```python
137-
FDict(self) -> None
138-
```
139-
Datatype for encoding dictionaries of files.
140-
141148
## `FieldType`
142149

143150
```python
@@ -159,6 +166,13 @@ File(self) -> None
159166
```
160167
Type for encoding files on disk.
161168

169+
## `FileDict`
170+
171+
```python
172+
FileDict(self) -> None
173+
```
174+
Datatype for encoding dictionaries of files.
175+
162176
## `JSON`
163177

164178
```python
@@ -195,20 +209,6 @@ PickleEncoder(self) -> None
195209
```
196210
Pickle encoder.
197211

198-
## `SDict`
199-
200-
```python
201-
SDict(self) -> None
202-
```
203-
Datatype for encoding dictionaries which are supported as dict by databackend.
204-
205-
## `SList`
206-
207-
```python
208-
SList(self) -> None
209-
```
210-
Datatype for encoding lists which are supported as list by databackend.
211-
212212
## `Saveable`
213213

214214
```python

‎docs/api/base/document.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Document(self,
1818
| db | The datalayer to use. |
1919
| kwargs | **kwargs for `dict` |
2020

21-
A wrapper around an instance of dict or a Encodable.
21+
A wrapper around a `dict` including a schema and encoding.
2222

2323
The document data is used to dump that resource to
2424
a mix of json-able content, ids and `bytes`

‎docs/api/base/encoding.md

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66

77
```python
88
EncodeContext(self,
9+
name: str = '__main__',
910
builds: Dict[str,
1011
dict] = <factory>,
1112
blobs: Dict[str,

‎docs/api/base/event.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Job(self,
3535
job_id: Optional[str] = <factory>,
3636
method: str,
3737
status: str = 'pending',
38-
dependencies: Sequence[str] = ()) -> None
38+
dependencies: List[str] = <factory>) -> None
3939
```
4040
| Parameter | Description |
4141
|-----------|-------------|

‎docs/api/components/application.md

+6-10
Original file line numberDiff line numberDiff line change
@@ -8,26 +8,22 @@
88
Application(self,
99
identifier: str,
1010
upstream: Optional[List[ForwardRef('Component')]] = None,
11-
cache: Optional[bool] = True,
12-
status: Optional[str] = None,
13-
build_variables: Optional[Dict] = None,
14-
build_template: str | None = None,
1511
db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None,
1612
*,
1713
components: List[superduper.components.component.Component],
18-
link: Optional[str] = None) -> None
14+
link: Optional[str] = None,
15+
build_variables: Optional[Dict] = None,
16+
build_template: str | None = None) -> None
1917
```
2018
| Parameter | Description |
2119
|-----------|-------------|
2220
| identifier | Identifier of the instance. |
2321
| upstream | A list of upstream components. |
24-
| cache | (Optional) If set `true` the component will not be cached during primary job of the component i.e on a distributed cluster this component will be reloaded on every component task e.g model prediction. |
25-
| status | What part of the lifecycle the component is in. |
26-
| build_variables | Variables which were supplied to a template to build. |
27-
| build_template | Template which was used to build. |
28-
| db | Datalayer instance. |
22+
| db | Datalayer instance. Datalayer instance. |
2923
| components | List of components to group together and apply to `superduper`. |
3024
| link | A reference link to web app serving the application i.e. streamlit, gradio, etc |
25+
| build_variables | Variables which were supplied to a template to build. |
26+
| build_template | Template which was used to build. |
3127

3228
A placeholder to hold list of components with associated funcionality.
3329

‎docs/api/components/cdc.md

+2-10
Original file line numberDiff line numberDiff line change
@@ -25,23 +25,15 @@ a component which ingests from the table (etc.).
2525
CDC(self,
2626
identifier: str,
2727
upstream: Optional[List[ForwardRef('Component')]] = None,
28-
cache: Optional[bool] = True,
29-
status: Optional[str] = None,
30-
build_variables: Optional[Dict] = None,
31-
build_template: str | None = None,
3228
db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None,
3329
*,
34-
cdc_table: str = <property object at 0x126b1c950>) -> None
30+
cdc_table: str = <property object at 0x11c42b3d0>) -> None
3531
```
3632
| Parameter | Description |
3733
|-----------|-------------|
3834
| identifier | Identifier of the instance. |
3935
| upstream | A list of upstream components. |
40-
| cache | (Optional) If set `true` the component will not be cached during primary job of the component i.e on a distributed cluster this component will be reloaded on every component task e.g model prediction. |
41-
| status | What part of the lifecycle the component is in. |
42-
| build_variables | Variables which were supplied to a template to build. |
43-
| build_template | Template which was used to build. |
44-
| db | Datalayer instance. |
36+
| db | Datalayer instance. Datalayer instance. |
4537
| cdc_table | Table which fires the triggers. |
4638

4739
Trigger actions when new data arrives in a table.

‎docs/api/components/component.md

+5-18
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,10 @@
22

33
[Source code](https://github.com/superduper-io/superduper/blob/main/superduper/components/component.py)
44

5-
## `build_uuid`
5+
## `ensure_setup`
66

77
```python
8-
build_uuid()
9-
```
10-
Build UUID.
11-
12-
## `ensure_initialized`
13-
14-
```python
15-
ensure_initialized(func)
8+
ensure_setup(func)
169
```
1710
| Parameter | Description |
1811
|-----------|-------------|
@@ -26,24 +19,18 @@ Decorator to ensure that the model is initialized before calling the function.
2619
Component(self,
2720
identifier: str,
2821
upstream: Optional[List[ForwardRef('Component')]] = None,
29-
cache: Optional[bool] = True,
30-
status: Optional[str] = None,
31-
build_variables: Optional[Dict] = None,
32-
build_template: str | None = None,
3322
db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None) -> None
3423
```
3524
| Parameter | Description |
3625
|-----------|-------------|
3726
| identifier | Identifier of the instance. |
3827
| upstream | A list of upstream components. |
39-
| cache | (Optional) If set `true` the component will not be cached during primary job of the component i.e on a distributed cluster this component will be reloaded on every component task e.g model prediction. |
40-
| status | What part of the lifecycle the component is in. |
41-
| build_variables | Variables which were supplied to a template to build. |
42-
| build_template | Template which was used to build. |
43-
| db | Datalayer instance. |
28+
| db | Datalayer instance. Datalayer instance. |
4429

4530
Base class for all components in superduper.io.
4631

4732
Class to represent superduper.io serializable entities
4833
that can be saved into a database.
4934

35+
:param db: Datalayer instance.
36+

0 commit comments

Comments
 (0)