Skip to content

Commit 4a0a80c

Browse files
AdamGlusteinpavithraes
AdamGlustein
and
pavithraes
authored
Fix up examples documentation and add the earthquake example (#346)
Surface end-to-end examples in Home Page and add a detailed table of examples --------- Signed-off-by: Pavithra Eswaramoorthy <[email protected]> Signed-off-by: Adam Glustein <[email protected]> Co-authored-by: Pavithra Eswaramoorthy <[email protected]>
1 parent 65207a4 commit 4a0a80c

File tree

10 files changed

+235
-28
lines changed

10 files changed

+235
-28
lines changed

‎docs/wiki/Home.md‎

Lines changed: 36 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,24 +16,55 @@ CSP (Composable Stream Processing) is a library for high-performance real-time e
1616
- **PyData interoperability:** Use your favorite libraries from the Scientific Python Ecosystem for numerical and statistical computations
1717
- **Functional/declarative style:** Write concise and composable code for stream processing by building graphs in Python
1818

19-
<!-- ## Applications -->
20-
2119
## Get Started
2220

2321
- [Install CSP](Installation) and [write your first CSP program](First-Steps)
2422
- Learn more about [nodes](CSP-Node), [graphs](CSP-Graph), and [execution modes](Execution-Modes)
2523
- Learn to extend CSP with [adapters](Adapters)
2624

27-
<!-- - Check out the [examples](Examples) for various CSP features and use cases -->
25+
## Applications
2826

29-
> \[!TIP\]
30-
> Find relevant docs with GitHub’s search function, use `repo:Point72/csp type:wiki <search terms>` to search the documentation Wiki Pages.
27+
<table>
28+
<tr>
29+
<td>Real-time public transportation</td>
30+
<td>Tracking Wikipedia page edits</td>
31+
<td>Processing seismic waveforms</td>
32+
<td>Visualizing global earthquakes</td>
33+
</tr>
34+
<tr>
35+
<td>
36+
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/mta.ipynb">
37+
<img src="images/mta-example.png" width="300px">
38+
</a>
39+
</td>
40+
<td>
41+
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/wikimedia.ipynb">
42+
<img src="images/wikimedia-example.png" width="300px">
43+
</a>
44+
</td>
45+
<td>
46+
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/seismic_waveform.ipynb">
47+
<img src="images/seismic-waveform-example.png" width="300px">
48+
</a>
49+
</td>
50+
<td>
51+
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/earthquake.ipynb">
52+
<img src="images/earthquake-example.png" width="300px">
53+
</a>
54+
</td>
55+
</tr>
56+
</table>
57+
58+
Check out the [examples directory](https://github.com/Point72/csp/tree/main/examples) for a comprehensive collection of CSP applications.
3159

3260
## Community
3361

3462
- [Contribute](Contribute) to CSP and help improve the project
3563
- Read about future plans in the [project roadmap](Roadmap)
3664

65+
> \[!TIP\]
66+
> Find relevant docs with GitHub’s search function, use `repo:Point72/csp type:wiki <search terms>` to search the documentation Wiki Pages.
67+
3768
## License
3869

3970
CSP is licensed under the Apache 2.0 license. See the [LICENSE](https://github.com/Point72/csp/blob/main/LICENSE) file for details.

‎docs/wiki/_Sidebar.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,8 @@ Notes for editors:
4949
- [`csp.Struct` API](csp.Struct-API)
5050
- [`csp.dynamic` API](csp.dynamic-API)
5151
- [`csp.profiler` API](csp.profiler-API)
52-
- [Examples](Examples)
5352
- [Glossary of Terms](Glossary)
53+
- [Examples](https://github.com/Point72/csp/tree/main/examples)
5454

5555
**Developer Guide**
5656

‎docs/wiki/images/earthquake-example.png‎

670 KB
Loading

‎docs/wiki/images/mta-example.png‎

475 KB
Loading

‎docs/wiki/images/seismic-waveform-example.png‎

69.2 KB
Loading

‎docs/wiki/images/wikimedia-example.png‎

429 KB
Loading

‎docs/wiki/references/Examples.md‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎examples/01_basics/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22

33
- [Simplest Possible Graph](./e1_basic.py)
44
- [Ticking Graphs](./e2_ticking.py)
5-
- [Complete Example (Trading)](./e3_trade_pnl.py)
6-
- [Visualizing a Graph](./e4_show_graph.py)
5+
- [Visualizing a Graph](./e3_show_graph.py)
6+
- [Complete Example (Trading)](./e4_trade_pnl.py)

‎examples/03_using_adapters/websocket/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
# Websocket Adapter
22

3+
- [Websocket Client](./e1_websocket_client.py)
34
- [Websocket Output](./e2_websocket_output.py)

‎examples/README.md‎

Lines changed: 195 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,197 @@
11
# `csp` examples
22

3-
- [Basics](./01_basics/)
4-
- [Intermediate](./02_intermediate/)
5-
- [Adapters](./03_using_adapters/)
6-
- [Writing Adapters](./04_writing_adapters/)
7-
- [Writing C++ Nodes and Adapters](./05_cpp/)
8-
- [Advanced](./06_advanced/)
9-
- [End to End Examples](./07_end_to_end/)
10-
- [Just for fun!](./98_just_for_fun/)
11-
- [Developer Tools](./99_developer_tools/)
3+
<table>
4+
<thead>
5+
<tr>
6+
<th>Type</th>
7+
<th>Example</th>
8+
<th>Description</th>
9+
</tr>
10+
</thead>
11+
<tbody>
12+
<!-- Basic -->
13+
<tr>
14+
<td rowspan=4><a href="./01_basics/">Basics</a></td>
15+
<td><a href="./01_basics/e1_basic.py">Basic Graph</a></td>
16+
<td>Basic sum of constant integers</td>
17+
</tr>
18+
<tr>
19+
<td><a href="./01_basics/e2_ticking.py">Ticking Graphs</a></td>
20+
<td>Cumulative sum of streaming integers</td>
21+
</tr>
22+
<tr>
23+
<td><a href="./01_basics/e3_show_graph.py">Visualizing a Graph</a></td>
24+
<td>Bid-ask spread of a trade with graph visualization</td>
25+
</tr>
26+
<tr>
27+
<td><a href="./01_basics/e4_trade_pnl.py">Complete Example (Trading)</a></td>
28+
<td>Volume weighted average price (VWAP) and profit and loss (PnL)</td>
29+
</tr>
30+
<!-- Intermediate -->
31+
<tr>
32+
<td rowspan=4><a href="./02_intermediate/">Intermediate</a></td>
33+
<td><a href="./02_intermediate/e1_feedback.py">Feedback Connections</a></td>
34+
<td>Add a feedback edge between nodes in a graph</td>
35+
</tr>
36+
<tr>
37+
<td><a href="./e02_intermediate/2_stats.py">Statistics Nodes</a></td>
38+
<td>Use the CSP statistics library on simulated trading data</td>
39+
</tr>
40+
<tr>
41+
<td><a href="./02_intermediate/e3_numpy_stats.py">Statistics Nodes with Numpy</a></td>
42+
<td>Rolling window statistics on a set of three symbols using NumPy arrays</td>
43+
</tr>
44+
<tr>
45+
<td><a href="./02_intermediate/e4_exprtk.py">Expression Nodes with <code>exprtk</code></a></td>
46+
<td>Calculate mathematical expressions with streaming data</td>
47+
</tr>
48+
<!-- Using Adapters -->
49+
<tr>
50+
<td rowspan=4><a href="./03_using_adapters/">Using Adapters</a></td>
51+
<td><a href="./03_using_adapters/kafka/e1_kafka.py">Kafka Adapter Example</a></td>
52+
<td>
53+
Stream data from a Kafka bus using KafkaAdapterManager and MessageMapper
54+
</td>
55+
</tr>
56+
<tr>
57+
<td><a href="./03_using_adapters/parquet/e1_parquet_write_read.py">Parquet Adapter Example</a></td>
58+
<td>
59+
Read from and write CSP data to Parquet files
60+
</td>
61+
</tr>
62+
<tr>
63+
<td><a href="./03_using_adapters/websocket/e1_websocket_client.py">Websocket Client</a></td>
64+
<td>
65+
Send and receive messages over a websocket connection
66+
</td>
67+
</tr>
68+
<tr>
69+
<td><a href="./03_using_adapters/websocket/e2_websocket_output.py">Websocket Output</a></td>
70+
<td>
71+
Send data over a websocket connection and view HTML output
72+
</td>
73+
</tr>
74+
<!-- Writing Adapters -->
75+
<tr>
76+
<td rowspan=8><a href="./04_writing_adapters/">Writing Adapters</a></td>
77+
<td><a href="./04_writing_adapters/e1_generic_push_adapter.py">Generic Push Adapter</a></td>
78+
<td>
79+
Push real-time data into a CSP graph
80+
</td>
81+
</tr>
82+
<tr>
83+
<td><a href="./04_writing_adapters/e2_pullinput.py">Pull Input Adapter</a></td>
84+
<td>
85+
Replay historical data using a pull adapter
86+
</td>
87+
</tr>
88+
<tr>
89+
<td><a href="./04_writing_adapters/e3_adaptermanager_pullinput.py">Pull Input Adapter with Adapter
90+
Manager</a></td>
91+
<td>
92+
Provide data to multiple input adapters from a single source
93+
</td>
94+
</tr>
95+
<tr>
96+
<td><a href="./04_writing_adapters/e4_pushinput.py">Push Input Adapter</a></td>
97+
<td>
98+
Write your own push adapter for real-time data
99+
</td>
100+
</tr>
101+
<tr>
102+
<td><a href="./04_writing_adapters/e5_adaptermanager_pushinput.py">Push Input Adapter with Adapter
103+
Manager</a></td>
104+
<td>
105+
Use an adapter manager with real-time data sources
106+
</td>
107+
</tr>
108+
<tr>
109+
<td><a href="./04_writing_adapters/e6_outputadapter.py">Output Adapter</a></td>
110+
<td>
111+
Create a custom writer for CSP output data
112+
</td>
113+
</tr>
114+
<tr>
115+
<td><a href="./04_writing_adapters/e7_adaptermanager_inputoutput.py">Complete Input/Output Adapter with
116+
Adapter Manager</a></td>
117+
<td>
118+
Manage input and output adapters with a single adapter manager
119+
</td>
120+
</tr>
121+
<tr>
122+
<td><a href="./07_end_to_end/earthquake.ipynb">Push-Pull Input Adapter for Earthquake Data</a></td>
123+
<td>
124+
Create a push-pull adapter which transitions from replay to live execution
125+
</td>
126+
</tr>
127+
<!-- Writing C++ Nodes and Adapters -->
128+
<tr>
129+
<td rowspan=2><a href="./05_cpp/">Writing C++ Nodes and Adapters</a></td>
130+
<td><a href="./05_cpp/1_cpp_node/">C++ Node</a></td>
131+
<td>
132+
Extend CSP with a pig latin C++ node
133+
</td>
134+
</tr>
135+
<tr>
136+
<td><a href="./05_cpp/2_cpp_node_with_struct/">C++ Node with <code>csp.Struct</code></a></td>
137+
<td>
138+
Write a C++ node with a csp.Struct input
139+
</td>
140+
</tr>
141+
<!-- Advanced -->
142+
<tr>
143+
<td rowspan=2><a href="./06_advanced/">Advanced</a></td>
144+
<td><a href="./06_advanced/e1_dynamic.py">Dynamic Graphs</a></td>
145+
<td>
146+
Update the shape of a graph at runtime
147+
</td>
148+
</tr>
149+
<tr>
150+
<td><a href="./06_advanced/e2_pandas_extension.py">Pandas Extension</a></td>
151+
<td>
152+
Use CSP within a pandas DataFrame
153+
</td>
154+
</tr>
155+
<!-- End-to-end examples -->
156+
<tr>
157+
<td rowspan=4><a href="./07_end_to_end/">End-to-end examples</a></td>
158+
<td><a href="./07_end_to_end/mta.ipynb">MTA Subway Data</a></td>
159+
<td>
160+
Access real-time New York City transit data
161+
</td>
162+
</tr>
163+
<tr>
164+
<td><a href="./07_end_to_end/seismic_waveform.ipynb">Seismic Data with obspy</a></td>
165+
<td>
166+
Analyze seismic waveforms and compare with batch processing methods
167+
</td>
168+
</tr>
169+
<tr>
170+
<td><a href="./07_end_to_end/wikimedia.ipynb">Wikipedia Updates and Edits</a></td>
171+
<td>
172+
Monitor live updates to all Wikimedia sites
173+
</td>
174+
</tr>
175+
<tr>
176+
<td><a href="./07_end_to_end/earthquake.ipynb">World Earthquake Dashboard</a></td>
177+
<td>
178+
Display recent earthquakes on a live-updating world map
179+
</td>
180+
</tr>
181+
<!-- Others -->
182+
<tr>
183+
<td><a href="./98_just_for_fun/">Just for fun!</a></td>
184+
<td><a href="./98_just_for_fun/e1_csp_nand_computer.py">NAND Computer</a></td>
185+
<td>
186+
Understand <code>csp.node</code> & <code>csp.graph</code> by connecting NAND logic gates
187+
</td>
188+
</tr>
189+
<tr>
190+
<td><a href="./99_developer_tools/">Developer Tools</a></td>
191+
<td><a href="./99_developer_tools/e1_profiling.py">Profiling <code>csp</code> code</a></td>
192+
<td>
193+
Profile a CSP Graph, view static attributes and runtime performance
194+
</td>
195+
</tr>
196+
</tbody>
197+
</table>

0 commit comments

Comments
 (0)