-
Notifications
You must be signed in to change notification settings - Fork 1
[Doc] Add reader docs #44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,55 @@ | ||||||
# Trace Reader | ||||||
|
||||||
We support a unified trace reader to open trace files in different format and read the requests. | ||||||
|
||||||
## Basic usage | ||||||
|
||||||
`TraceReader` class is the core of this functionality. When we create an instance of `TraceReader`, we open a trace file for read requests. | ||||||
|
||||||
`TraceReader` accepts three arguments: | ||||||
- `trace: str | TraceReader`: A trace path or other trace instance. The trace path can be a file path on your local machine (e.g., ~/data/trace.oracleGeneral.zst) or an S3 URI (e.g., s3://cache-datasets/cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst). | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The type hint for the
Suggested change
|
||||||
- `trace_type: TraceType` (optional): If not given, it will be infered according to the file name. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
- `reader_init_params: ReaderInitParam` (optional): If not given, will use default params for reader initialization. | ||||||
|
||||||
Here is an example to load one trace via an S3 URI. | ||||||
|
||||||
```python | ||||||
import libcachesim as lcs | ||||||
|
||||||
# Open a trace hosted on S3 (find more via https://github.com/cacheMon/cache_dataset) | ||||||
URI = "s3://cache-datasets/cache_dataset_oracleGeneral/2007_msr/msr_hm_0.oracleGeneral.zst" | ||||||
reader = lcs.TraceReader( | ||||||
trace = URI, | ||||||
trace_type = lcs.TraceType.ORACLE_GENERAL_TRACE, | ||||||
reader_init_params = lcs.ReaderInitParam(ignore_obj_size=False) | ||||||
) | ||||||
``` | ||||||
|
||||||
Then we can walk through the trace. | ||||||
|
||||||
```python | ||||||
for req in reader: | ||||||
print(req.obj_id, req.obj_size) | ||||||
``` | ||||||
|
||||||
## Reader slicing | ||||||
|
||||||
`TraceReader` support slicing and index access. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
|
||||||
```python | ||||||
# Read the first 100 reqs | ||||||
for req in reader[:100]: | ||||||
print(req.obj_id, req.obj_size) | ||||||
``` | ||||||
|
||||||
```python | ||||||
# Read 100 reqs after the first 100 reqs | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
for req in reader[100:200]: | ||||||
print(req.obj_id, req.obj_size) | ||||||
``` | ||||||
|
||||||
```python | ||||||
# Read last 100 reqs | ||||||
for req in reader[-100:]: | ||||||
print(req.obj_id, req.obj_size) | ||||||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a small grammatical error here. It should be 'formats' instead of 'format'.