Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating docs & nb #1128

Merged
merged 15 commits into from
Oct 9, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/exceptions.rst
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ Help debugging

If you would like help debugging an error that you are encountering, please feel free to share your code, objects and exceptions report with us.

An easy way to do this is to post a notebook with your code to the :ref:`coop` and share the link with us at info@expectedparrot.com.
An easy way to do this is to post a notebook with your code to the :ref:`coop` and **share the link with us at info@expectedparrot.com**.
You can use the following code to generate a link to your notebook:

.. code-block:: python
@@ -36,7 +36,7 @@ For example, you may intend for the answer to be formatted as a list but receive
Or a question may be unanswered and the model has returned `None`.
These exceptions are typically raised by the `Question` class and are subclassed from `QuestionAnswerValidationError`.

A useful starting point for debugging these exceptions is to check the `Settings` class for the `Questions` model.
A useful starting point for debugging these exceptions is to check the `Settings` class for the `Questions` model (https://github.com/expectedparrot/edsl/blob/main/edsl/questions/settings.py).
The default settings (which can be modified) are as follows:

.. code-block:: python
54 changes: 30 additions & 24 deletions docs/filestore.rst
Original file line number Diff line number Diff line change
@@ -3,27 +3,30 @@
File Store
==========

`FileStore` is a module for storing and sharing data on the Coop that you want to use in your EDSL projects, such as survey data, PDFs, CSVs or images.
It can be particularly useful for storing files to be used as as `Scenario` objects with a survey, and allows you to include code for retrieving and processing the files in your EDSL project.
`FileStore` is a module for storing and sharing data on the Coop to use in EDSL projects, such as survey data, PDFs, CSVs, docs or images.
It can be particularly useful for storing data intended to be used with surveys as `Scenario` objects, such as in data labeling tasks,
and allows you to include code for retrieving and processing the data files in your EDSL project to facilitate collaboration and replication of results.


File types
----------

The following file types are supported by the FileStore:
The following file types are currently supported by the FileStore:

- CSV
- PDF
- PNG
* CSV
* PDF
* PNG (image)


Posting a file
--------------

To store a file, you need to create a `FileStore` object with the path to the file you want to store.
You can then use the `push` method to store the file on the Coop and get a URL for accessing the file.
To post a file, import the `FileStore` type (`CSVFileStore`, `PDFFileStore` or `PNGFileStore`) and create an object with the path to the file.
Then call the `push` method to store the file on the Coop and get a URL and uuid for accessing it.
You can optionally pass a `description` and `visibility` parameter to the `push` method (Coop objects can be *public*, *private* or *unlisted* by default).

CSV example:
CSV example
^^^^^^^^^^^

.. code-block:: python

@@ -34,7 +37,7 @@ CSV example:
print(info) # display the URL and Coop uuid of the stored file for retrieving it later


Example output:
Example output (showing the default description and visibility setting):

.. code-block:: python

@@ -46,7 +49,8 @@ Example output:
'visibility': 'unlisted'}


PDF example:
PDF example
^^^^^^^^^^^

.. code-block:: python

@@ -69,7 +73,8 @@ Example output:
'visibility': 'unlisted'}


PNG example:
PNG example
^^^^^^^^^^^

.. code-block:: python

@@ -95,22 +100,22 @@ Example output:
Retrieving and using a file
---------------------------

To retrieve a file, you need to create a `FileStore` object with the Coop uuid of the file you want to retrieve.
You can then use the `pull` method to retrieve the file from the Coop.
To retrieve a file, create a `FileStore` object (`CSVFileStore`, `PDFFileStore` or `PNGFileStore`)
and pass it the Coop uuid of the file you want to retrieve and the Expected Parrot URL.
Then call the `pull` method to retrieve the file from the Coop.

Once you have retrieved a file, you can use it in your EDSL project as a `Scenario` object.
Each file type has a method for converting the file into a scenario:
Once retrieved, a file can be converted into scenarios by calling the relevant method on a `ScenarioList` object:

- `from_csv` for CSV files
- `from_pdf` for PDF files
- `from_image` for PNG files
* `ScenarioList.from_csv()` for CSV files
* `ScenarioList.from_pdf()` for PDF files
* `ScenarioList.from_image()` for PNG files


CSV example
^^^^^^^^^^^

For CSV files we use the `from_csv` method of the `ScenarioList` class.
They keys are the column names of the CSV file, which can be modified with the `rename` method.
Here we retrieve the CSV file posted above and then convert it into a `ScenarioList` object with the `from_csv()` method.
The keys are the column names of the CSV file, which can be modified with the `rename` method.

.. code-block:: python

@@ -125,7 +130,7 @@ They keys are the column names of the CSV file, which can be modified with the `
PDF example
^^^^^^^^^^^

For PDF files we use the `from_pdf` method of the `ScenarioList` class.
Here we retrieve the PDF file posted above and then convert it into a `ScenarioList` object with the `from_pdf()` method.
The default keys are `filename`, `page`, `text`, which can be modified with the `rename` method.

.. code-block:: python
@@ -155,7 +160,8 @@ Output:
PNG example
^^^^^^^^^^^

For PNG files we use the `from_image` method which takes the PNG file and (optionally) the name of a key to use for the scenario object.
Here we retrieve the PNG file posted above and then convert it into a `ScenarioList` object with the `from_image()` method.
We can optionally pass the name of a key to use for the scenario object, or edit the key later.

.. code-block:: python

@@ -170,7 +176,7 @@ For PNG files we use the `from_image` method which takes the PNG file and (optio
Working with scenarios
----------------------

Before using the scenario, verify the key and value of the scenario object (e.g., by printing), and rename the key as desired to use in survey questions.
Before using the scenario, we can verify the key and value of the scenario object (e.g., by printing), and rename the key as desired to use in survey questions.

For a single `Scenario` we can check the key:

20 changes: 16 additions & 4 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -8,18 +8,27 @@ EDSL: AI-Powered Research
EDSL is developed by `Expected Parrot <https://www.expectedparrot.com>`_ and available under the MIT License.

This page provides documentation, tutorials and demo notebooks for the EDSL package and the `Coop <https://www.expectedparrot.com/explore>`_: a platform for creating, storing and sharing AI research.
The contents are organized into key sections to help you get started:
The contents are organized into key sections to help you get started.


Researchers
-----------

**Are you using EDSL for a research project? We'd love to hear about your experience!**

Send us an email at info@expectedparrot.com and we'll provide credits to run your project or a gift card for your time.


Links
-----

- Download the current version of EDSL at `PyPI <https://pypi.org/project/edsl>`_.
- Get the latest EDSL updates at `GitHub <https://github.com/expectedparrot/edsl>`_.
- Create a `Coop account <https://www.expectedparrot.com/login>`_ to store and share your research, and access special features:
- Create a `Coop account <https://www.expectedparrot.com/login>`_ to store and share your research and access special features, including:

* :ref:`survey_builder`: An interface for launching hybrid human-AI surveys
* :ref:`remote_inference` and :ref:`remote_caching`: Work with EDSL on the Expected Parrot server
* :ref:`remote_inference`: Run surveys on the Expected Parrot server
* :ref:`remote_caching`: Automatically store results and API calls on the Expected Parrot server
* :ref:`filestore`: Store and share files for use in EDSL projects

- Explore research at the `Coop <https://www.expectedparrot.com/explore>`_.
@@ -33,14 +42,15 @@ Introduction
- :ref:`overview`: An overview of the purpose, concepts and goals of the EDSL package.
- :ref:`whitepaper`: A whitepaper about the EDSL package (*in progress*).
- :ref:`citation`: How to cite the package in your work.
- :ref:`papers`: Research papers and articles that use EDSL.


Technical Setup
---------------

- :ref:`installation`: Instructions for installing the EDSL package.
- :ref:`coop`: Create, store and share research on the Expected Parrot server.
- :ref:`api_keys`: Instructions for storing API keys to use EDSL locally.
- :ref:`api_keys`: Instructions for storing API keys to use EDSL locally (*optional*).


Getting Started
@@ -125,7 +135,9 @@ Information about additional functionality for developers.
:hidden:

overview
whitepaper
citation
papers

.. toctree::
:maxdepth: 2
Loading