Skip to content

Commit

Permalink
[CLEANUP]
Browse files Browse the repository at this point in the history
  • Loading branch information
Kye committed Apr 27, 2024
1 parent 1b64ef0 commit a59a39c
Show file tree
Hide file tree
Showing 54 changed files with 497 additions and 639 deletions.
6 changes: 3 additions & 3 deletions docs/diy_your_own_agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,11 +152,11 @@ The Agent class provides built-in support for long-term memory, allowing agents

```python

from swarms.memory import AbstractVectorDatabase
from swarms.memory import BaseVectorDatabase
from swarms import Agent


class CustomMemory(AbstractVectorDatabase):
class CustomMemory(BaseVectorDatabase):

    def __init__(self, *args, **kwargs):

Expand Down Expand Up @@ -196,7 +196,7 @@ class MyCustomAgent(Agent):

```

In the example above, we define a new `CustomMemory` class that inherits from the `AbstractVectorDatabase` class provided by the Agent class framework. Within the `CustomMemory` class, you can implement specialized memory management logic, such as custom indexing, retrieval, and storage mechanisms.
In the example above, we define a new `CustomMemory` class that inherits from the `BaseVectorDatabase` class provided by the Agent class framework. Within the `CustomMemory` class, you can implement specialized memory management logic, such as custom indexing, retrieval, and storage mechanisms.

Next, within the `MyCustomAgent` class, we initialize an instance of the `CustomMemory` class and assign it to the `self.long_term_memory` attribute. This custom memory instance can then be utilized within the overridden `run` method, where you can query the memory and process the results as needed.

Expand Down
4 changes: 2 additions & 2 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,10 +167,10 @@ nav:
- groupchatmanager: "swarms/structs/groupchatmanager.md"
- MajorityVoting: "swarms/structs/majorityvoting.md"
- swarms.memory:
- Building Custom Vector Memory Databases with the AbstractVectorDatabase Class: "swarms/memory/diy_memory.md"
- Building Custom Vector Memory Databases with the BaseVectorDatabase Class: "swarms/memory/diy_memory.md"
- ShortTermMemory: "swarms/memory/short_term_memory.md"
- Guides:
- Building Custom Vector Memory Databases with the AbstractVectorDatabase Class: "swarms/memory/diy_memory.md"
- Building Custom Vector Memory Databases with the BaseVectorDatabase Class: "swarms/memory/diy_memory.md"
- How to Create A Custom Language Model: "swarms/models/custom_model.md"
- Deploying Azure OpenAI in Production, A Comprehensive Guide: "swarms/models/azure_openai.md"
- DIY Build Your Own Agent: "diy_your_own_agent.md"
Expand Down
2 changes: 1 addition & 1 deletion docs/swarms/agents/toolagent.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The primary objective of the `ToolAgent` class is to amplify the efficiency of d
The `ToolAgent` class has the following definition:

```python
class ToolAgent(AbstractLLM):
class ToolAgent(BaseLLM):
def __init__(
self,
name: str,
Expand Down
4 changes: 2 additions & 2 deletions docs/swarms/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1073,9 +1073,9 @@ agent.run(task=task, img=img)

### Swarms Compliant Model Interface
```python
from swarms import AbstractLLM
from swarms import BaseLLM

class vLLMLM(AbstractLLM):
class vLLMLM(BaseLLM):
def __init__(self, model_name='default_model', tensor_parallel_size=1, *args, **kwargs):
super().__init__(*args, **kwargs)
self.model_name = model_name
Expand Down
44 changes: 22 additions & 22 deletions docs/swarms/memory/diy_memory.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# Building Custom Vector Memory Databases with the AbstractVectorDatabase Class
# Building Custom Vector Memory Databases with the BaseVectorDatabase Class

In the age of large language models (LLMs) and AI-powered applications, efficient memory management has become a crucial component. Vector databases, which store and retrieve data in high-dimensional vector spaces, have emerged as powerful tools for handling the vast amounts of data generated and consumed by AI systems. However, integrating vector databases into your applications can be a daunting task, requiring in-depth knowledge of their underlying architectures and APIs.

Enter the `AbstractVectorDatabase` class, a powerful abstraction layer designed to simplify the process of creating and integrating custom vector memory databases into your AI applications. By inheriting from this class, developers can build tailored vector database solutions that seamlessly integrate with their existing systems, enabling efficient storage, retrieval, and manipulation of high-dimensional data.
Enter the `BaseVectorDatabase` class, a powerful abstraction layer designed to simplify the process of creating and integrating custom vector memory databases into your AI applications. By inheriting from this class, developers can build tailored vector database solutions that seamlessly integrate with their existing systems, enabling efficient storage, retrieval, and manipulation of high-dimensional data.

In this comprehensive guide, we'll explore the `AbstractVectorDatabase` class in detail, covering its core functionality and diving deep into the process of creating custom vector memory databases using popular solutions like PostgreSQL, Pinecone, Chroma, FAISS, and more. Whether you're a seasoned AI developer or just starting to explore the world of vector databases, this guide will provide you with the knowledge and tools necessary to build robust, scalable, and efficient memory solutions for your AI applications.
In this comprehensive guide, we'll explore the `BaseVectorDatabase` class in detail, covering its core functionality and diving deep into the process of creating custom vector memory databases using popular solutions like PostgreSQL, Pinecone, Chroma, FAISS, and more. Whether you're a seasoned AI developer or just starting to explore the world of vector databases, this guide will provide you with the knowledge and tools necessary to build robust, scalable, and efficient memory solutions for your AI applications.

## Understanding the AbstractVectorDatabase Class
## Understanding the BaseVectorDatabase Class

Before we dive into the implementation details, let's take a closer look at the `AbstractVectorDatabase` class and its core functionality.
Before we dive into the implementation details, let's take a closer look at the `BaseVectorDatabase` class and its core functionality.

The `AbstractVectorDatabase` class is an abstract base class that defines the interface for interacting with a vector database. It serves as a blueprint for creating concrete implementations of vector databases, ensuring a consistent and standardized approach to database operations across different systems.
The `BaseVectorDatabase` class is an abstract base class that defines the interface for interacting with a vector database. It serves as a blueprint for creating concrete implementations of vector databases, ensuring a consistent and standardized approach to database operations across different systems.

The class provides a set of abstract methods that define the essential functionality required for working with vector databases, such as connecting to the database, executing queries, and performing CRUD (Create, Read, Update, Delete) operations.

Here's a breakdown of the abstract methods defined in the `AbstractVectorDatabase` class:
Here's a breakdown of the abstract methods defined in the `BaseVectorDatabase` class:

1\. `connect()`: This method establishes a connection to the vector database.

Expand All @@ -34,22 +34,22 @@ Here's a breakdown of the abstract methods defined in the `AbstractVectorDatabas

9\. `delete(message)`: This method deletes a record from the vector database.

By inheriting from the `AbstractVectorDatabase` class and implementing these abstract methods, developers can create concrete vector database implementations tailored to their specific needs and requirements.
By inheriting from the `BaseVectorDatabase` class and implementing these abstract methods, developers can create concrete vector database implementations tailored to their specific needs and requirements.

## Creating a Custom Vector Memory Database

Now that we have a solid understanding of the `AbstractVectorDatabase` class, let's dive into the process of creating a custom vector memory database by inheriting from this class. Throughout this guide, we'll explore various vector database solutions, including PostgreSQL, Pinecone, Chroma, FAISS, and more, showcasing how to integrate them seamlessly into your AI applications.
Now that we have a solid understanding of the `BaseVectorDatabase` class, let's dive into the process of creating a custom vector memory database by inheriting from this class. Throughout this guide, we'll explore various vector database solutions, including PostgreSQL, Pinecone, Chroma, FAISS, and more, showcasing how to integrate them seamlessly into your AI applications.

### Step 1: Inherit from the AbstractVectorDatabase Class
### Step 1: Inherit from the BaseVectorDatabase Class

The first step in creating a custom vector memory database is to inherit from the `AbstractVectorDatabase` class. This will provide your custom implementation with the foundational structure and interface defined by the abstract class.
The first step in creating a custom vector memory database is to inherit from the `BaseVectorDatabase` class. This will provide your custom implementation with the foundational structure and interface defined by the abstract class.

```python

from abc import ABC, abstractmethod
from swarms import AbstractVectorDatabase
from swarms import BaseVectorDatabase

class MyCustomVectorDatabase(AbstractVectorDatabase):
class MyCustomVectorDatabase(BaseVectorDatabase):

    def __init__(self, *args, **kwargs):

Expand All @@ -59,17 +59,17 @@ class MyCustomVectorDatabase(AbstractVectorDatabase):

```

In the example above, we define a new class `MyCustomVectorDatabase` that inherits from the `AbstractVectorDatabase` class. Within the `__init__` method, you can add any custom initialization logic specific to your vector database implementation.
In the example above, we define a new class `MyCustomVectorDatabase` that inherits from the `BaseVectorDatabase` class. Within the `__init__` method, you can add any custom initialization logic specific to your vector database implementation.

### Step 2: Implement the Abstract Methods

The next step is to implement the abstract methods defined in the `AbstractVectorDatabase` class. These methods provide the core functionality for interacting with your vector database, such as connecting, querying, and performing CRUD operations.
The next step is to implement the abstract methods defined in the `BaseVectorDatabase` class. These methods provide the core functionality for interacting with your vector database, such as connecting, querying, and performing CRUD operations.

```python
from swarms import AbstractVectorDatabase
from swarms import BaseVectorDatabase


class MyCustomVectorDatabase(AbstractVectorDatabase):
class MyCustomVectorDatabase(BaseVectorDatabase):

    def __init__(self, *args, **kwargs):

Expand Down Expand Up @@ -146,7 +146,7 @@ PostgreSQL is a powerful open-source relational database management system that
```python

import psycopg2
from swarms import AbstractVectorDatabase
from swarms import BaseVectorDatabase

class PostgreSQLVectorDatabase(MyCustomVectorDatabase):

Expand Down Expand Up @@ -209,7 +209,7 @@ Pinecone is a managed vector database service that provides efficient storage, r
```python

import pinecone
from swarms import AbstractVectorDatabase
from swarms import BaseVectorDatabase


class PineconeVectorDatabase(MyCustomVectorDatabase):
Expand Down Expand Up @@ -263,7 +263,7 @@ Chroma is an open-source vector database library that provides efficient storage
```python

from chromadb.client import Client
from swarms import AbstractVectorDatabase
from swarms import BaseVectorDatabase

class ChromaVectorDatabase(MyCustomVectorDatabase):

Expand Down Expand Up @@ -467,12 +467,12 @@ By following these best practices and considering potential challenges, you can

# Conclusion

In this comprehensive guide, we've explored the `AbstractVectorDatabase` class and its role in simplifying the process of creating custom vector memory databases. We've covered the core functionality of the class, walked through the step-by-step process of inheriting and extending its functionality, and provided examples of integrating popular vector database solutions like PostgreSQL, Pinecone, Chroma, and FAISS.
In this comprehensive guide, we've explored the `BaseVectorDatabase` class and its role in simplifying the process of creating custom vector memory databases. We've covered the core functionality of the class, walked through the step-by-step process of inheriting and extending its functionality, and provided examples of integrating popular vector database solutions like PostgreSQL, Pinecone, Chroma, and FAISS.

Building custom vector memory databases empowers developers to create tailored and efficient data management solutions that seamlessly integrate with their AI applications. By leveraging the power of vector databases, you can unlock new possibilities in data storage, retrieval, and manipulation, enabling your AI systems to handle vast amounts of high-dimensional data with ease.

Remember, the journey of building custom vector memory databases is an iterative and collaborative process that requires continuous learning, adaptation, and refinement. Embrace the challenges, stay up-to-date with the latest developments in vector databases and AI, and continuously strive to optimize and enhance your implementations.

As you embark on this journey, keep in mind the importance of scalability, performance, data quality, security, and compliance. Foster an environment of collaboration, knowledge sharing, and community engagement to ensure that your custom vector memory databases are robust, reliable, and capable of meeting the ever-evolving demands of the AI landscape.

So, dive in, leverage the power of the `AbstractVectorDatabase` class, and create the custom vector memory databases that will drive the future of AI-powered applications.
So, dive in, leverage the power of the `BaseVectorDatabase` class, and create the custom vector memory databases that will drive the future of AI-powered applications.
22 changes: 11 additions & 11 deletions docs/swarms/models/base_llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@

## 1. Introduction <a name="introduction"></a>

The Language Model Interface (`AbstractLLM`) is a flexible and extensible framework for working with various language models. This documentation provides a comprehensive guide to the interface, its attributes, methods, and usage examples. Whether you're using a pre-trained language model or building your own, this interface can help streamline the process of text generation, chatbots, summarization, and more.
The Language Model Interface (`BaseLLM`) is a flexible and extensible framework for working with various language models. This documentation provides a comprehensive guide to the interface, its attributes, methods, and usage examples. Whether you're using a pre-trained language model or building your own, this interface can help streamline the process of text generation, chatbots, summarization, and more.

## 2. Abstract Language Model <a name="abstract-language-model"></a>

### Initialization <a name="initialization"></a>

The `AbstractLLM` class provides a common interface for language models. It can be initialized with various parameters to customize model behavior. Here are the initialization parameters:
The `BaseLLM` class provides a common interface for language models. It can be initialized with various parameters to customize model behavior. Here are the initialization parameters:

| Parameter | Description | Default Value |
|------------------------|-------------------------------------------------------------------------------------------------|---------------|
Expand Down Expand Up @@ -82,7 +82,7 @@ The `AbstractLLM` class provides a common interface for language models. It can

### Methods <a name="methods"></a>

The `AbstractLLM` class defines several methods for working with language models:
The `BaseLLM` class defines several methods for working with language models:

- `run(task: Optional[str] = None, *args, **kwargs) -> str`: Generate text using the language model. This method is abstract and must be implemented by subclasses.

Expand Down Expand Up @@ -156,18 +156,18 @@ get_generation_time() -> float`: Get the time taken for text generation.

## 3. Implementation <a name="implementation"></a>

The `AbstractLLM` class serves as the base for implementing specific language models. Subclasses of `AbstractLLM` should implement the `run` method to define how text is generated for a given task. This design allows flexibility in integrating different language models while maintaining a common interface.
The `BaseLLM` class serves as the base for implementing specific language models. Subclasses of `BaseLLM` should implement the `run` method to define how text is generated for a given task. This design allows flexibility in integrating different language models while maintaining a common interface.

## 4. Usage Examples <a name="usage-examples"></a>

To demonstrate how to use the `AbstractLLM` interface, let's create an example using a hypothetical language model. We'll initialize an instance of the model and generate text for a simple task.
To demonstrate how to use the `BaseLLM` interface, let's create an example using a hypothetical language model. We'll initialize an instance of the model and generate text for a simple task.

```python
# Import the AbstractLLM class
from swarms.models import AbstractLLM
# Import the BaseLLM class
from swarms.models import BaseLLM

# Create an instance of the language model
language_model = AbstractLLM(
language_model = BaseLLM(
model_name="my_language_model",
max_tokens=50,
temperature=0.7,
Expand All @@ -188,7 +188,7 @@ In this example, we've created an instance of our hypothetical language model, c

## 5. Additional Features <a name="additional-features"></a>

The `AbstractLLM` interface provides additional features for customization and control:
The `BaseLLM` interface provides additional features for customization and control:

- `batch_run`: Generate text for a batch of tasks efficiently.
- `arun` and `abatch_run`: Asynchronous versions of `run` and `batch_run` for concurrent text generation.
Expand All @@ -199,7 +199,7 @@ These features enhance the flexibility and utility of the interface in various a

## 6. Performance Metrics <a name="performance-metrics"></a>

The `AbstractLLM` class offers methods for tracking performance metrics:
The `BaseLLM` class offers methods for tracking performance metrics:

- `_tokens_per_second`: Calculate tokens generated per second.
- `_num_tokens`: Calculate the number of tokens in a text.
Expand All @@ -224,4 +224,4 @@ The `track_resource_utilization` method is a placeholder for tracking and report

## 9. Conclusion <a name="conclusion"></a>

The Language Model Interface (`AbstractLLM`) is a versatile framework for working with language models. Whether you're using pre-trained models or developing your own, this interface provides a consistent and extensible foundation. By following the provided guidelines and examples, you can integrate and customize language models for various natural language processing tasks.
The Language Model Interface (`BaseLLM`) is a versatile framework for working with language models. Whether you're using pre-trained models or developing your own, this interface provides a consistent and extensible foundation. By following the provided guidelines and examples, you can integrate and customize language models for various natural language processing tasks.
Loading

0 comments on commit a59a39c

Please sign in to comment.