Skip to content
This repository has been archived by the owner on Aug 13, 2024. It is now read-only.

Commit

Permalink
Rename cil to code-indexer-loop
Browse files Browse the repository at this point in the history
  • Loading branch information
ricklamers committed Sep 7, 2023
1 parent 6bb55ac commit 853304c
Show file tree
Hide file tree
Showing 11 changed files with 25 additions and 22 deletions.
9 changes: 5 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
all: clean build publish

build:
python setup.py sdist bdist_wheel
flit build

clean:
rm -rf build dist my_project.egg-info
rm -rf dist
rm -rf build

publish: build
twine upload dist/*
flit publish

.PHONY: build clean publish all
.PHONY: build clean publish all
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
# CIL: Code Indexer Loop
# Code Indexer Loop

[![PyPI version](https://badge.fury.io/py/cil.svg)](https://pypi.org/project/cil/)
[![License](https://img.shields.io/github/license/definitive-io/cil)](LICENSE)
[![Forks](https://img.shields.io/github/forks/definitive-io/cil)](https://github.com/definitive-io/cil/network)
[![Stars](https://img.shields.io/github/stars/definitive-io/cil)](https://github.com/definitive-io/cil/stargazers)
[![PyPI version](https://badge.fury.io/py/code-indexer-loop.svg)](https://pypi.org/project/code-indexer-loop/)
[![License](https://img.shields.io/github/license/definitive-io/code-indexer-loop)](LICENSE)
[![Forks](https://img.shields.io/github/forks/definitive-io/code-indexer-loop)](https://github.com/definitive-io/code-indexer-loop/network)
[![Stars](https://img.shields.io/github/stars/definitive-io/code-indexer-loop)](https://github.com/definitive-io/code-indexer-loop/stargazers)
[![Twitter](https://img.shields.io/twitter/url/https/twitter.com?style=social&label=Follow%20%40DefinitiveIO)](https://twitter.com/definitiveio)
[![Discord](https://dcbadge.vercel.app/api/server/CPJJfq87Vx?compact=true&style=flat)](https://discord.gg/CPJJfq87Vx)


**CIL** is a Python library designed to index and retrieve code snippets.
**Code Indexer Loop** is a Python library designed to index and retrieve code snippets.

It uses the useful indexing utilities of the **LlamaIndex** library and the multi-language **tree-sitter** library to parse the code from many popular programming languages. **tiktoken** is used to count tokens in a code snippet and **LangChain** to obtain embeddings (defaults to **OpenAI**'s `text-embedding-ada-002`) and store them in an embedded **ChromaDB** vector database. It uses **watchdog** for continuous updating of the index based on file system events.

## Installation:
Use `pip` to install Code Indexer Loop from PyPI.
```
pip install cil
pip install code_indexer_loop
```

## Usage:
1. Import necessary modules:
```python
from cil.api import CodeIndexer
from code_indexer_loop.api import CodeIndexer
```
2. Create a CodeIndexer object and have it watch for changes:
```python
Expand Down Expand Up @@ -76,4 +76,4 @@ Run the unit tests by invoking `pytest` in the root.
Please see the LICENSE file provided with the source code.

## Attribution
We'd like to thank the Sweep AI for publishing their ideas about code chunking. Read their blog posts about the topic [here](https://docs.sweep.dev/blogs/chunking-2m-files) and [here](https://docs.sweep.dev/blogs/chunking-improvements). The implementation in `cil` is modified from their original implementation mainly to limit based on tokens instead of characters and to achieve perfect document reconstruction (`"".join(chunks) == original_source_code`).
We'd like to thank the Sweep AI for publishing their ideas about code chunking. Read their blog posts about the topic [here](https://docs.sweep.dev/blogs/chunking-2m-files) and [here](https://docs.sweep.dev/blogs/chunking-improvements). The implementation in `code_indexer_loop` is modified from their original implementation mainly to limit based on tokens instead of characters and to achieve perfect document reconstruction (`"".join(chunks) == original_source_code`).
File renamed without changes.
6 changes: 3 additions & 3 deletions cil/api.py → code_indexer_loop/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@
from watchdog.events import FileSystemEventHandler
from watchdog.observers import Observer

from cil.code_splitter import CodeSplitter
from cil.constants import EXTENSION_TO_TREE_SITTER_LANGUAGE
from cil.utils import hash_md5
from code_indexer_loop.code_splitter import CodeSplitter
from code_indexer_loop.constants import EXTENSION_TO_TREE_SITTER_LANGUAGE
from code_indexer_loop.utils import hash_md5


class CodeIndexer:
Expand Down
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion cil/test_api.py → code_indexer_loop/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import pytest

from cil.code_splitter import (CodeSplitter, MaxChunkLengthExceededError,
from code_indexer_loop.code_splitter import (CodeSplitter, MaxChunkLengthExceededError,
TokenCounter)

THIS_FILE_DIR = os.path.dirname(os.path.realpath(__file__))
Expand Down
File renamed without changes.
12 changes: 7 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
[build-system]
build-backend = "flit_core.buildapi"
requires = ["flit_core >=3.8.0,<4"]

[project]
name = "cil"
description = "CIL: Code Indexer Loop"
name = "code-indexer-loop"
description = "Code Indexer Loop"
authors = [
{name = "Rick Lamers", email = "[email protected]"}
]
Expand All @@ -27,9 +31,7 @@ dev = [
"autoflake ~=2.2.0",
"ruff ~=0.0.284",
"pytest ~=7.4.1",
"twine ~=4.0.2",
"setuptools",
"wheel",
"flit >=3.8.0,<4",
]
test = [
"pytest-cov ~=3.0.0",
Expand Down

0 comments on commit 853304c

Please sign in to comment.