Skip to content

Commit 1154e57

Browse files
authored
Merge pull request #11 from Siege-Software/add-docstrings-and-update-readme
Add docstrings and update readme
2 parents 30333e0 + d821f89 commit 1154e57

File tree

7 files changed

+248
-205
lines changed

7 files changed

+248
-205
lines changed

CONTRIBUTING.md

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
## Contribution
2+
django-typesense’s git main branch should always be stable, production-ready & passing all tests.
3+
4+
### Setting up the project
5+
```
6+
# clone the repo
7+
git clone https://gitlab.com/siege-software/packages/django_typesense.git
8+
git checkout -b <your_branch_name> stable/1.x.x
9+
10+
# Set up virtual environment
11+
python3.8 -m venv venv
12+
source venv/bin/activate
13+
14+
pip install -r requirements-dev.txt
15+
16+
# Enable automatic pre-commit hooks
17+
pre-commit install
18+
```
19+
20+
### Running Tests
21+
```
22+
cd tests
23+
pytest .
24+
```
25+
26+
### Building the package
27+
`python -m build`
28+
29+
### Installing the package from build
30+
` pip install path/to/django_typesense-0.0.1.tar.gz`

README.md

+125-138
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
# django typesense
2+
3+
[![Build](https://github.com/Siege-Software/django-typesense/workflows/build/badge.svg?branch=main)](https://github.com/Siege-Software/django-typesense/actions?workflow=CI)
24
[![codecov](https://codecov.io/gh/Siege-Software/django-typesense/branch/main/graph/badge.svg?token=S4W0E84821)](https://codecov.io/gh/Siege-Software/django-typesense)
35
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
46
![PyPI download month](https://img.shields.io/pypi/dm/django-typesense.svg)
57
[![PyPI version](https://badge.fury.io/py/django-typesense.svg)](https://pypi.python.org/pypi/django-typesense/)
68
![Python versions](https://img.shields.io/badge/python-%3E%3D3.8-brightgreen)
7-
![Django Versions](https://img.shields.io/badge/django-%3E%3D4-brightgreen)
9+
![Django Versions](https://img.shields.io/badge/django-%3E%3D3.2-brightgreen)
10+
[![PyPI License](https://img.shields.io/pypi/l/django-typesense.svg)](https://pypi.python.org/pypi/django-typesense/)
811

912

1013
> [!WARNING]
@@ -13,140 +16,158 @@
1316
## What is it?
1417
Faster Django Admin powered by [Typesense](https://typesense.org/)
1518

16-
## TODOs
17-
- Performance comparison stats
1819

19-
## Note on ForeignKeys and OneToOneFields
20-
- While data from foreign keys can be indexed, displaying them on the admin will trigger database queries that will negatively affect performance.
21-
- We recommend indexing the string representation of the foreignkey as a model property to enable display on admin.
2220

23-
## How to use
21+
## Quick Start Guide
22+
### Installation
2423
`pip install django-typesense`
2524

26-
Install directly from github to test the most recent version
27-
```
28-
pip install git+https://github.com/SiegeSoftware/django-typesense.git
29-
```
25+
or install directly from github to test the most recent version
26+
27+
`pip install git+https://github.com/SiegeSoftware/django-typesense.git`
3028

3129
Add `django_typesense` to the list of installed apps.
32-
You will need to set up the typesense server on your machine.
3330

34-
### Update the model to inherit from the Typesense model mixin
31+
Follow this [guide](https://typesense.org/docs/guide/install-typesense.html#option-1-typesense-cloud) to install and run typesense
32+
33+
### Create Collections
34+
Throughout this guide, we’ll refer to the following models, which comprise a song catalogue application:
35+
36+
```
37+
from django.db import models
38+
39+
40+
class Genre(models.Model):
41+
name = models.CharField(max_length=100)
42+
43+
def __str__(self):
44+
return self.name
45+
46+
47+
class Artist(models.Model):
48+
name = models.CharField(max_length=200)
49+
50+
def __str__(self):
51+
return self.name
3552
53+
54+
class Song(models.Model):
55+
title = models.CharField(max_length=100)
56+
genre = models.ForeignKey(Genre, on_delete=models.CASCADE)
57+
release_date = models.DateField(blank=True, null=True)
58+
artists = models.ManyToManyField(Artist)
59+
number_of_comments = models.IntegerField(default=0)
60+
number_of_views = models.IntegerField(default=0)
61+
duration = models.DurationField()
62+
description = models.TextField()
63+
64+
def __str__(self):
65+
return self.title
66+
67+
@property
68+
def release_date_timestamp(self):
69+
# read https://typesense.org/docs/0.25.0/api/collections.html#indexing-dates
70+
return self.release_date.timestamp() if self.release_date else self.release_date
71+
72+
def artist_names(self):
73+
return list(self.artists.all().values_list('name', flat=True))
74+
3675
```
37-
from django_typesense.models import TypesenseModelMixin, TypesenseQuerySet
38-
39-
class MyModelManager(models.Manager):
40-
"""Manager for class :class:`.models.MyModelName`
41-
"""
42-
field1 = models...
43-
field2 = models...
44-
field3 = models...
45-
date_created = models...
76+
77+
For such an application, you might be interested in improving the search and load times on the song records list view.
78+
79+
```
80+
from django_typesense.collections import TypesenseCollection
81+
from django_typesense import fields
82+
83+
84+
class SongCollection(TypesenseCollection):
85+
# At least one of the indexed fields has to be provided as one of the `query_by_fields`. Must be a CharField
86+
query_by_fields = 'title,artist_names'
4687
47-
typesense_fields = [
48-
{
49-
"name": "field1", "type": "string",
50-
},
51-
{
52-
"name": "field2", "type": "int64"
53-
},
54-
{
55-
"name": "field3", "type": "string[]"
56-
},
57-
{
58-
"name": "date_created", "type": "int64"
59-
}
60-
]
61-
62-
typesense_default_sorting_field = 'date_created'
63-
query_by_fields = ','.join(
64-
[
65-
'field1', 'field2', 'date_created'
66-
]
67-
)
68-
69-
def get_typesense_dict(self):
70-
"""
71-
Create a data structure that can be serialized as JSON for Typesense fields.
72-
73-
Normalize the structure if required.
74-
75-
Returns:
76-
dict: JSON-serializable data structure
77-
"""
78-
79-
typesense_dict = {
80-
'id': str(self.id),
81-
'field1': self.field1,
82-
'field2': self.field2,
83-
'field3': self.field3,
84-
'date_created': self.date_created.timestamp()
85-
}
86-
87-
return typesense_dict
88-
89-
def get_queryset(self):
90-
"""
91-
Get an optimized queryset.
92-
93-
Returns:
94-
django.db.models.query.QuerySet: Queryset with instances of \
95-
:class:`.models.Work`
96-
"""
97-
return TypesenseQuerySet(
98-
self.model, using=self._db
99-
)
88+
title = fields.TypesenseCharField()
89+
genre_name = fields.TypesenseCharField(value='genre.name')
90+
genre_id = fields.TypesenseSmallIntegerField()
91+
release_date = fields.TypesenseDateField(value='release_date_timestamp', optional=True)
92+
artist_names = fields.TypesenseArrayField(base_field=fields.TypesenseCharField(), value='artist_names')
93+
number_of_comments = fields.SmallIntegerField(index=False, optional=True)
94+
number_of_views = fields.SmallIntegerField(index=False, optional=True)
95+
duration = fields.DurationField()
96+
```
10097

98+
It's okay to store fields that you don't intend to search but to display on the admin. Such fields should be marked as un-indexed e.g:
10199

102-
class MyModelName(TypesenseModelMixin)
100+
number_of_views = fields.SmallIntegerField(index=False, optional=True)
101+
102+
Update the song model as follows:
103+
```
104+
class Song(models.Model):
105+
...
106+
collection_class = SongCollection
103107
...
104-
105-
objects = MyModelManager()
106108
```
107109

108-
`TypesenseQuerySet` is required to automatically index model changes on create, update and delete
110+
How the value of a field is retrieved from a model instance:
111+
1. The collection field name is called as a property of the model instance
112+
2. If `value` is provided, it will be called as a property or method of the model instance
113+
114+
Where the collections live is totally dependent on you but we recommend having a `collections.py` file in the django app where the model you are creating a collection for is.
109115

110-
### Admin Setup
111-
To update a model admin to display and search from the model Typesense collection, the admin class should inherit from the TypesenseSearchAdminMixin
116+
> [!NOTE]
117+
> We recommend displaying data from ForeignKey or OneToOne fields as string attributes using the display decorator to avoid triggering database queries that will negatively affect performance.
118+
119+
### Admin Integration
120+
To make a model admin display and search from the model's Typesense collection, the admin class should inherit `TypesenseSearchAdminMixin`
112121

113122
```
114123
from django_typesense.admin import TypesenseSearchAdminMixin
115124
116-
class MyModelAdmin(TypesenseSearchAdminMixin):
117-
pass
125+
@admin.register(Song)
126+
class SongAdmin(TypesenseSearchAdminMixin):
127+
...
128+
list_display = ['title', 'genre_name', 'release_date', 'number_of_views', 'duration']
129+
130+
@admin.display(description='Genre')
131+
def genre_name(self, obj):
132+
return obj.genre.name
133+
...
118134
119135
```
120136

121-
### Bulk indexing typesense collections
122-
To update or delete collection documents in bulk. Bulk updating is multi-threaded.
123-
You might encounter poor performance when indexing large querysets. Suggestions on how to improve are welcome.
137+
### Indexing
138+
For the initial setup, you will need to index in bulk. Bulk updating is multi-threaded. Depending on your system specs, you should set the `batch_size` keyword argument.
124139

125140
```
126-
from django_typesense.methods import bulk_delete_typsense_records, bulk_update_typsense_records
127-
from .models import MyModel
128-
from django_typesense.typesense_client import client
141+
from django_typesense.utils import bulk_delete_typsense_records, bulk_update_typsense_records
129142
130-
model_qs = Model.objects.all().order_by('date_created') # querysets should be ordered
131-
bulk_update_typesense_records(model_qs) # for bulk document indexing
132-
bulk_delete_typsense_records(model_qs) # for bulk document deletiom
143+
model_qs = Song.objects.all().order_by('id') # querysets should be ordered
144+
bulk_update_typesense_records(model_qs, batch_size=1024)
133145
```
134146

135147
# Custom Admin Filters
136148
To make use of custom admin filters, define a `filter_by` property in the filter definition.
137-
Define boolean typesense field `has_alien` that gets it's value from a model property.
149+
Define boolean typesense field `has_views` that gets it's value from a model property. This is example is not necessarily practical but for demo purposes.
138150

139151
```
140-
@property
141-
def has_alien(self):
142-
# moon_aliens and mars_aliens are reverse foreign keys
143-
return self.moon_aliens.exists() or self.mars_aliens.exists()
152+
# models.py
153+
class Song(models.Model):
154+
...
155+
@property
156+
def has_views(self):
157+
return self.number_of_views > 0
158+
...
159+
160+
# collections.py
161+
class SongCollection(TypesenseCollection):
162+
...
163+
has_views = fields.TypesenseBooleanField()
164+
...
144165
```
145166

146167
```
147-
class HasAlienFilter(admin.SimpleListFilter):
148-
title = _('Has Alien')
149-
parameter_name = 'has_alien'
168+
class HasViewsFilter(admin.SimpleListFilter):
169+
title = _('Has Views')
170+
parameter_name = 'has_views'
150171
151172
def lookups(self, request, model_admin):
152173
return (
@@ -158,54 +179,20 @@ class HasAlienFilter(admin.SimpleListFilter):
158179
def queryset(self, request, queryset):
159180
# This is used by the default django admin
160181
if self.value() == 'True':
161-
return queryset.filter(Q(mars_aliens__isnull=False) | Q(moon_aliens__isnull=False))
182+
return queryset.filter(number_of_views__gt=0)
162183
elif self.value() == 'False':
163-
return queryset.filter(mars_aliens__isnull=True, moon_aliens__isnull=True)
184+
return queryset.filter(number_of_views=0)
164185
165186
return queryset
166187
167188
@property
168189
def filter_by(self):
169190
# This is used by typesense
170191
if self.value() == 'True':
171-
return {"has_alien": "=true"}
192+
return {"has_views": "=true"}
172193
elif self.value() == 'False':
173-
return {"has_alien": "!=false"}
194+
return {"has_views": "!=false"}
174195
175196
return {}
176197
```
177198

178-
179-
## Release Process
180-
Each release has its own branch, called stable/version_number and any changes will be issued from those branches.
181-
The main branch has the latest stable version
182-
183-
## Contribution
184-
TBA
185-
186-
```
187-
# clone the repo
188-
git clone https://gitlab.com/siege-software/packages/django_typesense.git
189-
git checkout -b <your_branch_name> stable/1.x.x
190-
191-
# Set up virtual environment
192-
python3.8 -m venv venv
193-
source venv/bin/activate
194-
195-
pip install -r requirements-dev.txt
196-
197-
# Enable automatic pre-commit hooks
198-
pre-commit install
199-
```
200-
201-
## Running Tests
202-
```
203-
cd tests
204-
pytest .
205-
```
206-
207-
## Building the package
208-
```python -m build```
209-
210-
## Installing the package from build
211-
``` pip install path/to/django_typesense-0.0.1.tar.gz```

django_typesense/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.1.1-alpha"
1+
__version__ = "0.1.1"

0 commit comments

Comments
 (0)