Skip to content

Commit 78025b7

Browse files
committed
README: maintenance and minor fixes
* Move docker installation section to the top * Remove outdated information about /tmp/ logs * Update manual installation to work with current Elixir * Update listed requirements * Update server information * Some formatting fixes * Update copyright date
1 parent 5a225c5 commit 78025b7

File tree

1 file changed

+84
-84
lines changed

1 file changed

+84
-84
lines changed

README.adoc

+84-84
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@ toc::[]
2222

2323
= Requirements
2424

25-
* Python >= 3.8
25+
* Python >= 3.9
2626
* Git >= 1.9
27-
* The Jinja2 and Pygments (>= 2.7) Python libraries
27+
* The Jinja2 and Pygments Python libraries
2828
* Berkeley DB (and its Python binding)
2929
* Universal Ctags
3030
* Perl (for non-greedy regexes and automated testing)
@@ -43,13 +43,63 @@ queries, respectively.
4343
When installing the system, you should test each layer manually and make
4444
sure it works correctly before moving on to the next one.
4545

46-
= Manual Installation
46+
= Recommended setup: Docker
47+
48+
Dockerfiles are provided in the `docker/` directory.
49+
To build the image, run the following commands:
50+
51+
# git clone https://github.com/bootlin/elixir.git ./elixir
52+
# docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
53+
54+
ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
55+
the option is used to pass a version string to Elixir.
56+
57+
You can then run the image using `docker run`.
58+
Here we mount a host directory as Elixir data:
59+
60+
# mkdir ./elixir-data
61+
# docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
62+
63+
The Docker image does not contain any repositories.
64+
To index a repository, you can use the `index-repository` script.
65+
For example, to add the https://musl.libc.org/[musl] repository, run:
66+
67+
# docker exec -it elixir-container index /srv/elixir-data musl
68+
69+
Or, to run indexing in a separate container:
70+
71+
# docker run -v ./elixir-data/:/srv/elixir-data \
72+
--entrypoint index elixir /srv/elixir-data musl
73+
74+
You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
75+
76+
After indexing is done, Elixir should be available under the following URL on your host:
77+
http://172.17.0.2/musl/latest/source
78+
79+
If 172.17.0.2 does not answer, you can check the IP address of the container by running:
80+
81+
# docker inspect elixir-container | grep IPAddress
82+
83+
== Automatic repository updates with Docker
84+
85+
The Docker image does not automatically update repositories by itself.
86+
You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
87+
from cron on the host to periodically update repositories.
88+
89+
== Using Docker image as a development server
90+
91+
You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
92+
into `/usr/local/elixir/` in the container when running `docker run elixir`.
93+
94+
Changes in the code made on the host should be automatically reflected in the container.
95+
You can use `apache2ctl` to restart Apache.
96+
Error logs are available in `/var/log/apache2/error.log` within the container.
97+
98+
= Manual Installation (without Docker)
4799

48100
== Install Dependencies
49101

50-
____
51-
For Debian
52-
____
102+
For Debian:
53103

54104
----
55105
sudo apt install python3-pip python3-venv libdb-dev python3-dev build-essential universal-ctags perl git apache2 libapache2-mod-wsgi-py3 libjansson4
@@ -134,23 +184,24 @@ cd /usr/local/elixir/
134184
./update.py <number of threads>
135185
----
136186

137-
____
138187
Generating the full database can take a long time: it takes about 15 hours on a Xeon E3-1245 v5 to index 1800 tags in the Linux kernel. For that reason, you may want to tweak the script (for example, by limiting the number of tags with a "head") in order to test the update and query commands. You can even create a new Git repository and just create one tag instead of using the official kernel repository which is very large.
139-
____
140188

141189
== Second Test
142190

143191
Verify that the queries work:
144192

145-
$ ./elixir/query.py v4.10 ident raw_spin_unlock_irq C
146-
$ ./elixir/query.py v4.10 file /kernel/sched/clock.c
193+
----
194+
python3 -m elixir.query v4.10 ident raw_spin_unlock_irq C
195+
python3 -m elixir.query v4.10 file /kernel/sched/clock.c
196+
----
147197

148198
NOTE: `v4.10` can be replaced with any other tag.
199+
149200
NOTE: Don't forget to activate the virtual environment!
150201

151202
== Configure httpd
152203

153-
The CGI interface (`web.py`) is meant to be called from your web
204+
The WSGI interface (`elixir/web.py`) is meant to be called from your web
154205
server. Since it includes support for indexing multiple projects,
155206
it expects a different variable (`LXR_PROJ_DIR`) which points to a
156207
directory with a specific structure:
@@ -162,24 +213,37 @@ directory with a specific structure:
162213
** `<project 2>`
163214
*** `data`
164215
*** `repo`
165-
** `<project 3>`
166-
*** `data`
167-
*** `repo`
168216

169217
It will then generate the other two variables upon calling the query
170218
command.
171219

172220
Now replace `/etc/apache2/sites-enabled/000-default.conf` with `docker/000-default.conf`.
173221
Note: If using httpd (RedHat/Centos) instead of apache2 (Ubuntu/Debian),
174222
the default config file to edit is `/etc/httpd/conf.d/elixir.conf`.
223+
Rember to replace the `SetEnv LXR_PROJ_DIR` value with path to your projects directory.
224+
225+
Then, enable rewrite module.
226+
227+
----
228+
a2enmod rewrite
229+
----
230+
231+
You will also need to allow git to read from repositories in project directory.
232+
An example gitconfig file that allows all access is available in `docker/gitconfig`.
233+
You can copy it to `/etc/gitconfig`.
234+
235+
NOTE: Placing this config file in /etc/gitconfig marks all git repositories as safe,
236+
for all users.
237+
This may not be acceptable, depending on your security requirements. Git version
238+
that allows leading path matches in gitconfig is not available in Debian yet,
239+
listing all repositories indexed by Elixir explicitly would be required.
175240

176241
Finally, start the httpd server.
177242

178243
----
179244
systemctl restart apache2
180245
----
181246

182-
183247
== Configure SELinux policy
184248

185249
When running systemd with SELinux enabled, httpd server can only visit limited directories.
@@ -200,22 +264,11 @@ In case you want to check SELinux log related with httpd, run the following code
200264
audit2why -a | grep httpd | less
201265
----
202266

203-
== Configure systemd log directory
204-
205-
By default, the error log of elixir will be put in /tmp/elixir-errors.
206-
However, systemd enables PrivateTmp by default.
207-
And, the final error directory will be like /tmp/systemd-private-xxxxx-httpd.service-xxxx/tmp/elixir-errors.
208-
If you want to disable it, configure httpd.service with the following attribute:
209-
----
210-
PrivateTmp=false
211-
----
212-
213267
== Configuration for other servers
214268

215269
Other HTTP servers (like nginx or lighthttpd) may not support WSGI and may require a separate WSGI server, like uWSGI.
216270

217-
Information about how to configure uWSGI with Lighthttpd can be found here:
218-
https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server
271+
Information about how to configure uWSGI with Lighthttpd can be found link:https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server[here]
219272

220273
Pull requests with example uWSGI configuration for Elixir are welcome.
221274

@@ -255,64 +308,12 @@ as a front-end to reduce the load on the server running the Elixir code.
255308
== Keeping Elixir databases up to date
256309

257310
To keep your Elixir databases up to date and index new versions that are released,
258-
we're proposing to use a script like `index /srv/elixir-data --all` which is called
259-
through a daily cron job.
311+
we're proposing to use a script like `utils/index /srv/elixir-data --all` which is
312+
called through a daily cron job.
260313

261314
You can set `$ELIXIR_THREADS` if you want to change the number of threads used by
262315
update.py for indexing (by default the number of CPUs on your system).
263316

264-
= Building Docker images
265-
266-
Dockerfiles are provided in the `docker/` directory.
267-
To build the image, run the following commands:
268-
269-
# git clone https://github.com/bootlin/elixir.git ./elixir
270-
# docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
271-
272-
ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
273-
the option is used to pass a version string to Elixir.
274-
275-
You can then run the image using `docker run`.
276-
Here we mount a host directory as Elixir data:
277-
278-
# mkdir ./elixir-data
279-
# docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
280-
281-
The Docker image does not contain any repositories.
282-
To index a repository, you can use the `index-repository` script.
283-
For example, to add the https://musl.libc.org/[musl] repository, run:
284-
285-
# docker exec -it elixir-container index /srv/elixir-data musl
286-
287-
Or, to run indexing in a separate container:
288-
289-
# docker run -v ./elixir-data/:/srv/elixir-data \
290-
--entrypoint index elixir /srv/elixir-data musl
291-
292-
You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
293-
294-
After indexing is done, Elixir should be available under the following URL on your host:
295-
http://172.17.0.2/musl/latest/source
296-
297-
If 172.17.0.2 does not answer, you can check the IP address of the container by running:
298-
299-
# docker inspect elixir-container | grep IPAddress
300-
301-
== Automatic repository updates
302-
303-
The Docker image does not automatically update repositories by itself.
304-
You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
305-
from cron on the host to periodically update repositories.
306-
307-
== Using Docker image as a development server
308-
309-
You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
310-
into `/usr/local/elixir/` in the container when running `docker run elixir`.
311-
312-
Changes in the code made on the host should be automatically reflected in the container.
313-
You can use `apache2ctl` to restart Apache.
314-
Error logs are available in `/var/log/apache2/error.log` within the container.
315-
316317
= Hardware requirements
317318

318319
Performance requirements depend mostly on the amount of traffic that you get
@@ -327,8 +328,7 @@ At Bootlin, here are a few details about the server we're using:
327328
* As of July 2019, our Elixir service consumes 17 GB of data (supporting all projects),
328329
or for the Linux kernel alone (version 5.2 being the latest), 12 GB for indexing data,
329330
and 2 GB for the git repository.
330-
* We're using an LXD instance with 8 GB of RAM on a cloud server with 8 CPU cores
331-
running at 3.1 GHz.
331+
* We're using a dedicated server with 16 GB of RAM on and 8 CPU cores running at 3.8 GHz.
332332

333333
= Contributing to Elixir
334334

@@ -459,5 +459,5 @@ issue comment] for another example of a similar situation.)
459459

460460
= License
461461

462-
Elixir is copyright (c) 2017--2020 its contributors. It is licensed AGPLv3.
462+
Elixir is copyright (c) 2017--2025 its contributors. It is licensed AGPLv3.
463463
See the `COPYING` file included with Elixir for details.

0 commit comments

Comments
 (0)