Skip to content

Commit cd012e4

Browse files
committed
README: maintenance and minor fixes
* Move docker installation section to the top * Remove outdated information about /tmp/ logs * Update manual installation to work with current Elixir * Update listed requirements * Update server information * Some formatting fixes * Update copyright date
1 parent 5a225c5 commit cd012e4

File tree

1 file changed

+87
-84
lines changed

1 file changed

+87
-84
lines changed

README.adoc

+87-84
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@ toc::[]
2222

2323
= Requirements
2424

25-
* Python >= 3.8
25+
* Python >= 3.9
2626
* Git >= 1.9
27-
* The Jinja2 and Pygments (>= 2.7) Python libraries
27+
* The Jinja2 and Pygments Python libraries
2828
* Berkeley DB (and its Python binding)
2929
* Universal Ctags
3030
* Perl (for non-greedy regexes and automated testing)
@@ -43,13 +43,66 @@ queries, respectively.
4343
When installing the system, you should test each layer manually and make
4444
sure it works correctly before moving on to the next one.
4545

46-
= Manual Installation
46+
= Recommended setup: Docker
47+
48+
Dockerfiles are provided in the `docker/` directory.
49+
To build the image, run the following commands:
50+
51+
# git clone https://github.com/bootlin/elixir.git ./elixir
52+
# docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
53+
54+
ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
55+
the option is used to pass a version string to Elixir.
56+
57+
You can then run the image using `docker run`.
58+
Here we mount a host directory as Elixir data:
59+
60+
# mkdir ./elixir-data
61+
# docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
62+
63+
The Docker image does not contain any repositories.
64+
To index a repository, you can use the `index-repository` script.
65+
For example, to add the https://musl.libc.org/[musl] repository, run:
66+
67+
# docker exec -it elixir-container index /srv/elixir-data musl
68+
69+
Or, to run indexing in a separate container:
70+
71+
# docker run -v ./elixir-data/:/srv/elixir-data \
72+
--entrypoint index elixir /srv/elixir-data musl
73+
74+
You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
75+
76+
After indexing is done, Elixir should be available under the following URL on your host:
77+
http://172.17.0.2/musl/latest/source
78+
79+
If 172.17.0.2 does not answer, you can check the IP address of the container by running:
80+
81+
# docker inspect elixir-container | grep IPAddress
82+
83+
== Automatic repository updates with Docker
84+
85+
The Docker image does not automatically update repositories by itself.
86+
You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
87+
from cron on the host to periodically update repositories.
88+
89+
== Using Docker image as a development server
90+
91+
You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
92+
into `/usr/local/elixir/` in the container when running `docker run elixir`.
93+
94+
Changes in the code made on the host should be automatically reflected in the container.
95+
You can use `apache2ctl` to restart Apache.
96+
Error logs are available in `/var/log/apache2/error.log` within the container.
97+
98+
= Manual Installation (without Docker)
99+
100+
Installation tutorial has been tested on Debian 12.
101+
Other distributions may require more adjustments.
47102

48103
== Install Dependencies
49104

50-
____
51-
For Debian
52-
____
105+
For Debian 12:
53106

54107
----
55108
sudo apt install python3-pip python3-venv libdb-dev python3-dev build-essential universal-ctags perl git apache2 libapache2-mod-wsgi-py3 libjansson4
@@ -134,23 +187,24 @@ cd /usr/local/elixir/
134187
./update.py <number of threads>
135188
----
136189

137-
____
138190
Generating the full database can take a long time: it takes about 15 hours on a Xeon E3-1245 v5 to index 1800 tags in the Linux kernel. For that reason, you may want to tweak the script (for example, by limiting the number of tags with a "head") in order to test the update and query commands. You can even create a new Git repository and just create one tag instead of using the official kernel repository which is very large.
139-
____
140191

141192
== Second Test
142193

143194
Verify that the queries work:
144195

145-
$ ./elixir/query.py v4.10 ident raw_spin_unlock_irq C
146-
$ ./elixir/query.py v4.10 file /kernel/sched/clock.c
196+
----
197+
python3 -m elixir.query v4.10 ident raw_spin_unlock_irq C
198+
python3 -m elixir.query v4.10 file /kernel/sched/clock.c
199+
----
147200

148201
NOTE: `v4.10` can be replaced with any other tag.
202+
149203
NOTE: Don't forget to activate the virtual environment!
150204

151205
== Configure httpd
152206

153-
The CGI interface (`web.py`) is meant to be called from your web
207+
The WSGI interface (`elixir/web.py`) is meant to be called from your web
154208
server. Since it includes support for indexing multiple projects,
155209
it expects a different variable (`LXR_PROJ_DIR`) which points to a
156210
directory with a specific structure:
@@ -162,24 +216,37 @@ directory with a specific structure:
162216
** `<project 2>`
163217
*** `data`
164218
*** `repo`
165-
** `<project 3>`
166-
*** `data`
167-
*** `repo`
168219

169220
It will then generate the other two variables upon calling the query
170221
command.
171222

172223
Now replace `/etc/apache2/sites-enabled/000-default.conf` with `docker/000-default.conf`.
173224
Note: If using httpd (RedHat/Centos) instead of apache2 (Ubuntu/Debian),
174225
the default config file to edit is `/etc/httpd/conf.d/elixir.conf`.
226+
Rember to replace the `SetEnv LXR_PROJ_DIR` value with path to your projects directory.
227+
228+
Then, enable rewrite module.
229+
230+
----
231+
a2enmod rewrite
232+
----
233+
234+
You will also need to allow git to read from repositories in project directory.
235+
An example gitconfig file that allows all access is available in `docker/gitconfig`.
236+
You can copy it to `/etc/gitconfig`.
237+
238+
NOTE: Placing this config file in /etc/gitconfig marks all git repositories as safe,
239+
for all users.
240+
This may not be acceptable, depending on your security requirements. Git version
241+
that allows leading path matches in gitconfig is not available in Debian yet,
242+
listing all repositories indexed by Elixir explicitly would be required.
175243

176244
Finally, start the httpd server.
177245

178246
----
179247
systemctl restart apache2
180248
----
181249

182-
183250
== Configure SELinux policy
184251

185252
When running systemd with SELinux enabled, httpd server can only visit limited directories.
@@ -200,22 +267,11 @@ In case you want to check SELinux log related with httpd, run the following code
200267
audit2why -a | grep httpd | less
201268
----
202269

203-
== Configure systemd log directory
204-
205-
By default, the error log of elixir will be put in /tmp/elixir-errors.
206-
However, systemd enables PrivateTmp by default.
207-
And, the final error directory will be like /tmp/systemd-private-xxxxx-httpd.service-xxxx/tmp/elixir-errors.
208-
If you want to disable it, configure httpd.service with the following attribute:
209-
----
210-
PrivateTmp=false
211-
----
212-
213270
== Configuration for other servers
214271

215272
Other HTTP servers (like nginx or lighthttpd) may not support WSGI and may require a separate WSGI server, like uWSGI.
216273

217-
Information about how to configure uWSGI with Lighthttpd can be found here:
218-
https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server
274+
Information about how to configure uWSGI with Lighthttpd can be found link:https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server[here]
219275

220276
Pull requests with example uWSGI configuration for Elixir are welcome.
221277

@@ -255,64 +311,12 @@ as a front-end to reduce the load on the server running the Elixir code.
255311
== Keeping Elixir databases up to date
256312

257313
To keep your Elixir databases up to date and index new versions that are released,
258-
we're proposing to use a script like `index /srv/elixir-data --all` which is called
259-
through a daily cron job.
314+
we're proposing to use a script like `utils/index /srv/elixir-data --all` which is
315+
called through a daily cron job.
260316

261317
You can set `$ELIXIR_THREADS` if you want to change the number of threads used by
262318
update.py for indexing (by default the number of CPUs on your system).
263319

264-
= Building Docker images
265-
266-
Dockerfiles are provided in the `docker/` directory.
267-
To build the image, run the following commands:
268-
269-
# git clone https://github.com/bootlin/elixir.git ./elixir
270-
# docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
271-
272-
ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
273-
the option is used to pass a version string to Elixir.
274-
275-
You can then run the image using `docker run`.
276-
Here we mount a host directory as Elixir data:
277-
278-
# mkdir ./elixir-data
279-
# docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
280-
281-
The Docker image does not contain any repositories.
282-
To index a repository, you can use the `index-repository` script.
283-
For example, to add the https://musl.libc.org/[musl] repository, run:
284-
285-
# docker exec -it elixir-container index /srv/elixir-data musl
286-
287-
Or, to run indexing in a separate container:
288-
289-
# docker run -v ./elixir-data/:/srv/elixir-data \
290-
--entrypoint index elixir /srv/elixir-data musl
291-
292-
You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
293-
294-
After indexing is done, Elixir should be available under the following URL on your host:
295-
http://172.17.0.2/musl/latest/source
296-
297-
If 172.17.0.2 does not answer, you can check the IP address of the container by running:
298-
299-
# docker inspect elixir-container | grep IPAddress
300-
301-
== Automatic repository updates
302-
303-
The Docker image does not automatically update repositories by itself.
304-
You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
305-
from cron on the host to periodically update repositories.
306-
307-
== Using Docker image as a development server
308-
309-
You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
310-
into `/usr/local/elixir/` in the container when running `docker run elixir`.
311-
312-
Changes in the code made on the host should be automatically reflected in the container.
313-
You can use `apache2ctl` to restart Apache.
314-
Error logs are available in `/var/log/apache2/error.log` within the container.
315-
316320
= Hardware requirements
317321

318322
Performance requirements depend mostly on the amount of traffic that you get
@@ -327,8 +331,7 @@ At Bootlin, here are a few details about the server we're using:
327331
* As of July 2019, our Elixir service consumes 17 GB of data (supporting all projects),
328332
or for the Linux kernel alone (version 5.2 being the latest), 12 GB for indexing data,
329333
and 2 GB for the git repository.
330-
* We're using an LXD instance with 8 GB of RAM on a cloud server with 8 CPU cores
331-
running at 3.1 GHz.
334+
* We're using a dedicated server with 16 GB of RAM on and 8 CPU cores running at 3.8 GHz.
332335

333336
= Contributing to Elixir
334337

@@ -459,5 +462,5 @@ issue comment] for another example of a similar situation.)
459462

460463
= License
461464

462-
Elixir is copyright (c) 2017--2020 its contributors. It is licensed AGPLv3.
465+
Elixir is copyright (c) 2017--2025 its contributors. It is licensed AGPLv3.
463466
See the `COPYING` file included with Elixir for details.

0 commit comments

Comments
 (0)