@@ -22,9 +22,9 @@ toc::[]
22
22
23
23
= Requirements
24
24
25
- * Python >= 3.8
25
+ * Python >= 3.9
26
26
* Git >= 1.9
27
- * The Jinja2 and Pygments (>= 2.7) Python libraries
27
+ * The Jinja2 and Pygments Python libraries
28
28
* Berkeley DB (and its Python binding)
29
29
* Universal Ctags
30
30
* Perl (for non-greedy regexes and automated testing)
@@ -43,13 +43,66 @@ queries, respectively.
43
43
When installing the system, you should test each layer manually and make
44
44
sure it works correctly before moving on to the next one.
45
45
46
- = Manual Installation
46
+ = Recommended setup: Docker
47
+
48
+ Dockerfiles are provided in the `docker/` directory.
49
+ To build the image, run the following commands:
50
+
51
+ # git clone https://github.com/bootlin/elixir.git ./elixir
52
+ # docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
53
+
54
+ ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
55
+ the option is used to pass a version string to Elixir.
56
+
57
+ You can then run the image using `docker run`.
58
+ Here we mount a host directory as Elixir data:
59
+
60
+ # mkdir ./elixir-data
61
+ # docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
62
+
63
+ The Docker image does not contain any repositories.
64
+ To index a repository, you can use the `index-repository` script.
65
+ For example, to add the https://musl.libc.org/[musl] repository, run:
66
+
67
+ # docker exec -it elixir-container index /srv/elixir-data musl
68
+
69
+ Or, to run indexing in a separate container:
70
+
71
+ # docker run -v ./elixir-data/:/srv/elixir-data \
72
+ --entrypoint index elixir /srv/elixir-data musl
73
+
74
+ You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
75
+
76
+ After indexing is done, Elixir should be available under the following URL on your host:
77
+ http://172.17.0.2/musl/latest/source
78
+
79
+ If 172.17.0.2 does not answer, you can check the IP address of the container by running:
80
+
81
+ # docker inspect elixir-container | grep IPAddress
82
+
83
+ == Automatic repository updates with Docker
84
+
85
+ The Docker image does not automatically update repositories by itself.
86
+ You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
87
+ from cron on the host to periodically update repositories.
88
+
89
+ == Using Docker image as a development server
90
+
91
+ You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
92
+ into `/usr/local/elixir/` in the container when running `docker run elixir`.
93
+
94
+ Changes in the code made on the host should be automatically reflected in the container.
95
+ You can use `apache2ctl` to restart Apache.
96
+ Error logs are available in `/var/log/apache2/error.log` within the container.
97
+
98
+ = Manual Installation (without Docker)
99
+
100
+ Installation tutorial has been tested on Debian 12.
101
+ Other distributions may require more adjustments.
47
102
48
103
== Install Dependencies
49
104
50
- ____
51
- For Debian
52
- ____
105
+ For Debian 12:
53
106
54
107
----
55
108
sudo apt install python3-pip python3-venv libdb-dev python3-dev build-essential universal-ctags perl git apache2 libapache2-mod-wsgi-py3 libjansson4
@@ -134,23 +187,24 @@ cd /usr/local/elixir/
134
187
./update.py <number of threads>
135
188
----
136
189
137
- ____
138
190
Generating the full database can take a long time: it takes about 15 hours on a Xeon E3-1245 v5 to index 1800 tags in the Linux kernel. For that reason, you may want to tweak the script (for example, by limiting the number of tags with a "head") in order to test the update and query commands. You can even create a new Git repository and just create one tag instead of using the official kernel repository which is very large.
139
- ____
140
191
141
192
== Second Test
142
193
143
194
Verify that the queries work:
144
195
145
- $ ./elixir/query.py v4.10 ident raw_spin_unlock_irq C
146
- $ ./elixir/query.py v4.10 file /kernel/sched/clock.c
196
+ ----
197
+ python3 -m elixir.query v4.10 ident raw_spin_unlock_irq C
198
+ python3 -m elixir.query v4.10 file /kernel/sched/clock.c
199
+ ----
147
200
148
201
NOTE: `v4.10` can be replaced with any other tag.
202
+
149
203
NOTE: Don't forget to activate the virtual environment!
150
204
151
205
== Configure httpd
152
206
153
- The CGI interface (`web.py`) is meant to be called from your web
207
+ The WSGI interface (`elixir/ web.py`) is meant to be called from your web
154
208
server. Since it includes support for indexing multiple projects,
155
209
it expects a different variable (`LXR_PROJ_DIR`) which points to a
156
210
directory with a specific structure:
@@ -162,24 +216,37 @@ directory with a specific structure:
162
216
** `<project 2>`
163
217
*** `data`
164
218
*** `repo`
165
- ** `<project 3>`
166
- *** `data`
167
- *** `repo`
168
219
169
220
It will then generate the other two variables upon calling the query
170
221
command.
171
222
172
223
Now replace `/etc/apache2/sites-enabled/000-default.conf` with `docker/000-default.conf`.
173
224
Note: If using httpd (RedHat/Centos) instead of apache2 (Ubuntu/Debian),
174
225
the default config file to edit is `/etc/httpd/conf.d/elixir.conf`.
226
+ Rember to replace the `SetEnv LXR_PROJ_DIR` value with path to your projects directory.
227
+
228
+ Then, enable rewrite module.
229
+
230
+ ----
231
+ a2enmod rewrite
232
+ ----
233
+
234
+ You will also need to allow git to read from repositories in project directory.
235
+ An example gitconfig file that allows all access is available in `docker/gitconfig`.
236
+ You can copy it to `/etc/gitconfig`.
237
+
238
+ NOTE: Placing this config file in /etc/gitconfig marks all git repositories as safe,
239
+ for all users.
240
+ This may not be acceptable, depending on your security requirements. Git version
241
+ that allows leading path matches in gitconfig is not available in Debian yet,
242
+ listing all repositories indexed by Elixir explicitly would be required.
175
243
176
244
Finally, start the httpd server.
177
245
178
246
----
179
247
systemctl restart apache2
180
248
----
181
249
182
-
183
250
== Configure SELinux policy
184
251
185
252
When running systemd with SELinux enabled, httpd server can only visit limited directories.
@@ -200,22 +267,11 @@ In case you want to check SELinux log related with httpd, run the following code
200
267
audit2why -a | grep httpd | less
201
268
----
202
269
203
- == Configure systemd log directory
204
-
205
- By default, the error log of elixir will be put in /tmp/elixir-errors.
206
- However, systemd enables PrivateTmp by default.
207
- And, the final error directory will be like /tmp/systemd-private-xxxxx-httpd.service-xxxx/tmp/elixir-errors.
208
- If you want to disable it, configure httpd.service with the following attribute:
209
- ----
210
- PrivateTmp=false
211
- ----
212
-
213
270
== Configuration for other servers
214
271
215
272
Other HTTP servers (like nginx or lighthttpd) may not support WSGI and may require a separate WSGI server, like uWSGI.
216
273
217
- Information about how to configure uWSGI with Lighthttpd can be found here:
218
- https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server
274
+ Information about how to configure uWSGI with Lighthttpd can be found link:https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server[here]
219
275
220
276
Pull requests with example uWSGI configuration for Elixir are welcome.
221
277
@@ -255,64 +311,12 @@ as a front-end to reduce the load on the server running the Elixir code.
255
311
== Keeping Elixir databases up to date
256
312
257
313
To keep your Elixir databases up to date and index new versions that are released,
258
- we're proposing to use a script like `index /srv/elixir-data --all` which is called
259
- through a daily cron job.
314
+ we're proposing to use a script like `utils/ index /srv/elixir-data --all` which is
315
+ called through a daily cron job.
260
316
261
317
You can set `$ELIXIR_THREADS` if you want to change the number of threads used by
262
318
update.py for indexing (by default the number of CPUs on your system).
263
319
264
- = Building Docker images
265
-
266
- Dockerfiles are provided in the `docker/` directory.
267
- To build the image, run the following commands:
268
-
269
- # git clone https://github.com/bootlin/elixir.git ./elixir
270
- # docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
271
-
272
- ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
273
- the option is used to pass a version string to Elixir.
274
-
275
- You can then run the image using `docker run`.
276
- Here we mount a host directory as Elixir data:
277
-
278
- # mkdir ./elixir-data
279
- # docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
280
-
281
- The Docker image does not contain any repositories.
282
- To index a repository, you can use the `index-repository` script.
283
- For example, to add the https://musl.libc.org/[musl] repository, run:
284
-
285
- # docker exec -it elixir-container index /srv/elixir-data musl
286
-
287
- Or, to run indexing in a separate container:
288
-
289
- # docker run -v ./elixir-data/:/srv/elixir-data \
290
- --entrypoint index elixir /srv/elixir-data musl
291
-
292
- You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
293
-
294
- After indexing is done, Elixir should be available under the following URL on your host:
295
- http://172.17.0.2/musl/latest/source
296
-
297
- If 172.17.0.2 does not answer, you can check the IP address of the container by running:
298
-
299
- # docker inspect elixir-container | grep IPAddress
300
-
301
- == Automatic repository updates
302
-
303
- The Docker image does not automatically update repositories by itself.
304
- You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
305
- from cron on the host to periodically update repositories.
306
-
307
- == Using Docker image as a development server
308
-
309
- You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
310
- into `/usr/local/elixir/` in the container when running `docker run elixir`.
311
-
312
- Changes in the code made on the host should be automatically reflected in the container.
313
- You can use `apache2ctl` to restart Apache.
314
- Error logs are available in `/var/log/apache2/error.log` within the container.
315
-
316
320
= Hardware requirements
317
321
318
322
Performance requirements depend mostly on the amount of traffic that you get
@@ -327,8 +331,7 @@ At Bootlin, here are a few details about the server we're using:
327
331
* As of July 2019, our Elixir service consumes 17 GB of data (supporting all projects),
328
332
or for the Linux kernel alone (version 5.2 being the latest), 12 GB for indexing data,
329
333
and 2 GB for the git repository.
330
- * We're using an LXD instance with 8 GB of RAM on a cloud server with 8 CPU cores
331
- running at 3.1 GHz.
334
+ * We're using a dedicated server with 16 GB of RAM on and 8 CPU cores running at 3.8 GHz.
332
335
333
336
= Contributing to Elixir
334
337
@@ -459,5 +462,5 @@ issue comment] for another example of a similar situation.)
459
462
460
463
= License
461
464
462
- Elixir is copyright (c) 2017--2020 its contributors. It is licensed AGPLv3.
465
+ Elixir is copyright (c) 2017--2025 its contributors. It is licensed AGPLv3.
463
466
See the `COPYING` file included with Elixir for details.
0 commit comments