@@ -22,9 +22,9 @@ toc::[]
22
22
23
23
= Requirements
24
24
25
- * Python >= 3.8
25
+ * Python >= 3.9
26
26
* Git >= 1.9
27
- * The Jinja2 and Pygments (>= 2.7) Python libraries
27
+ * The Jinja2 and Pygments Python libraries
28
28
* Berkeley DB (and its Python binding)
29
29
* Universal Ctags
30
30
* Perl (for non-greedy regexes and automated testing)
@@ -43,13 +43,63 @@ queries, respectively.
43
43
When installing the system, you should test each layer manually and make
44
44
sure it works correctly before moving on to the next one.
45
45
46
- = Manual Installation
46
+ = Recommended setup: Docker
47
+
48
+ Dockerfiles are provided in the `docker/` directory.
49
+ To build the image, run the following commands:
50
+
51
+ # git clone https://github.com/bootlin/elixir.git ./elixir
52
+ # docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
53
+
54
+ ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
55
+ the option is used to pass a version string to Elixir.
56
+
57
+ You can then run the image using `docker run`.
58
+ Here we mount a host directory as Elixir data:
59
+
60
+ # mkdir ./elixir-data
61
+ # docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
62
+
63
+ The Docker image does not contain any repositories.
64
+ To index a repository, you can use the `index-repository` script.
65
+ For example, to add the https://musl.libc.org/[musl] repository, run:
66
+
67
+ # docker exec -it elixir-container index /srv/elixir-data musl
68
+
69
+ Or, to run indexing in a separate container:
70
+
71
+ # docker run -v ./elixir-data/:/srv/elixir-data \
72
+ --entrypoint index elixir /srv/elixir-data musl
73
+
74
+ You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
75
+
76
+ After indexing is done, Elixir should be available under the following URL on your host:
77
+ http://172.17.0.2/musl/latest/source
78
+
79
+ If 172.17.0.2 does not answer, you can check the IP address of the container by running:
80
+
81
+ # docker inspect elixir-container | grep IPAddress
82
+
83
+ == Automatic repository updates with Docker
84
+
85
+ The Docker image does not automatically update repositories by itself.
86
+ You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
87
+ from cron on the host to periodically update repositories.
88
+
89
+ == Using Docker image as a development server
90
+
91
+ You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
92
+ into `/usr/local/elixir/` in the container when running `docker run elixir`.
93
+
94
+ Changes in the code made on the host should be automatically reflected in the container.
95
+ You can use `apache2ctl` to restart Apache.
96
+ Error logs are available in `/var/log/apache2/error.log` within the container.
97
+
98
+ = Manual Installation (without Docker)
47
99
48
100
== Install Dependencies
49
101
50
- ____
51
- For Debian
52
- ____
102
+ For Debian:
53
103
54
104
----
55
105
sudo apt install python3-pip python3-venv libdb-dev python3-dev build-essential universal-ctags perl git apache2 libapache2-mod-wsgi-py3 libjansson4
@@ -134,23 +184,24 @@ cd /usr/local/elixir/
134
184
./update.py <number of threads>
135
185
----
136
186
137
- ____
138
187
Generating the full database can take a long time: it takes about 15 hours on a Xeon E3-1245 v5 to index 1800 tags in the Linux kernel. For that reason, you may want to tweak the script (for example, by limiting the number of tags with a "head") in order to test the update and query commands. You can even create a new Git repository and just create one tag instead of using the official kernel repository which is very large.
139
- ____
140
188
141
189
== Second Test
142
190
143
191
Verify that the queries work:
144
192
145
- $ ./elixir/query.py v4.10 ident raw_spin_unlock_irq C
146
- $ ./elixir/query.py v4.10 file /kernel/sched/clock.c
193
+ ----
194
+ python3 -m elixir.query v4.10 ident raw_spin_unlock_irq C
195
+ python3 -m elixir.query v4.10 file /kernel/sched/clock.c
196
+ ----
147
197
148
198
NOTE: `v4.10` can be replaced with any other tag.
199
+
149
200
NOTE: Don't forget to activate the virtual environment!
150
201
151
202
== Configure httpd
152
203
153
- The CGI interface (`web.py`) is meant to be called from your web
204
+ The WSGI interface (`elixir/ web.py`) is meant to be called from your web
154
205
server. Since it includes support for indexing multiple projects,
155
206
it expects a different variable (`LXR_PROJ_DIR`) which points to a
156
207
directory with a specific structure:
@@ -162,24 +213,37 @@ directory with a specific structure:
162
213
** `<project 2>`
163
214
*** `data`
164
215
*** `repo`
165
- ** `<project 3>`
166
- *** `data`
167
- *** `repo`
168
216
169
217
It will then generate the other two variables upon calling the query
170
218
command.
171
219
172
220
Now replace `/etc/apache2/sites-enabled/000-default.conf` with `docker/000-default.conf`.
173
221
Note: If using httpd (RedHat/Centos) instead of apache2 (Ubuntu/Debian),
174
222
the default config file to edit is `/etc/httpd/conf.d/elixir.conf`.
223
+ Rember to replace the `SetEnv LXR_PROJ_DIR` value with path to your projects directory.
224
+
225
+ Then, enable rewrite module.
226
+
227
+ ----
228
+ a2enmod rewrite
229
+ ----
230
+
231
+ You will also need to allow git to read from repositories in project directory.
232
+ An example gitconfig file that allows all access is available in `docker/gitconfig`.
233
+ You can copy it to `/etc/gitconfig`.
234
+
235
+ NOTE: Placing this config file in /etc/gitconfig marks all git repositories as safe,
236
+ for all users.
237
+ This may not be acceptable, depending on your security requirements. Git version
238
+ that allows leading path matches in gitconfig is not available in Debian yet,
239
+ listing all repositories indexed by Elixir explicitly would be required.
175
240
176
241
Finally, start the httpd server.
177
242
178
243
----
179
244
systemctl restart apache2
180
245
----
181
246
182
-
183
247
== Configure SELinux policy
184
248
185
249
When running systemd with SELinux enabled, httpd server can only visit limited directories.
@@ -200,22 +264,11 @@ In case you want to check SELinux log related with httpd, run the following code
200
264
audit2why -a | grep httpd | less
201
265
----
202
266
203
- == Configure systemd log directory
204
-
205
- By default, the error log of elixir will be put in /tmp/elixir-errors.
206
- However, systemd enables PrivateTmp by default.
207
- And, the final error directory will be like /tmp/systemd-private-xxxxx-httpd.service-xxxx/tmp/elixir-errors.
208
- If you want to disable it, configure httpd.service with the following attribute:
209
- ----
210
- PrivateTmp=false
211
- ----
212
-
213
267
== Configuration for other servers
214
268
215
269
Other HTTP servers (like nginx or lighthttpd) may not support WSGI and may require a separate WSGI server, like uWSGI.
216
270
217
- Information about how to configure uWSGI with Lighthttpd can be found here:
218
- https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server
271
+ Information about how to configure uWSGI with Lighthttpd can be found link:https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI#Python-WSGI-apps-via-uwsgi-SCGI-FastCGI-or-HTTP-using-the-uWSGI-server[here]
219
272
220
273
Pull requests with example uWSGI configuration for Elixir are welcome.
221
274
@@ -255,64 +308,12 @@ as a front-end to reduce the load on the server running the Elixir code.
255
308
== Keeping Elixir databases up to date
256
309
257
310
To keep your Elixir databases up to date and index new versions that are released,
258
- we're proposing to use a script like `index /srv/elixir-data --all` which is called
259
- through a daily cron job.
311
+ we're proposing to use a script like `utils/ index /srv/elixir-data --all` which is
312
+ called through a daily cron job.
260
313
261
314
You can set `$ELIXIR_THREADS` if you want to change the number of threads used by
262
315
update.py for indexing (by default the number of CPUs on your system).
263
316
264
- = Building Docker images
265
-
266
- Dockerfiles are provided in the `docker/` directory.
267
- To build the image, run the following commands:
268
-
269
- # git clone https://github.com/bootlin/elixir.git ./elixir
270
- # docker build -t elixir --build-arg ELIXIR_VERSION=`git rev-parse --short HEAD` -f ./elixir/docker/Dockerfile ./elixir
271
-
272
- ELIXIR_VER build argument is optional. Since .git directory is not copied into Docker image by default,
273
- the option is used to pass a version string to Elixir.
274
-
275
- You can then run the image using `docker run`.
276
- Here we mount a host directory as Elixir data:
277
-
278
- # mkdir ./elixir-data
279
- # docker run -v ./elixir-data/:/srv/elixir-data -d --name elixir-container elixir
280
-
281
- The Docker image does not contain any repositories.
282
- To index a repository, you can use the `index-repository` script.
283
- For example, to add the https://musl.libc.org/[musl] repository, run:
284
-
285
- # docker exec -it elixir-container index /srv/elixir-data musl
286
-
287
- Or, to run indexing in a separate container:
288
-
289
- # docker run -v ./elixir-data/:/srv/elixir-data \
290
- --entrypoint index elixir /srv/elixir-data musl
291
-
292
- You can also use `index /srv/elixir-data --all` to start indexing all officially supported repositories.
293
-
294
- After indexing is done, Elixir should be available under the following URL on your host:
295
- http://172.17.0.2/musl/latest/source
296
-
297
- If 172.17.0.2 does not answer, you can check the IP address of the container by running:
298
-
299
- # docker inspect elixir-container | grep IPAddress
300
-
301
- == Automatic repository updates
302
-
303
- The Docker image does not automatically update repositories by itself.
304
- You can, for example, start `index /srv/elixir-data --all` in the container (or in a separate container, with Elixir data volume/directory mounted)
305
- from cron on the host to periodically update repositories.
306
-
307
- == Using Docker image as a development server
308
-
309
- You can easily use the Docker image as a development server by following the steps above, but mounting Elixir source directory from the host
310
- into `/usr/local/elixir/` in the container when running `docker run elixir`.
311
-
312
- Changes in the code made on the host should be automatically reflected in the container.
313
- You can use `apache2ctl` to restart Apache.
314
- Error logs are available in `/var/log/apache2/error.log` within the container.
315
-
316
317
= Hardware requirements
317
318
318
319
Performance requirements depend mostly on the amount of traffic that you get
@@ -327,8 +328,7 @@ At Bootlin, here are a few details about the server we're using:
327
328
* As of July 2019, our Elixir service consumes 17 GB of data (supporting all projects),
328
329
or for the Linux kernel alone (version 5.2 being the latest), 12 GB for indexing data,
329
330
and 2 GB for the git repository.
330
- * We're using an LXD instance with 8 GB of RAM on a cloud server with 8 CPU cores
331
- running at 3.1 GHz.
331
+ * We're using a dedicated server with 16 GB of RAM on and 8 CPU cores running at 3.8 GHz.
332
332
333
333
= Contributing to Elixir
334
334
@@ -459,5 +459,5 @@ issue comment] for another example of a similar situation.)
459
459
460
460
= License
461
461
462
- Elixir is copyright (c) 2017--2020 its contributors. It is licensed AGPLv3.
462
+ Elixir is copyright (c) 2017--2025 its contributors. It is licensed AGPLv3.
463
463
See the `COPYING` file included with Elixir for details.
0 commit comments