Merge branch 'to-v1.4'

kamax-matrix · Apr 9, 2019 · ee21f05 · ee21f05
2 parents 1587103 + 6cc17ab
commit ee21f05
Show file tree

Hide file tree

Showing 93 changed files with 4,804 additions and 373 deletions.
diff --git a/README.md b/README.md
@@ -31,7 +31,7 @@ users. 3PIDs can be anything that uniquely and globally identify a user, like:
 - Twitter handle
 - Facebook ID
 
-If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.
+If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.  
 
 # Features
 [Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://matrix.org/docs/spec/identity_service/r0.1.0.html#general-principles):
@@ -53,6 +53,7 @@ As an enhanced Identity service:
   - Central Matrix Identity servers
 - [Session Control](docs/threepids/session/session.md): Extensive control of where 3PIDs are transmitted so they are not
   leaked publicly by users
+- [Registration control](docs/features/registration.md): Control and restrict user registration based on 3PID patterns or criterias, like a pending invite
 - [Authentication](docs/features/authentication.md): Use your Identity stores to perform authentication in [synapse](https://github.com/matrix-org/synapse)
   via the [REST password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
 - [Directory search](docs/features/directory.md) which allows you to search for users within your organisation,

diff --git a/build.gradle b/build.gradle
@@ -145,13 +145,24 @@ dependencies {
 
     // HTTP server
     compile 'io.undertow:undertow-core:2.0.16.Final'
+
+    // Command parser for AS interface
+    implementation 'commons-cli:commons-cli:1.4'
 
     testCompile 'junit:junit:4.12'
     testCompile 'com.github.tomakehurst:wiremock:2.8.0'
     testCompile 'com.unboundid:unboundid-ldapsdk:4.0.9'
     testCompile 'com.icegreen:greenmail:1.5.9'
 }
 
+jar {
+    manifest {
+        attributes(
+            'Implementation-Version': mxisdVersion()
+        )
+    }
+}
+
 shadowJar {
     baseName = project.name
     classifier = null

diff --git a/docs/features/authentication.md b/docs/features/authentication.md
@@ -21,7 +21,7 @@ It allows to use Identity stores configured in mxisd to authenticate users on yo
 
 Authentication is divided into two parts:
 - [Basic](#basic): authenticate with a regular username.
-- [Advanced](#advanced): same as basic with extra ability to authenticate using a 3PID.
+- [Advanced](#advanced): same as basic with extra abilities like authenticate using a 3PID or do username rewrite.
 
 ## Basic
 Authentication by username is possible by linking synapse and mxisd together using a specific module for synapse, also
@@ -145,7 +145,49 @@ Your VirtualHost should now look similar to:
 </VirtualHost>
 ```
 
+##### nginx
+
+The specific configuration to add under the relevant `server`:
+
+```nginx
+location /_matrix/client/r0/login {
+    proxy_pass http://localhost:8090;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $remote_addr;
+}
+```
+
+Your `server` section should now look similar to:
+
+```nginx
+server {
+    listen 443 ssl;
+    server_name matrix.example.org;
+    
+    # ...
+    
+    location /_matrix/client/r0/login {
+        proxy_pass http://localhost:8090;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
 #### DNS Overwrite
+
 Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
 the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
 
@@ -165,6 +207,12 @@ In case the hostname is the same as your Matrix domain and `server.name` is not
 
 `value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
 
+### Optional features
+
+The following features are available after you have a working Advanced setup:
+
+- Username rewrite: Allows you to rewrite the username of a regular login/pass authentication to a 3PID, that then gets resolved using the regular lookup process. Most common use case is to allow login with numerical usernames on synapse, which is not possible out of the box.
+
 #### Username rewrite
 In mxisd config:
 ```yaml

diff --git a/docs/features/experimental/application-service.md b/docs/features/experimental/application-service.md
@@ -1,65 +1,65 @@
-# Integration as an Application Service
+# Application Service
 **WARNING:** These features are currently highly experimental. They can be removed or modified without notice.  
-All the features requires a Homeserver capable of connecting Application Services.
+All the features requires a Homeserver capable of connecting [Application Services](https://matrix.org/docs/spec/application_service/r0.1.0.html).
 
-## Email notification for Room invites by Matrix ID
-This feature allows for users found in Identity stores to be instantly notified about Room Invites, regardless if their
-account was already provisioned on the Homeserver.
+The following capabilities are provided in this feature:
+- [Admin commands](#admin-commands)
+- [Email Notification about room invites by Matrix IDs](#email-notification-about-room-invites-by-matrix-ids)
+- [Auto-reject of expired 3PID invites](#auto-reject-of-expired-3pid-invites)
 
-### Requirements
-- [Identity store(s)](../../stores/README.md) supporting the Profile feature
-- At least one email entry in the identity store for each user that could be invited.
+## Setup
+> **NOTE:** Make sure you are familiar with [configuration format and rules](../../configure.md).
+
+Integration as an Application service is a three steps process:
+1. Create the baseline mxisd configuration to allow integration.
+2. Integrate with the homeserver.
+3. Configure the specific capabilities, if applicable.
 
 ### Configuration
-In your mxisd config file:
-```yaml
-matrix:
-  listener:
-    url:  '<URL TO THE CS API OF THE HOMESERVER>'
-    localpart: 'appservice-mxisd'
-    token:
-      hs: 'HS_TOKEN_CHANGE_ME'
+#### Variables
+Under the `appsvc` namespace:
 
-synapseSql:
-  enabled: false ## Do not use this line if Synapse is used as an Identity Store
-  type: '<DB TYPE>'
-  connection: '<DB CONNECTION URL>'
-```
+| Key                   | Type    | Required | Default | Purpose                                                        |
+|-----------------------|---------|----------|---------|----------------------------------------------------------------|
+| `enabled`             | boolean | No       | `true`  | Globally enable/disable the feature                            |
+| `user.main`           | string  | No       | `mxisd` | Localpart for the main appservice user                         |
+| `endpoint.toHS.url`   | string  | Yes      | *None*  | Base URL to the Homeserver                                     |
+| `endpoint.toHS.token` | string  | Yes      | *None*  | Token to use when sending requests to the Homeserver           |
+| `endpoint.toAS.url`   | string  | Yes      | *None*  | Base URL to mxisd from the Homeserver                          |
+| `endpoint.toAS.token` | string  | Yes      | *None*  | Token for the Homeserver to use when sending requests to mxisd |
 
-The `synapseSql` section is optional. It is used to retrieve display names which are not directly accessible in this mode.
-For details about `type` and `connection`, see the [relevant documentation](../../stores/synapse.md).
-If you do not configure it, some placeholders will not be available in the notification, like the Room name.
+#### Example
+```yaml
+appsvc:
+  endpoint:
+    toHS:
+      url: 'http://localhost:8008'
+      token: 'ExampleTokenToHS-ChangeMe!'
+    toAS:
+      url: 'http://localhost:8090'
+      token: 'ExampleTokenToAS-ChangeMe!'
+```
+### Integration
+#### Synapse
+Under the `appsvc.registration.synapse` namespace:
 
-You can also change the default template of the notification using the `generic.matrixId` template option.  
-See [the Template generator documentation](../../threepids/notification/template-generator.md) for more info.
+| Key    | Type   | Required | Default            | Purpose                                                                  |
+|--------|--------|----------|--------------------|--------------------------------------------------------------------------|
+| `id`   | string | No       | `appservice-mxisd` | The unique, user-defined ID of this application service. See spec.       |
+| `file` | string | Yes      | *None*             | If defined, the synapse registration file that should be created/updated |
 
-### Homeserver integration
-#### Synapse
-Create a new appservice registration file. Futher config will assume it is in `/etc/matrix-synapse/appservice-mxisd.yaml`
+##### Example 
 ```yaml
-id: "appservice-mxisd"
-url: "http://127.0.0.1:8090"
-as_token: "AS_TOKEN_CHANGE_ME"
-hs_token: "HS_TOKEN_CHANGE_ME"
-sender_localpart: "appservice-mxisd"
-namespaces:
-  users:
-    - regex: "@*"
-      exclusive: false
-  aliases: []
-  rooms: []
+appsvc:
+  registration:
+    synapse:
+      file: '/etc/matrix-synapse/mxisd-appservice-registration.yaml'
 ```
-`id`: An arbitrary unique string to identify the AS.  
-`url`: mxisd to reach mxisd. This ideally should be HTTP and not going through any reverse proxy.  
-`as_token`: Arbitrary value used by mxisd when talking to the HS. Not currently used.  
-`hs_token`: Arbitrary value used by synapse when talking to mxisd. Must match `token.hs` in mxisd config.
-`sender_localpart`: Username for the mxisd itself on the HS. Default configuration should be kept.  
-`namespaces`: To be kept as is.  
 
 Edit your `homeserver.yaml` and add a new entry to the appservice config file, which should look something like this:
 ```yaml
 app_service_config_files:
-  - '/etc/matrix-synapse/appservice-mxisd.yaml'
+  - '/etc/matrix-synapse/mxisd-appservice-registration.yaml'
   - ...
 ```
 
@@ -68,5 +68,54 @@ Restart synapse when done to register mxisd.
 #### Others
 See your Homeserver documentation on how to integrate.
 
-### Test
+## Capabilities
+### Admin commands
+#### Setup
+Min config:
+```yaml
+appsvc:
+  feature:
+    admin:
+      allowedRoles:
+        - '+aMatrixCommunity:example.org'
+        - 'SomeLdapGroup'
+        - 'AnyOtherArbitraryRoleFromIdentityStores'
+```
+
+#### Use
+The following steps assume:
+- `matrix.domain` set to `example.org`
+- `appsvc.user.main` set to `mxisd` or not set
+
+1. Invite `@mxisd:example.org` to a new direct chat
+2. Type `!help` to get all available commands
+
+### Email Notification about room invites by Matrix IDs
+This feature allows for users found in Identity stores to be instantly notified about Room Invites, regardless if their
+account was already provisioned on the Homeserver.
+
+#### Requirements
+- [Identity store(s)](../../stores/README.md) supporting the Profile feature
+- At least one email entry in the identity store for each user that could be invited.
+
+#### Configuration
+In your mxisd config file:
+```yaml
+synapseSql:
+  enabled: false ## Do not use this line if Synapse is used as an Identity Store
+  type: '<DB TYPE>'
+  connection: '<DB CONNECTION URL>'
+```
+
+The `synapseSql` section is optional. It is used to retrieve display names which are not directly accessible in this mode.
+For details about `type` and `connection`, see the [relevant documentation](../../stores/synapse.md).
+If you do not configure it, some placeholders will not be available in the notification, like the Room name.
+
+You can also change the default template of the notification using the `generic.matrixId` template option.  
+See [the Template generator documentation](../../threepids/notification/template-generator.md) for more info.
+
+#### Test
 Invite a user which is part of your domain while an appropriate Identity store is used.
+
+### Auto-reject of expired 3PID invites
+*TBC*
diff --git a/docs/features/identity.md b/docs/features/identity.md
@@ -1,6 +1,13 @@
 # Identity
 Implementation of the [Identity Service API r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html).
 
+- [Lookups](#lookups)
+- [Invitations](#invitations)
+  - [Expiration](#expiration)
+  - [Policies](#policies)
+  - [Resolution](#resolution)
+- [3PIDs Management](#3pids-management)
+
 ## Lookups
 If you would like to use the central matrix.org Identity server to ensure maximum discovery at the cost of potentially
 leaking all your contacts information, add the following to your configuration:
@@ -12,8 +19,78 @@ forward:
 **NOTE:** You should carefully consider enabling this option, which is discouraged.  
 For more info, see the [relevant issue](https://github.com/kamax-matrix/mxisd/issues/76).
 
-## Room Invitations
-Resolution can be customized using the following configuration:
+## Invitations
+### Expiration
+#### Overview
+Matrix does not provide a mean to remove/cancel pending 3PID invitations with the APIs. The current reference
+implementations also do not provide any mean to do so. This leads to 3PID invites forever stuck in rooms.
+
+To provide this functionality, mxisd uses a workaround: resolve the invite to a dedicated User ID, which can be
+controlled by mxisd or a bot/service that will then reject the invite.
+
+If this dedicated User ID is to be controlled by mxisd, the [Application Service](experimental/application-service.md)
+feature must be configured and integrated with your Homeserver, as well as the *Auto-reject 3PID invite capability*.
+
+#### Configuration
+```yaml
+invite:
+  expiration:
+    enabled: true/false
+    after: 5
+    resolveTo: '@john.doe:example.org'
+```
+`enabled`
+- Purpose: Enable or disable the invite expiration feature.
+- Default: `true`
+
+`after` 
+- Purpose: Amount of minutes before an invitation expires.
+- Default: `10080` (7 days)
+
+`resolveTo`
+- Purpose: Matrix User ID to resolve the expired invitations to.
+- Default: Computed from `appsvc.user.inviteExpired` and `matrix.domain`
+
+### Policies
+3PID invite policies are the companion feature of [Registration](registration.md). While the Registration feature acts on
+requirements for the invitee/register, this feature acts on requirement for the one(s) performing 3PID invites, ensuring
+a coherent system.
+
+It relies on only allowing people with specific [Roles](profile.md) to perform 3PID invites. This would typically allow
+a tight-control on a server setup with is "invite-only" or semi-open (relying on trusted people to invite new members).
+
+It's a middle ground between a closed server, where every user must be created or already exists in an Identity store,
+and an open server, where anyone can register.
+
+#### Integration
+Because Identity Servers do not control 3PID invites as per Matrix spec, mxisd needs to intercept a set of Homeserver
+endpoints to apply the policies.
+
+##### Reverse Proxy
+###### nginx
+**IMPORTANT**: Must be placed before your global `/_matrix` entry:
+```nginx
+location ~* ^/_matrix/client/r0/rooms/([^/]+)/invite$ {
+    proxy_pass		    http://127.0.0.1:8090;
+    proxy_set_header	Host $host;
+    proxy_set_header	X-Forwarded-For $remote_addr;
+}
+```
+
+#### Configuration
+The only policy currently available is to restrict 3PID invite to users having a specific (set of) role(s), like so:
+
+```yaml
+invite:
+  policy:
+    ifSender:
+      hasRole:
+        - '<THIS_ROLE>'
+        - '<OR_THIS_ROLE>'
+```
+
+### Resolution
+Resolution of 3PID invitations can be customized using the following configuration:
 
 `invite.resolution.recursive`  
 - Default value: `true`  
@@ -26,5 +103,5 @@ Resolution can be customized using the following configuration:
 - Default value: `1`  
 - Description: How often, in minutes, mxisd should try to resolve pending invites.
 
-## 3PID addition to user profile
+## 3PIDs Management
 See the [3PID session documents](../threepids/session)