Selects logo a praymentis.

I AM SELECT

Feb 03
2021
Combining the self hosted cloud synchronization Nextcloud with the end to end encrypted chat API Matrix is a dream come true for a decentralized, self hosted infrastructure.

Integrating Matrix with Nextcloud

Matrix is a decentralized chat network similar to email. You can join Matrix on any domain that provides a Matrix API to chat locally and with a global network. Conversations are end to end encrypted by default, there are several nice looking clients (full disclosure I also created one :) with all features you would expect from a modern chat application.

Nextcloud on the other hand is a personal cloud solution that provides everything you expect from a cloud service like syncing: files, calendars, contacts, bookmarks, a plugin system, file encryption, version history and so much more.

To integrate Matrix with Nextcloud you will need a server that runs the Matrix API. The most popular choice is currently Synapse, created by the developers of the Matrix protocol. Most likely you also want a simple way to use Matrix with a browser based client, configured to use your Matrix API. The most popular choice for a client is currently Element, also developed by the Matrix team. I will show the set up of both of these applications, but I just want to mention that there are more server and clients to choose from.

To connect the Matrix Synapse server with the Nextcloud server Nextcloud can act as an OpenID Connect / OAuth2 provider and Matrix as the client. This means the user management is done by Nextcloud, where you can benefit from their app store to integrate even more authentication systems.

This article assumes the following setup as base.

  • Ubuntu/Linux (Vserver/raspberry pi/ ...)
  • Nginx
  • Nextcloud

Nextcloud and Matrix will be setup on 2 (sub-)domains:

  • nextcloud.example.org Nextcloud (personal preference)
  • example.org Matrix API and Matrix client (to get usernames like @user:example.org)

If you have a different setup just skip through and find the parts you need and adapt them.

(Hint: I will use domain names as file and folder names, do not get confused by that, to make it more obvious what is what I added emoji icons next to 📄files, 📁folders, and 🔗URLs)

Matrix Synapse installation

Install Matrix Synapse from their .deb package repository.

wget -qO - https://matrix.org/packages/debian/repo-key.asc | sudo apt-key add -
sudo add-apt-repository https://matrix.org/packages/debian/
sudo apt update
sudo apt install matrix-synapse -y

Nginx config

The Matrix client Element can be served as static files (.html, .css, .js) by Nginx. It must be located on the same (sub-)domain as the API since the Nextcloud login flow returns a Content-Security-Policy form-action header that only allows to redirect to the same domain as the client (on Chrome / Blink browsers).

The Matrix API provided by Synapse needs to expose the following routes.

  • 🔗http://example.org/_matrix and
  • 🔗http://example.org/_synapse

Nginx is used as a reverse proxy, forwarding all requests for these URLs to a local Synapse server (running on port 8008)

📄/etc/nginx/sites-enabled/example.org
server {
    server_name example.org
    …
    # Matrix client
    root 📁/var/www/matrix.example.org;
    index index.html index.htm;
    # Matrix API
    location ~ ^/(\_matrix|\_synapse)/ {
        proxy_pass http://127.0.0.1:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
    }
    location /.well-known/matrix/server {
        return 200 '{"m.server": "example.org:443"}';
        add_header Content-Type application/json;
  }
  location /.well-known/matrix/client {
        return 200 '{"m.homeserver": {"base_url": "https://example.org"},"m.identity_server": {"base_url": "https://vector.im"}}';
        add_header Content-Type application/json;
        add_header "Access-Control-Allow-Origin" *;
  }
}

Test the config and reload Nginx.

nginx -t
systemctl reload nginx.service

Matrix client Element installation

Download the latest release and unpacked it to 📁/var/www/matrix.example.org (as in the Nginx config).

Element is set up to prevent registration since the user management should be done solely by Nextcloud.

📄/var/www/matrix.example.org/config.json
{
    "default_server_config": {
        "m.homeserver": {
            "base_url": "https://example.org",
            "server_name": "example.org"
        },},
    "disable_custom_urls": true,
    "disable_guests": true,
    "disable_3pid_login": true,"brand": "Example.org","roomDirectory": {
        "servers": [
            "example.org"
        ]
    },}

Element can be customized with a favicon 📁/var/www/matrix.example.org/vector-icons and a background image, by replacing 📁/var/www/matrix.example.org/themes/element/img/backgrounds/lake.jpg with another image called lake.jpg.

Nextcloud web admin config

To set up Nextcloud as an OpenID / OAuth identity provider go to your Nextcloud admin account and open Settings > Admin > Security to add a new OAuth client. The redirect URL should correspond to your Nginx settings for the Matrix API above.

🔗https://nextcloud.example.org/settings/admin/security
OAuth 2.0 clients
Redirection URI     https://example.org/_synapse/client/oidc/callback/

On creations the client ID and secret are generated. They will be needed in the Synapse server configuration below.

With the update to synapse v1.27.0 the redirect URL changed from _synapse/oidc/callback/ to _synapse/client/oidc/callback/.

To show a link to the Matrix client in Nextcloud menu use the External sites app. Configure it under Settings > Admin > External.

"Chat"  "https://matrix.example.org"
…
Redirect: ✅ This site does not allow embedding

In Nextcloud a Matrix client can only be embedded as an iframe (as far as I know, I tried the External sites app and the Element app ). This however is causing the OAuth handshake to fail, due to the browser security restrictions. Therefore the Matrix client must run in a seperate tab when using OAuth.

Mapping the user identity data

Nextcloud and Synapse do not have the same data structure for user identity information. A mapping plugin is needed that can be used by Synapse to translate the identity data. Create the following Python script.

(File path: your Python version and Synapse environment may be different)

📄/opt/venvs/matrix-synapse/lib/python3.8/site-packages/nextcloud_oidc_mapping_provider.py
from synapse.handlers.oidc_handler import OidcMappingProvider
class NextcloudOidcMappingProvider(OidcMappingProvider):
        def __init__(self, config):
                self._config = config

        @staticmethod
        def parse_config(config):
                return {}

        def get_remote_user_id(self, userinfo):
                return userinfo["ocs"]["data"]["id"]

        async def map_user_attributes(self, userinfo, token, failures):
                localpart = userinfo["ocs"]["data"]["id"]
                if failures: # if the user id exists add a suffix
                        localpart += '-rdp'p
                display_name = userinfo["ocs"]["data"]["display-name"]
                return {"localpart": localpart, "display_name": display_name}

        async def get_extra_attributes(self, userinfo, token):
                extras = {}
                return extras

Matrix Synapse server config

Configure the server to only be reachable on localhost / 127.0.0.1 (since Nginx exposes the server to the outside), and add the OpenID Connect / OAuth settings to the Matrix server config.

📄/etc/matrix-synapse/homeserver.yaml
listeners:
  - port: 8008bind_addresses: ['::1', '127.0.0.1']oidc_config:
    enabled: true
    client_id: "XXXXXXXXXXXX"
    client_secret: "XXXXXXXXXXXX"
    scopes: ["profile", "email"]
    issuer: "https://nextcloud.example.org"
    discover: false
    authorization_endpoint: "https://nextcloud.example.org/index.php/apps/oauth2/authorize"
    userinfo_endpoint: "https://nextcloud.example.org/ocs/v2.php/cloud/user?format=json"
    token_endpoint: "https://nextcloud.example.org/index.php/apps/oauth2/api/v1/token"
    user_mapping_provider:
        module: nextcloud_oidc_mapping_provider.NextcloudOidcMappingProvider
        config: {}

The configuration above corresponds to the mapping plugin file name nextcloud_oidc_mapping_provider and the class name in the file NextcloudOidcMappingProvider.

PostgreSQL Database

Postgres is not necessary for a basic setup, it is however needed by extensions like the Signal bridge. To Install Postgres run:

sudo apt update
sudo apt install postgresql postgresql-contrib

By switching to the Postgres user you change to an extended enviornment that provides additional commands like psql and createuser

su - postgres
createuser --pwprompt synapse_user

Store the password from createuser in your password database for the next step. Create a database for matrix, Synapse will create the necessary tables.

psql
> CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse_user;

Enable Postgres in the Synapse config.

📄/etc/matrix-synapse/homeserver.yaml
database:
    name: psycopg2
    args:
        user: synapse_user
        password: XXXXXXXXXXXXXXXX
        database: synapse
        host: localhost
        cp_min: 5
        cp_max: 10

Creating Matrix users

Creating a user is not necessary since Synapse is configured to only allow logins via OIDC / OAuth. Users will be automatically created once they login over OAuth for the first time.

I had some troubles logging in initially, when the first user was not a Nextcloud admin user. After the admin user was initialized everything started to work just fine for other users.

Starting Matrix Synapse

Use the Systemd tools to start the matrix-synapse service and enable it to automatically (re)start it on boot and crashes.

sudo systemctl start matrix-synapse
sudo systemctl enable matrix-synapse

Verifying the installation / troubleshooting

Show the status of the Synapse service.

sudo systemctl status matrix-synapse

Show if it's listening on the port you specified in your Nginx reverse proxy config.

sudo ss -plntu

Show the log messages, in case the OIDC python plugin throws errors etc.

sudo journalctl -u matrix-synapse -b -e

Restart the Synapse service.

sudo systemctl restart matrix-synapse.service

After changing domain names for the Matrix API the matrix-synapse service would not start. To start it again I deleted the database and recreated it.

su - postgres
psql
> DROP DATABASE synapse;
> CREATE DATABASE synapse … ;

Changing the domain name also means reconfiguring Element, Synapse, and Nextcloud.

The server name of Synapse can be changed in the following file after installation: 📄/etc/matrix-synapse/conf.d/server_name.yaml.

Sauce

Discussion

Let me know if you have feedback to this article on Mastodon.