Originally published on 30 July 2021
Last updated on 15 August 2021
Contents
============================================
UPDATE from 15 August 2021:
Please see my updated post on how to create an XMPP server with Prosody which is now based on documentation directly from the project. Some parts of this original post still apply and are referenced in the updated article.
============================================
UPDATE from 07 August 2021:
I’ve now written a little bit more on the feedback I received to my original post and how this serves as a great example of how to run an open source project.
============================================
UPDATE from 31 July 2021:
Shortly after sharing this post on Mastodon, I received some really good feedback on how to improve many of the steps regarding the installation of Prosody, especially on how to better handle the private key and certificate to improve security. Therefore, I would urge people interested in installing Prosody to refer directly to the excellent documentation on their site.
I hope to provide additional details on how to properly install Prosody in the near future.
The process to implement Nginx with Let’s Encrypt should still be accurate.
============================================
For the past few days, I’ve been playing around with the XMPP protocol by creating accounts on various services, installing a number of desktop and iOS clients and generally poking around some chatrooms. Even though I don’t personally know anyone with an XMPP account, I’ve been intrigued at how it works and operates. XMPP is not a new protocol, having been introduced over 20 years ago originally as Jabber. XMPP works as a decentralized, federated network, similar to email, Mastodon or Matrix. In terms of features, capabilities, and weight, it sits somewhere between IRC, which I really like, and Matrix, which I really want to like. Assuming that I’ll be able to convince some friends or family members to try XMPP or, even as just a technical exercise, I thought I would try to host my own XMPP server.
Like most things open source, there are a number of choices available when it comes to running an XMPP server. Some of the more popular options include ejabberd and Openfire but for the purposes of what I’m trying to do, I went with Prosody. The documentation for all of these projects is well done but I just felt that Prosody was a better fit for my very limited use case.
Most of what I document below is based on two excellent Digital Ocean tutorials. The first one is by Elliot Cooper on how to install Prosody on Ubuntu. Because I also wanted to have the same VPS serve web content on the same domain that I’d be using for the XMPP server, I also used Erin Glass’s tutorial on configuring Nginx on Ubuntu. If what you read here doesn’t make sense or is incorrect, that is solely my fault and would recommend you consult back to both of these great articles. With that out of the way, let’s get started!
Before doing anything in the Ubuntu 20.04 VPS I was using, I first had to configure some DNS records to point my domain to the IP address of the VPS. Using an Azure DNS zone, I configured the following settings:
Name: @
Type: A
TTL: 3600 (seconds)
Value: IP AddressFor now, I have just settled on using DNS A records but want to look further into how to use SRV records.
I also ensured that port 80 to allow HTTP traffic was open in the Networking tab of my VPS:
Source: Any
Source port ranges: *
Destination: Any
Service: HTTP
Destination port ranges: 80
Protocol: TCP
Action: Allow
Priority: 200
Name: HTTP
Description: HTTPPort 22, for ssh was already open.
As I mentioned earlier, I want use the example.com domain both as the domain for any XMPP addresses (e.g. user1@example.com) as well as the domain from which to serve web pages. These pages could be about the XMPP server itself, what it’s used for, or who can join. There are probably better ways to set this up than what I do here which is to just install an Nginx server on the same VPS as the XMPP server. For example, you could host the web server and XMPP server in separate containers or even separate VPSs and then use DNS to tie everything together. Perhaps at one point I will go down one of these routes but for now, based on my limited skillset, I’ve decided to run everything together in one place.
Install Nginx using the command:
$ sudo apt install nginx
Next, create the directory from where you want to serve any web pages associated with your domain and change the ownership and permissions so that you can write to them:
$ sudo mkdir -p /var/www/example.com/html $ sudo chown -R $USER:$USER /var/www/example.com/html $ sudo chmod -R 755 /var/www/example.com
Create a sample index.html page at /var/www/example.com/html/index.html.
In order for Nginx to serve this content, it’s necessary to create a server block with the correct directives. Instead of modifying the default configuration file directly, make a new one at /etc/nginx/sites-available/example.com:
$ sudo vim /etc/nginx/sites-available/example.com
Paste in the following configuration block, which is similar to the default, but updated for our new directory and domain name:
server {
listen 80;
listen [::]:80;
root /var/www/example.com/html;
index index.html index.htm index.nginx-debian.html;
server_name example.com;
location / {
try_files $uri $uri/ =404;
}
}For now, our web server will listen only on port 80 for HTTP. Later on, we’ll install a Let’s Encrypt certificate to allow us to serve content over port 443 using HTTPS.
Enable the file by creating a link from it to the sites-enabled directory, which Nginx reads from during startup:
$ sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
To avoid a possible hash bucket memory problem that can arise from adding additional server names, it is necessary to adjust a single value in the /etc/nginx/nginx.conf file. Open the file:
$ sudo vim /etc/nginx/nginx.conf
Find the server_names_hash_bucket_size directive and remove the # symbol to uncomment the line.
...
http {
...
server_names_hash_bucket_size 64;
...
}
...Save and close the file when you are finished.
Next, test to make sure that there are no syntax errors in any of your Nginx files:
$ sudo nginx -t
If there aren’t any problems, restart Nginx to enable your changes:
$ sudo systemctl restart nginx
Test the configuration by going to http://example.com to see if Nginx can serve your page. Depending on your browser, you may get a warning that the site is not secure. We’ll fix that in the next section.
Update VPS Networking details to allow HTTPS traffic via port 443:
Source: Any
Source port ranges: *
Destination: Any
Service: HTTPS
Destination port ranges: 443
Protocol: TCP
Action: Allow
Priority: 200
Name: HTTPS
Description: HTTPSInstall certbot and request your certificate:
$ sudo apt install certbot python3-certbot-nginx $ sudo certbot --nginx -d example.com
If all goes well, you should see a “Congratulations” message indicating that your certificates were correctly installed. Check that you can now reach your site at https://example.com with the lock icon displayed in your browser’s address bar.
Let’s Encrypt’s certificates are only valid for 90 days. This is to encourage users to automate their certificate renewal process. The certbot package we installed takes care of this for us by adding a systemd timer that will run twice a day and automatically renew any certificate that’s within 30 days of expiration.
You can query the status of the timer with systemctl:
$ sudo systemctl status certbot.timer
You should see output similar to this:
● certbot.timer - Run certbot twice daily
Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled)
Active: active (waiting) since Sun 2021-07-25 21:11:25 UTC; 4 days ago
Trigger: Fri 2021-07-30 23:22:09 UTC; 4h 18min left
Triggers: ● certbot.service
Jul 25 21:11:25 hostname systemd[1]: Started Run certbot twice daily.Now let’s get onto the fun part of installing our XMPP server.
I elected to install the most current version of Prosody using their APT repository:
$ echo deb http://packages.prosody.im/debian $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list $ wget https://prosody.im/files/prosody-debian-packages.key -O- | sudo apt-key add - $ sudo apt update $ sudo apt install prosody prosody-modules lua-dbi-sqlite3 lua-event
Prosody uses TLS certificates to encrypt the connections between the server and the clients. These certificates are the same ones that you use any time you visit a website with an HTTPS URL. We already obtained our Let’s Encrypt TLS certificates from a prior step; we just now need to configure a few things to allow Prosody to use them.
Change the group owner of the private keys to the Prosody server’s group prosody with the following commands:
$ sudo chgrp prosody /etc/letsencrypt/live/example.com/privkey.pem
The chgrp utility changes the group owner of files and directories. Here, we changed the group from the default root to prosody. To verify permission change:
$ sudo readlink -f /etc/letsencrypt/live/example.com/privkey.pem /etc/letsencrypt/archive/example.com/privkey1.pem $ sudo ls -all /etc/letsencrypt/archive/example.com/privkey1.pem -rw------- 1 root prosody 1704 Jul 21 13:59 /etc/letsencrypt/archive/example.com/privkey1.pem
For the next few steps, I’m going to copy almost verbatim from Elliot’s Digital Ocean Tutorial because of how critical the steps are.
Change the permissions of the directories that contain the TLS certificate files to 0755. These directories are owned by the root user and the root group. The following commands will change the permissions on these directories:
$ sudo chmod 0755 /etc/letsencrypt/archive $ sudo chmod 0755 /etc/letsencrypt/live
The new permissions of 0755 on these directories mean that the root user has read, write, and execute permissions. Members of the root group have read and execute permissions. All other users and groups on the system have read and execute permissions.
Now, change the permissions of the TLS private key:
$ sudo chmod 0640 /etc/letsencrypt/live/example.com/privkey.pem
The chmod utility modifies which users and groups have read, write, and execute permissions on files and directories.
The 0640 permissions on these files mean that the root user has read and write permissions on the file. Members of the prosody group have read permissions on the file. The prosody group has one member, the prosody user. This is the user that the Prosody server runs as and the user it will access the file as. All other users on the system have no permission to access the file.
You can test that Prosody can read the private keys by using sudo to read the private key files with cat as the prosody user:
$ sudo -u prosody cat /etc/letsencrypt/live/example.com/privkey.pem
If this is successful then you will see the contents of the key file displayed on your screen.
Prosody uses a single file containing the certificate and private key to encrypt the file upload and download connections. This file is not created by certbot automatically so you must create it manually.
You will first move into the directory that contains the key and certificate files, then use cat to combine their contents into a new file key-and-cert.pem:
$ cd /etc/letsencrypt/live/example.com/ $ sudo sh -c 'cat privkey.pem fullchain.pem >key-and-cert.pem'
The beginning of this command, sudo sh -c, opens a new sub-shell that has root user’s permissions and so can write the new file to /etc/letsencrypt/live/example.com/.
Now, change the group and permissions of this new file to match the group and permission that you set for the other private key file with the following commands:
$ sudo chmod 0640 key-and-cert.pem $ sudo chgrp prosody key-and-cert.pem
Finally, this file must be re-created every time the certificate is renewed or it will contain an expired certificate.
Certbot comes with a mechanism called a “hook” that allows a script to be run before or after a certificate is renewed. You can use this mechanism to run a script that will re-create the command you ran after every certificate renewal.
Open the new script file called /etc/letsencrypt/renewal-hooks/post/key-and-cert-combiner.sh with a text editor:
$ sudo vim /etc/letsencrypt/renewal-hooks/post/key-and-cert-combiner.sh
Then, add the following lines into the file:
#!/usr/bin/env bash
set -e
# combines the certificate and key into a single file with cat
cat /etc/letsencrypt/live/example.com/privkey.pem \
/etc/letsencrypt/live/example.com/fullchain.pem \
>/etc/letsencrypt/live/example.com/key-and-cert.pemChange the script’s permission to allow it be an executable:
$ sudo chmod +x /etc/letsencrypt/renewal-hooks/post/key-and-cert-combiner.sh
Next, test that the certificates are installed correctly and that the post-renewal hook script is working by running the following certbot command:
$ sudo certbot renew --dry-run
This command tells certbot to renew the certificates but with the --dry-run option that stops certbot from making any changes. If everything is successful then you will see the following output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for example.com
Waiting for verification...
Cleaning up challenges
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/example.com/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/example.com/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Running post-hook command: /etc/letsencrypt/renewal-hooks/post/key-and-cert-combiner.shNow we can edit Prosody’s main configuration file.
I first made a backup of the original configuration file for reference and then opened up the file to make changes:
$ sudo cp /etc/prosody/prosody.cfg.lua /etc/prosody/prosody.cfg.lua.original $ sudo vim /etc/prosody/prosody.cfg.lua
Within the file, I updated the admins line to include myself as a server administrator:
admins = { "samir@example.com" }
In the modules_enabled section, I uncommented the following modules to enable them:
mam: Stores chat messages on the server so users can retrieve them.csi_simple: Enables optimizations for mobile clients.http_files: Enables file sharing.groups: Enables user visibility.announce: Enables admins to send announcements to all users.motd: message of the dayThere are more modules that you can configure but for now, these are the only ones I have selected.
To select SQLite as the message database, enable the following two lines by removing the leading -- as shown following:
...
storage = "sql" -- Default is "internal"
...
sql = { driver = "SQLite3", database = "prosody.sqlite" } -- Default. 'database' is the filename.
...You can decide how long the server will store old chat messages by editing the following line:
...
archive_expires_after = "1w" -- Remove archived messages after 1 week
...The default period is 1w (one week). Use d for days, w for weeks, and y for years.
The https_certificate line tells Prosody where to look for the combined certificate and key we created earlier to use for file transfers. Edit it so that it uses the path to our combined file:
...
https_certificate = "/etc/letsencrypt/live/example.com/key-and-cert.pem"
...In the default configuration, Prosody listens on localhost or 127.0.0.1 for chat connections. This is not needed on a remote server. Disable this behavior by adding -- to the line so that it looks like the following after editing:
...
--VirtualHost "localhost"
...The groups module that we enabled in the modules section allows chat clients to see each other. The groups module reads a file that holds the group names and their members. Set the location and name of the file by adding the following lines to the bottom of the configuration:
...
-- The groups module reads a file that holds the group names and their members.
-- This line configures Prosody to read a file to gather group information.
groups_file = "/etc/prosody/sharedgroups.txt"
...This line configures Prosody to read a file at /etc/prosody/sharedgroups.txt to gather group information. We will populate this file with users and groups in a subsequent step.
Create this file with the following command in a different terminal:
$ sudo touch /etc/prosody/sharedgroups.txt
The touch utility creates an empty file when no additional options are used.
Prosody uses a block of configuration that begins with VirtualHost to start the chat server that uses our hostname. Add the following configuration block to the bottom of the configuration:
...
-- Prosody uses a block of configuration that begins with VirtualHost to
-- start the chat server that uses your hostname.
VirtualHost "example.com"
ssl = {
key = "/etc/letsencrypt/live/chat234.kt/privkey.pem";
certificate = "/etc/letsencrypt/live/example.com/fullchain.pem";
}
...Restart the service for the configuration changes to take effect:
$ sudo systemctl restart prosody.service
Now, we can create our first XMPP user:
$ sudo prosodyctl register username example.com Enter new password: Retype new password:
Edit the shared group files we created earlier to add our new user:
$ sudo vim /etc/prosody/sharedgroups.txt
Add the following lines:
[Everyone]
username@example.comRestart the service once more:
$ sudo systemctl restart prosody.service
In addition to the standard ports 80 and 443 that we opened earlier for HTTP and HTTPS, respectively, Prosody also listens on a number of additional ports which we may also have to open. To verify which ones we really need, I ran the following commands:
$ sudo apt install net-tools $ sudo netstat -lnptu | grep lua tcp 0 0 0.0.0.0:5269 0.0.0.0:* LISTEN 160841/lua5.2 tcp 0 0 0.0.0.0:5280 0.0.0.0:* LISTEN 160841/lua5.2 tcp 0 0 0.0.0.0:5281 0.0.0.0:* LISTEN 160841/lua5.2 tcp 0 0 0.0.0.0:5222 0.0.0.0:* LISTEN 160841/lua5.2 tcp6 0 0 :::5269 :::* LISTEN 160841/lua5.2 tcp6 0 0 :::5280 :::* LISTEN 160841/lua5.2 tcp6 0 0 :::5281 :::* LISTEN 160841/lua5.2 tcp6 0 0 :::5222 :::* LISTEN 160841/lua5.2
As you can see from the output, we also need to open ports 5222, 5269, 5280 and 5281. In the Azure control panel, these are the settings I used in the VPS Networking tab:
Source: Any
Source port ranges: *
Destination: Any
Service: Custom
Destination port ranges: 5222
Protocol: TCP
Action: Allow
Priority: 225
Name: Client_connections
Description: Prosody XMPP Client Connections
Source: Any
Source port ranges: *
Destination: Any
Service: Custom
Destination port ranges: 5269
Protocol: TCP
Action: Allow
Priority: 230
Name: Server-to-server_connections
Description: Prosody XMPP Server-to-server connections
Source: Any
Source port ranges: *
Destination: Any
Service: Custom
Destination port ranges: 5280
Protocol: TCP
Action: Allow
Priority: 235
Name: Prosody_HTTP
Description: Prosody HTTP
Source: Any
Source port ranges: *
Destination: Any
Service: Custom
Destination port ranges: 5281
Protocol: TCP
Action: Allow
Priority: 240
Name: Prosody_HTTPS
Description: Prosody HTTPSThat pretty much wraps up our installation! Now all we have to do is connect to the server using our client of choice.
Because XMPP is an open protocol, there are a number of clients you can choose from to connect to the server, chat with people, or join multi-user chatrooms. On a Linux desktop, the venerable chat client Pidgin is a solid choice if you’re already using it for things like IRC. Or, you can go for a more XMPP-specific client such as Gajim, which has an interface similar to Pidgin, or something more modern like Dino. Personally, I’ve been using the terminal-based client Profanity which serves my needs just fine.
On iOS, I’ve been sticking to Monal while also trying out Siskin IM and ChatSecure. If you run Android on your mobile phone, Conversations appears to be the go-to app.
This is about as far as I’ve gotten so far based on my limited sysadmin skills and knowledge of XMPP. Looking ahead, there are a few things I’d like to address or otherwise clean up:
Look into using SRV records instead of just A record. As I mentioned above, I think the way that I have installed Nginx and Prosody on the same server using the same TLS certificates seems, for a lack of a better word, wrong. I’m not familiar with containers but may look into running these services on separate servers with the web server at example.com and the XMPP server at chat.example.com and then tying them together via DNS records somehow.
Improve my servers’s compliance score. There are a few sites out there that check your server’s compliance to the various XMPP protocols and specifications. Two of the more popular ones are compliance.conversations.im and xmpp.net. I’m not looking for perfect scores or complete coverage (since this server is just for personal use) but it would be nice to learn how to get it more compliant.
Active some multi-user chat (MUC) rooms. These rooms, similar to IRC channels allow multiple people to chat with each other. This way, you could invite friends or family members into a room to allow everyone to converse with each other.
Figure out Omemo encryption. Or rather, encryption in general. So far, everything I have configured only allows me to chat in plain text. I’ve tried experimenting with turning on encryption with various clients using verbose keys but I’ve not yet been successful. I guess this is why IRC has been able to remain such a simple protocol as it doesn’t even bother dealing with encryption (and why the Matrix client Element is such a pain to use, because it does try dealing with end-to-end encryption, backing up keys, etc.)
I am still working on my own individual setup and testing it out. Once it’s ready for “production”, I’ll share my XMPP address! In the meantime, I would love to hear how all of you are using XMPP!