NestJS
on systemd

Everything in this guide runs on one server. SSH in as root or your cloud user once, then do the rest as a normal user with sudo. We use two accounts on purpose: deploy is the human who logs in and ships code, letterpad is the system user the app runs as. The app user cannot log in and owns nothing it does not need.

Patch the box and set the clock

Update packages first, then pin the timezone so log timestamps match your team. Unattended upgrades keep security patches flowing without you.

sudo apt update && sudo apt full-upgrade -y
sudo apt install -y curl git build-essential ca-certificates unattended-upgrades
sudo timedatectl set-timezone Asia/Kolkata
sudo dpkg-reconfigure -plow unattended-upgrades

$ timedatectl | grep "Time zone"
                Time zone: Asia/Kolkata (IST, +0530)

Create the two users

deploy gets a home, a shell and sudo. letterpad is a --system account with no home and nologin as its shell, so even a leaked password opens nothing.

# You: a normal login user with sudo
sudo adduser deploy
sudo usermod -aG sudo deploy
sudo mkdir -p /home/deploy/.ssh
sudo cp ~/.ssh/authorized_keys /home/deploy/.ssh/
sudo chown -R deploy:deploy /home/deploy/.ssh && sudo chmod 700 /home/deploy/.ssh

# The app: a system user that can never log in
sudo useradd --system --no-create-home --shell /usr/sbin/nologin letterpad

$ id letterpad
uid=998(letterpad) gid=998(letterpad) groups=998(letterpad)

Open a second terminal and confirm ssh deploy@your-server works before you close the root session. From here on, every command runs as deploy.

Ubuntu ships an old Node, so take the current LTS from NodeSource. The three data services come straight from Ubuntu's own repository, and each one installs as a systemd service that starts on boot by itself.

Node.js

curl -fsSL https://deb.nodesource.com/setup_24.x -o /tmp/nodesource_setup.sh
sudo -E bash /tmp/nodesource_setup.sh
sudo apt install -y nodejs

$ which node && node -v
/usr/bin/node
v24.x.x

Note the path /usr/bin/node. The systemd unit in module 05 calls Node by its full path, because services do not read your shell PATH.

PostgreSQL, Redis and RabbitMQ

sudo apt install -y postgresql redis-server rabbitmq-server
sudo systemctl enable --now postgresql redis-server rabbitmq-server

$ systemctl list-units --type=service "postgresql*" redis-server rabbitmq-server
  UNIT                       LOAD   ACTIVE SUB     DESCRIPTION
  postgresql.service         loaded active exited  PostgreSQL RDBMS
  postgresql@16-main.service loaded active running PostgreSQL Cluster 16-main
  rabbitmq-server.service    loaded active running RabbitMQ Messaging Server
  redis-server.service       loaded active running Advanced key-value store

Two PostgreSQL units show up. postgresql.service is an umbrella that only exits; the database itself is postgresql@16-main.service. Remember that name, the app unit waits on it.

Each service gets its own credentials for letterpad.io and nothing else. Generate passwords with openssl rand -hex 24 so they contain no characters that need URL escaping later.

PostgreSQL: a role and a database

The role owns its database and nothing more. PostgreSQL on Ubuntu listens on localhost by default, so there is nothing to change in postgresql.conf.

sudo -u postgres psql -c "CREATE ROLE letterpad WITH LOGIN PASSWORD 'CHANGE_ME_PG';"
sudo -u postgres createdb --owner=letterpad letterpad

$ psql "postgresql://letterpad:CHANGE_ME_PG@127.0.0.1:5432/letterpad" -c "select 1;"
 ?column? 
----------
        1
(1 row)

Redis: bind, password, memory policy

Open /etc/redis/redis.conf with sudo nano and set these lines. Each one already exists in the file, so search for it and edit in place.

bind 127.0.0.1 -::1
protected-mode yes
supervised systemd
requirepass CHANGE_ME_REDIS
maxmemory 256mb
maxmemory-policy noeviction

sudo systemctl restart redis-server
REDISCLI_AUTH=CHANGE_ME_REDIS redis-cli ping

$ REDISCLI_AUTH=CHANGE_ME_REDIS redis-cli ping
PONG

RabbitMQ: a vhost, a user, no guest

A vhost is RabbitMQ's namespace, like a database in PostgreSQL. Give letterpad.io its own, then delete the default guest account.

sudo rabbitmqctl add_vhost letterpad
sudo rabbitmqctl add_user letterpad 'CHANGE_ME_MQ'
sudo rabbitmqctl set_permissions -p letterpad letterpad ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest

$ sudo rabbitmqctl list_users
Listing users ...
user       tags
letterpad  []

Then pin the AMQP listener to localhost in /etc/rabbitmq/rabbitmq.conf (create the file if it is missing) and restart.

listeners.tcp.default = 127.0.0.1:5672

$ sudo systemctl restart rabbitmq-server && sudo ss -ltnp | grep 5672
LISTEN 0  128  127.0.0.1:5672  0.0.0.0:*  users:(("beam.smp",pid=1312,fd=35))

Two small changes in the app first

systemd stops a service by sending SIGTERM. NestJS ignores it unless you turn on shutdown hooks, and then it closes your PostgreSQL pool, Redis client and RabbitMQ channel before exiting. Listen on 127.0.0.1 so the only way in is through nginx.

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.enableShutdownHooks();                       // run onModuleDestroy on SIGTERM
  app.getHttpAdapter().getInstance().set('trust proxy', 'loopback'); // real client IP from nginx
  await app.listen(Number(process.env.PORT ?? 3000), process.env.HOST ?? '127.0.0.1');
}
bootstrap();

Add a tiny health route. nginx, the deploy script and any uptime monitor will all call it. For real dependency checks, @nestjs/terminus can ping PostgreSQL, Redis and RabbitMQ from the same route.

import { Controller, Get } from '@nestjs/common';

@Controller('health')
export class HealthController {
  @Get()
  check() {
    return { status: 'ok', uptime: Math.round(process.uptime()) };
  }
}

Clone and build on the server

The code lives in /srv/letterpad, owned by deploy and readable by the letterpad group. Adding deploy to that group lets your deploy script read the env file later.

sudo mkdir -p /srv/letterpad
sudo chown deploy:letterpad /srv/letterpad
sudo usermod -aG letterpad,systemd-journal deploy   # log out and back in after this

git clone git@github.com:your-org/letterpad.io.git /srv/letterpad
cd /srv/letterpad
npm ci
npm run build
npm prune --omit=dev

$ ls /srv/letterpad/dist/main.js
/srv/letterpad/dist/main.js

Secrets in one env file

systemd reads this file with EnvironmentFile=. One KEY=value per line, no export, no quotes needed. It is owned by root and readable by the letterpad group, so the app can read it and nobody else can.

NODE_ENV=production
HOST=127.0.0.1
PORT=3000
DATABASE_URL=postgresql://letterpad:CHANGE_ME_PG@127.0.0.1:5432/letterpad
REDIS_URL=redis://:CHANGE_ME_REDIS@127.0.0.1:6379/0
RABBITMQ_URL=amqp://letterpad:CHANGE_ME_MQ@127.0.0.1:5672/letterpad

sudo install -d -m 750 -o root -g letterpad /etc/letterpad
sudo nano /etc/letterpad/letterpad.env
sudo chown root:letterpad /etc/letterpad/letterpad.env
sudo chmod 640 /etc/letterpad/letterpad.env

$ ls -l /etc/letterpad/
-rw-r----- 1 root letterpad 312 Oct  9 10:42 letterpad.env

Run the migrations once by hand now, with the env file loaded into this shell only: set -a; source /etc/letterpad/letterpad.env; set +a; npm run migration:run. Use whatever migration script your ORM gives you.

Create the file with sudo nano /etc/systemd/system/letterpad.service. The name before .service becomes the name you use with every systemctl command.

[Unit]
Description=letterpad.io NestJS API
After=network-online.target postgresql@16-main.service redis-server.service rabbitmq-server.service
Wants=network-online.target postgresql@16-main.service redis-server.service rabbitmq-server.service
StartLimitIntervalSec=300
StartLimitBurst=5

[Service]
Type=simple
User=letterpad
Group=letterpad
WorkingDirectory=/srv/letterpad
EnvironmentFile=/etc/letterpad/letterpad.env
ExecStart=/usr/bin/node dist/main.js
Restart=always
RestartSec=5
KillSignal=SIGTERM
TimeoutStopSec=30
SyslogIdentifier=letterpad
LimitNOFILE=65535
StateDirectory=letterpad
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true

[Install]
WantedBy=multi-user.target

Line by line: [Unit]

Line by line: [Service]

Line by line: [Install]

Load it, enable it, start it

sudo systemd-analyze verify /etc/systemd/system/letterpad.service
sudo systemctl daemon-reload
sudo systemctl enable --now letterpad
systemctl status letterpad

$ systemctl status letterpad
● letterpad.service - letterpad.io NestJS API
     Loaded: loaded (/etc/systemd/system/letterpad.service; enabled; preset: enabled)
     Active: active (running) since Fri 2026-10-09 10:51:07 IST; 4s ago
   Main PID: 2241 (node)
      Tasks: 11 (limit: 4558)
     Memory: 92.4M (peak: 95.1M)
     CGroup: /system.slice/letterpad.service
             └─2241 /usr/bin/node dist/main.js

Oct 09 10:51:08 web-1 letterpad[2241]: [Nest] LOG [NestApplication] Nest application successfully started

verify prints nothing when the file is clean. daemon-reload makes systemd read the new file, and you run it again after every edit. enable --now both starts the service and turns on start at boot.

Prove the auto restart works

Kill the process the hard way, the way an out of memory kill would. Within five seconds systemd brings it back with a new PID.

sudo systemctl kill --signal=SIGKILL letterpad
sleep 6
systemctl show letterpad -p MainPID -p NRestarts -p ActiveState

$ systemctl show letterpad -p MainPID -p NRestarts -p ActiveState
MainPID=2318
NRestarts=1
ActiveState=active

Now the reboot test: sudo reboot, SSH back in, and run systemctl is-active letterpad. It should print active without you touching anything.

systemd-analyze security letterpad

Change a setting with a drop-in

Never edit a unit that a package installed, and prefer drop-ins for your own too. systemctl edit opens an empty override file, saves it as /etc/systemd/system/letterpad.service.d/override.conf, and reloads systemd for you.

sudo systemctl edit letterpad

[Service]
# Give a slow shutdown more time and cap memory
TimeoutStopSec=60
MemoryMax=768M

Then sudo systemctl restart letterpad and check systemctl cat letterpad shows both files.

Nest's logger, console.log and any crash stack trace go to stdout or stderr. systemd captures both, tags them letterpad, and stores them in the journal. No log files, no rotation scripts for the app. For production, log JSON lines with a logger such as pino so you can filter by field later.

Read the logs

Set retention once

Ubuntu keeps the journal on disk already, but its limits are percentages of the disk. Set explicit ones in a drop-in rather than editing the main journald.conf.

[Journal]
Storage=persistent
Compress=yes
SystemMaxUse=2G
SystemKeepFree=4G
SystemMaxFileSize=128M
MaxRetentionSec=30day
MaxFileSec=1week
RateLimitIntervalSec=30s
RateLimitBurst=20000

sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/retention.conf
sudo systemctl restart systemd-journald
journalctl --disk-usage

$ journalctl --disk-usage
Archived and active journals take up 184.0M in the file system.

Clean up by hand when you need to

sudo journalctl --vacuum-time=14d    # drop everything older than 14 days
sudo journalctl --vacuum-size=500M   # shrink archived files to 500 MB
sudo journalctl --verify             # check the files are not corrupted

nginx is the exception: it writes its own files under /var/log/nginx/, and Ubuntu's /etc/logrotate.d/nginx already rotates them daily and keeps 14. Change rotate 14 there if you need longer.

Point the DNS A records for letterpad.io and www.letterpad.io at the server first. Certbot needs them to prove you own the domain.

Install nginx and get a certificate

certonly fetches the certificate without touching your config, and the deploy hook reloads nginx after every renewal so the new certificate is picked up.

sudo apt install -y nginx certbot python3-certbot-nginx
sudo ufw allow 'Nginx Full'          # module 09 explains this
sudo certbot certonly --nginx -d letterpad.io -d www.letterpad.io \
  --deploy-hook "systemctl reload nginx"

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/letterpad.io/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/letterpad.io/privkey.pem
Certbot has set up a scheduled task to automatically renew this certificate in the background.

The site config

Create /etc/nginx/sites-available/letterpad.io with this content.

upstream letterpad_api {
    server 127.0.0.1:3000;
    keepalive 32;
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      '';
}

server {
    listen 80;
    listen [::]:80;
    server_name letterpad.io www.letterpad.io;
    return 301 https://letterpad.io$request_uri;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name letterpad.io www.letterpad.io;

    ssl_certificate     /etc/letsencrypt/live/letterpad.io/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/letterpad.io/privkey.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 1d;

    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header X-Content-Type-Options "nosniff" always;
    client_max_body_size 10m;

    access_log /var/log/nginx/letterpad.access.log;
    error_log  /var/log/nginx/letterpad.error.log warn;

    location / {
        proxy_pass http://letterpad_api;
        proxy_http_version 1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade           $http_upgrade;
        proxy_set_header Connection        $connection_upgrade;
        proxy_connect_timeout 5s;
        proxy_read_timeout    60s;
        proxy_send_timeout    60s;
    }

    location = /health {
        proxy_pass http://letterpad_api;
        access_log off;
    }
}

Line by line

Enable the site and reload

sudo ln -s /etc/nginx/sites-available/letterpad.io /etc/nginx/sites-enabled/
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t
sudo systemctl reload nginx
curl -I https://letterpad.io/health

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
$ curl -I https://letterpad.io/health
HTTP/2 200
content-type: application/json; charset=utf-8
strict-transport-security: max-age=31536000

Always run nginx -t before reload. A reload with a broken config is refused and the old config keeps serving, but a restart with one takes the site down. Check renewal works with sudo certbot renew --dry-run.

ufw is a friendly front end to the kernel firewall. The order below matters: allow SSH before you enable the firewall, or you lock yourself out of the server.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw limit OpenSSH              # allow SSH, but slow down brute force
sudo ufw allow 'Nginx Full'         # 80 and 443
sudo ufw logging low
sudo ufw enable
sudo ufw status verbose

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)

To                         Action      From
--                         ------      ----
22/tcp (OpenSSH)           LIMIT IN    Anywhere
80,443/tcp (Nginx Full)    ALLOW IN    Anywhere
22/tcp (OpenSSH (v6))      LIMIT IN    Anywhere (v6)
80,443/tcp (Nginx Full (v6)) ALLOW IN    Anywhere (v6)

Check what is actually listening

The firewall is the second wall. The first is that the services only listen on localhost. ss shows both at once.

sudo ss -tlnp

$ sudo ss -tlnp
State   Local Address:Port   Process
LISTEN  0.0.0.0:22           sshd
LISTEN  0.0.0.0:80           nginx
LISTEN  0.0.0.0:443          nginx
LISTEN  127.0.0.1:3000       node
LISTEN  127.0.0.1:5432       postgres
LISTEN  127.0.0.1:6379       redis-server
LISTEN  127.0.0.1:5672       beam.smp
LISTEN  0.0.0.0:25672        beam.smp
LISTEN  0.0.0.0:4369         epmd
# 25672 and 4369 are public on the socket but blocked by ufw

Rules you will need later

If PostgreSQL, Redis or RabbitMQ are managed services instead, nothing changes here: outgoing traffic is already allowed. Remove their units from After= and Wants= in module 05.

#!/usr/bin/env bash
set -euo pipefail
cd /srv/letterpad

git fetch --prune origin
git reset --hard origin/main
npm ci
npm run build

set -a; source /etc/letterpad/letterpad.env; set +a
npm run migration:run
npm prune --omit=dev

sudo systemctl restart letterpad

for i in $(seq 1 20); do
  if curl -fsS http://127.0.0.1:3000/health > /dev/null; then
    echo "letterpad is healthy"; exit 0
  fi
  sleep 1
done

echo "health check failed"
journalctl -u letterpad -n 50 --no-pager
exit 1

Let deploy restart only this service

A sudoers rule lets the script restart letterpad without a password, and nothing else. Always edit sudoers with visudo: it refuses to save a file with a syntax error.

sudo visudo -f /etc/sudoers.d/letterpad-deploy

deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart letterpad, /usr/bin/systemctl status letterpad

$ chmod +x scripts/deploy.sh && ./scripts/deploy.sh
added 612 packages in 14s
> letterpad@1.4.0 build
> nest build
No migrations are pending
letterpad is healthy

A 502 from nginx almost always means Nest is not listening: check systemctl status letterpad first, then the journal. A crash loop that stops after five tries means the start limit fired, so fix the cause, run reset-failed, and start it again.