7. mTLS authentication

Used in this guide:
Next.js 15.3.4
Payload CMS
PostgreSQL
pgBouncer
DigitalOcean Droplet

Keep pgBouncer public, but require a client certificate to even complete TLS. Only holders of your client cert & key (your Vercel) can connect; everyone else gets dropped at handshake.


1. Setup Root CA

SSH into your droplet and make a new directory called mtls, run:

sudo mkdir -p /etc/pgbouncer/mtls

Give (or change) owner to the root postgres user, run:

sudo chown postgres:postgres /etc/pgbouncer/mtls

Change permissions of the directory

sudo chmod 750 /etc/pgbouncer/mtls

Generate private CA key, run:

sudo -u postgres openssl genrsa -out /etc/pgbouncer/mtls/ca.key 4096

Create self-signed CA certificate (for pgBouncer)

sudo -u postgres openssl req -x509 -new -nodes \
  -key /etc/pgbouncer/mtls/ca.key \
  -sha256 -days 3650 \
  -out /etc/pgbouncer/mtls/ca.crt \
  -subj "/CN=pgbouncer-client-ca-2025"	

Set security permission (for the key), run:

sudo chmod 600 /etc/pgbouncer/mtls/ca.key

Another security permission (for the cert), run:

sudo chmod 644 /etc/pgbouncer/mtls/ca.crt

Check/Verify the certificate, run:

openssl x509 -in /etc/pgbouncer/mtls/ca.crt -noout -subject -issuer -dates

You successfully created a self-signed Certificate Authority (CA) on your droplet. This CA is a set of files (ca.key and ca.crt) that will serve as the root of trust for your application. This is the first and most critical step in setting up Mutual TLS (mTLS) to secure your database connection from Vercel without opening the port to the public.




2. Issue the client certificate (for Vercel)

Generate client private key, run:

For a new project, change the "vercel-client.key" part to a different name.

sudo -u postgres openssl genrsa -out /etc/pgbouncer/mtls/vercel-client.key 2048

Create a CSR request for the client cert, run:

For a new project, change the "vercel-client.key", "vercel-client.csr", and "vercel-db-client-1" parts to different names.

sudo -u postgres openssl req -new \
  -key /etc/pgbouncer/mtls/vercel-client.key \
  -out /etc/pgbouncer/mtls/vercel-client.csr \
  -subj "/CN=vercel-db-client-1"

Create configuration file for the client certificate on your droplet, run:

Skip this process for a new project. We can reuse the old one.

sudo -u postgres tee /etc/pgbouncer/mtls/client-ext.cnf >/dev/null <<'EOF'
basicConstraints=CA:FALSE
keyUsage=digitalSignature
extendedKeyUsage=clientAuth
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
EOF

Use your CA's private key to sign the Certificate Signing Request (CSR) you just made, producing the vercel-client.crt file.

For a new project, change the "vercel-client.csr" and "vercel-client.crt" parts to different names.

sudo -u postgres openssl x509 -req \
  -in /etc/pgbouncer/mtls/vercel-client.csr \
  -CA /etc/pgbouncer/mtls/ca.crt -CAkey /etc/pgbouncer/mtls/ca.key -CAcreateserial \
  -out /etc/pgbouncer/mtls/vercel-client.crt \
  -days 825 -sha256 -extfile /etc/pgbouncer/mtls/client-ext.cnf

Lock down permissions

For a new project, change the "vercel-client.key" and "vercel-client.crt" parts to different names.

sudo chmod 600 /etc/pgbouncer/mtls/vercel-client.key
sudo chmod 644 /etc/pgbouncer/mtls/vercel-client.crt

Quick verify, run:

For a new project, change the "vercel-client.crt" part to a different names.

openssl x509 -in /etc/pgbouncer/mtls/vercel-client.crt -noout -subject -issuer -text | \
  egrep -i 'Subject:|Issuer:|Extended Key Usage|TLS Web Client Authentication'

What You've Accomplished so far:

  • You've created a unique private key (vercel-client.key) and a Certificate Signing Request (CSR) for your Vercel app.
  • You've used your "tiny CA's" private key to sign the CSR, which resulted in the final client certificate (vercel-client.crt). The Certificate request self-signature ok message and Subject and Issuer lines in your terminal output confirm that the certificate was created successfully and was signed by your CA.
  • You've set the correct permissions for the client key and certificate to ensure they are secure.
  • The final egrep command shows that the certificate is correctly configured for TLS Web Client Authentication, which is the crucial setting that tells pgBouncer this certificate is for a client connecting to a server.

For a new project, once you reach this point, skip to Copy the client cert & key to your Vercel and use them




3. Tell pgBouncer to require client cert

Heads-up: after this reload, connections without the client cert (e.g. your current Vercel deploy) will fail until we finish Step 4.

Edit pgBouncer config:

sudo nano /etc/pgbouncer/pgbouncer.ini

Under [pgbouncer], make sure you have these (keep your existing cert paths):

client_tls_cert_file = /etc/pgbouncer/pgbouncer.crt
client_tls_key_file  = /etc/pgbouncer/pgbouncer.key
 
# NEW: require client certs signed by our CA
client_tls_sslmode   = verify-ca
client_tls_ca_file   = /etc/pgbouncer/mtls/ca.crt
 
# Keep password auth as second factor
auth_type            = scram-sha-256
auth_file            = /etc/pgbouncer/userlist.txt

Reload pgBouncer:

sudo systemctl reload pgbouncer

Checking unauthorized connection (it should fail)

psql "host=127.0.0.1 port=6432 dbname=test_payload_auth_v3 user=test_payload_auth_admin_v3 sslmode=require" -c "select 1;"

You've successfully created all the necessary cryptographic files for your Vercel application to connect securely to your database using Mutual TLS (mTLS).




4. Copy the client cert & key to your Vercel and use them

Run and copy the cert result:

base64 -w0 /etc/pgbouncer/mtls/vercel-client.crt

Run and copy the key result:

base64 -w0 /etc/pgbouncer/mtls/vercel-client.key

Go to Vercel and add 2 environment variables called PGCLIENT_KEY_B64 and PGCLIENT_CERT_B64 and paste their corresponding content.

Do not add these variables to env local development. It contains sensitive information

Do not include the "==" part from the cert nor the "root@..." part from the cert and key. Even though Vercel is smart enough to exclude it but it's better not to include

In .env.local, (local dev) change DATABASE_URI_VERCEL by removing the sslmode=verify-full part.

DATABASE_URI_VERCEL=postgresql://<db_user>:<db_user_password>@<your_sub_domain>:6432/<db_name>

Reconfigure payload.config.ts

root/payload.config.ts
import sharp from 'sharp'
import { lexicalEditor } from '@payloadcms/richtext-lexical'
import { postgresAdapter } from '@payloadcms/db-postgres'
import { buildConfig } from 'payload'
import Users from './payload/collections/users'
import { nodemailerAdapter } from '@payloadcms/email-nodemailer'
import Profiles from './payload/collections/profiles'
 
const isDev = process.env.NODE_ENV === 'development';
 
const buildPool = () => {
  if (isDev) {
    return { connectionString: process.env.DATABASE_URI_LOCAL! };
  }
 
  // decode client cert/key from base64
  const cert = Buffer.from(process.env.PGCLIENT_CERT_B64!, 'base64');
  const key  = Buffer.from(process.env.PGCLIENT_KEY_B64!,  'base64');
 
  // parse DSN for host/port/db/user/pass (no sslmode here)
  const dsn = new URL(process.env.DATABASE_URI_VERCEL!);
  const host = dsn.hostname;
  const port = Number(dsn.port || '6432');
  const database = dsn.pathname.replace(/^\//, '');
  const user = decodeURIComponent(dsn.username);
  const password = decodeURIComponent(dsn.password);
 
  return {
    host,
    port,
    database,
    user,
    password,
    ssl: {
      cert,                 // Buffer — makes Node TLS actually send the client cert
      key,                  // Buffer
      rejectUnauthorized: true,
      servername: host,     // must match your LE cert (<your_sub_domain>)
      // ca: optional; LE is already trusted by Node
    },
  };
};
 
export default buildConfig({
  editor: lexicalEditor(),
 
  collections: [
    Users,
    Profiles,
  ],
  globals: [
 
  ],
  secret: process.env.PAYLOAD_SECRET || '',
  db: postgresAdapter({
    // Postgres-specific arguments go here.
    // `pool` is required.
    pool: buildPool(),
    schemaName: 'payload',
  }),
  sharp,
  email: nodemailerAdapter({
    defaultFromName: 'Your Website Name',
    defaultFromAddress: 'no-reply@yourdomain.com',
    transportOptions: {
      host: process.env.SMTP_HOST, // Your SMTP server host (e.g., smtp.gmail.com)
      port: parseInt(process.env.SMTP_PORT || "587"), // 587 for TLS, 465 for SSL
      secure: process.env.SMTP_SECURE === 'true', // Set to true if port is 465, false for 587 or 25
      auth: {
        user: process.env.SMTP_USER, // Your SMTP username (email)
        pass: process.env.SMTP_PASS, // Your SMTP password
      },
    },
  }),
  admin: {
    autoLogin: process.env.NODE_ENV === "development"
      ? {
        email: "<login_email_for_admin_panel>",
        password: "<password>",
        prefillOnly: true,
      }
      : false
  },
  // serverURL: process.env.NEXT_PUBLIC_ROOT_URL_PROD
})

Make sure all the env variables used in payload.config.ts match what's in .env.local

Be aware that we set SSL cert and key inside payload.config.ts and not in the connection string in .env.local

Final Check

Check if mTLS handshake is working

openssl s_client -starttls postgres \
  -connect <your_sub_domain>:6432 \
  -servername <your_sub_domain> \
  -cert /etc/pgbouncer/mtls/vercel-client.crt \
  -key  /etc/pgbouncer/mtls/vercel-client.key \
  -CAfile /etc/ssl/certs/ca-certificates.crt -brief

If you get CONNECTION ESTABLISHED and Verification: OK somehwere in the result, it means the mTLS mechanism is working and the rest is up to your Vercel and Next.js setup.


You are ready to deploy via Vercel.

Just don't forget to give Vercel PGCLIENT_KEY_B64 and PGCLIENT_CERT_B64




Debug

  • Make sure all environment variables used in payload.config.ts match .env.local
  • Make sure to give Vercel PGCLIENT_KEY_B64 and PGCLIENT_CERT_B64
  • Make sure there are no duplicates in pgbouncer.ini

Double check in:

sudo nano /etc/pgbouncer/pgbouncer.ini
  • Make sure the userlist.txt has your new user (the owner of the database), run:
sudo nano /etc/pgbouncer/userlist.txt

And add:

"database_owner" "owner_password"
  • Makesure pgbouncer.ini has your new database, open:
sudo nano /etc/pgbouncer/pgbouncer.ini

Under [databases], add:

<database_name> = host=127.0.0.1 port=5432 dbname=<database_name> user=<database_user/owner_name>

Restart pgBouncer:

sudo systemctl restart pgbouncer

JKT

Stay focused, and the rest will follow

©Jakkrit Turner. All rights reserved