A Real-World Project: Building a Self-Hosted Workflow Automation Platform with Docker Compose, PostgreSQL, and Caddy
Introduction: Why I Built This n8n Deployment
In today's fast-paced business environment, efficiency isn't just an advantage =>it's a necessity. For this production-grade Docker project, I chose to deploy n8n (pronounced "n-eight-n"), a powerful, self-hostable, open-source workflow automation platform. This wasn't just a learning exercise; it was about solving a real problem: eliminating manual, repetitive tasks that drain productivity from Micro, Small, and Medium Enterprises (MSMEs).
What is n8n?
n8n is a workflow automation platform that connects different apps and services to create complex, customized workflows without extensive coding. Think of it as a central nervous system for your business.
it makes your applications communicate with each other, handling routine operations automatically.
Why n8n Matters for MSMEs
I chose n8n for this deployment project because it addresses critical business needs:
💰 Cost Efficiency: Being open-source and self-hostable means significantly lower operational costs compared to proprietary SaaS automation services like Zapier or Make.com. For budget-conscious smaller businesses, this can mean thousands of dollars in annual savings.
🔒 Data Control & Security: Self-hosting gives complete control over sensitive business data and credentials. In an age of data breaches and privacy concerns, knowing exactly where your data lives and who has access to it is invaluable.
📈 Scalability for Growth: The containerized, microservices architecture I've implemented ensures the platform can scale from a single user to an enterprise-level operation without major re-architecture.
Real-World Automation Use Cases
Through n8n, businesses can automate:
Marketing Automation: Automatically add leads from web forms to CRM systems, notify sales teams via Slack, WhatsApp and trigger personalized welcome email sequences.
Data Synchronization: Keep inventory numbers, customer lists, and project statuses consistent across Google Sheets, databases, and accounting software in real-time.
Internal Operations: Automate notification systems, generate scheduled reports, perform data cleanup tasks, and manage approval workflows.
Why This Deployment Architecture?
For my first Docker production deployment, I needed an architecture that was not only robust and secure but also manageable and educational. Here's the technical stack I chose and why each component matters.
The Power of Docker Compose
Docker Compose allows us to define and orchestrate multi-container applications using a single declarative configuration file. For this n8n deployment, I manage three services — n8n application, PostgreSQL database, and Caddy reverse proxy as a unified system.
Why Docker Compose?
Manageability: The entire infrastructure is defined in a single docker-compose.yml file, making it version-controllable, reproducible, and easy to understand. Anyone reviewing my project can see exactly how services are configured and connected.
Isolation: Each service runs in its own container with defined resource boundaries and network isolation, improving security and preventing conflicts.
Portability: The same configuration works on any system with Docker installed — whether it's my development machine, a production VPS, or a cloud provider.
Scalability: While starting with a single instance, this containerized architecture provides a clear migration path to orchestration platforms like Kubernetes when scaling becomes necessary.
Choosing PostgreSQL Over SQLite for Production
One of the most important architectural decisions was selecting the database backend. While n8n defaults to SQLite for simple setups, a production environment demands more.
Why PostgreSQL?
🔄 Concurrency: SQLite locks the entire database file during write operations, which severely limits performance when multiple workflows execute simultaneously. PostgreSQL handles multiple concurrent connections and read/write operations efficiently using its Multi-Version Concurrency Control (MVCC) system.
✅ Reliability & ACID Compliance: PostgreSQL offers superior transaction management with full ACID (Atomicity, Consistency, Isolation, Durability) guarantees. This is crucial when dealing with workflow execution history and sensitive credential storage where data integrity cannot be compromised.
📦 Data Encapsulation: PostgreSQL runs as a separate, dedicated service with its own container, providing better separation of concerns. This architecture simplifies backup and restore operations compared to file-based databases.
🚀 Performance at Scale: PostgreSQL provides advanced query optimization, sophisticated indexing capabilities, and efficient resource management that becomes critical as workflow complexity and execution volume grow.
PostgreSQL 16 Alpine specifically offers:
- Latest stable release with performance improvements
- Long-term support until November 2028
- Smaller container image (~240MB vs ~380MB for standard images)
- Reduced attack surface due to Alpine Linux's minimal design
Caddy: Simplified Security with Automatic HTTPS
For my first production deployment, I wanted security to be robust but not complex. Caddy emerged as the perfect reverse proxy choice.
Why Caddy?
🔐 Automatic HTTPS: When you configure a domain name, Caddy automatically obtains, installs, and renews SSL certificates from Let's Encrypt —> no manual certificate management, no cron jobs, no expired certificates causing downtime.
⚡ Zero-Configuration SSL: Unlike traditional web servers (Apache, Nginx) that require complex SSL configuration, Caddy makes HTTPS the default with minimal configuration.
🛡️ Security by Default: Caddy includes modern security headers, HTTP/2 and HTTP/3 support, and secure TLS configurations out of the box.
🔄 Graceful Reloads: Configuration changes can be applied without service interruption —> critical for production environments.
Simplicity: The Caddyfile configuration syntax is intuitive and readable, making it perfect for a first production project where understanding every component is important.
Prerequisites: What You Need Before Starting
Before beginning this deployment, ensure your production server meets these requirements:
Required Software
Docker (Version 20.10 or higher):
docker --version
Docker Compose (Version 2.0 or higher):
docker compose version
Installation (if needed for Ubuntu/Debian):
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
System Requirements
- Operating System: Linux (Ubuntu 20.04+, Debian 11+, CentOS 8+)
- RAM: Minimum 2GB, Recommended 4GB+ (based on expected workflow complexity)
- Storage: Minimum 10GB free space for containers and data
- Network: Public IP address for external access
Optional (For Production Domain Deployment)
- Domain Name: DNS A record pointing to your server's IP address
- Firewall Configuration: Ports 80 (HTTP) and 443 (HTTPS) open for incoming connections
Architecture Overview: How Everything Connects
Understanding the architecture was crucial for my learning journey. Here's how the three services interact:
┌─────────────────────────────────────────────────────────────┐
│ Internet │
└──────────────────────────┬──────────────────────────────────┘
│
│ HTTP/HTTPS (Ports 80/443)
│
┌────────▼─────────┐
│ │
│ Caddy Proxy │ ← Automatic HTTPS
│ (Alpine Linux) │ SSL Termination
│ │ Reverse Proxy
└────────┬─────────┘
│
│ Internal Network (Default)
│ HTTP to n8n:5678
┌────────▼─────────┐
│ │
│ n8n Application │ ← Workflow Engine
│ (Node.js) │ REST API
│ │ Web Interface
└────────┬─────────┘
│
│ Internal Network (Isolated)
│ PostgreSQL Protocol :5432
┌────────▼─────────┐
│ │
│ PostgreSQL 16 │ ← Database
│ (Alpine Linux) │ Data Persistence
│ │ Credential Storage
└──────────────────┘
Network Architecture Explained
Two Isolated Networks:
Default Network (Exposed):
- Connects Caddy (exposed to internet) with n8n
- Caddy receives external HTTP/HTTPS requests
- Forwards internally to n8n on port 5678
Internal Network (Isolated):
- Connects n8n with PostgreSQL
- Completely isolated from internet access
- Database port 5432 not exposed externally
- Security benefit: Database cannot be directly attacked from internet
Request Flow:
User Browser → HTTPS/HTTP
↓
Caddy (Ports 80/443) → SSL Termination
↓
n8n (Port 5678) → Workflow Processing
↓
PostgreSQL (Port 5432) → Data Storage
Data Persistence Strategy
All critical data is stored in local directory bind mounts under ./data/:
/home/user/n8n/
├── docker-compose.yml
├── Caddyfile
├── .env
└── data/
├── postgres/
│ └── pgdata/
├── n8n/
│ ├── .n8n.json
│ ├── credentials/
│ └── workflows/
└── caddy/
├── data/
└── config/
Why Local Directories Instead of Docker Volumes?
- Easy Backups: Simple filesystem operations (
cp, rsync, tar)
- Direct Access: No need for
docker volume commands to inspect data
- Portability: Easy migration between servers
- Transparency: Clear visibility of where data resides
- Version Control: Can selectively track configurations (excluding sensitive data)
Step 1: Setting Up docker-compose.yml
This file is the heart of the deployment, defining all services, their configurations, and how they interconnect. Let me break down each component with the reasoning behind every configuration choice.
Complete docker-compose.yml
services:
postgres:
image: postgres:16-alpine
container_name: n8n_postgres
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- ./data/postgres:/var/lib/postgresql/data
networks:
- internal
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
interval: 10s
timeout: 5s
retries: 5
n8n:
image: n8nio/n8n:stable
container_name: n8n_app
restart: always
environment:
N8N_HOST: ${N8N_HOST}
N8N_PROTOCOL: ${N8N_PROTOCOL}
WEBHOOK_URL: ${N8N_PROTOCOL}://${N8N_HOST}/
DB_TYPE: postgresdb
DB_POSTGRESDB_HOST: postgres
DB_POSTGRESDB_PORT: 5432
DB_POSTGRESDB_DATABASE: ${POSTGRES_DB}
DB_POSTGRESDB_USER: ${POSTGRES_USER}
DB_POSTGRESDB_PASSWORD: ${POSTGRES_PASSWORD}
N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY}
EXECUTIONS_DATA_PRUNE: "true"
EXECUTIONS_DATA_MAX_AGE: 168
volumes:
- ./data/n8n:/home/node/.n8n
networks:
- internal
- default
depends_on:
postgres:
condition: service_healthy
caddy:
image: caddy:2-alpine
container_name: n8n_caddy
restart: always
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- ./data/caddy/data:/data
- ./data/caddy/config:/config
networks:
- default
depends_on:
- n8n
networks:
internal:
driver: bridge
default:
driver: bridge
PostgreSQL Service Deep Dive
postgres:
image: postgres:16-alpine
container_name: n8n_postgres
restart: always
Configuration Explained:
image: postgres:16-alpine: Uses PostgreSQL 16 with Alpine Linux base (lightweight, security-focused)container_name: n8n_postgres: Friendly name for easier management and log identificationrestart: always: Container automatically restarts on failure or system reboot —> critical for production availability
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
PGDATA: /var/lib/postgresql/data/pgdata
Why Each Variable Matters:
POSTGRES_USER: Creates the database superuser account (loaded from .env for security)POSTGRES_PASSWORD: Secures database access — must be strong and uniquePOSTGRES_DB: Database name created on first startup (default: n8n_db)PGDATA: Specifies exact data directory path — required when using bind mounts to avoid permission issues
volumes:
- ./data/postgres:/var/lib/postgresql/data
Data Persistence:
./data/postgres: Local directory on host machine (created automatically)/var/lib/postgresql/data: PostgreSQL's internal data directory- Bind Mount: Direct mapping ensures data survives container removal/recreation
- Critical: Without this, all workflow execution history would be lost on container restart!
networks:
- internal
Network Isolation:
- Connected only to internal network
- Not exposed to default network (internet-facing)
- Database port 5432 never directly accessible from outside
- Security benefit: Prevents external database attacks
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
interval: 10s
timeout: 5s
retries: 5
Why Health Checks?
pg_isready: PostgreSQL utility that checks if database accepts connectionsinterval: 10s: Check every 10 secondstimeout: 5s: Wait maximum 5 seconds for responseretries: 5: Must pass 5 consecutive checks before considered healthy- Purpose: Prevents n8n from starting before database is fully ready, avoiding connection errors
n8n Application Service Deep Dive
n8n:
image: n8nio/n8n:stable
container_name: n8n_app
restart: always
Image Selection:
n8nio/n8n:stable: Official n8n image on the stable release channel- Why stable tag? Avoids unexpected changes from
latest builds, ensuring predictable production behavior
environment:
N8N_HOST: ${N8N_HOST}
N8N_PROTOCOL: ${N8N_PROTOCOL}
WEBHOOK_URL: ${N8N_PROTOCOL}://${N8N_HOST}/
Public Access Configuration:
N8N_HOST: How n8n is accessed externally
- IP-based:
:80 (just the port)
- Domain-based:
n8n.yourdomain.com
N8N_PROTOCOL: http (IP access) or https (domain with SSL)WEBHOOK_URL: Full URL for external services to send webhook callbacks
- Example:
https://n8n.yourdomain.com/ for webhooks from Stripe, GitHub, etc.
DB_TYPE: postgresdb
DB_POSTGRESDB_HOST: postgres
DB_POSTGRESDB_PORT: 5432
DB_POSTGRESDB_DATABASE: ${POSTGRES_DB}
DB_POSTGRESDB_USER: ${POSTGRES_USER}
DB_POSTGRESDB_PASSWORD: ${POSTGRES_PASSWORD}
Database Integration:
DB_TYPE: postgresdb: Tells n8n to use PostgreSQL instead of default SQLiteDB_POSTGRESDB_HOST: postgres: Uses Docker service name (Docker's internal DNS resolves this to the container IP)DB_POSTGRESDB_PORT: 5432: Standard PostgreSQL port on internal network- Credentials must match PostgreSQL service configuration exactly
N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY}
🔐 MOST CRITICAL SECURITY PARAMETER:
- Encrypts all sensitive credentials (API keys, passwords, OAuth tokens) stored in PostgreSQL
- Must be set before first run
- DO NOT LOSE THIS KEY: All encrypted credentials become permanently unrecoverable if lost
- Generate using:
openssl rand -base64 32
EXECUTIONS_DATA_PRUNE: "true"
EXECUTIONS_DATA_MAX_AGE: 168
Data Retention Management:
EXECUTIONS_DATA_PRUNE: "true": Enables automatic cleanup of old workflow execution logsEXECUTIONS_DATA_MAX_AGE: 168: Retention period in hours (168 hours = 7 days)- Why this matters: Prevents database from growing infinitely — execution history can consume significant space over time
- Customization: Adjust based on compliance requirements and storage capacity
volumes:
- ./data/n8n:/home/node/.n8n
Application Data Storage:
./data/n8n: Local directory for n8n application data/home/node/.n8n: n8n's internal data directory (runs as node user, UID 1000)- Stores: Custom node packages, local file storage, configuration cache
- Note: Actual workflow definitions and credentials are in PostgreSQL, not here
networks:
- internal
- default
Dual Network Connection:
internal: Communicates with PostgreSQL databasedefault: Receives proxied requests from Caddy- Bridge role: n8n sits between the internet-facing proxy and isolated database
depends_on:
postgres:
condition: service_healthy
Startup Order Control:
- Waits for PostgreSQL service
- Critical:
condition: service_healthy ensures database health checks pass before n8n starts
- Prevents: Database connection errors during startup
Caddy Reverse Proxy Service Deep Dive
caddy:
image: caddy:2-alpine
container_name: n8n_caddy
restart: always
Image Choice:
caddy:2-alpine: Caddy 2.x with Alpine Linux base- Benefits: Small image size (~50MB), reduced attack surface, same powerful features
ports:
- "80:80"
- "443:443"
Port Exposure (Only Exposed Ports):
"80:80": HTTP traffic (required for Let's Encrypt validation and HTTP-to-HTTPS redirect)"443:443": HTTPS traffic (secure encrypted connections)- Format:
"host_port:container_port"
- Critical: These are the ONLY ports accessible from internet—>everything else is internal
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- ./data/caddy/data:/data
- ./data/caddy/config:/config
Volume Mounts Explained:
./Caddyfile:/etc/caddy/Caddyfile:ro: Configuration file (:ro = read-only for security)./data/caddy/data:/data: SSL certificates and persistent data (Let's Encrypt certificates stored here)./data/caddy/config:/config: Runtime configuration cache- Bind mounts: All data accessible on host for easy backup and inspection
networks:
- default
Network Connection:
- Connected to default network (shared with n8n, exposed to internet)
- Not connected to internal network (doesn't need database access)
depends_on:
- n8n
Dependency:
- Starts after n8n application is running
- Ensures reverse proxy target is available when Caddy starts accepting traffic
Step 2: Configuring the Caddyfile
The Caddyfile defines how Caddy handles incoming web traffic. Its simplicity is deceptive —> behind this clean syntax, Caddy automatically manages SSL certificates, security headers, and request forwarding.
Flexible Caddyfile for IP and Domain Access
# Option 1: IP-based access (initial deployment)
# For accessing via http://YOUR_SERVER_IP
:80 {
reverse_proxy n8n:5678 {
header_up Host {host}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
# Option 2: Domain-based access with automatic HTTPS
# Uncomment and replace with your domain when ready
# n8n.yourdomain.com {
# reverse_proxy n8n:5678 {
# header_up Host {host}
# header_up X-Real-IP {remote_host}
# header_up X-Forwarded-For {remote_host}
# header_up X-Forwarded-Proto {scheme}
# }
# }
Configuration Breakdown
IP-Based Access Block:
:80 {
:80: Listens on port 80 (HTTP) without a specific hostname- Use case: Initial deployment when accessing via
http://123.456.789.012
- No SSL: Caddy only enables automatic HTTPS when a domain name is specified
reverse_proxy n8n:5678 {
n8n: Docker service name (resolved via Docker DNS to n8n container IP)5678: n8n's internal application port- Function: Forwards all incoming requests to n8n application
header_up Host {host}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
Header Forwarding (Why Each Matters):
Host {host}: Preserves original hostname from request — n8n needs this for webhook URL generationX-Real-IP {remote_host}: Real client IP address (not Caddy's internal IP)X-Forwarded-For {remote_host}: Standard header for proxied requests, used for logging and securityX-Forwarded-Proto {scheme}: Tells n8n whether original request was HTTP or HTTPS — critical for proper redirects
Domain-Based Access (Production):
When you have a domain pointing to your server:
n8n.yourdomain.com {
reverse_proxy n8n:5678 {
# Same headers as above
}
}
What Changes When Domain is Configured:
- Caddy automatically contacts Let's Encrypt
- Validates domain ownership via HTTP-01 challenge
- Obtains SSL certificate
- Enables HTTPS on port 443
- Automatically redirects HTTP (port 80) to HTTPS (port 443)
- Sets up automatic renewal (certificates renewed before 30-day expiration)
No manual certificate management required!
Step 3: Critical Security - Data Encryption
Security was a top priority in this deployment. n8n stores sensitive credentials (API keys, OAuth tokens, database passwords) that must be protected.
Understanding N8N_ENCRYPTION_KEY
What It Does:
- Encrypts all credentials before storing in PostgreSQL database
- Uses AES-256-GCM encryption (industry-standard, highly secure)
- Each credential is encrypted individually with authentication
Why It's Critical:
- Lose this key = Lose all credentials permanently
- No recovery mechanism exists — encrypted data cannot be decrypted without the exact key
- Changing the key invalidates all existing encrypted credentials
Generating a Secure Encryption Key
Option 1: Base64 Encoded (Recommended)
openssl rand -base64 32
Output example: Xk9pL2mN3qR5sT7vW9yZ1bC4dF6gH8jKlMnPqRsTuVwXyZ
Option 2: Hexadecimal
openssl rand -hex 32
Output example: a3f5c7b9d1e2f4g6h8i0j2k4l6m8n0o2p4q6r8s0t2u4v6w8x0y2z4
Option 3: Alphanumeric Only
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1
Output example: 8kY3mQ7nB2xR9tL6wV4pS1zF5cH0jN3g
Best Practices:
- Minimum 32 characters (256 bits of entropy)
- Store in password manager immediately after generation
- Never commit to version control
- Back up securely (encrypted backup storage recommended)
Step 4: Environment Configuration (.env File)
The .env file contains all sensitive configuration. This file must never be committed to version control.
Complete .env File Structure
# ============================================
# n8n Public Access Configuration
# ============================================
# For IP-based access (initial deployment):
N8N_HOST=:80
N8N_PROTOCOL=http
# For domain-based access (production):
# N8N_HOST=n8n.yourdomain.com
# N8N_PROTOCOL=https
# ============================================
# PostgreSQL Database Configuration
# ============================================
POSTGRES_USER=n8n_user
POSTGRES_PASSWORD=your_strong_database_password_here
POSTGRES_DB=n8n_db
# ============================================
# n8n Security Configuration
# ============================================
# CRITICAL: Encryption key for credentials (generate with: openssl rand -base64 32)
N8N_ENCRYPTION_KEY=your_generated_encryption_key_here
# JWT Secret (must be different from encryption key)
N8N_USER_MANAGEMENT_JWT_SECRET=your_unique_jwt_secret_here
# Session duration (hours users stay logged in)
N8N_USER_MANAGEMENT_JWT_DURATION_HOURS=24
N8N_USER_MANAGEMENT_JWT_REFRESH_TIMEOUT_HOURS=24
# ============================================
# Login Security & Brute Force Protection
# ============================================
# Maximum failed login attempts before lockout
N8N_LOGIN_MAX_ATTEMPTS=5
# Lockout duration in minutes
N8N_LOGIN_LOCKOUT_DURATION=30
# ============================================
# Password Policy
# ============================================
N8N_USER_MANAGEMENT_PASSWORD_MIN_LENGTH=12
N8N_USER_MANAGEMENT_PASSWORD_REQUIRE_UPPERCASE=true
N8N_USER_MANAGEMENT_PASSWORD_REQUIRE_LOWERCASE=true
N8N_USER_MANAGEMENT_PASSWORD_REQUIRE_NUMBER=true
N8N_USER_MANAGEMENT_PASSWORD_REQUIRE_SPECIAL=true
# ============================================
# Additional Security Headers & CORS
# ============================================
N8N_SECURITY_HEADERS_ENABLED=true
# For domain-based deployment:
# N8N_ALLOWED_ORIGINS=https://n8n.yourdomain.com
# ============================================
# Workflow Execution Settings
# ============================================
# Maximum workflow execution timeout (seconds)
EXECUTIONS_TIMEOUT=600
EXECUTIONS_TIMEOUT_MAX=3600
# ============================================
# Logging Configuration
# ============================================
N8N_LOG_LEVEL=warn
N8N_LOG_OUTPUT=json
Configuration Variable Explanations
Session Management:
N8N_USER_MANAGEMENT_JWT_DURATION_HOURS=24
- Purpose: How long users remain logged in without activity
- 24 hours: Balances security with user convenience
- Shorter values (1-8 hours): Higher security, more frequent logins
- Why 24? Provides full workday access without requiring re-authentication
N8N_LOGIN_MAX_ATTEMPTS=5
- Purpose: Limits failed login attempts before account lockout
- 5 attempts: Industry standard for brute-force protection
- Why? After 5 failed attempts, probability of legitimate user is very low
- Protection: Makes password guessing attacks impractical
N8N_LOGIN_LOCKOUT_DURATION=30
- Purpose: Lockout duration in minutes after exceeding max attempts
- 30 minutes: Long enough to deter automated attacks, short enough to not permanently block legitimate users
- Why? Provides cooldown period while not creating excessive user friction
Password Policy:
N8N_USER_MANAGEMENT_PASSWORD_MIN_LENGTH=12
- 12 characters: Minimum for modern password security standards
- Why? Provides sufficient entropy against brute-force attacks
- Each additional character exponentially increases crack time
N8N_USER_MANAGEMENT_PASSWORD_REQUIRE_*=true
- Enforces composition: Uppercase, lowercase, numbers, special characters
- Why all four? Creates passwords resistant to dictionary attacks
- Example compliant password:
MyN8n@Pass2024!
Workflow Execution Safety:
EXECUTIONS_TIMEOUT=600
- Purpose: Maximum execution time in seconds (600 = 10 minutes)
- Why? Prevents runaway workflows from consuming excessive resources
- Customization: Adjust based on your longest legitimate workflow duration
EXECUTIONS_DATA_MAX_AGE=168
- Purpose: How long to keep execution history (168 hours = 7 days)
- Why 7 days? Balances troubleshooting needs with database size management
- Automatic cleanup: Prevents database from growing infinitely
Step 5: Deployment - Bringing It All Together
With all configuration in place, it's time to deploy. Here's the step-by-step process I followed:
Pre-Deployment Checklist
docker --version
docker compose version
cd /home/user/n8n
ls -la
Initial Deployment
docker compose up -d
What Happens:
Network Creation:
[+] Network n8n_internal Created
[+] Network n8n_default Created
Two isolated networks established for security
Container Startup:
[+] Container n8n_postgres Started
[+] Container n8n_app Starting... (waiting for postgres health)
[+] Container n8n_caddy Starting... (waiting for n8n)
Health Checks:
- PostgreSQL health checks begin immediately
- After 5 successful checks (~50 seconds), marked healthy
- n8n starts connecting to database
- Caddy starts accepting traffic
Verify Deployment Success
docker compose ps
Expected Output:
NAME IMAGE STATUS
n8n_postgres postgres:16-alpine Up (healthy)
n8n_app n8nio/n8n:stable Up
n8n_caddy caddy:2-alpine Up
All containers should show "Up" status.
docker compose logs -f
Look for:
- PostgreSQL:
database system is ready to accept connections
- n8n:
Editor is now accessible via: http://...
- Caddy:
serving initial configuration
Press Ctrl+C to stop viewing logs (containers continue running)
Access Your n8n Instance
curl -4 ifconfig.me
Open in browser:
http:
Example: http://123.456.789.012
First-Time Setup
When accessing n8n for the first time, you'll complete the initial owner account creation:
- Email address for owner account
- Strong password (must meet policy requirements from
.env)
- Workspace name (optional)
- Usage preferences (optional telemetry)
This is your admin account -> credentials are encrypted using N8N_ENCRYPTION_KEY
Need Help with n8n Deployment or Custom Automation?
If you're looking for professional assistance with:
- n8n Installation & Configuration: Production-ready deployments with security best practices
- Custom Workflow Design: Tailored automation solutions for your specific business needs
- Migration Services: Moving from Zapier, Make.com, or other platforms to self-hosted n8n
- Ongoing n8n Management: Server maintenance, updates, monitoring, and troubleshooting
- Process Automation Consulting: Identifying automation opportunities in your business
I can help! With experience in server administration and proven expertise in n8n automation (40% reduction in manual tasks for my current organization), I specialize in designing and implementing workflow automation that drives real business value.
📧 Email: push1697@gmail.com
💼 LinkedIn: linkedin.com/in/pushpendra16
📱 WhatsApp: +91 8619274820
🌐 Location: Jaipur, Rajasthan, India (Remote services available)
Real-World Deployment Challenges: Lessons Learned
During my actual deployment, I encountered several issues that taught me valuable lessons about production Docker deployments. Here's what went wrong and how I fixed it.
Challenge 1: Permission Errors (n8n Container)
Error Encountered:
Error: EACCES: permission denied, open '/home/node/.n8n/config'
What Happened:
The n8n container runs as user node with UID 1000. The mounted ./data/n8n directory had restrictive permissions that prevented the container from writing configuration files.
Root Cause:
When using bind mounts (local directories), the container user must have write permissions to the mounted directory. Docker doesn't automatically handle this like it does with named volumes.
Solution:
chmod -R 777 data/n8n
Better Solution (More Secure):
sudo chown -R 1000:1000 data/n8n
chmod -R 755 data/n8n
Lesson Learned:
Always consider container user IDs when using bind mounts. Check the container's documentation for the default user UID/GID.
Challenge 2: Port Binding Error (Caddy)
Error Encountered:
Error: cannot expose privileged port 80: permission denied
What Happened:
My Docker installation was running in rootless mode (security-enhanced). Rootless Docker cannot bind to privileged ports (< 1024) without special system configuration.
Root Cause:
Linux restricts binding to ports below 1024 to root user. Rootless Docker intentionally runs without root privileges for enhanced security.
Initial Workaround:
Modified docker-compose.yml to use non-privileged ports:
ports:
- "8080:80"
- "8443:443"
Permanent Solution:
echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
docker compose down
docker compose up -d
Lesson Learned:
Security features (like rootless Docker) sometimes conflict with standard port conventions. Understanding the trade-offs between security and convenience is crucial for production deployments.
Challenge 3: Caddyfile Syntax Error
Error Encountered:
Error: unrecognized global option: reverse_proxy
What Happened:
Initially, I attempted to use environment variable substitution in the Caddyfile, which caused syntax confusion.
Initial (Broken) Configuration:
${N8N_HOST} {
reverse_proxy n8n:5678
}
Root Cause:
Caddyfile doesn't support environment variable substitution in the same way docker-compose does. The ${N8N_HOST} was being interpreted as a literal string, not replaced with the value.
Solution:
Use explicit configuration based on deployment type:
For IP-based access:
:80 {
reverse_proxy n8n:5678 {
header_up Host {host}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
For domain-based access:
n8n.yourdomain.com {
reverse_proxy n8n:5678 {
# same headers
}
}
Lesson Learned:
Configuration file syntaxes vary between tools. What works in docker-compose.yml doesn't necessarily work in Caddyfile. Always reference official documentation for each component.
Challenge 4: Docker Compose Version Warning
Warning Encountered:
WARN[0000] the attribute 'version' is obsolete
What Happened:
Modern Docker Compose (v2.x) no longer requires or uses the version: field at the top of docker-compose.yml.
Original File:
version: '3.8'
services:
postgres:
Solution:
Simply removed the version field:
services:
postgres:
Why This Changed:
Docker Compose v2 automatically uses the latest spec features. The version field was only necessary for Docker Compose v1.x to determine feature compatibility.
Lesson Learned:
Tools evolve. Configuration patterns that were best practices in 2020 may be obsolete in 2024. Stay updated with latest documentation.
Final Working Configuration
After resolving all issues, here's what successfully deployed:
Access Information:
http:
Status:
NAME IMAGE STATUS PORTS
n8n_app n8nio/n8n:stable Up 5678/tcp
n8n_caddy caddy:2-alpine Up 0.0.0.0:8080->80/tcp, 0.0.0.0:8443->443/tcp
n8n_postgres postgres:16-alpine Up (healthy) 5432/tcp
All containers running successfully! ✅
Real-World Automation Examples I've Built
As someone who actively uses n8n in production environments, here are some automation workflows I've designed and implemented:
1. Email-to-Ticket Automation System
Problem: Support requests from multiple email accounts needed manual consolidation
Solution: n8n workflow monitoring multiple IMAP mailboxes, creating tickets in project management system with intelligent categorization
Result: 60% reduction in ticket processing time, zero missed support requests
Problem: Customer data scattered across Google Sheets, CRM, and accounting software
Solution: Bi-directional sync workflows with conflict resolution and audit logging
Result: Single source of truth for customer data, eliminated duplicate entry work
3. AI-Powered Content Moderation
Problem: Manual review of user-generated content was time-consuming
Solution: n8n workflow integrating AI APIs for content analysis, automatic flagging, and notification system
Result: 85% of content automatically processed, moderation team focuses only on flagged items
4. Automated Backup & Reporting Pipeline
Problem: Weekly server backups and reports required manual execution
Solution: Scheduled n8n workflows with error handling, Slack notifications, and report generation
Result: 100% backup reliability, management receives automated insights every Monday
Want similar automation for your business? These are just examples =>every business has unique processes that can benefit from intelligent automation. Let's discuss how n8n can transform your operations.
📧 Contact me: push1697@gmail.com
Migration Path: From IP to Domain with HTTPS
One of the design goals was making it easy to transition from initial IP-based deployment to production domain-based deployment with automatic HTTPS. Here's how this migration works seamlessly.
Current State (IP-Based Access)
Configuration:
N8N_HOST=:80
N8N_PROTOCOL=http
Access: http://123.456.789.012:8080
Limitations:
- No encryption (HTTP only)
- IP address not user-friendly
- No automatic SSL certificate management
Migration Steps to Domain-Based HTTPS
Point your domain's A record to your server IP:
DNS Configuration:
Type: A
Name: n8n
Value: 123.456.789.012
TTL: 3600
Result: n8n.yourdomain.com → 123.456.789.012
Verify DNS Propagation:
nslookup n8n.yourdomain.com
dig n8n.yourdomain.com +short
Expected Output: 123.456.789.012
Wait Time: DNS propagation typically takes 5-15 minutes, can be up to 48 hours in rare cases
Step 2: Update Environment Configuration
nano .env
Change from IP-based:
N8N_HOST=:80
N8N_PROTOCOL=http
To domain-based:
N8N_HOST=n8n.yourdomain.com
N8N_PROTOCOL=https
Also update CORS if configured:
N8N_ALLOWED_ORIGINS=https://n8n.yourdomain.com
Save: Ctrl+O, Enter, Ctrl+X
Step 3: Update Caddyfile
Edit Caddyfile:
nano Caddyfile
Comment out IP-based block:
# :80 {
# reverse_proxy n8n:5678 {
# header_up Host {host}
# header_up X-Real-IP {remote_host}
# header_up X-Forwarded-For {remote_host}
# header_up X-Forwarded-Proto {scheme}
# }
# }
Uncomment domain-based block:
n8n.yourdomain.com {
reverse_proxy n8n:5678 {
header_up Host {host}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
Step 4: Restart Services
docker compose down
docker compose up -d
Step 5: Watch Caddy Obtain SSL Certificate
docker compose logs -f caddy
Look for these log messages:
[INFO] Obtaining SSL certificate
[INFO] Validating domain ownership via HTTP-01 challenge
[INFO] Certificate obtained successfully
[INFO] Enabling automatic HTTPS
This process takes 10-60 seconds depending on Let's Encrypt response time
Step 6: Verify HTTPS Access
Open in browser:
https:
Verify:
- Browser shows padlock icon 🔒
- Certificate issued by "Let's Encrypt"
- No certificate warnings
- HTTP automatically redirects to HTTPS
Check certificate details:
curl -vI https://n8n.yourdomain.com 2>&1 | grep -E 'SSL|TLS'
What Caddy Does Automatically
- Certificate Request: Contacts Let's Encrypt ACME API
- Domain Validation: Responds to HTTP-01 challenge on port 80
- Certificate Installation: Stores certificate in
./data/caddy/data
- HTTPS Enablement: Configures TLS with modern cipher suites
- HTTP Redirect: Automatically redirects all HTTP traffic to HTTPS
- Renewal Scheduling: Sets up automatic renewal before 30-day expiration
- OCSP Stapling: Enables for faster certificate validation
No manual intervention required for renewals! Caddy handles everything.
Migration Benefits
Zero Data Loss:
- All workflows preserved
- All credentials remain encrypted
- Execution history intact
- No database migration needed
No Downtime Required:
- Can be done during low-traffic period
- Total downtime: ~10 seconds (during restart)
Improved Security:
- All traffic encrypted end-to-end
- Protection against man-in-the-middle attacks
- Automatic security header injection
Backup Strategy: Protecting Your Work
Production deployments require reliable backup strategies. Here's the comprehensive approach I implemented.
What Needs Backing Up
Critical Data:
- PostgreSQL Database - Workflows, credentials, execution history
- n8n Data Directory - Custom nodes, file storage, local configuration
- Caddy Data - SSL certificates (can be regenerated, but backup prevents rate limits)
- Configuration Files -
.env, docker-compose.yml, Caddyfile
Manual Backup Script
Save as ~/n8n-backup.sh:
#!/bin/bash
BACKUP_DIR=~/n8n-backups
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_PATH="$BACKUP_DIR/$TIMESTAMP"
N8N_DIR=/home/user/n8n
mkdir -p "$BACKUP_PATH"
cd "$N8N_DIR" || exit 1
if [ -f .env ]; then
export $(grep -v '^#' .env | xargs)
fi
echo "Backing up PostgreSQL database..."
docker compose exec -T postgres pg_dump -U "${POSTGRES_USER}" "${POSTGRES_DB}" > "$BACKUP_PATH/database.sql"
echo "Backing up configuration files..."
cp .env "$BACKUP_PATH/.env"
cp docker-compose.yml "$BACKUP_PATH/docker-compose.yml"
cp Caddyfile "$BACKUP_PATH/Caddyfile"
echo "Backing up data directories..."
tar -czf "$BACKUP_PATH/data-backup.tar.gz" \
--exclude='./data/postgres/pgdata/postmaster.pid' \
--exclude='./data/postgres/pgdata/*.pid' \
./data
BACKUP_SIZE=$(du -sh "$BACKUP_PATH" | cut -f1)
RETENTION_DAYS=7
echo "Removing backups older than $RETENTION_DAYS days..."
find "$BACKUP_DIR" -maxdepth 1 -type d -mtime +$RETENTION_DAYS -exec rm -rf {} \;
echo "[$(date)] Backup completed: $BACKUP_PATH (Size: $BACKUP_SIZE)"
ls -lh "$BACKUP_PATH"
Make executable:
chmod +x ~/n8n-backup.sh
Run manually:
~/n8n-backup.sh
Automated Daily Backups
Set up cron job for automatic backups:
crontab -e
0 2 * * * ~/n8n-backup.sh >> ~/n8n-backup.log 2>&1
Verify crontab:
crontab -l
Check backup logs:
tail -f ~/n8n-backup.log
Restore from Backup
Save as ~/n8n-restore.sh:
#!/bin/bash
BACKUP_DATE="20240101_120000"
BACKUP_PATH=~/n8n-backups/$BACKUP_DATE
N8N_DIR=/home/user/n8n
cd "$N8N_DIR" || exit 1
echo "Stopping services..."
docker compose down
echo "Creating safety backup of current data..."
if [ -d data ]; then
mv data "data.old.$(date +%Y%m%d_%H%M%S)"
fi
echo "Restoring configuration files..."
cp "$BACKUP_PATH/.env" ./
cp "$BACKUP_PATH/docker-compose.yml" ./
cp "$BACKUP_PATH/Caddyfile" ./
echo "Restoring data directories..."
tar -xzf "$BACKUP_PATH/data-backup.tar.gz" -C "$N8N_DIR"
echo "Fixing permissions..."
sudo chown -R 1000:1000 ./data/n8n
sudo chown -R $USER:$USER ./data
chmod -R 755 ./data
echo "Starting services..."
docker compose up -d
echo "Waiting for services to initialize..."
sleep 15
docker compose ps
echo "================================"
echo "Restore completed from backup: $BACKUP_DATE"
echo "Previous data backed up to: data.old.*"
echo "Verify everything works, then remove old data:"
echo " rm -rf data.old.*"
echo "================================"
Make executable:
chmod +x ~/n8n-restore.sh
To restore:
nano ~/n8n-restore.sh
~/n8n-restore.sh
Monitoring and Maintenance
Viewing Logs
All services:
docker compose logs -f
Specific service:
docker compose logs -f n8n
docker compose logs -f postgres
docker compose logs -f caddy
Last 100 lines:
docker compose logs --tail=100 n8n
Filter by time:
docker compose logs --since=1h n8n
Container Health Monitoring
Quick status:
docker compose ps
Resource usage:
docker stats
Detailed container inspection:
docker inspect n8n_app
Database Maintenance
Access PostgreSQL CLI:
docker compose exec postgres psql -U n8n_user -d n8n_db
Useful database commands:
SELECT pg_size_pretty(pg_database_size('n8n_db'));
SELECT
schemaname,
tablename,
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;
VACUUM ANALYZE;
\q
Updating n8n
Check for updates:
docker compose pull
Apply updates:
~/n8n-backup.sh
docker compose down
docker compose up -d
docker compose exec n8n n8n --version
Professional n8n Services Available
Don't want to manage this yourself? I offer comprehensive n8n services for businesses:
🚀 Deployment Services
- Production-ready n8n installation with security hardening
- AWS/GCP/Azure cloud deployment
- High-availability configurations
- Custom domain setup with SSL
🔧 Workflow Design & Implementation
- Business process analysis and automation strategy
- Custom workflow development for your specific needs
- Integration with existing tools (CRM, ERP, Marketing platforms)
- API integration and custom node development
🛡️ Managed Services
- 24/7 monitoring and incident response
- Regular updates and security patches
- Performance optimization
- Backup management and disaster recovery
📚 Training & Consultation
- Team training on n8n best practices
- Workflow design workshops
- Technical documentation
- Ongoing support and troubleshooting
Pricing: Flexible packages available based on your requirements
Response Time: Within 24 hours for inquiries
Experience: 2+ years managing production n8n deployments
📧 Email: push1697@gmail.com
💼 LinkedIn: linkedin.com/in/pushpendra16
📱 WhatsApp: +91 8619274820
Manageability Checklist
| Action | Tool / Method | Frequency | Benefit |
| Backups | Automated cron script | Daily at 2 AM | Quick recovery from failures |
| Updates | docker compose pull && up -d | Monthly | Security patches, new features |
| Log Monitoring | docker compose logs -f | As needed | Debugging, performance tracking |
| Health Checks | docker compose ps | Weekly | Early problem detection |
| Database Vacuum | PostgreSQL VACUUM | Monthly | Maintain query performance |
| SSL Renewal | Caddy automatic | Automatic | Continuous HTTPS availability |
| Disk Space | df -h & docker system df | Weekly | Prevent storage issues |
| Security Audit | Review .env settings | Quarterly | Maintain security posture |
Key Takeaways from This Project
Technical Accomplishments
- Production-Ready Architecture: Deployed a multi-container application with proper network isolation and security
- Automatic HTTPS: Implemented zero-configuration SSL with automatic renewal
- Data Persistence: Configured durable storage for database and application data
- Security Best Practices: Encrypted credentials, strong password policies, session management
- Operational Excellence: Automated backups, comprehensive logging, easy updates
Lessons Learned
Docker Fundamentals:
- Understanding container user IDs and filesystem permissions
- Difference between bind mounts and named volumes
- Importance of health checks for service dependencies
- Network isolation for security
Configuration Management:
- Keep sensitive data in
.env files (never commit to git)
- Each tool has its own syntax (docker-compose vs Caddyfile)
- Version specifications matter (stable vs latest tags)
- Documentation is your friend —> always reference official docs
Production Considerations:
- Security isn't optional —> encryption keys, password policies, session management all matter
- Backups aren't optional either —> automate them from day one
- Monitoring and logging are essential for debugging production issues
- Always have a rollback plan (backups, version pinning)
Real-World Challenges:
- Things break in unexpected ways (permission errors, port conflicts)
- Troubleshooting skills are as important as initial setup knowledge
- Understanding the "why" behind configurations helps fix issues faster
- Community resources and documentation are invaluable
Conclusion
This n8n deployment project represents a complete journey through modern DevOps practices —> from architecture design to production deployment, from handling real-world errors to implementing operational best practices.
What makes this deployment production-ready:
- ✅ Robust database backend (PostgreSQL instead of SQLite)
- ✅ Automated security (Caddy with Let's Encrypt SSL)
- ✅ Data encryption (N8N_ENCRYPTION_KEY for credentials)
- ✅ Network isolation (internal network for database)
- ✅ Automated backups (daily cron job with retention policy)
- ✅ Comprehensive monitoring (logs, health checks, resource metrics)
- ✅ Easy migration path (IP to domain without data loss)
- ✅ Disaster recovery plan (restore scripts and procedures)
Access your deployed n8n instance:
https:
Start automating workflows, connecting APIs, and building the integrations that make businesses more efficient.
Happy Automating! 🚀
Additional Resources
- n8n Official Documentation: https://docs.n8n.io/
- Docker Documentation: https://docs.docker.com/
- PostgreSQL Documentation: https://www.postgresql.org/docs/
- Caddy Documentation: https://caddyserver.com/docs/
- n8n Community Forum: https://community.n8n.io/
- n8n Workflow Templates: https://n8n.io/workflows/
