A disaster recovery (DR) plan is the document you write now so you can recover quickly when something goes wrong at 3am. For a production Hong Kong VPS, “disaster” encompasses everything from accidental database deletion to hardware failure to ransomware — events that require systematic recovery procedures rather than panicked improvisation. This guide builds a practical DR framework with defined recovery objectives, tested runbooks, and realistic RTO targets.
Recovery Objectives: Define Before a Crisis
RTO: Recovery Time Objective
How long can your application be down before it causes unacceptable business impact? Define this per scenario:
| Failure Scenario | Realistic RTO Target | Notes |
|---|---|---|
| Misconfigured Nginx (site returns 502) | 5–15 minutes | Revert config, reload Nginx |
| Application crash (process stopped) | 1–5 minutes | Systemd restarts automatically; monitor detects if not |
| Database corruption (moderate) | 30–120 minutes | Restore from backup; depends on backup frequency |
| Accidental data deletion | 30–240 minutes | Point-in-time restore; time depends on backup granularity |
| VPS hardware failure (provider issue) | 2–8 hours | Provision new VPS; restore from off-site backup |
| Data centre incident | 4–24 hours | Failover to secondary VPS in different region |
| Ransomware / complete compromise | 4–24 hours | Full rebuild from backups on new VPS |
RPO: Recovery Point Objective
How much data loss is acceptable? Your backup frequency determines your practical RPO:
| Backup Frequency | RPO | Data loss in worst case |
|---|---|---|
| Continuous replication (streaming WAL) | Seconds | <30 seconds of transactions |
| Every 4 hours (our recommended schedule) | 4 hours | Up to 4 hours of data |
| Daily backups | 24 hours | Up to 24 hours of data |
| Weekly backups | 7 days | Up to 7 days of data — usually unacceptable for production |
For most applications, 4-hour RPO is acceptable. For financial applications (transactions, payments), implement streaming replication to a secondary server for near-zero RPO.
Runbook 1: Nginx Configuration Rollback
<code"># INCIDENT: Site returning 500/502/504 after Nginx configuration change # EXPECTED RTO: 5 minutes # Step 1: Check what's wrong systemctl status nginx nginx -t 2>&1 # Show configuration error tail -20 /var/log/nginx/error.log # Step 2: Option A — Revert to last known working config # (Requires you have backup of working config — always make one before changes!) cp /etc/nginx/sites-available/mysite.bak /etc/nginx/sites-available/mysite nginx -t && systemctl reload nginx # Step 3: Option B — Revert to VPS snapshot # Via Server.HK control panel: # VPS Management → Snapshots → Restore [most recent pre-change snapshot] # (Takes 5–15 minutes; restores entire server state) # Step 4: Verify recovery curl -sI https://yourdomain.com | head -3 # Should show: HTTP/2 200 echo "RECOVERY COMPLETE: $(date)"
Runbook 2: Database Restoration
<code"># INCIDENT: Accidental table drop or data corruption
# EXPECTED RTO: 30–120 minutes
# Step 1: Stop application to prevent further writes
systemctl stop myapp gunicorn laravel-octane 2>/dev/null
# (Prevents writing to a corrupt database)
# Step 2: Identify the most recent valid backup
ls -lt /opt/backups/postgres/ | head -10
# Step 3: Option A — Restore specific tables from backup (minimal downtime)
# Extract and restore only the affected table
gunzip -c /opt/backups/postgres/pg_all_2026-07-30_020000.sql.gz | \
grep -A 1000000 "COPY public.orders" | \
grep -B 1000000 "^\\\\.$" | \
sudo -u postgres psql -d production_db
# Step 4: Option B — Full database restore (for major corruption)
systemctl stop postgresql
# Backup current corrupted data (just in case)
sudo -u postgres pg_dumpall | gzip > /tmp/corrupted_$(date +%s).sql.gz
# Restore from backup
gunzip -c /opt/backups/postgres/pg_all_2026-07-30_020000.sql.gz | \
sudo -u postgres psql
systemctl start postgresql
# Step 5: Verify data integrity
sudo -u postgres psql -d production_db << 'SQL'
SELECT
tablename,
n_live_tup as rows
FROM pg_stat_user_tables
ORDER BY n_live_tup DESC;
SQL
# Step 6: Restart application
systemctl start myapp
systemctl status myapp
echo "DB RECOVERY COMPLETE: $(date)"
echo "DATA LOSS WINDOW: from $(ls -lt /opt/backups/postgres/ | sed -n '2p' | awk '{print $9}') to incident time"Runbook 3: Full Server Rebuild
<code"># INCIDENT: Hardware failure, ransomware, or catastrophic VPS corruption
# EXPECTED RTO: 4–8 hours
# PHASE 1: Provision new VPS (15–30 minutes)
# 1. Log in to Server.HK control panel
# 2. Create new VPS: same plan, Ubuntu 22.04, Hong Kong region, CN2 GIA
# 3. Note new VPS IP address
# PHASE 2: Restore from off-site backup (Cloudflare R2)
# Run this on the NEW VPS:
# 2.1 Install rclone
curl https://rclone.org/install.sh | bash
# 2.2 Configure R2 access
rclone config # Add R2 remote using stored credentials
# 2.3 List available backups
rclone ls r2:hk-vps-backups/postgres/ | sort | tail -5
# 2.4 Download most recent database backup
BACKUP_DATE="2026-07-30_020000"
rclone copy r2:hk-vps-backups/postgres/pg_all_${BACKUP_DATE}.sql.gz /tmp/
# 2.5 Download application files backup
rclone copy r2:hk-vps-backups/files/files_${BACKUP_DATE}.tar.gz /tmp/
# PHASE 3: Reinstall server stack (60–90 minutes)
# Run your server setup playbook or follow the setup guides:
apt update && apt upgrade -y
# Install: Nginx, PHP/Node.js/Python, PostgreSQL, Redis
# (Reference your specific tech stack setup guide)
# PHASE 4: Restore data (30–60 minutes)
# Restore database
gunzip -c /tmp/pg_all_${BACKUP_DATE}.sql.gz | sudo -u postgres psql
# Restore application files
tar xzf /tmp/files_${BACKUP_DATE}.tar.gz -C /
# PHASE 5: Update DNS (5 minutes)
# Change A record to NEW VPS IP in your DNS provider
# With 5-minute TTL (pre-lowered): propagates within 5–15 minutes
# PHASE 6: Verify and monitor (30 minutes)
curl -sI https://yourdomain.com | head -3
tail -f /var/log/nginx/access.log
echo "FULL RECOVERY COMPLETE: $(date)"Runbook 4: Secondary Region Failover
For applications requiring sub-1-hour RTO for data centre incidents, maintain a warm standby in a secondary region:
<code"># Architecture: HK VPS (primary) + SG VPS (warm standby) # PostgreSQL streaming replication: HK primary → SG replica # INCIDENT: HK VPS completely unreachable # EXPECTED RTO: 15–30 minutes # Step 1: Verify HK primary is genuinely down (not a false alarm) ping -c 5 HK_VPS_IP curl -sI https://yourdomain.com # Confirmed: unreachable # Step 2: Promote SG replica to primary ssh root@SG_VPS_IP "sudo -u postgres pg_ctl promote -D /var/lib/postgresql/16/main" # Wait for promotion ssh root@SG_VPS_IP "sudo -u postgres psql -c \"SELECT pg_is_in_recovery();\"" # Should return: f (false = now primary) # Step 3: Update application configuration on SG VPS ssh root@SG_VPS_IP " sed -i 's/DB_HOST=HK_VPS_IP/DB_HOST=localhost/' /opt/myapp/.env systemctl restart myapp " # Step 4: Update DNS to SG VPS IP # Change A record from HK_IP to SG_IP in your DNS provider # Propagation: depends on TTL (should have lowered to 300s per backup guide) # Step 5: Verify traffic flowing to SG VPS ssh root@SG_VPS_IP "tail -f /var/log/nginx/access.log" echo "FAILOVER COMPLETE: $(date)" echo "Users now served from Singapore standby" echo "TODO: When HK VPS recovers, resync data and fail back"
Disaster Recovery Testing Schedule
| Test | Frequency | Who | Criteria |
|---|---|---|---|
| Backup file verification (test restore) | Monthly | On-call engineer | Backup restores to test environment successfully |
| Nginx rollback drill | Quarterly | Any engineer | Recovery completed in under 10 minutes |
| Database restore to test environment | Quarterly | Senior engineer | Restored DB row counts match production |
| Full rebuild simulation | Annually | Engineering team | New VPS serving traffic within RTO |
| Failover to secondary region | Annually | Engineering + Management | Traffic flows to secondary within 30 minutes |
<code"># Monthly backup verification script
cat > /opt/test-backup-restore.sh << 'EOF' #!/bin/bash DATE=$(date +%Y%m%d) LATEST_BACKUP=$(ls -t /opt/backups/postgres/*.sql.gz | head -1) echo "Testing backup restore: $LATEST_BACKUP" # Restore to test database TEST_DB="restore_test_${DATE}" sudo -u postgres createdb $TEST_DB 2>/dev/null
gunzip -c $LATEST_BACKUP | sudo -u postgres psql -d $TEST_DB -q
# Verify row counts
PROD_COUNT=$(sudo -u postgres psql -t -c "SELECT count(*) FROM users;" production_db)
TEST_COUNT=$(sudo -u postgres psql -t -c "SELECT count(*) FROM users;" $TEST_DB)
if [ "$PROD_COUNT" -eq "$TEST_COUNT" ]; then
echo "PASS: User count matches ($TEST_COUNT rows)"
else
echo "FAIL: Count mismatch — Production: $PROD_COUNT, Restored: $TEST_COUNT"
# Alert engineer
fi
sudo -u postgres dropdb $TEST_DB
EOF
chmod +x /opt/test-backup-restore.shIncident Communication Template
<code"># Status page update template for incidents: cat > /opt/incident-templates/service-degraded.txt << 'EOF' INCIDENT: [Service Name] - [Brief Description] STATUS: Investigating / Identified / Monitoring / Resolved IMPACT: [Affected services and users] DETECTED: [Time] LAST UPDATED: [Time] WHAT HAPPENED: [Brief technical description] WHAT WE'RE DOING: [Current actions] NEXT UPDATE: [Estimated time] For urgent inquiries: [contact method] EOF
Conclusion
A disaster recovery plan for your Hong Kong VPS is worthless if it exists only as a document — it must be tested, updated, and rehearsed before a crisis forces you to use it. The runbooks above cover the most common failure scenarios in order of frequency: configuration errors (most common), database issues, full server rebuild, and regional failover (rare but highest impact). Define your RTO and RPO, implement the backup strategy, test it monthly, and your team will be able to recover from any incident with confidence rather than panic.
Protect your production system: Browse Server.HK Hong Kong VPS plans — all plans support snapshots for rapid recovery; contact support for advice on secondary region standby configurations.