Solving the Cloudflare + Directus Proxy Puzzle: A DevOps Deep Dive

Next.js static generation randomly failing during npm run build with no server logs. Here's how to debug invisible network issues and implement a reliable solution.

Problem: Silent Build Failures

$ npm run build

> reelabilities@1.0.0 build
> next build

- info Linting and checking validity of types
- info Creating an optimized production build
- info Compiled successfully
- info Collecting page data  .
  × Failed to collect page data for /toronto/films/[slug]
    Error: fetch failed
        at Object.fetch (node:internal/deps/undici/undici:11457:11)
        at async fetchFilmById (/app/.next/server/chunks/123.js:1:234)

Symptoms:

• Development: ✅ Perfect
• Runtime: ✅ Perfect
• Build time: ❌ Random failures
• Server logs: 📝 Empty
• CDN logs: 📝 Empty

Diagnosis

# Directus container logs - nothing suspicious
docker logs directus-container -f --tail=100

# Caddy reverse proxy logs - clean
docker logs caddy-container -f --tail=100

# Cloudflare dashboard - security events empty
# Analytics dashboard - no unusual traffic patterns

Root Cause: Cloudflare's bot protection silently blocking build-time API requests.

Evidence:

• Build-time requests are concurrent and automated
• CDN security treats this as suspicious traffic
• No logs because blocks happen at edge level

Solution: /etc/hosts Bypass

Bypass Cloudflare during builds while keeping it for runtime traffic.

# /etc/hosts
YOUR_SERVER_IP cmsdomain.com

Server Configuration:

# Caddyfile - Handle direct HTTPS with IP restrictions
cmsdomain.com:443 {
    # Restrict access to build server IP only
    @allowed remote_ip YOUR_BUILD_SERVER_IP
    handle @allowed {
        reverse_proxy directus:8055
    }
    
    # Deny all other IPs
    handle {
        respond "Access Denied" 403
    }
    
    # TLS handled by Caddy with Let's Encrypt
    tls your-email@domain.com
    
    # CORS headers for API access
    header Access-Control-Allow-Origin *
    header Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE, OPTIONS"
    header Access-Control-Allow-Headers "Authorization, Content-Type, Accept"
    
    # Handle preflight requests
    @cors_preflight method OPTIONS
    respond @cors_preflight 204
}

# Still handle Cloudflare traffic on standard port (no IP restrictions for public access)
cmsdomain.com:80 {
    reverse_proxy directus:8055
}

Build Environment:

# Build environment variables
NODE_ENV=production
CI=true
DIRECTUS_URL=https://cmsdomain.com

# Critical: Allow insecure SSL connections for direct server access
# This bypasses SSL certificate validation during builds
NODE_TLS_REJECT_UNAUTHORIZED=0

// Build-time API configuration
const DIRECTUS_URL = process.env.NODE_ENV === 'production' && process.env.CI
  ? 'https://cmsdomain.com'     // Direct to server during builds
  : process.env.DIRECTUS_URL;        // Regular runtime configuration

export const directus = createDirectus(DIRECTUS_URL)
  .with(authentication())
  .with(rest());

Results

$ npm run build

> reelabilities@1.0.0 build
> next build

✓ Linting and checking validity of types
✓ Creating an optimized production build
✓ Compiled successfully
✓ Collecting page data
✓ Generating static pages (247/247)
✓ Finalizing page optimization

Build completed successfully!

100% build success rate.

Production Setup

# CI/Build server /etc/hosts
YOUR_SERVER_IP cmsdomain.com

# Build environment variables
NODE_ENV=production
CI=true
DIRECTUS_URL=https://cmsdomain.com

# Critical for direct server access
NODE_TLS_REJECT_UNAUTHORIZED=0

Caddy Config:

cmsdomain.com {
    # IP restriction for build server access
    @build_server remote_ip YOUR_BUILD_SERVER_IP
    
    handle @build_server {
        # Handle build server traffic (bypassing Cloudflare)
        reverse_proxy directus:8055 {
            # Optimize for build-time requests
            timeout 30s
            header_up Host {upstream_hostport}
            header_up X-Forwarded-Proto {scheme}
        }
    }
    
    handle {
        # Handle regular traffic through Cloudflare
        reverse_proxy directus:8055 {
            timeout 30s
        }
    }
    
    # Compression
    encode gzip
    
    # CORS
    header Access-Control-Allow-Origin *
    header Access-Control-Allow-Methods *
    header Access-Control-Allow-Headers *
}

Final Results

• 100% build success over 3+ months
• Faster builds (no CDN latency)
• Security maintained (IP restrictions)
• Runtime performance unchanged

Debug Methodology

1. Eliminate layers systematically
2. Use /etc/hosts for CDN bypass testing
3. Check build vs runtime contexts
4. Monitor patterns, not just individual failures
5. Test in isolated environments

Key Takeaways

• CDN bot protection can be invisible and inconsistent
• Build-time traffic patterns differ from user traffic
• /etc/hosts is the fastest way to isolate CDN issues
• NODE_TLS_REJECT_UNAUTHORIZED=0 works for controlled environments
• Dual-stack approaches let you optimize per context

Common Signs of CDN Interference

• Works in development, fails in CI
• Intermittent failures with no server logs
• Concurrent requests failing more than sequential
• Different behavior across CDN edge locations