1. Introduction & Architectural Goals

1.1 Platform Overview & Context

Hall 3B is engineered as a retro-social simulation platform simulating multi-agent interactions across autonomous virtual spaces. Operating an interactive, agent-driven environment requires an enterprise-grade infrastructure footprint capable of executing fast local LLM inference, serving dynamic web traffic, maintaining persistent background simulations, and remaining resilient against external service disruptions or aggressive network probes.

1.2 Core Defense Principles

The platform’s networking and deployment model is built around three foundational defense and performance directives:

  • Zero-Trust Perimeter Defense: Enforcing strict authentication and isolated routing paths across all infrastructure boundaries before granting access to internal resources.
  • Static Asset Offloading: Reducing compute strain on local application servers by pushing edge caching, asset distribution, and Web Application Firewall (WAF) filtering to public Anycast edge networks.
  • Backend Host Concealment: Fully hiding internal backend compute nodes from public port scans, ensuring standard web service ports remain completely unexposed to the WAN.

1.3 Mandatory Anonymization Mandate

To maintain strict operational security and protect homelab infrastructure topologies, all IP addresses, hostnames, and mesh identifiers across this blueprint are sanitized using generalized descriptors (e.g., [Public Edge IP], [Private Mesh IP], [Compute Host Tailnet IP], and example.com).

2. End-to-End Hop-by-Hop Traffic Routing Architecture

2.1 Overview Table

Hop Number Layer / Component Functional Role Network / Security Boundary

Hop 1 Cloudflare Global Anycast Edge DDoS mitigation, WAF filtering, SSL/TLS termination, CDN static caching Public Internet → Edge CDN Boundary

Hop 2 Public Ingress Gateway Reverse proxy routing, SSL termination, Umami telemetry decoupling, Authelia SSO integration Public Edge Host ([Public Edge IP])

Hop 3 Private Overlay Transit Encrypted WireGuard tunnel, origin port isolation, interface binding Private Overlay Mesh ([Compute Host Tailnet IP])

Hop 4 Backend Compute Runtime Containerized Node.js/Express app, SQLite WAL persistence Isolated Local Compute Runtime Enclave

2.2 Hop 1: Cloudflare Global Anycast Edge

Inbound client traffic first contacts the outermost edge tier, which provides DDoS mitigation, Web Application Firewall (WAF) rule execution, public SSL/TLS termination, and static asset CDN caching. By terminating public client connections at the Anycast edge, no public origin IP address is exposed to WAN scans or target probes, completely abstracting the physical deployment site.

2.3 Hop 2: Public Ingress Gateway (Reverse Proxy & Analytics Decoupling)

From the Anycast edge, validated requests are forwarded to a public reverse proxy gateway running on an edge host ([Public Edge IP]). This layer performs three critical system functions:

  1. Forward-Authentication Verification: The gateway integrates with Authelia SSO to execute forward-authentication checks. Requests must pass identity verification before being forwarded downstream, injecting sanitized identity headers (X-Forwarded-User, X-Forwarded-Proto, X-Forwarded-Host) into the proxy pipeline.
  2. First-Party Telemetry Preservation: To maintain system observability without compromising client privacy or suffering payload loss, client-side tracking endpoints for Umami are reverse-proxied through first-party domain paths. Bypassing client-side ad-blocker drops ensures continuous performance telemetry while stripping sensitive outbound client headers prior to reaching internal analytics storage.
  3. TLS & Request Scrubbing: The gateway terminates transport encryption from Cloudflare, verifies incoming request signatures, and scrubs malformed headers before piping clean requests across the private transit boundary.

2.4 Hop 3: Private Overlay Transit (Encrypted WireGuard Mesh)

Traffic passes from the public ingress gateway to the backend host ([Compute Host Tailnet IP]) across an encrypted WireGuard mesh overlay (such as Tailscale).

The backend compute node operates under strict origin isolation:

  • Zero WAN Exposure: Standard web ports 80 and 443 remain completely closed on public-facing network interfaces. Host firewall rules drop all incoming unsolicited WAN connections.
  • Interface Binding: The Node.js application process binds exclusively to the private mesh overlay interface ([Private Mesh IP]) and loopback adapter.
  • Routing Policies: When active client workstations or edge proxies utilize exit-node routing across the mesh, local network bypass policies (—exit-node-allow-lan-access=true) are explicitly enforced to ensure internal LAN traffic and overlay mesh packets do not drop or route symmetrically out through unexpected external hops.

2.5 Hop 4: Backend Compute Runtime & Persistence

The terminal layer of the pipeline is a containerized Node.js application executing an Express server (mapped internally to local port 3002). Application persistence relies on an embedded SQLite database configured for high-concurrency read and write operations.

To guarantee zero database locking under heavy multi-process access, the runtime executes specific performance PRAGMAs during database connection initialization:

— SQLite Performance & Concurrency Runtime Initialization

PRAGMA journal_mode = WAL;

PRAGMA synchronous = NORMAL;

PRAGMA busy_timeout = 5000;

PRAGMA temp_store = MEMORY;

Node.js Concurrency Engine Architecture

To manage simultaneous database access between HTTP application handlers and autonomous background daemons without file-lock contention:

  • Driver Selection: The application utilizes better-sqlite3 due to its synchronous, high-throughput C++ binding interface, executing queries directly on the V8 event loop thread.
  • Transaction Isolation: Background worker writes employ explicit IMMEDIATE transactions (db.transaction(…).immediate()), reserving a write lock immediately to prevent write-lock escalation deadlocks.
  • Deferred Readers: Web server HTTP request handlers use standard DEFERRED read transactions, allowing them to read uninterrupted from the WAL shared-memory file (.shm) concurrently while background write processes append to the WAL log (.wal).

3. Dynamic 3-Tier AI Swarm Engine

3.1 Engine Architecture Overview

Multi-agent interactions in Hall 3B rely on a dynamic 3-tier fallback AI engine. This architecture guarantees high availability, sub-second agent responses, and complete elimination of HTTP 504 gateway timeouts, even during local compute saturation or cloud API rate-limiting events.

Inbound User / Agent Prompt

       │
       ▼

[ Tier 1: Local AI ] ──(Success: <400ms)──► Return Agent Response

       │

(Timeout / Latency Spikes)

       │
       ▼

[ Tier 2: Cloud API Endpoints ] ──(Success: <800ms)──► Return Agent Response

       │
  (Cloud Outage / Rate Limit)
       │
       ▼

[ Tier 3: Heuristic Lore Engine ] ──────────────► Instant Fallback (<10ms)

3.2 Tier 1: Local-First Inference via Private Mesh

Under nominal conditions, agent prompts route over the private overlay mesh to a local AI runtime on local compute resources ([Compute Host Tailnet IP]). The daemon executes quantized open-weight local AI models. This local-first paradigm eliminates API operational costs and keeps private interaction data contained within the internal network enclave.

3.3 Tier 2: Cloud Model Failover & Circuit Breaker

If local GPU or CPU inference queues saturate and latency exceeds operational bounds, an automated circuit breaker trips within a sub-400ms window to fail over seamlessly to cloud model endpoints.

// Node.js Sub-400ms Circuit Breaker Request Wrapper

async function fetchAgentInference(prompt, systemContext) {

const controller = new AbortController();

const timeoutId = setTimeout(() => controller.abort(), 400);

try {

const response = await fetch('http://[Compute Host Tailnet IP]:PORT/api/chat', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  signal: controller.signal,
  body: JSON.stringify({
    model: 'LOCAL_MODEL',
    messages: [{ role: 'system', content: systemContext }, { role: 'user', content: prompt }],
    stream: false
  })
});
clearTimeout(timeoutId);
const data = await response.json();
return data.message.content;

} catch (error) {

clearTimeout(timeoutId);
// Circuit breaker tripped: Execute Tier 2 Cloud Failover Handler
return await executeCloudFailover(prompt, systemContext);

}

}

3.4 Tier 3: Procedural Heuristic Lore Fallback Engine

If cloud provider endpoints experience widespread outages, network routing failures, or API rate-limit errors, traffic falls back to the Tier 3 safety net: an embedded procedural lore engine.

Executing in under 10 milliseconds, this component bypasses neural network inference entirely:

  • Engine Design: Utilizes deterministic finite-state machines combined with pre-compiled lore template lookup tables and rule-based pattern matching (Markov chains and structured state trees).
  • Guaranteed Uptime: Contextually appropriate, lore-compliant dialogue strings are assembled in memory and delivered instantly to the client layer. This guarantees 100% platform availability and ensures HTTP 504 gateway timeouts are mathematically impossible.

3.5 Tier Comparison Table

Tier Level Execution Target Latency Profile Failure Trigger

Tier 1: Local Local AI runtime ([Compute Host Tailnet IP]) Variable (150ms – 380ms) Sub-400ms latency timeout / Queue saturation

Tier 2: Cloud External Cloud AI API Endpoints Low / Predictable (300ms – 800ms) API HTTP 5xx error / Rate limit (429) / WAN fault

Tier 3: Heuristic Embedded Procedural State Engine Instant (<10ms) Tier 2 failure / Total WAN disconnect

4. Decoupled Autonomous World Heartbeat

4.1 Cron-Driven Background Worker

Platform autonomy—simulating background hallway chatter, dorm room events, and multi-agent social interactions—is architecturally decoupled from inbound user HTTP traffic. An autonomous background daemon executes on fixed cron schedules, driving simulation state ticks independently of active web client sessions.

[ Cron Event Trigger ]

      │
      ▼

[ Heartbeat Worker Process ]

      │
      ├─► 1. Evaluate Agent State Vectors
      ├─► 2. Execute Tier 1/2 Inference Batch
      └─► 3. Append State Updates to Database

4.2 Asynchronous State Persistence

To prevent background simulation processes from causing frame drops or query latency for active web users, the heartbeat worker leverages the asynchronous performance features of the SQLite WAL architecture:

  1. Non-Blocking Write Queue: Simulation updates, agent movements, and chatter logs are batched in Node.js process memory before committing to disk.
  2. Immediate Lock Reservation: Write transactions explicitly open in IMMEDIATE mode, committing bulk updates in contiguous blocks to minimize the duration of write locks on the shared .wal file.
  3. Memory Mapped I/O: The web application and heartbeat process share database memory pages via SQLite’s shared-memory (.shm) map, allowing HTTP read operations to query active state changes milliseconds after the background worker commits them.
  4. Key Takeaways & Homelab Engineering Principles

5.1 Summary of Architectural Principles

  1. Strict Perimeter Isolation: Concealing internal compute nodes behind private overlay mesh networks and public proxy gateways prevents exposure to WAN port scans and targeted edge probes.
  2. Resilient AI Fallback Chains: Combining local quantized inference engines with automated sub-400ms circuit breakers, cloud failovers, and deterministic fallback heuristics guarantees 100% uptime and eliminates HTTP 504 gateway timeouts.
  3. Concurrency-Friendly Local Storage: Operating embedded SQLite in Write-Ahead Logging (WAL) mode with tuned PRAGMA parameters (synchronous=NORMAL, busy_timeout=5000) delivers low-overhead, high-concurrency local persistence without the complexity of managing heavy database server clusters.
  4. First-Party Telemetry Preservation: Reverse proxying internal analytics payloads through primary domain routes prevents client-side ad-blocker suppression while enforcing strict header sanitization before reaching analytics storage.