Architecture
MicroLog is split across two deployment targets that communicate over the internet. No log data is stored on the cloud server — only authentication metadata lives there. All logs land on your local machine.
Deployment targets
- Server (cloud VPS) — runs
mosquitto,ingest-go, andhub. Holds a SQLite DB for auth and alert rules only. - Local PC — runs
collectoranddashboard. All logs land in./local-logs/<projectId>/<serverId>/YYYY-MM-DD.logas NDJSON. - Monitored server(s) — runs
agent-goas a Docker container. Can be the same machine as the cloud VPS or any other server.
Data flow
MQTT topics follow the pattern raw/{serverId}/{kind} (agent → ingest) and live/{projectId}/{serverId}/{kind} (ingest → hub → subscribers). The kind is one of metrics, heartbeat, logs, or containers.
Components
agent-go
Runs as a Docker container on every server you want to monitor. Reads /host/proc for CPU, memory, load, disk I/O, and network stats. Tails /host/log/syslog and auth.log. Optionally polls the Docker socket for container state. Publishes all events to mosquitto over MQTT with TLS.
Source: apps/agent-go/ · MQTT topic: raw/{serverId}/{kind}
mosquitto
MQTT broker. Exposes two listeners: :8883 (external, TLS) for agents, and :1883 (internal Docker network, plain) for ingest and hub. Access is controlled via a password file and an ACL file that grants agents write-only access to raw/# and restricts hub to read-only on live/#.
ingest-go
Subscribes to raw/# on mosquitto. For each message, validates the serverId against SQLite (with a 60 s in-memory cache). On success, injects projectId into the payload and republishes to live/{projectId}/{serverId}/{kind}. Rejected messages are silently dropped.
Source: apps/platform/ingest-go/
hub
Central broker and HTTP API. Subscribes to live/# via MQTT. Buffers the last 600 metric points per server in memory. Runs alert rules from DB every 30 s. Detects offline servers every 5 s (15 s heartbeat threshold). Broadcasts all events to connected dashboard and collector clients over WebSocket.
Also serves: / (landing), /docs, /register, /api/login/request, /api/login/poll, /magic, /servers, /agent-install.sh, /agent-bundle.tar.gz.
collector
Connects to the hub as a WebSocket client. Writes every received event to NDJSON log files at ./local-logs/{projectId}/{serverId}/YYYY-MM-DD.log. Runs indefinitely and handles hub reconnection automatically.
Source: apps/platform/collector/collector.ts
dashboard
Static frontend served by bun x serve. Connects to hub via WebSocket using a JWT obtained via magic-link login (/api/login/request + /api/login/poll). Displays real-time charts (CPU, memory, disk, network), container status cards, and a live log stream. No build step — plain JS files.
Source: apps/dashboard/frontend/
Environment variables
agent-go
| Variable | Default | Description |
|---|---|---|
| API_KEY | — | Required. API key assigned to this server in the DB. |
| SERVER_ID | — | Required. UUID of this server in the DB. |
| MQTT_BROKER | ssl://gramastudio.cl:8883 | MQTT broker address for the agent. |
| MQTT_AGENT_PASS | agent-secret | Password for the shared agent MQTT user. |
| PROC_ROOT | /host/proc | Mount point for the host /proc filesystem. |
| LOG_ROOT | /host/log | Mount point for the host /var/log filesystem. |
| DOCKER_CONTAINERS | — | all or comma-separated container names to monitor. |
| DOCKER_SOCKET | /var/run/docker.sock | Path to the Docker Unix socket inside the container. |
ingest-go
| Variable | Default | Description |
|---|---|---|
| MQTT_BROKER | tcp://mosquitto:1883 | Internal broker address. |
| MQTT_USER | ingest | MQTT username for ingest. |
| MQTT_PASS | — | Required. MQTT password for ingest. |
| SQLITE_PATH | /app/data/microlog.db | Path to the SQLite database. |
| AUTH_DEBUG | 0 | Set to 1 to log auth decisions. |
hub
| Variable | Default | Description |
|---|---|---|
| HUB_PORT | 4001 | HTTP/WebSocket port. |
| SQLITE_PATH | auto-detected | Resolved in order: env → /app/data/microlog.db → ./platform/database/microlog.db. |
| MQTT_BROKER | mqtt://mosquitto:1883 | Internal broker address. |
| MQTT_HUB_USER | hub | MQTT username for hub. |
| MQTT_HUB_PASS | — | MQTT password for hub. |
| HUB_WRITE_ALERTS | true | Set to false in production to skip writing alert log files. |
| RESEND_API_KEY | — | Optional. Enables email notifications via Resend. |
collector
| Variable | Default | Description |
|---|---|---|
| HUB_URL | ws://localhost:4001 | WebSocket URL of the hub. |
| PROJECT_ID | — | Required. UUID of the project to subscribe to. |
| LOG_ROOT | ./local-logs | Root directory for NDJSON log files. |
Admin CLI
All commands run with bun run apps/platform/admin/cli.ts <command> [flags].
Alert --rule values match metric keys from the agent payload: cpu, memory, diskUsage, load, etc. The special rule offline triggers on heartbeat timeout (no threshold required).
Scripts
All scripts live in scripts/, load a .env file from the project root if present, and prompt interactively for any missing required values. Run them from the root of the repo.
start-server.sh
Launches mosquitto + ingest-go + hub via infra/docker-compose.server.yml. Before starting, checks that the three TLS certificate files (chain.pem, fullchain.pem, privkey.pem) exist in infra/mosquitto/certs/ and aborts with instructions if any are missing. Prompts interactively for MOSQUITTO_INGEST_PASS and MOSQUITTO_HUB_PASS.
Accepts -d to run in background.
init-mosquitto.sh
Creates the ingest, hub and agent MQTT users inside the running mosquitto container and reloads the password file. Run after start-server.sh on first deploy, or whenever passwords change. Prompts for the three passwords interactively.
The agent password set here must match the MQTT_AGENT_PASS used in start-agent.sh.
start-agent.sh
Builds the Go agent image from apps/agent-go/ (requires the repo to be present on the server), removes any existing microlog-agent container, and starts a new one. Prompts for API_KEY, SERVER_ID, MQTT broker, agent password, and optionally which Docker containers to monitor.
Default broker: ssl://gramastudio.cl:8883. Accepts -d to run in background.
start-dashboard.sh
Launches collector + dashboard via infra/docker-compose.local.yml. Prompts for HUB_URL (default: ws://log.gramastudio.cl:4001), PROJECT_ID, and optionally an auth token. The collector writes NDJSON logs to ./local-logs/; the dashboard is served at http://localhost:5178.
Accepts -d to run in background.
Self-hosted setup sequence
Docker Compose
| File | Target | Services |
|---|---|---|
| infra/docker-compose.server.yml | Cloud VPS | mosquitto, ingest (Go), hub (Bun) |
| infra/docker-compose.local.yml | Local PC | collector, dashboard |
Database schema
The SQLite schema lives in apps/platform/database/schema.sqlite.sql and is auto-applied on first run if the users table is absent. Tables: users → projects → servers (holds api_key), alerts, invite_tokens, config.