Tech Stack

Data Pipeline

Data flows through five stages on a continuous loop:

Kalshi API ──▶ Ingestion ──▶ PostgreSQL ──▶ Scoring ──▶ Alerts ──▶ Email
                  │                            │           │
              Every 5 min                  Every 5 min   Every 5 min
              (markets,                    (6 signals     (if score
               trades,                     computed,      exceeds
               orderbooks)                 weighted,      threshold)
                                           labeled)

1. Ingestion Layer

The ingestion layer pulls data from the Kalshi public API and stores it in PostgreSQL.

Market Sync

Streams all open markets from Kalshi page-by-page via async generators (200 per batch) to stay under the 512MB RAM limit. Skips the Sports category. Maps each market to its event category. Marks markets as eligible or ineligible based on volume and open interest thresholds.

Trade Sync

For each eligible market, fetches recent trades since the last known timestamp. Stores individual trade records with price, size, side, and timestamp.

Orderbook Sync

Snapshots the current bid/ask/spread for eligible markets. Hardened against malformed API data with try/except guards around float conversions.

API Client

Async httpx client with cursor-based pagination, automatic rate limit handling (auto-retry on HTTP 429), and batched async generators to keep memory usage low.

2. Scoring Engine

The core intelligence. Computes a composite anomaly score (0–100) for each eligible market using six weighted signals. Each signal compares the market's current behavior to its own 7-day historical baseline.

A confirmation gate prevents false positives: at least one primary signal (Trade Size or Price Impact) must score 60+ and one secondary signal must score 40+ for a market to reach "High" or above. Markets failing the gate are capped at 54.99.

For full scoring details, see How It Works.

3. Alert System

Alert Generator

After each scoring cycle, checks if any market exceeds the user-configured score threshold (default: 70). Includes deduplication: won't re-alert the same market within 4 hours unless the score jumps by 15+ points (escalation detection).

Email Delivery

Sends via the Resend API using async httpx. Supports multiple comma-separated recipients. Two email types:

• Instant alerts — fired per-market the moment a threshold is crossed
• Daily recap — top 10 alerts from the past 24 hours, sent at 8pm ET

4. Database Schema

PostgreSQL with connection resilience: pool_pre_ping, 300s pool recycle, and 5-attempt startup retry with backoff.

5. Web Interface

Server-rendered HTML with Jinja2 templates and Pico CSS. HTMX provides interactive features (watch/dismiss buttons) without a JavaScript framework.

All timestamps display in 12-hour AM/PM Eastern time via a custom Jinja2 filter that converts UTC to US/Eastern with proper EST/EDT handling.

6. Background Scheduler

All background jobs run in-process via APScheduler. Each job is wrapped in an independent try/except so one failure doesn't cascade to the others.

7. Deployment

GitHub push ──▶ Railway auto-deploy ──▶ Docker build ──▶ Container start
                                                          ├── Uvicorn (web server)
                                                          ├── APScheduler (background jobs)
                                                          └── PostgreSQL (Railway service)

Single-container architecture — the web server and all background jobs run in one process. This keeps costs minimal ($5/month on Railway Hobby) but means a redeploy briefly pauses both the UI and data ingestion.

Memory Management

Railway's container has a 512MB RAM limit. To stay within this:

• Market data is streamed page-by-page via async generators instead of loaded all at once
• Sports markets are skipped entirely during ingestion (they account for a large portion of Kalshi's catalog)
• gc.collect() is called after each batch to free memory promptly
• Database connection pool is kept small (3 connections + 5 overflow)

Resilience

• Database: 5-attempt startup retry with exponential backoff, pool_pre_ping to detect stale connections, 300s pool recycle
• API: Automatic retry on rate limits (HTTP 429) with backoff
• Jobs: Each scheduled job runs in independent try/except — one failure won't take down the others
• Timezone: Uses ZoneInfo("US/Eastern") for proper EST/EDT handling instead of hardcoded offsets