Hourly freebuff bot-sweep dry-run endpoint

[jahooma]

Summary

• New admin endpoint POST /api/admin/bot-sweep, guarded by a BOT_SWEEP_SECRET bearer token (timing-safe compare). A new hourly GitHub Action hits it; the endpoint scans active/queued free_session rows, ranks likely-bot users by a tiered score (24-7 usage, heavy msg volume, young accounts, alias-email patterns, etc.), and emails a report to james@codebuff.com. It's a DRY RUN — no bans are issued automatically.
• Adds two helper scripts used during the 2026-04-21 manual ban sweep: scripts/inspect-freebuff-active.ts (snapshots current sessions with clustering signals) and scripts/unban-freebuff-users.ts (reverse of the ban script). Also promotes FREEBUFF_ROOT_AGENT_IDS to a shared constant so both the CLI script and the production sweep count the same agent IDs.

Test plan

• Set BOT_SWEEP_SECRET in Infisical (prod) and as a GitHub repo secret
• Deploy web and manually run the GitHub Action via workflow_dispatch
• Confirm the email lands in james@codebuff.com and the column formatting survives the Loops template
• Verify unauthorized requests get 401 and missing-secret returns 503

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR adds a read-only hourly bot-detection sweep for freebuff abuse: a new POST /api/admin/bot-sweep endpoint (guarded by a timing-safe bearer-token check) queries active/queued free_session rows, scores users across several heuristics (24/7 usage, heavy message volume, young accounts, email-alias patterns, device-fingerprint sharing, creation clusters), and emails a ranked report to james@codebuff.com. A GitHub Actions workflow fires it every hour. Two manual-sweep helper scripts (inspect-freebuff-active.ts, unban-freebuff-users.ts) and a shared FREEBUFF_ROOT_AGENT_IDS constant are also included.

Key findings:

• Email failure is silently ignored (P1): when sendBasicEmail fails, the route still returns HTTP 200. The GitHub Actions workflow only checks the status code, so a broken Loops configuration would cause the sweep to run indefinitely with no reports delivered while all GH Actions runs appear green.
• Hourly emails with zero suspects (P2): the email is sent unconditionally — if there are no sessions or no flagged users, a "0 medium suspects" email fires every hour.
• distinctHours24h undercounts across day boundaries (P2): EXTRACT(HOUR FROM …) returns the clock-hour (0–23), so the same hour can appear twice in a rolling 24h window that spans two calendar days. DATE_TRUNC('hour', …) would give a unique value per calendar-hour slot and avoid the undercount.

Confidence Score: 4/5

Safe to merge with low risk — read-only reporting feature with no automated bans; issues are operational reliability concerns only

The P1 concern (silent email failure returning 200) is an operational reliability gap but causes no data loss, incorrect bans, or security exposure. The P2 heuristic issue causes false negatives in edge cases only. Auth, scoring logic, cluster detection, env schema, and GH workflow are all solid.

web/src/app/api/admin/bot-sweep/route.ts — silent email failure and unconditional email send

Important Files Changed

Filename	Overview
web/src/app/api/admin/bot-sweep/route.ts	New admin endpoint with timing-safe auth; silently returns 200 when email delivery fails (GH Actions won't alert) and sends hourly emails even with 0 suspects
web/src/server/free-session/abuse-detection.ts	Core heuristic engine; EXTRACT(HOUR) for distinctHours24h can undercount across day boundaries — DATE_TRUNC('hour') would be more precise
.github/workflows/bot-sweep.yml	Hourly GH Actions workflow; correctly handles missing secret, sets max-time, and exits non-zero on non-200 — only issue is it won't detect silent email failures (see route.ts comment)
common/src/constants/free-agents.ts	Promotes FREEBUFF_ROOT_AGENT_IDS to a shared constant so scripts and prod code use the same agent list
packages/internal/src/env-schema.ts	Adds optional BOT_SWEEP_SECRET (min 16 chars) to the server env schema; correctly optional so dev environments can start without it
scripts/inspect-freebuff-active.ts	Manual inspection script for active/queued sessions; safe read-only tool
scripts/unban-freebuff-users.ts	Safe unban helper; dry-run by default with explicit --commit flag

Sequence Diagram

sequenceDiagram
    participant GHA as GitHub Actions (hourly)
    participant API as POST /api/admin/bot-sweep
    participant DB as PostgreSQL (free_session + messages)
    participant Loops as Loops (email)
    participant Reviewer as james@codebuff.com

    GHA->>API: POST with Bearer BOT_SWEEP_SECRET
    API->>API: timingSafeEqual auth check
    alt secret missing
        API-->>GHA: 503
    else invalid token
        API-->>GHA: 401
    else auth OK
        API->>DB: SELECT free_session LEFT JOIN user
        DB-->>API: sessions[]
        API->>DB: SELECT message stats (24h FILTER + lifetime COUNT)
        DB-->>API: msgStats[]
        API->>API: score & tier suspects, findCreationClusters()
        API->>Loops: sendBasicEmail(subject, message)
        Loops-->>API: { success: true|false }
        API-->>GHA: 200 { ok, suspectCount, emailSent }
        Loops-->>Reviewer: ranked suspect report (if success)
    end

Loading

_{Reviews (1): Last reviewed commit: "Hourly freebuff bot-sweep dry run + ban ..." | Re-trigger Greptile}

[greptile-apps]

Email failure silently returns HTTP 200

When sendBasicEmail fails (e.g., LOOPS_API_KEY is unset or the Loops API is down), the route logs the error but still returns { ok: true, emailSent: false } with HTTP 200. The GitHub Actions workflow only checks for a non-200 status to decide whether to fail the run:

if [ "$status" != "200" ]; then
  exit 1
fi

This means a configuration problem (missing API key, wrong transactional template ID, etc.) would cause the sweep to run for weeks with no emails delivered, while GitHub Actions reports every run as green.

Consider returning a non-2xx status when the email fails:

if (!emailResult.success) {
  logger.error({ error: emailResult.error }, 'Failed to email bot-sweep report')
  return NextResponse.json(
    { ok: false, error: 'report generated but email failed', emailSent: false },
    { status: 502 },
  )
}

[greptile-apps]

Email sent every hour even with zero suspects

The email is dispatched unconditionally after every sweep. When there are no active/queued sessions or no suspects match any flag, the email subject will be [freebuff bot-sweep] 0 medium suspects (0 active+queued). Over time this produces hourly noise in the inbox even during quiet periods.

Consider skipping the email when there is nothing actionable:

const report = await identifyBotSuspects({ logger })

if (report.suspects.length === 0 && report.creationClusters.length === 0) {
  logger.info({ totalSessions: report.totalSessions }, 'bot-sweep: no suspects found, skipping email')
  return NextResponse.json({ ok: true, totalSessions: report.totalSessions, suspectCount: 0, emailSent: false })
}

const { subject, message } = formatSweepReport(report)
// ... rest of email send

[greptile-apps]

EXTRACT(HOUR) deduplicates across calendar-day boundaries — distinctHours24h can be undercounted

EXTRACT(HOUR FROM finished_at) returns an integer 0–23 (the clock-hour component). Within the rolling 24-hour window the same clock-hour can appear twice — e.g., messages at yesterday 14:00 and today 14:05 both extract to 14, so COUNT(DISTINCT EXTRACT(HOUR …)) counts them as a single distinct hour.

The 24-7-usage flag fires at distinctHours24h >= 20, so borderline bots near that threshold might be missed. DATE_TRUNC('hour', …) produces a unique value per calendar-hour slot and avoids the undercount:

distinctHours24h: sql<number>`COUNT(DISTINCT DATE_TRUNC('hour', ${schema.message.finished_at})) FILTER (WHERE ${schema.message.finished_at} >= ${cutoff})`,