Roadmap

Defined the flat `NormalizedRecord` shape every connector maps its source's payload into, so contract awards, grant opportunities, and entity registrations can share one filterable, sortable list instead of living in four separate views. Source-specific fields that don't map to a column survive in an `extra` bag rather than being dropped, so nothing is lost on the way in.

USAspending's award-search API was the right one to build first: no API key, JSON rather than XML, and it already covers most of the contract-award data FPDS carries. Paging is bounded by configuration rather than fetching the full corpus, which is a deliberate limit the removal-detection step below has to unwind later.

The core of the product. Every poll hashes each normalized record and compares it to what's stored: new records are inserted and logged as `created`, changed records are updated in place and logged as `updated` with the exact field-level before/after values, and unchanged records just bump `lastSeenAt` without generating noise. Every change event is tied to the poll run that produced it, so both "what changed in this poll" and "what is this record's full history" are answerable from the same table.

A per-record detail page rendering the full change timeline — every field that moved, its previous value, its new value, and when the poll caught it. This is the view that proves the pipeline is doing something the source systems can't do at all.

Flagging the records that actually matter, so the list can be narrowed to them. On its own this is just a filter; it exists now because it's the thing notifications will eventually hang off, and getting the data model right early is cheaper than retrofitting it.

Free-text search plus filters on source, record type, agency, amount range, and changed-since, with sorting on every column that supports it. CSV export accepts the same parameters and streams the filtered view with CRM-friendly column names rather than raw API field names — the first, manual version of the CRM pipeline planned for Open Beta.

SQLite was a dev-only placeholder and doesn't survive Vercel's serverless functions, which have no persistent filesystem. Switched the schema to Postgres and added a docker-compose Postgres for local development so dev and production run the same engine. Only the database runs in Docker — the dev server still runs on the host, so hot reload stays fast.

A GitHub Actions workflow calls the poll endpoint on a schedule, and that endpoint requires a shared secret when one is configured so it can't be triggered by anyone who finds the URL. Change-tracking is worth nothing if the polls only happen when someone remembers to click a button.

The positioning, audience, and phase plan — this roadmap, the README, and the planning and project-management conventions the repo follows. Deliberately done while the project is small enough that the answers can still change cheaply.

A reverse-chronological view of every change across every source, filterable down to watched records only, source, and change type. This is the first and cheapest form of notification: no delivery infrastructure, no accounts, no email reputation to manage — the change data already existed in ChangeEvent and just needed somewhere to be read. Email and chat delivery are later phases precisely because this has to prove the alerts are worth receiving first.

Verified the search2 request/response shape live against the running API — it matched what the connector already assumed. Found and fixed one real bug in the process: search2 returns dates as MM/DD/YYYY, not ISO, and the connector was passing them straight into periodStart/periodEnd (documented and typed as ISO everywhere else in the app). Added a converter, added pagination (previously single-page only, silently capped at 100 results), and added per-request instrumentation matching the USAspending connector's pattern. This is what makes the "one list" claim true for grant seekers rather than just contractors.

The list rendered eight columns out of the twenty the API was already returning, and the API itself was still dropping currency, PSC code, and description on the way out. Now every field is a column and the table scrolls wide, with the watch toggle and title frozen to the left edge so a row stays identifiable at any scroll position. No column controls yet, deliberately: during dogfooding you can't know which fields matter until you've lived with all of them, and saved column preferences are the right way to narrow this later — not a smaller default now.

The detail API already returned every normalized field plus the raw source payload, parsed and ready. The page declared a fourteen-field interface and rendered about six of them — so the raw payload crossed the wire on every request and was discarded by the client. Now every normalized field renders, the `extra` bag gets its own section, and the raw JSON payload is available in a collapsible panel — the remaining half of the principle in DESIGN.md: a user who can't see a field has no way to know it changed.

The CSV export wrote seventeen curated, CRM-shaped columns and silently omitted currency, PSC code, description, and the entire `extra` bag — the place every field that didn't map to a column ends up. The CRM shape stayed, as an option (`?format=crm`) rather than as the only way out — the default export now includes every field we hold on the records that were filtered, extra bag and raw payload included.

`POLL_TRIGGER_SECRET` was optional even in production, and when it was unset the poll endpoint ran for anyone who found the URL. `POST /api/poll` now refuses to run (503) when `NODE_ENV=production` and the secret isn't set; it stays optional in development, where hammering the endpoint costs nothing.

A `PollRun` recorded one row per run: did it finish, how many records came back. That couldn't answer "how many API calls actually worked," because a single poll makes many paginated HTTP requests and none of them were recorded individually. Added a `PollRequest` model (connector endpoint, status code, latency, attempt number, error) tied to `PollRun`; the USAspending connector logs one row per paginated call, including the calls that succeeded before a later one failed. Makes three things visible that were invisible before: real success rates, latency degrading before it becomes failure, and partial failures, where a run reports success having quietly dropped some pages. It's also the hard prerequisite for the public status page in Open Beta; the page is easy, the data underneath it was the work.

Every run already recorded its source, timing, counts, and error message — none of it was visible anywhere but the database. A silently failing daily poll looked exactly like a quiet week, which is the single worst failure mode this product has. A new `/poll-history` page reads PollRun history plus the per-request data above — aggregated request counts, failure counts, and average latency per run.

There were none. Added Vitest, extracted the diffing logic out of `sync.ts` into a pure, database-free `diff.ts` module, and covered it: identical records, single/multiple field changes, null/undefined equivalence, the `extra` bag, and that diffing only ever inspects the tracked field list. Connector normalization for USAspending and Grants.gov is covered against recorded fixture payloads, asserting full-field mapping and that unmapped fields land in `extra` rather than being dropped. `npm run test` now gates CI alongside lint, typecheck, and build.

The API already supported record-type, amount-range, and changed-since filtering plus page/pageSize — none of it was reachable from the UI, and neither the record list nor the change feed had a way to move past their first 50 results. Added the missing filter controls, Prev/Next pagination with a page-size selector, and made clicking a column header sort by it (previously sorting only lived in a dropdown disconnected from the table it sorted). Filter/sort/page state now lives in the URL, so a filtered view is a link that can be bookmarked or handed to a teammate instead of a private arrangement of controls that resets on reload.

Three per-browser display preferences, all localStorage-backed: a Columns panel (checkbox to hide, arrows to reorder, reset to default), a comfortable/compact row density toggle, and a light/dark mode switch for basic accessibility. These are deliberately local-only stand-ins for the real per-account saved views in the `columns-and-views` Closed Beta step below, not a replacement for it — see that step's note.

The per-event change timeline already existed, but it only ever showed one hop at a time. Added a "drift since first seen" section to the record detail page comparing the oldest ChangeEvent's snapshot against the record's current normalized state — the actual answer to "what's different about this record today versus the day we started watching it," which is the thing a spreadsheet snapshot fundamentally cannot produce. Comparing against the raw Prisma columns instead of the normalized snapshot produced false-positive drift on date fields (different serialization of the same date) — caught in manual testing before it shipped.

Moved every tool page under `/app/*` and built a real static marketing page at `/` — mission statement, the four personas as a teaser, a short differentiation pitch — instead of a bare redirect into the tool. This is preparation for two things this project will need soon: a real marketing presence, and an auth gate in front of the tool once Closed Beta needs accounts. The tool's nav and any tool-specific banners are scoped to `/app/*` now, not the root layout, since the marketing page is a narrative surface per DESIGN.md and shouldn't carry the tool's chrome.

Production has never had a real `DATABASE_URL` (see `production-deploy` below), so `/api/records` was 500ing there — the app looked broken to anyone who visited before that's fixed. Rather than leave it that way, the app now falls back automatically to a static data snapshot whenever `DATABASE_URL` is unset: zero-config, and it switches back to live data the moment a real database is connected. A visible banner says plainly that the data is a static snapshot and from when, and watchlist/poll actions refuse with a clear message instead of failing silently, since there's nowhere for them to write. This is a stopgap, not a substitute for `production-deploy` — the snapshot only ever moves forward when someone re-runs the export script by hand.

Audited every page at a phone width and fixed the two real horizontal-overflow bugs found (the nav bar and a couple of long unbreakable IDs on the record detail page forcing the whole page to scroll sideways). Beyond just "not broken," added a genuinely mobile-native way to work through records: a swipeable card view, one record at a time, defaulting on under a 640px-wide screen. Swipe left or right (or tap explicit buttons, for anyone not on touch) to skip or watch; which direction means which is a saved user preference rather than an assumed convention, since swipe apps don't agree on this. The wide table stays the tool for a desktop-width screen — DESIGN.md's density rules are about the work surface, not a mandate that every view has to be a table.

The first version of the card view swapped records instantly on a decision, with no real transition — confusing rather than decisive, and easy to lose track of what you'd already gone through. Added a real exit animation (the current card flies off in the swipe direction while the next one, already peeking behind it scaled down, grows into place), a typewriter reveal on each card's title and description, and a toast confirming which action was taken. Also added a third gesture, swipe up, for sharing via the Web Share API (native OS share sheet) with a clipboard fallback where that's unavailable — plus explicit "Copy for LLM" and "Share" buttons producing a clean plain-text block, since a phone user's next move is often pasting a record into a chat rather than exporting a CSV. The filter bar now collapses behind a toggle in card mode only, so the card gets near-fullscreen room to swipe in.

FPDS's public site was decommissioned in February 2026 and its ATOM feed retires later in FY2026, replaced by the SAM.gov Contract Awards API (https://open.gsa.gov/api/contract-awards/). This is more urgent than it looks: the scaffold's `fpds` connector still carries instructions to build an XML parser against a feed with months to live, so the plan is actively wrong rather than merely unbuilt. The migration is a net simplification — REST/JSON instead of ATOM/XML, and it collapses FPDS and SAM.gov into one connector family. GSA publishes a field-level variance document between the two.

SBIR.gov publishes a free, key-less API covering both awards and open solicitations (https://www.sbir.gov/api). It's the best value-per-effort connector available: no authentication, JSON, and it serves the small-contractor and grant-seeker audiences at the same time — SBIR/STTR is often the first federal money a small technical company ever wins, which makes it the exact record type our earliest users care about.

Built against the documented field schema for both the Award and Solicitation APIs, but left disabled: SBIR.gov's own docs page states the APIs are "currently undergoing maintenance," and every request returned 403 regardless of headers — confirmed this wasn't specific to our environment, since other .gov APIs worked fine from the same network at the same time. Genuinely blocked on the upstream API coming back, not on more work here. Verify a live response actually matches the documented shape before enabling once it's back.

Full keyboard navigation of the list — move between rows, toggle a watch, jump to the filter, open a record without reaching for the mouse. This is what separates a dense table from a fast one, and it's the cheapest large win available because it needs no accounts and no new data. See DESIGN.md: we are replacing a spreadsheet, and spreadsheets are keyboard-driven.

A record vanishing from a source is a change, and often the most interesting one. The schema already carries `removedAt` and nothing sets it. Deliberately scoped to a single connector that can fetch its full corpus rather than to the whole pipeline: a windowed poll can't distinguish "removed" from "outside this page," and the unbounded-polling work that would fix that generally is an Open Beta concern. Proving the mechanic on one complete source is buildable now; making it universal is not.

Slack, Teams, Jira, webhook and email as configurable per-organization integrations that a watch delivers through, plus a delivery log — because "we sent it" and "it arrived" are different claims, and only the second matters when a customer says they never got the alert.

Partially shipped, and the UI says which parts: webhook and Slack deliver, Teams and Jira and email are marked "not sending yet". Offering a channel that silently never delivers is worse than not offering it — the user wires it up, tells their team alerts are on, and finds out weeks later they weren't.