Substack and GitHub trade manual queries for operational evidence

Substack employees now use AI agents to query company data, replacing manual SQL commands with plain English requests to understand metrics like creator engagement (Sharon Goldman from Ground Level AI). Chris Best, Substack's CEO, credits this shift with preventing product missteps, such as a planned mobile app feature that only 1.5% of creators would have used. To ensure accuracy, the data team maintains a Slack channel where analysts verify AI-generated answers and correct the system's understanding of business context, a process that now supports roughly 70% of employees monthly (Sharon Goldman from Ground Level AI).

GitHub engineers migrated 430,000 lines of production code from TypeScript to Rust to decouple their Copilot runtime from the command-line interface, a task that would have historically required a large team over several years (ByteByteGo Newsletter). By using AI agents and the Copilot CLI, a single developer completed the transition in months, eliminating the 100 MB memory overhead previously required for Node and V8 machinery. This migration allows services to load the runtime directly into their own processes, improving performance for applications with strict resource constraints.

Forward-deployed engineers define project success by documenting who decides when a task is done and identifying where they will observe actual usage before writing code (The AI Engineer). They use answer keys, such as a set of six categorized emails, to score model performance against ground truth labels rather than relying on synthetic benchmarks (The AI Engineer). "You kind of have to go 10 or 20 layers deep before you find something that’s really important," Best noted, emphasizing that deep data investigation is often inaccessible without automated assistance.