Notes-to-self on the build-pulse pipeline: how raw git history becomes the landing chart, and why each cleanup exists. Scroll - the corner chart runs each step as you go.
Raw layer kept permissive on purpose: one row per commit, all authors, all branches. Team monorepos run to tens of thousands of commits, so the early years are mostly other people.
Biggest filter, applied first. A substring match on a company domain once grabbed the entire team. Bitwala commits used a domain-hack email; 2013–16 sat under old Gmail identities. Fix: a precise nine-identity allowlist. Red = the team coming off.
Commit counts aren’t comparable across a squash-merge switch: before, a PR is many tiny commits; after, it’s one. Early-2018 Bitwala - ~202 commits/mo collapse to ~6 PRs once squash turns on. Solution: count first-parent integration events (PR or direct commit), workflow-proof. Red = the granular commit-count removed.
No usable git (the repo was a 2021 archive dump), but 8,000+ files kept real 2016 mtimes. Treated as an estimate, not commits: per-day effort from file size × type, spikes winsorized, smoothed, clipped to the real window, scaled below the commit lenses. Green = the estimate. Flagged.
Contract started Oct 2017; git only from Mar 2018 - the old-codebase months were never captured. Filled by sampling typical (spike-excluded) Bitwala months. Green = the gap fill. Flagged estimate.
Two last calls: log compression (ln(1+x)) so a 200-event month doesn’t bury a 4-event one, and direct commits weighted 0.4 of a PR so direct-to-master personal work stays comparable to PR-merged professional work. This is the version that ships.