One Month of Running a Weekly Accessibility Audit in Public: Every Number, Including the Embarrassing Ones


We spent one month running weekly accessibility scans and publishing the results. Here are every number we have, including the embarrassing ones.

Start with the production side, because that is the part we are least embarrassed about. In four weeks we ran four cohort scans with axe-core, published the summaries, and shipped 34 dev.to articles plus 9 Hashnode crossposts plus the blog itself. 80 sites got scanned across the four cohorts. 191 WCAG violations got written down with rule IDs, severity, and node counts. Of the 80 sites we scanned, zero were clean in the way we initially hoped someone would be.

The cohorts, in order:

Cohort 1 — Claude-generated UI components. Five components, generated by Claude Sonnet from a fixed prompt set, then audited. Three hits on region (content outside a <main> landmark) was the bulk of it. Small sample on purpose — this was the pilot for the whole format, and the question was “does AI ship accessible code out of the box” more than “how many violations can we find.” Answer: no, not really, and the failure pattern was consistent enough to be a template problem rather than a Claude problem.

Cohort 2 — SaaS pricing pages. Thirty sites. Twenty-one failed. 65 violations total, 548 affected DOM nodes. Color-contrast was the #1 rule, hitting 40% of the cohort (12 of 30). Linear, Render, and Intercom tied at 5 violations each at the top of the leaderboard. Nine sites came back clean — Figma, Netlify, Twilio, Webflow, Grammarly, Miro, Loom, Calendly, Zendesk. Roughly a third of the cohort produced no automated violations. No subsequent cohort has matched that rate. (full write-up)

Cohort 3 — our own blog. Sixteen pages on blog.a11yfix.dev. 27 violations across 15 of 16 pages, 1 clean page (the homepage). The irony that got into the title: we failed color-contrast on /blog/color-contrast-guide/. One shared newsletter component (.privacy-note, #898a8f on #f0f4ff, ratio 3.13:1) was responsible for 14 of the 15 color-contrast node failures. One <aside class="related-resources"> nested inside <main> was responsible for 11 landmark-complementary-is-top-level hits. Three code changes, about 25 of 27 violations fixed. We wrote a retrospective about exactly that, and then wrote this about writing that. (full write-up)

Cohort 4 — AI product landing pages. Twenty-nine sites: AI coders, AI search and assistants, writing tools, inference hosts. 29 out of 29 failed. 96 violations total, 512 DOM nodes. region (12 sites) and landmark-unique (7 sites) were the headliners — more than a third of the cohort shipped content living outside any landmark at all. We ran this one early because a reader suggested it on our SaaS pricing post, which matters more than the scan itself does. (full write-up)

Cumulative: 80 sites, 191 violations, and the “number of sites we audited that turned out to be completely clean” counter is stuck at nine, all of them from Cohort 2.

Now the part that is harder to look at.

Dev.to month one: 34 articles, 853 total views, 22 total reactions, 10 total comments. The math is easy and unkind — roughly 25 views per article averaged across the month. Our top post by views was the ARIA labels explainer at 224 views with 2 reactions. Our top post by reactions was a 3-reaction tie across the alt text guide and a 5-minute screen reader post, both of which pulled under 30 views each. The single most valuable post in terms of actual engagement was probably the SaaS pricing scan, because that is the one Ali Afana commented on, and that comment is what turned into the AI product landing cohort the following week. Comments beat reactions for us on that post. Honestly, comments beat reactions for us pretty much anywhere the comment came from a real person.

Hashnode: 9 crossposts, 10 total views. Not 10 views per post. 10 views across all 9 posts combined. That is not a typo and we did not forget a zero. Crossposting to Hashnode is costing us the 30 seconds it takes to paste the URL into the Distributor job, and the return on that 30 seconds is a number so small it is basically a reality check on what crossposting actually does when you do not have a Hashnode audience to start from. We are going to keep doing it because the cost is rounding error, but we are not going to count it as distribution any more.

Kit (newsletter) subscribers at end of month one: 1. That is me. Well, it is the A11yFix team account that we use to verify the double opt-in still works. A charitable reading is that month one was about testing whether the cohort-audit format produced articles anyone wanted to read at all, and we found out the answer is “sometimes, for ARIA Labels and the top EAA violations piece, yes.” An uncharitable reading is that 853 people read us on dev.to and exactly zero of them wanted more of this in their inbox, which means the funnel from reading to subscribing is broken somewhere we have not looked yet.

X followers: one. Still one. We’ll update this when the number moves.

So here is what is actually going on. We have a production engine that reliably ships a cohort scan and a summary article every week. We have a distribution channel that is slightly worse than we thought (dev.to views are real but thin, Hashnode is a rounding error, AI tag on dev.to seems genuinely dead, Reddit is blocked on us pending the API migration). And we have a conversion funnel that has not produced a single organic subscriber yet. All three of those things can be true at the same time and the first one does not rescue the other two.

The thing we are changing in month two is not the cohort scan schedule. That is working, in the sense that a weekly drop of real numbers is the thing we know how to do and the thing that at least produces artifacts that live on the web and accumulate. The thing we are changing is the mode. Month one was broadcast — we wrote summaries and pushed them out. The one time something conversational happened (Ali’s comment on SaaS pricing → our reply → the AI product landing scan the following week → the write-up crediting him in the opener), it produced more forward motion than the other 33 posts combined. It is the only piece of evidence we have about what month two should look like, and we are taking it seriously.

So month two: every scan article gets written with one specific reader in mind, not a general dev.to audience. Every comment on a post gets a real reply, not a thank-you. Topic selection for cohorts 5 and 6 is going to come from reader suggestions the same way cohort 4 did. If that means we publish slightly less and reply slightly more, we are OK with that trade.

We are also going to look hard at the reading-to-subscribing step. 853 views and 1 (me) subscriber is not a “keep doing what you are doing” number. It is a “something between the article and the subscribe button is not working” number. Maybe the signup block is in the wrong place. Maybe the offer is wrong. Maybe “subscribe for more a11y scans” is just not a compelling enough reason for someone who got what they needed from a single article. We will report back in the month two numbers post.

One more honest thing before we wrap. The 80-sites-audited number sounds bigger than it is, because we are counting the five Claude-generated components in cohort 1 alongside the 29 AI landing pages in cohort 4, and those are not the same kind of audit. The real “cohort scan of production sites” number is 75, not 80. The “WCAG violations documented with rule IDs that anyone could verify from the JSON we saved” number is 191, which we stand behind. The “sites we ran into that were genuinely clean” number is 9, all from one cohort, which means “clean” is rarer than we want to believe and also that our own blog is not in that 9.

Month one was mostly us learning that publishing a lot and converting anyone at all are very different problems. Month two is us trying to solve the second one.

Numbers log, so this is easy to compare against next month: 4 cohorts, 80 sites, 191 violations, 34 dev.to articles, 853 dev.to views, 22 reactions, 10 comments, 9 Hashnode posts, 10 Hashnode views, 1 Kit subscriber, 1 X follower, 9 clean sites out of 80.

See you at day 60.