JDOT #8: AIMAC Results and AI in 2026

Accessibility and Gen. AI Podcast & other appearances

Before I get to our benchmark, let me catch you up on the Accessibility and Gen AI Podcast episodes that Eamon McErlean and I have launched since the last newsletter, as well as a couple of activities I'm involved in.

Episodes #10-13 feature Josh Miller, CEO of 3Play Media; Victor Tsaran at Google; Molly Lazarus (Levitt) Director at Remarkable Tech and her guests Arman Nobari and Anthony Gibson from Good Trouble. Last but not least we just launched the episode with Eugene Woo at Venngage.

Correction, October 5, 2026: The original range was episodes 10 to 14. This January 5 edition covered episodes 10 to 13; episode 14 was published on February 16, 2026.

• Watch episode 10 with Josh Miller
• Watch episode 11 with Victor Tsaran
• Watch episode 12 with Molly Lazarus, Arman Nobari and Anthony Gibson
• Watch episode 13 with Eugene Woo

Screenshot of Eugene, Molly, Victor and Josh's thumbnails on YouTube

If you haven't checked out our podcast yet, have a listen and give us feedback!

I was interviewed by Tobias Macey for the AI Engineering Podcast.

Tomorrow, Tuesday January 6, I'll be moderating a panel at #CES with Peter Korn, Marilyn Morgan Westner, Rebecca Rosenberg and Daniela Braga, PhD. Be sure to catch up with us there.

AIMAC, the AI Model Accessibility Checker

Which AI coding models write HTML that is accessible to People with Disabilities?

AIMAC.ai looked at:

• 36 AI models
• 28 categories
• 1008 AI-generated HTML pages

The Landscape

If you're not in the weeds on who the contenders are when it comes to AI coding models, let's start by explaining the landscape. If you are an AI pro, you can skip over this section right to The Results.

You've almost certainly heard of ChatGPT from OpenAI. They kicked off the AI frenzy around November 2022 when people realized that AI was starting to sound human.

Correction, October 5, 2026: The original paragraph said November 2023. ChatGPT launched on November 30, 2022; the year is now corrected.

The first major AI product outside of chatbots to impact the world was coding. OpenAI has been the leader in the quality of their models, but some important employees left OpenAI and started other companies. The most important and successful offshoot at the moment is Anthropic. They were concerned that OpenAI didn't put enough energy behind AI safety and decided the only way for them to safeguard AI was to have their own company.

Google is another interesting story. Their employees invented the transformer technology that powers most of the AI technology you've heard of. They felt quite challenged and left out by these developments at first. However, they have all the structural components to win this game.

AI is expensive, but they have the cash. They own TPUs, the only chips outside of NVIDIA's GPUs that are needed to train models. They have some of the best AI talent in the world under the Google DeepMind division. And they have an enormous audience.

AI is Google's game to lose. If they do, it will be because of their corporate culture. It has ossified to a large degree, which will happen when your technical founders peace out while their cash cow, search, pumps out more cash than virtually any other company in history.

Those are the 3 major players in the AI coding game.

Trailing at 4th is xAI, Elon's company. They started very late and still lag by most measures. However, they have built a massive infrastructure using NVIDIA GPUs that may make them an eventual contender. They are probably number 1 when it comes to realtime data.

We had to disqualify two of their three models because, like many Open Source models that were disqualified, they output tiny HTML pages under 10KB. There's not enough content to make many accessibility mistakes, so it's not fair to compare them to most of the models in our benchmark. That doesn't mean these models are bad. It's just unfair to compare Apples to Oranges.

That leads neatly to the 5th group of models. The independent labs, with mostly Open Source models, by companies that have few resources compared to the Big 4. Most of these companies are Chinese, though before floundering with Llama 4, Meta had one of the best Open Source models out there. France has Mistral AI which produced some compelling models in the past.

Now that you understand the background, let's explain what a benchmark is and why create one?

Why do we need an accessibility benchmark?

AI model companies compete on benchmarks.

By the end of 2025, it's pretty clear that AI models have overtaken humans in terms of volume of code written. 2026 is poised to fundamentally change coding forever.

The bleeding edge of AI coding is a phenomenon that is moving faster than any technology I've ever seen. We'll discuss that a bit more later.

If the code that AI writes is inaccessible, it will be disastrous for users of Assistive Technology. It is imperative that the AI Model companies prioritize accessibility. And frankly, they are not. And they will not unless the community makes it clear that we will not accept any less.

I'm in the weeds every day. Coding from the moment I wake up, until the time I go to sleep. I coded in bed last night and I'm writing this from bed at 6am this morning.

When I tag employees at these companies, I get about a 30% response rate. Except when the topic is accessibility. So far, and I'm not exaggerating, I've gotten a 0% response rate from employees of AI companies when the topic is accessibility.

I'm not ascribing this to malice. Everyone is so busy innovating, they really don't want to be bothered or slowed down by making their coding models accessible. The results, for the most part, bear this out.

The Results

OpenAI was the winner. By a mile. They swept the Top 4 spots. Even their Open Source model came in at 11th. Nobody else was close.

The Top 10 Models on the AIMAC Benchmark

The next winner was totally unexpected. The independent lab MiniMax's M1 model tied for 4th! They are keeping pace with OpenAI at a fraction of the budget.

Other small and Open Source models owned the middle. Alibaba Group's Qwen and Kimi (Moonshot AI) both had models that beat Anthropic's best model.

Anthropic's Claude models were middle of the road. As a regular Claude Code user, I wasn't that surprised because as much as I love their product, it requires a lot of tricks and oversight to wrangle accessible output.

I had high hopes, considering how much their identity is wrapped around being the moral AI company, but unfortunately they have expensive models with mediocre accessibility.

Screenshot of Claude Code

The biggest shock of all was Google. Gemini 3.0 Pro performed so well on many benchmarks. But they came in dead last on AIMAC. 36 of 36. They own Lighthouse. They should have trained their models on it and had a perfect score. Let's hope they do better on the next iteration. cc Christopher Patnoe

Other uses for AIMAC

We implemented a couple of cool features to this benchmark that you won't find anywhere else. We included an emdash benchmark. How many emdashes does each model generate?

It's probably only of minor concern to accessibility, but if anyone wants to run a screen reader test against AI generated HTML with a ton of emdashes, they can use our benchmark for the data and/or guidance.

The other cool feature is that you can look at a model and compare how it performs across 28 different categories. Or you can pick a category and see how each model's designs look across the 36 AI Models tested.

There's nowhere else that lets you compare over 1,000 AI-generated designs.

Correction, October 5, 2026: The original sentence said over 1,000 AI models. The benchmark tested 36 models across 28 categories, producing 1,008 HTML pages. The sentence now refers to the generated designs.

How AI Coding will change the world in 2026

Andrej Karpathy, one of the best programmers in the world, wrote on December 26, 2025:

"I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between."

He goes on to say that he could 10X his productivity if he strung together the tools people are building. Personally, I've been using Steve Yegge's Beads, which is like Jira for AI, on top of Claude Code. As great as it is, Beads is being absolutely roasted right now as being "malware" for embedding itself too deeply in your system. A fair critique. There are third party viewers, none of which I like btw, on top of Beads.

Why do I mention it? Because the first commit on this project was ~3 months ago. The ecosystem is moving so quickly because only AI would allow you to build something so big, and so critical, so fast.

Using Claude Code is the most fun I've had as a programmer....ever. But thanks to a viral post by 🦄 Peter Steinberger, "Shipping at Inference Speed", paired with the results from AIMAC, I'm in the process of moving to OpenAI's Codex as my main driver. OpenAI's models are not the best communicators, but they are the best coders and the most accessible.

If you want to follow someone who is cranking out AI-generated open source projects that are simply breathtaking, follow Peter's work.

I'm also using Eric Provencher's excellent product, Repo Prompt, within Claude Code to hand over the hard work to Codex. To give you a small window into my daily workflow, here is how the image at the top of this edition was created.

Inside Claude Code, I used a slash command to ask Repo Prompt to ask Codex to make a plan, use the OpenRouter API I've set up for AIMAC, to hit Google's API, asking it to scrape AIMAC's website to garner insights on the benchmark, to generate that image. Other than asking it to focus on the Top 3 players, I didn't get specific on what the image should look like. Google's Nano Banana Pro is just that good.

2024 was the year of the AI IDE, 2025 the year of AI in CLIs, what is 2026?

Cursor dominated 2024 by bringing AI to the IDE, which is an interface programmers use to code. In 2025, Anthropic brought AI to the CLI, which you can think of as text messaging for programmers. Where you type text in a box and ask the AI to write your code.

What does 2026 look like?

I saw a demo of Vibe Kanban on YouTube the other day and am going to try it out this week, likely replacing Beads with it. In 2024 we really needed to write a lot of code, with AI. In 2025, we needed to spend time letting AI write most of the code, while we had to look at the "diffs", the changes in code. That's why we lived in the CLI.

But as AI crosses the Rubicon yet again, a term I use for when AI gets better at a task than humans, there will be little need for the CLI or to look at the diffs anymore. Vibe Kanban seems like a good paradigm of what's coming.

You configure all the AI models you pay for. The cheap models. The quick ones. The best coders who are slow. And the best hybrids. You write down your task. And you set up an army of agents that check the quality of the code that is written. They talk to each other. Jeffrey Emanuel created an email agent product (Open Source) that allows the agents to coordinate, discuss and fix code.

That's where AI is going in 2026. Swarms of AI writing the code, and fixing existing code. The second a bug report comes in, code will be written to fix it, and it will go through security checks, accessibility checks, and conversations between AIs who will converge on the best solution.

Check back in January 2027, and we'll see if I was right.

New Year, New AI and Accessibility Projects

Now that the new version of AIMAC has been released, I'm starting the New Year thinking about what new projects to work on. 2026 is not only going to be a big year for AI, but also for Accessibility.

First, let's name the elephant in the room. People's jobs have already been affected and it hurts. I know too many people out of work. Heck I got laid off myself due to AI and I had AI in my title!

Many want to put their heads in the sand. Stop AI right now. I totally get it. There are AI Doomers and AI Accelerationists. All seem positive about what the future of AI brings. But the reality is that no one knows. Personally I think it will be great and awful. I fear the bad. But I refuse to pretend that AI can be stopped today. It is too late. So we can go all in and have a say. Or bow out and be impotent watchers.

We need to understand that there's an opportunity here that didn't exist before. We spent a few decades as an industry, trying to make digital products better for People with Disabilities. And yet, we get shockingly awful results from the WebAIM Million Report and validation that the numbers are real from the SOMAA Report by ArcTouch. If we can make it easier for developers and designers to make more accessible experiences, that's our duty.

That's why as conflicted as I feel about the good and the bad of AI, particularly for accessibility, I'm going all in. Hoping, and expecting, to make accessibility far, far better from now on.

I have so many ideas that were simply impossible one year ago. Whether I know you from the world of accessibility or from CTO Mixers and the tech scene in LA, drop me a line if you need help setting up the kind of AI orchestration I mentioned above, or simply how to strategize about the AI space.

Call to Action

#GAAD was such a success because it engaged the community. Accessibility leaders in companies created educational events on the day. And people reached out to the media to cover the event.

Let's continue that tradition. Share the results of AIMAC with your team, with the AI leadership at your organizations. Pick your AI models on the basis of how accessible they are. Nudge people in the right direction. It isn't that hard for the AI Model companies to train their models for accessibility. But they need to feel the competitive pressure, as always with accessibility and technology.

If you are a journalist, or know journalists that cover the disability space, speak to them about the results. I'd be happy to support the conversation.

Speaking of GAAD, this is an effort under the auspices of the GAAD (Global Accessibility Awareness Day) Foundation, partnered with ServiceNow, who have supported the creation of this benchmark. Special thanks to Eamon McErlean who named AIMAC and whose encouragement made it possible.

Another shoutout to the accessibility team at Microsoft who are helping us with the next version, to be announced before the next GAAD. Thank you to Yumeng Ma who is working on the next iteration, which will include a research paper.

Wishing you all a wonderful 2026! If you're here for CES, reach out and say Hi!

Related reading

1. FutureProof podcast: Jeremy Goldman interviews Joe Devon about AI and accessibility