About the coalition
Good judgment comes from people.
AI is reaching into every part of our lives, and its biggest problems are too big for any one company or field. So we bring the right people together to work on them.
Why we started
AI's biggest challenges don't belong to any one company, lab, or field. Keeping AI safe, secure, fair, and trustworthy takes researchers, builders, security experts, lawyers, policy experts, and the people who see AI at work every day.
So we bring those people together in one place, work on real problems side by side, and credit everyone for what they contribute.
How we work
- Bring a challengeAnyone can raise an AI problem worth solving, from a gap in safety research to a risk showing up at work.
- Form a working groupPeople with the right mix of experience join in, from inside and outside the AI field.
- Work it out togetherGroups research, test, and talk it through, then share what they find with everyone.
- Credit everyoneEvery contribution goes on the member's own record, and published work carries their name.
Our first project
Real-world tests for AI
We don't hand someone a driver's license because they aced a quiz in a parking lot. We test them on real roads, in real traffic. AI deserves the same kind of test.
So our members write real situations where AI needs good judgment, and at least two people from different fields review each one. We use them to see how AI actually behaves, then publish them in a free library with credit to everyone who built them.
Why scenarios retire: a test only works when the AI hasn't seen the questions before. Once we've used a scenario to test AI, it isn't fresh anymore. So we retire it, replace it with a new one, and publish the old one for everyone to learn from. It's the same reason a teacher doesn't give the same exam twice.
Our first working groups
Our real-world testing starts where AI decisions already touch people's health, money, and jobs. More groups will form as members bring new challenges.
Finance
Lending, claims, and fraud, where a wrong call can cost someone their savings.
Healthcare
Care, medication, and follow-up, where a missed hand-off can hurt someone.
HR
Hiring, scheduling, and workplace decisions that shape people's jobs.
Three ways to take part
Supporter
Follow along
Get updates, join our open calls, and read the library.
Contributor
Work on challenges
Bring your experience to a working group, from research to real-world tests. Everything you do is credited on your own record.
Fellow
Lead
Invited from our most active contributors. Fellows lead working groups and co-author our research.
Who's behind it
The coalition is a program of Humanity Systems, a California nonprofit building alignment between humanity and the technology we create.
Everything we make together is for the public good. Every scenario, finding, and paper the coalition publishes is free for anyone to read and use, and it always will be. Some test scenarios stay private while they're in use, and we publish them as soon as they retire.
Joining is free, with no dues, no fees, and no credit card. Our work is funded by donations from people who believe in it.
The scenarios our members write are used in our AI Behavior Benchmark™, a free public look at how the most widely used AI models behave. Then they're published in our free library, so everyone can learn from them.