Last week Anthropic co-founder Dario Amodei cautioned the rate and pace of frontier artificial intelligence (AI) in what was the week’s most viewed business newsletter. His remarks were echoed by OpenAI’s Sam Altman, and to a lesser extent by Elon Musk. Both Amodei’s and Altman’s companies are said to have suffered lab leaks in recent months – programs escaping their confines (the latter with Hugging Face) or programs learning to communicate with other programs and then (apparently) conspiring.

Amodei was commended for acting responsibly, irrespective of the micro-rout in tech stocks his comments aggravated. But what he did next, though, was more important, and must be explored.

As the very first part of a three-step plan to caution frontier AI development, Amodei proposed employee-like entry access for groups such as third-party evaluators METR (Model Evaluation and Threat Research).

Until now, AI has enjoyed a good run of confident, handsome capital without much involvement from health and safety inspectors. But the way he expressed this suggestion left little room for doubt: it appears he wants METR itself in every firm. In bold, he claimed he had committed to this arrangement as a priority.

So who is METR?

The familiarity is uncanny. What claims to be a “research non-profit” is actually just the latest identity for the generation of self-appointed social enforcers who used to moderate comments on Twitter or worked in the mis- or disinformation industrial complexes in the days when it was fashionable. Another expression of our secret addiction to regulation, they’ve had a makeover, found new big words to describe themselves – and are now demanding a seat at the future of AI’s table.

Nerd Cartel

METR is a San Francisco Bay Area/Oxford University nerd cartel – interfering and menacing people who are far too autistic for their own good (as an autistic myself, I am allowed to say that). Having studied both places and the kind of people they pump out intimately, I am left only to conclude such an arrangement is akin to Gert van Rooyen signing a memorandum of understanding with the Wemmer Pan serial killer.

Beth Barnes is the CEO. She didn’t go to Oxford, but Exeter, where she founded the Effective Altruism (EA) sect (more about this shortly). During a TED Talk a decade ago, Beth advanced open borders; when it comes to AI, Beth apparently prefers rigid limitations.

Hjalmar Wijk, METR’s chief scientist, did go to Oxford. He sounds incredibly clever, but Neil Ferguson, the United Kingdom’s chief Covid alarmist, was also clever, and renowned for his modelling. That was until Microsoft had to allegedly dispatch a team of Seattle researchers in the middle of the night to address the sorry state of his code, ~13 000 lines or so.

In addition to going to Oxford, Pip Arnott (they/them) also modelled stuff. They work on “eval execution and maintaining evaluation tooling and infrastructure at METR”, which sounds fascinating, and very kind of them.

Next is Megan Kinniment. She studied physics at Oxford, where she got her Master’s, before going onto work for something called 1DaySooner. “Ending Infectious Diseases, One Day Sooner” 1DaySooner’s website boasts, before leading viewers into an application section for “High Impact Clinical Trials” — making it sound more like 28 Days Later.

There’s another woman there called Jasmine Dhaliwal. Previously, she claims to have worked as a “British diplomat” and was seconded to multilateral organisations. So, Gen Z boss and a NATO, possibly.

Effective Altruism

EA is over METR like a rash. Before FTX’s Sam Bankman-Fried scammed investors, few had heard of EA, which claims to “use evidence to find ways to do the most good possible”. Previously “doing the most good possible” involved the foundation representing this group buying a grand old country manor called Wytham Abbey (near Oxford) for £15 million, so that effective altruists could swan around and think altruistically. Two years later, thanks to Sam, it had to be sold.

It's not clear why these people – who describe themselves as “rationalists” – have to talk and behave like Moonies in pursuit of doing “good”, and the process appears to suffer some sociopathic dimensions i.e. a demeanour of goodness concealing black-pilled ultra conflict theory.

The idea of an Oxford-Bay Area power centre influencing AI policy is sure to stymie or even kill the technology’s potential, particularly as it relates to African access, and the sovereign models nations are eyeing. Imposing a group of ideologically homogeneous and overeducated doomers onto innovation also smacks of European Union and British speech regulation, demonstrably exploited for sets of obscure, probably dark purposes.

When Elon Musk purchased Twitter, or when Christopher Steele presented a fantasy dossier, sunlight exposed tracks left by shadowy forces, including the state. There could be no easier organ on the planet to infiltrate than a group of “radical empathists” who evidently seek control of one of humanity’s most promising advances. Amodei is a brilliant man, but brilliant men are just as easily captured as fools.

Complex Questions

Which makes his issue, and the way he has raised it, and the solutions he has suggested, not so much like acid rain, African bees, Y2K, or oil running out, but financial anxiety parading around in morality clothing. That – there being no more or less money to throw at AI development – is a complex question and involves the rhythms of capital markets and the threat of over-indexing; when a problem is raised alongside a group like METR, the emphasis shifts and incorporates other themes. Since Amodei’s newsletter, considerably less has been spoken about investment, and more about “guard rails”.

AI is set to improve even more. Faster, more intelligent models are set to emerge. AI is not the enemy for a comparatively underfunded continent like Africa. Doomer people who hate humanity, on the other hand…