Last week, I attended a networking event for people in the “AI Safety” field. I use scare quotes because I didn’t previously recognize how somewhat formalized the institutional and network structure is around this.
I previously only understood it as a general directional view of tech/AI impacts, and not a field or even industry one could go into.1
Below are some thoughts following my first contact with the Safety natives:
The interest is sincere: I don’t know what I was expecting, but coming from Washington where earnestness is viewed as a weakness (or at least a giveaway that you’re new), it was confusing at first.
Broad agreement / different views: There was something resembling a meeting of young political party operatives.2 I got the sense of a very top-level consensus that better understanding AI risks at a practical level, especially as can be incorporated into company practices, is a righteous mission. But within that the emphases varied: policy advocacy, communications, technical model evaluation research, general talent development…
Washington is a mystery: as with all of this, impressions are based on a non-representative sample. That said, of those who've never directly engaged in federal policy, their description of where things stand (“the current debate”) were largely informed by a passive TwiXter vibe, with incorrect conclusions, such as:
Musk is still part of determinative policy machinery
No one was paying attention to AI safety concerns, including cybersecurity risks, until very recently. I mean in the last couple weeks.
Sen. Sanders’ announcement of a proposed bill to…do something, honestly I don’t even know, meaningfully moves the needle on the near-term policy outcome.3
Builder culture is real: I assumed the entrepreneur, “imagine it-do it” culture was overstated, papered over by the gobs of money and equity grants sloshing around. But these were fairly young people, some with a demonstrable track record of successfully building organizations and fundraising, many of whom were looking toward a pivot in focus that likely meant less income going forward. There's something admirable in that.
“Don't wait to jump in too long”: several people had moved to SF without a promised job, or even a permanent visa. They just knew it was the place to be for what they wanted to do. And not just arrived without a job — they didn’t want a job, they wanted to found an organization and just did it! Sure Washington has (or had) some of that blind faith—youngsters arriving on interest, without a promise in hand. But I don’t recall ever meeting someone who said, “I’m interested in policy/politics, so I moved here in order to build an organization to serve an unmet need.”
The jargon…: I have a longer post forthcoming on the oppression of jargon, and it’s a universal problem. At the same time it’s necessary because acronyms are so much faster. An organization name came up multiple times, so I asked a rando what it was. They were polite and didn’t betray any obvious exasperation, but there a slight bemusement at how someone at this event didn’t know what Constellation was.
This person agrees jargon is a barrier. In fact I get a kick out of them referring to human persons having “correct context” and “alignment” in terms of their jobs.
A Mature Industry: As of about a year ago when I left Washington, “AI Safety” was mostly a not-flattering description of a loose group of opinion-havers. At most a general opinion, and at worst a smug dismissal used by a16z-types. That’s certainly changed some #becauseevents.
But here it’s not only not that: it’s a whole vocation, a set of skills, and a career path all in one. A Congressional chief of staff friend said it sounded like nothing if not a maturing industry, which makes perfect sense. Perhaps in another venue, on another night, folks who reject this safety POV would react differently.
Credentials are whatever: experience/successes are more important than degree, headline company name, etc.
I have no opinion of whether this particular arrangement all taken together is optimal for AI Safety success. But I’d offer the following:
#1 is a necessary condition (however weird it feels to me), and
#3 is solvable
Beyond that I got nothing but I’m curious what other non-natives think or have experienced themselves.
I’m choosing not to identify participants, though I’m not sure that it was actually off-the-record in that respect, because people are incapable of being chill in re: safety v. acceleration. Additionally it was by invite only.
I assume. I’ve never worked on a campaign or attended a capital P political event, beyond small group candidate PAC fundraisers in my role as a registered lobbyist.
In fact, that Sanders meaningfully indicates the direction of policy in general is a more widespread view than I would've thought. I say this without any strong feelings about him or his policies. But it made sense that X/social media as a proxy would lead to this conclusion.