Going Native

Last week, I attended an event in San Francisco for people in the “AI Safety” field. I use scare quotes because I didn’t previously recognize how formalized the institutional and network structure is around this.¹

I previously only understood it as a general directional view of AI impacts, and not a field one could go into.²

Below are some thoughts following my first contact with the Safety natives:

1. The interest is sincere: I don’t know what I was expecting, but coming from Washington where earnestness is viewed as a weakness (or at least a giveaway that you’re new), it was confusing at first.

2. Broad agreement / different emphases: There was something resembling a meeting of young political party operatives.³ I got the sense of a very top-level consensus that better understanding AI risks at a practical level — especially as can be incorporated into company practices — is a righteous mission. But within that the emphasis varied: policy advocacy, communications, technical model evaluation research, general talent development, recruitment…

3. Washington is a mystery: (as with all of this, impressions are based on a non-representative sample) Of those who've never directly engaged in federal policy, their description of where things stand (“the current debate”) were largely informed by a passive TwiXter vibe, with incorrect conclusions, such as:

   • Musk is still part of determinative policy machinery

   • No one was paying attention to AI safety concerns, including cybersecurity risks, until very recently. I mean like in the last couple weeks.

   • Sen. Sanders’ announcement of a proposed bill to…do something (honestly I don’t even know), meaningfully moves the needle on near-term policy.⁴

   I mentioned that a leading Democratic House voice on AI is a car salesman from Virginia, and got the usual eye-roll, before continuing…who’s actively pursuing a Master’s in Machine Learning.

   Then dropped, his closest GOP counterpart is a former game developer …with degrees in computer science and AI from CalTech and UCLA.

4. Builder culture is real: I assumed the entrepreneur, “imagine it-do it” culture was overstated, papered over by the gobs of money and equity grants sloshing around. But these were fairly young people, some with a demonstrable track record of successfully building organizations and fundraising, many of whom were looking toward a pivot that likely meant less income going forward. There's something admirable in that.

5. “Don't wait to jump in too long”: several people had moved to SF without a promised job, or even a permanent visa. They just knew it was the place to be for what they wanted to do. And not just arrived without a job — they didn’t want a job, they wanted to found an organization and just did it! Sure Washington has (or had) some of that blind faith—youngsters arriving on interest, without a job in hand. But I don’t recall ever meeting someone who said, “I’m interested in policy/politics, so I moved here in order to start an organization serving an unmet need.”

6. The jargon…: I have a longer post forthcoming on the oppression of jargon, and it’s a universal problem. At the same time it’s necessary because acronyms are so much faster. A name came up multiple times, so I asked someone what it was. They were polite and didn’t betray any obvious exasperation, but there was a slight bemusement at how someone at this event didn’t know what Constellation was.

   This person agrees jargon is a barrier. In fact I get a kick out of them referring to human persons having “correct context” and “alignment” in terms of their jobs.

7. A Mature Industry: As of about a year ago when I left Washington, “AI Safety” was mostly a not-flattering description of a loose group of opinion-havers. At most a general opinion, and at worst a smug dismissal used by a16z-types. That’s certainly changed some #becauseevents.

   But here it’s not only not that: it’s a whole vocation, a set of skills, and a career path all in one. A Congressional chief of staff friend said it sounded like nothing if not a maturing industry, which makes perfect sense.

8. Credentials are whatever: experience and successes are more important than degree, headline company name, etc.

I have no opinion of whether this particular arrangement all taken together is optimal for AI Safety success. But offer the following:

• #1 is a necessary condition (however weird it feels to me)

• #3 is solvable

• #7 is encouraging but risks falling into the ossification trap of any industry.

I’m curious what other non-natives think or have experienced themselves.

1. There's even the people urging against youngsters going straight into AI safety. Which is a classic of any mature, fashionable industry. Reminds of that pattern from the 2010s: anytime the New York Times declared a trend, New York Magazine would immediately declare the trend over.

2. I’m choosing not to identify participants — though I’m not sure that it was actually off-the-record in that respect — because people are incapable of being chill in re: safety v. acceleration. Additionally it was by invite only.

3. I assume. I’ve never worked on a campaign or attended a capital P political event, beyond small group candidate PAC fundraisers in my former role as a registered lobbyist.

4. In fact, that Sanders meaningfully indicates the direction of policy in general is a more widespread view than I would've thought. I say this without any strong feelings about him or his policies.

In nearly 20 years in the Senate, he's had 3 bills become law (1 was a postal office naming).

But it made sense that X/social media as a proxy would lead to this conclusion. His office was next to mine for several years though, so there’s that.