jacob tsimerman

[jacob tsimerman @Jacob_Tsimerman]

I’m honoured to be on the advisory board of the new Alignment Journal, and we open for submissions today! The field of AI safety needs more canonical, high-quality spaces for exposition (despite the wonderful efforts of many, many people, including Victoria Krakovna who is on the advisory board as well!) and I think this could be a great addition.

Please take a look, and if you're doing research on AI safety then (Thank You! And) consider submitting!

[AlignmentJournal @AlignmentJrnl]

The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone.

What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- Review discussion is confidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.

We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.

We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.

You can submit your preprint in any format. Submissions are rolling and open now on the website.

If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.

Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️

[jacob tsimerman @Jacob_Tsimerman]

I’m excited to announce that I will be working at the Mathematical AI Safety Institute (MAISI) full-time as its founding Scientific Director! This means I will not be joining OpenAI as an employee, though I will still be collaborating with them, and I’m happy to announce that OpenAI will be supporting MAISI. They have offered us $10MM in API credits, and access to their latest models for our research use. Moreover, they will be participating in a multi-lab AI safety workshop we are organizing for next year with other frontier AI labs.

As an independent research institute, we are also in conversations with other labs and AI safety orgs about how MAISI can collaborate with them and contribute quickly to AI safety research across the industry. As we’ve seen over the past few weeks, the need for action on AI safety is extremely urgent. Research is an important part of that, and mathematicians have an important role to play!

[jacob tsimerman @Jacob_Tsimerman]

I'm incredibly excited to announce the founding of the Mathematical AI safety Institute (MAISI) https://t.co/7h28dBocA6. AI safety needs more foundational theoretical development, and mathematicians have the skills and the mindset to help! MAISI is an independent institute with visiting positions ranging from 1 semester to 2 years. Our goal is to get mathematicians up to speed and working on research directions in AI safety as quickly as possible. There is important work to be done, and there is real progress to be made. YOU can help!

Applications are open now! MAISI is aiming to hire 10-30 mathematicians to join us in the Bay Area by January, and scale up to 30-100 for September 2027. If you're a mathematician interested in channeling your skills toward the most important problem of our time, please apply today! https://t.co/PTWQh0MK3T

[Geoffrey Irving @geoffreyirving]

https://t.co/9inEcHkGBl

[jacob tsimerman @Jacob_Tsimerman]

@littmath Bjorn Poonen did like 20 years ago and had partial results (in unpublished notes that briefly circulated). Seems hard to make it work (I also tried and failed).

[jacob tsimerman @Jacob_Tsimerman]

@AmmannNora @Iliad_research Yup, I link to Iliad in the organizations tab :) Unless you mean something more specific?

[jacob tsimerman @Jacob_Tsimerman]

A couple of websites

I want to share a couple websites aimed specifically at mathematicians regarding AI-safety and AI-risk .

The first one I made myself: https://t.co/Ye2ZF5CmJj. This is designed to be a very high level and accessible resource describing some research agendas, collections of problems, and organizations in the field of AI-safety. I hope that for mathematicians considering directing some of their effort in this direction, this eliminates some of the friction. I plan to expand it quite a bit, but I hope it's already useful! I'd be happy and grateful to hear (constructive) feedback on it :-)

1/2

[jacob tsimerman @Jacob_Tsimerman]

The second is a https://t.co/8igrZbtH65. This is a very thoughtful post by Xiaoyu He, giving a very coherent and accessible summary of the case for AI-risk. It is very well written and definitely worth a read!

[jacob tsimerman @Jacob_Tsimerman]

I'm very happy about the https://t.co/XVQBOKxlch letter. I think developing the ability to stop, pause, or slow down AI research and capabilities is extremely important, and pretty low-hanging fruit in AI safety efforts.

Besides being good for its own sake, it will necessitate the creation of valuable social, corporate and political infrastructure, and force us to think through questions like:
"What does this mean, practically?"
"How do we decide (and which concrete events trigger) when we stop/pause/slowdown?"
"How do we create proxies to measure 'AI research and capabilities'?"

These questions are complicated and there's a lot to figure out. Cynics often point to this as evidence that coordinating on something like this is impossible. I think that's false. We can achieve complicated goals, but it's important to get started in advance.

[jacob tsimerman @Jacob_Tsimerman]

Tim Gowers has written an extremely thoughtful account of his thoughts about AI and its effect on mathematics which I strongly agree with. He phrases it around the Leiden Declaration, but to me the ideas and concerns he expresses are the more interesting part.

[Timothy Gowers @wtgowers @wtgowers]

https://t.co/OTVQworI4O

[jacob tsimerman @Jacob_Tsimerman]

@littmath @eye_of_newton @wildlyramified @Jess_Riedel @littmath This is beautiful but is there an easy way to see the source (X) is isomorphic to A^3? I can follow a proof but seems somewhat computational

[jacob tsimerman @Jacob_Tsimerman]

Re: problem-solving, I agree that we have a different experience [I enjoy it and you don't]. I'm curious whether you actually disagree with my 20-80% estimate?

I would be curious what your picture of continuing to do mathematics in an AI-is-better-than-us world looks like, and whether its meaningfully different than being an undergraduate mathematician and digesting tons of math [I don't say this derogatorily, I loved undergrad!]. Asking someone to spell out a vision of the future is a big ask, so even-less-pressure-than-usual to actually do this.

[jacob tsimerman @Jacob_Tsimerman]

I want to clarify my thoughts on problem-solving in mathematics, and the potential consequences of AI for the field. For context, I’m quoting here my post in reply to Daniel Litt (who, echoing others, I find very clear, grounded, and insightful in his thinking).

The claim

The short version is that I think problem-solving is an immense, and pervasive part of modern mathematical research. Consequently, if human problem-solving disappears by virtue of the AIs becoming strictly and substantially better at it, then most of the time currently spent by modern mathematical researchers will have to be spent on an activity that is altogether pretty different. Whether such an activity is viable as a professional endeavour is something I am unsure of, but strongly encourage others to think about and try to envision, so that if/when the time comes, we can steer such a future into being.

Allow me to make this somewhat concrete: by problem-solving I mean questions of the form “is T true? If so find a proof. If not, find a disproof.” where T is a precise mathematical statement. I’ll also include “find an example of S, if there is one” where S is some structure (variety/category/property/isomorphism/….).

The argument

Ok. Now as I said (and some have echoed) I spend ~all of my time problem-solving as my primary goal. This has sub-goals, but my entire main research field disappears if someone solves the Zilber-Pink Conjecture in its more general form. This is a single conjecture (precisely stated!) and lots of mathematicians, postdocs, and graduate students are engaged in picking apart special cases of it, trying strategies, finding analogies to develop intuition, etc.. Of course, lots of motivation and intuition and analogizing and understanding have gone into deciding to make the ZP conjecture a focus! But the fact remains that this is now what is being worked on ~all of the time by this community. This is true of many mathematicians. They have a problem (or ten) and spend most of their time doing it. If someone solves it, they have to find a different problem. This can be a big, disorienting process involving a lot of energy, and is neither trivial nor always fun (though often rewarding in the end).

People have written a lot about Theory building vs. Problem-solving, and I want to first of all clarify I have nothing against theory building or theory builders! It is a valuable part of mathematics, and while there are differences in perspective between the “camps” there is way more mutual respect and agreement.

However, I gather there is a perception that theory-builders spend most of their time not-problem-solving, and I think this is largely untrue. Now I’m not a theory-builder primarily (though I’ve partaken a LITTLE BIT by necessity) so I am outside of my comfort zone. As such, I apologize for mistakes and welcome corrections!

But theory-building constantly runs through problem-solving. Let’s say you want to define the right notion of a cohomology theory. Of course you must make candidate definitions. But then what does it mean for it to be the right one? Well, you start asking if it has natural properties. These are T statements. Does it satisfy a Kunneth formula? Is it functorial in the right way? When you have the wrong one you have to find the properties it’s missing, and when you have the right one you have to prove that it indeed has those properties.

Again, I am not saying nor do I believe that this makes problem-solving “real math” and theory-building lesser. I am just trying to draw attention to the way I think research mathematicians operate, and mathematics is practiced.

To put all this a different way, imagine you had access to an AI oracle that could resolve statements T, but somehow lacked any creativity to build technology or make definitions (I think this is unlikely, but for the purpose of this thought experiment lets imagine it). How would your mathematics change, if you were a theory builder? Well, you make a definition, and want to know if it’s the right one. You immediately ask your oracle a thousand questions. From “are these basic properties true” to “ooh, so is this deep conjecture true?” and start getting back answers, and amending your definitions. You could invent and resolve entire research directions in days. But the confusion you would have had to push through to flesh out your theory would largely (probably not entirely) be instantly resolved and the whole process sped up tremendously by your oracle. A big part of the process would be gone. This is very very different to modern mathematics.

One more thought

This post is too long already, but I’ve seen some people say that they only do mathematics to find truth and others valourize that as the only virtuous way to be. I do not do mathematics only to find truth. I do it largely because I enjoy it and I am good at it. I also find it beautiful and am grateful I get to spend my days understanding beautiful things. But I enjoy the challenge, the process, resolving confusions, finding strategies, grappling with problems. I would like to push for this being de-stigmatized. Mathematicians are people who need money, housing, food, love, exercise, and a great deal of other stuff including various forms of meaning.

There are many people whose primary enjoyment of math comes through problem solving in one of its incarnations. If that disappears, that is not a trivial issue and many of them might not want to do it anymore (even if there were some way to proceed).

[jacob tsimerman @Jacob_Tsimerman]

Hey @littmath , I've seen you post this sentiment a lot, and want to push back a bit (in my formal twitter-posting debut!). So my math career has almost entirely been "solve this problem". Now, of course there is an enormous amount of other activities such as formulating toy problems, identifying which ones are worthwhile, theory building, and selection and formulation of the original problems. But typically for 80%+ of my time, I know what problem I'm trying to solve and am just trying to solve it.

I've seen the view expressed a lot that this is sort of not-the-main-point, and is just a part of figuring out the appropriate mathematical structure and phenomena. It's not that this is untenable, but I feel this is a somewhat overstated perspective. For one thing, talks (almost) always start/end with "here is the theorem I have proven".

Its not that the view you have of math isn't coherent, but I could equally formulate the point of math as

1. Find a fun phenomenon
2. Make a problem capturing it as nicely as possible
3. Solve it, possibly by building theories and formulating sub-problems.

so that problem-solving become the central point of math, and theory building as a side-effect. Indeed, mathematicians offer measure the value of a theory by the problems it can solve. This is at least an important part of a theory.

I actually feel you and I are not far apart in our mathematical taste, so I'm curious how much we actually disagree here (I suspect the answer is "some").

Sorry for the overly long post! I am very much a twitter-newbie and will have to adjust.

[jacob tsimerman @Jacob_Tsimerman]

Hey @littmath , I've seen you post this sentiment a lot, and want to push back a bit (in my formal twitter-posting debut!). So my math career has almost entirely been "solve this problem". Now, of course there is an enormous amount of other activities such as formulating toy problems, identifying which ones are worthwhile, theory building, and selection and formulation of the original problems. But typically for 80%+ of my time, I know what problem I'm trying to solve and am just trying to solve it.

I've seen the view expressed a lot that this is sort of not-the-main-point, and is just a part of figuring out the appropriate mathematical structure and phenomena. It's not that this is untenable, but I feel this is a somewhat overstated perspective. For one thing, talks (almost) always start/end with "here is the theorem I have proven".

Its not that the view you have of math isn't coherent, but I could equally formulate the point of math as

1. Find a fun phenomenon
2. Make a problem capturing it as nicely as possible
3. Solve it, possibly by building theories and formulating sub-problems.

so that problem-solving become the central point of math, and theory building as a side-effect. Indeed, mathematicians offer measure the value of a theory by the problems it can solve. This is at least an important part of a theory.

I actually feel you and I are not far apart in our mathematical taste, so I'm curious how much we actually disagree here (I suspect the answer is "some").

Sorry for the overly long post! I am very much a twitter-newbie and will have to adjust.

[jacob tsimerman @Jacob_Tsimerman]

Me? Im here.

[jacob tsimerman @Jacob_Tsimerman]

@ClareMalone I'm really sorry to hear that. I always thought you did a fantastic job on the 538 podcast putting all the numbers in perspective, and adding much needed social context. I really hope youll be able to stay on in some form as suggested, but if not I'll follow whatever you do next!

[jacob tsimerman @Jacob_Tsimerman]

@ZachReino FYI its now on https://t.co/Jtmx2CSVsT, which is an awesome platform!

[jacob tsimerman @Jacob_Tsimerman]

@ZachReino Hey i just found the "fatal decision" series and I love it! IMDB lists 52 episodes but I just found the 9 on YouTube (and go90 appears to not be available anymore). Do you know if the other episodes are available anywhere? Thanks!