Ankit Shah

Research Scientist at The Boston Dynamics AI Institute. Prev at Brown CS, Ph.D. from MIT. Making robots easy to program and deploy.

[MIT CSAIL @MIT_CSAIL]

A new handheld interface from MIT gives anyone the ability to train a robot for tasks in fields like manufacturing.

The versatile tool can teach a robot new skills using one of three approaches: natural teaching, kinesthetic training, & teleoperation: https://t.co/SQDdh0FAuE

[Experimental Philosophy @xphilosopher]

Yale Philosophy offers a course on “Formal Philosophical Methods” — a broad introduction to probability, logic, formal semantics, etc.

Instructor Calum McNamara has now made all materials for the course (78 pages) freely available

https://t.co/3ayerQBpHz

[Gary Marcus @GaryMarcus]

Wow!

The core finding in the much-maligned Apple paper from @ParshinShojaee et al – that reasoning models generalize poorly in the face of complexity – has been conceptually replicated three times in three weeks.

C. Opus sure didn’t see that coming. And a lot of people owe Ms. Shojaee an apology.

[Find me on bsky @colin-fraser.net @colin_fraser]

Ok I’ve read it now and as I expected the complaints about it are ill founded. It seems fine. I think it convincingly shows a lot of what I’ve been saying about these thinking models.

[Palash @ABiggerSpalash]

Friends, need your help.

@antarikshB, a senior from IIT B has launched an incredible project of organizing all Sanskrit literature in one place, in a user-friendly manner.

The service is free, not-for-profit, created purely out of passion. Media coverage will go a long way in ensuring the service reaches the right people.

Could you help by RT-ing and perhaps tag the right people?

(link below)

[Ankit Shah @ankitjs]

@sama Marginal improvements for exponential cost. Welcome to the world of computational complexity. Plenty of research directions abandoned because they would never scale

[Ankit Shah @ankitjs]

@deedydas I see in a later tweet you mention that that trends hold up over the years, so it would be interesting if you could try and set up null hypothesis test

[Ankit Shah @ankitjs]

@deedydas In expectation, the country should not change the odds, but a lottery is a single sample from what should be a Dirichlet distribution. What you have posted are raw numbers, what I am missing is... Is this sample truly unlikely under the null (a truly random lottery)?

[Ankit Shah @ankitjs]

@chris_j_paxton Lack of novelty manifests in three ways:
1) reinventing things we know failed
2) things tabled to be looked at later given better hardware/manufacturing or compute
3) multiple parallel efforts
3) promising efforts that died because it just wasn't the time
Optimus is first three?

[Ankit Shah @ankitjs]

@chris_j_paxton Agree with you by and large, but with the enshittification of products (ads everywhere, anandoned great products) the early (but unsustainable) preview felt better than what was delivered

[Ankit Shah @ankitjs]

@chris_j_paxton I think that's what I was trying to get at with "where else does it work" systems based approaches and e2e generalize qualitatively differently. My research bet is along the lines of systems-based backbone and learned features.

[Chris Paxton @chris_j_paxton]

I like how they use the infinite rotation to make planning easier. Taking advantage of how your humanoid doesn't need to be human

[Jason Liu @HRI @jasonxyliu]

How can robots understand spatiotemporal language in novel environments without retraining? 🗣️🤖

In our #IROS2024 paper, we present a modular system that uses LLMs and a VLM to ground spatiotemporal navigation commands in unseen environments described by multimodal semantic maps

[Kenneth Stanley @kenneth0stanley]

Recent results like Apple’s show that LLMs (even o1) flub on reasoning with simple changes to problems that shouldn’t matter. A consensus is building that it shows they are “just pattern matching.” But that metaphor is misleading: good reasoning itself can also be framed as “just pattern matching” at each step. The issue is not that we are merely seeing pattern matching, but that we are seeing *bad* pattern matching, at the wrong level of abstraction. If you think about it, that is a more serious pathology because it doesn’t separate when it works vs. when it doesn’t work into conveniently distinct buckets of computational tasks, In a sense, calling it “just pattern matching” implies an easier fix than there really is, as if all it will take is a better o1.

[Hadas Kress-Gazit 🪷 @HadasKressGazit]

Evaluation in robot learning papers, or, please stop using only success rate
a paper and a 🧵
https://t.co/3cUad03GUl

[Subbarao Kambhampati (కంభంపాటి సుబ్బారావు) @rao2z]

My (pure) speculation about what OpenAI o1 might be doing

[Caveat: I don't know anything more about the internal workings of o1 than the handful of lines about what they are actually doing in that blog post--and on the face of it, it is not more informative than "It uses Python er.. RL".. But here is what I told my students as one possible way it might be working]

There are two things--RL and "Private CoT" that are mentioned in the writeup. So imagine you are trying to transplant a "generalized AlphaGo"--let's call it GPTGo--onto the underlying LLM token prediction substate.

To do this, you need to know

(1) What are the GPTGo moves? For AlphaGo, we had GO moves). What would be the right moves when the task is just "expand the prompt".. ?

(2) Where is it getting its external success/failure signal from? for AlphaGo, we had simulators/verifiers giving the success/failure signal. The most interesting question in glomming the Self-play idea for general AI agent is where is it getting this signal? (See e.g. https://t.co/hPITPurBrx )

My guess is that the moves are auto-generated CoTs (thus the moves have very high branching factor). Let's assume--for simplification--that we have a CoT-generating LLM, that generates these CoTs conditioned on the prompt.

The success signal is from training data with correct answers. When the expanded prompt seems to contain the correct answer (presumably LLM-judged?), then it is success. If not failure.

The RL task is: Given the original problem prompt, generate and select a CoT, and use it to continue to extend the prompt (possibly generating subgoal CoTs after every few stages). Get the final success/failure signal for the example (for which you do have answer).

Loop on a gazillion training examples with answers, and multiple times per example. [The training examples with answers can either be coming from benchmarks, or from synthetic data with problems and their solutions--using external solvers; see https://t.co/hPITPurBrx]

Let RL do its thing to figure out credit-blame assignment for the CoTs that were used in that example. Incorporate this RL backup signal into the CoT genertor's weights (?).

<At this point, you now have a CoT generator that is better than before the RL stage>

During inference, stage, you can basically do rollouts (a la the original AlphaGo) to further improve the effectiveness of the moves ("internal CoT's"). The higher the roll out, the longer the time.

My guess is that what O1 is printing as a summary is just a summary of the "winning path" (according to it)--rather than the full roll out tree.

===
Assuming I am on the right path here in guessing what o1 is doing, a couple corollaries:

1. This can at least be better than just fine tuning on the synthetic data (again see https://t.co/hPITPurBrx)--we are getting more leverage out of the data by learning move (auto CoT) generators. [Think behavior cloning vs. RL..]

2. There will not still be any guarantees that the answers provided are "correct"--they may be probabilistically a little more correct (subject to the training data). If you want guarantees, you still will need some sort of LLM-Modulo approach even on top of this (c.f. https://t.co/mREKgH8mxk).

3. It is certainly not clear that anyone will be willing to really wait for long periods of time during inference (it is already painful to wait for 10 sec for a 10 word last letter concatenation!). See https://t.co/IewJWHB9Yz

The kind of people who will wait for longer periods would certainly want guarantees--and there are deep and narrow System 2's a plenty that can be used for many such cases.

4. There is a bit of a Ship of Theseus feel to calling o1 an LLM--considering how far it is from the other LLM models (all of which essentially have teacher-forced training and sub-real-time next token prediction. That said, this is certainly an interesting way to build a generalized system 2'ish component on top of LLM substrates--but without guarantees. I think we will need to understand how this would combine with other efforts to get System 2 behavior--including LLM-Modulo (https://t.co/mREKgH8mxk) that give guarantees for specific classes.

to be contd..

[bigAI @BrownBigAI]

@jasonxyliu will present their @IJCAIconf survey paper on robotic language grounding. Please check out his talk (8/8 11:30) if you are at #IJCAI2024

In colab w/ @VanyaCohen, Raymond Mooney from @UTAustin, @StefanieTellex from @BrownCSDept, @drdavidjwatkins from The AI Institute

[Jason Liu @HRI @jasonxyliu]

All the talk recordings are available at https://t.co/YoNmbilDce

[Ankit Shah @ankitjs]

@kenneth0stanley @khademinori To expand, to me reasoning involves grounding a linguistic concept reliably to a reusable primitive, either a computational one (such as addition, multiplication etc.) or a physical one (pick up an object). LLMs seem to fail at following recipes they generate

[Ankit Shah @ankitjs]

@kenneth0stanley @khademinori To rephrase: It can generate an accurate textual description of how to compute a fourier series yet fail to compute it correctly?