Sailik Sengupta

Senior Scientist @ Amazon | Alignment tuning and decoding of LLMs, multi-agent orchestration with LLMs. Opinions are my own.

[Subbarao Kambhampati (కంభంపాటి సుబ్బారావు) @rao2z]

With some bay area Yochanites.. 🤗

[Sailik Sengupta @sailiks]

Finally, Meta’s workforce gets the same level of privacy Meta has always given its users. Better late than never! 🙂

https://t.co/NW2W910ieV

[Sailik Sengupta @sailiks]

@nikunj @trq212 @grinich @saradu Love the energy and starting to agree with the "Gap between idea and execution has never been smaller" part.

Wasn't sure if you folks had slides / content to share; would love to take a look! :)

[Sailik Sengupta @sailiks]

🧵(11/5) "Every lie we tell incurs a debt to the truth. Sooner or later that debt is paid." — Chernobyl

What debt are we accumulating when AI's biggest breakthroughs happen behind NDAs?

When does the bill come due?

[Sailik Sengupta @sailiks]

🧵(6/5) The Prisoner's Dilemma breaks down when prisoners get paid millions to stay silent.

Welcome to AI research, where @JitendraMalikCV's game theoretic analogy meets venture capital in the environment design.

Thread 👇

[Jitendra MALIK @JitendraMalikCV]

(5/5)As it is becoming abundantly clear that the current hyper-scaling paradigm for LLMs is not going to lead to AGI / superintelligence, perhaps it is time to return to a “cooperate” framework? Publish research results openly, and build on what each other has published.

[Sailik Sengupta @sailiks]

🧵(10/5)

Not all is gloom and doom.

Different reward functions are emerging. Open Source models (e.g. @allen_ai, @deepseek_ai) and some researchers ARE leaving the secret-clubs—slowly realizing the yacht comes with golden handcuffs.

The environment is shifting.

[Sailik Sengupta @sailiks]

@AnthropicAI Nice to see this direction being explored; reminded me of our work we started in 2020 where we explored AI manager (assistants) being able to explain their task allocation decision to workers.

Link: https://t.co/XZGezNO2Bv

[Sailik Sengupta @sailiks]

📢 Our paper on an AI-system coming up with a task allocation and, when questioned, explaining its rationale to a dissatisfied team member has been accepted to AAAI, 2024! 🎉

Joint work w/ @ZahraZ__ @rao2z

1/n

[Subbarao Kambhampati (కంభంపాటి సుబ్బారావు) @rao2z]

Imagination that is based on imitation sans common sense will always perpetuate stereotypes--be it all male engg. profs or women with unbuttoned shirts.. That we have moved from GANs to diffusion models doesn't change the equation--it just makes the pics HD.. See our old work: https://t.co/P4tTTC9Wxo (cc: @niharikajain_az @sailiks @maidylm & @_aolmo_ )

[Sailik Sengupta @sailiks]

A few years back, @nytimes quoted our paper on how AI models propagate (& exacerbate) social biases: https://t.co/k8Pe2EJqGZ

Interesting to see that modern day AI tools hasn't improved since 🙃

Read the amazing work (w/ @rao2z) lead by @niharikajain_az : https://t.co/5OOCS2Zijz

[Sasha Luccioni, PhD 🦋🌎✨🤗 @SashaMTL]

Guess what: AI enhancement tools of any kind will have a tendency to add things that weren't there to begin with... like unbuttoning women's shirts and showing their bras🙃

[Sailik Sengupta @sailiks]

@rao2z @kayastechly Unrolling the turing complete CoT Transformer: https://t.co/mFJt66ECev 🫠

Eager to see how week the O1 model converts PSPACE planning problems into undecidable halting problems 😉

Now, I have a $$ idea on how we stop these rouge AGIs! 😎

[Sailik Sengupta @sailiks]

@deedydas Hey, thanks for sharing!

Also, wanted to highlight some related work on this topic from @AmazonScience:
- Decoding-time Alignment
https://t.co/zjYpIXnSMK
- Beating larger models on Plan Adherence
https://t.co/9fIcS2BQxZ
- On function generation
https://t.co/zw06XPDOvB

[Sailik Sengupta @sailiks]

Told ya decoding-time methods are going to be the next big thing!

Here's OpenAI's new "strawberry" product promises (needs 10-20s before giving the response) -- https://t.co/x72kQLShTW

Learn more form our paper on how much more you can do with DeAL 😉
https://t.co/16NoWfiv4U

[AK @_akhaliq]

DeAL

Decoding-time Alignment for Large Language Models

paper page: https://t.co/6ena1OiOcI

Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. Current work focuses on alignment at model training time, through techniques such as Reinforcement Learning with Human Feedback (RLHF). However, it is unclear if such methods are an effective choice to teach alignment objectives to the model. First, the inability to incorporate multiple, custom rewards and reliance on a model developer's view of universal and static principles are key limitations. Second, the residual gaps in model training and the reliability of such approaches are also questionable (e.g. susceptibility to jail-breaking even after safety training). To address these, we propose DeAL, a framework that allows the user to customize reward functions and enables Decoding-time Alignment of LLMs (DeAL). At its core, we view decoding as a heuristic-guided search process and facilitate the use of a wide variety of alignment objectives. Our experiments with programmatic constraints such as keyword and length constraints (studied widely in the pre-LLM era) and abstract objectives such as harmlessness and helpfulness (proposed in the post-LLM era) show that we can DeAL with fine-grained trade-offs, improve adherence to alignment objectives, and address residual gaps in LLMs. Lastly, while DeAL can be effectively paired with RLHF and prompting techniques, its generality makes decoding slower, an optimization we leave for future work.

[Sailik Sengupta @sailiks]

If you wanted to know a bit more details about how OpenAI is promising structured outputs for apis (https://t.co/oDxKNw7C7n), but only got a high level overview from their "constrained decoding" section, here is our paper that helps you reduce such errors to zero! 😎

[Sailik Sengupta @sailiks]

🤔 How to benchmark LLMs calling APIs/Tools & prevent `silly` errors?

💡Our @emnlpmeeting paper on measuring and mitigating constraint violations for utterance-to-API semantic parsing answers this.

📜 https://t.co/k4AAtzji6i

w/ @shufan_wang_, Sebastien, @jmgung, @nik0spapp, Yi

[Sailik Sengupta @sailiks]

Turns out you can do a lot more than just structured outputs for APIs 😉

For example, precise and faithful planning based on Standard Operating Procedures (SOPs).

See https://t.co/bi01HHi4cP

Paper link: https://t.co/G4FSyTaR0G

[Shamik Roy @ShamiikRoy]

#NAACL2024 paper alert 🚨: FLAP: Flow-Adhering Planning with Constrained Decoding in LLMs

Feel free to stop by our oral presentation session today if you are visiting @naaclmeeting in person. 😃

When: Today (June 17) at 4 PM Mexico City Time
Where: DON ALBERTO 2
Track: Dialogue

[Sailik Sengupta @sailiks]

@gneubig Neat! Also, wanted to highlight our paper on the same topic: https://t.co/zjYpIXoqCi

[Amazon Science @AmazonScience]

For all their success, LLMs have trouble following prescribed sequences of operations, whether operational workflows or API dependencies. At @naaclmeeting, Amazon researchers showed how to address this using dependency graphs and constrained decoding. #ConvAI #NAACL2024

[Subbarao Kambhampati (కంభంపాటి సుబ్బారావు) @rao2z]

Whoever thought there would be a (mini) Yochan Reunion at #ACL2024 😱 (with @sailiks in Bangkok..)