Shachar Rosenman ✈️ NeurIPS

Research Scientist at Intel Labs. Previously: MSc @ BIU NLP

[Millie Marconi @MillieMarconnni]

🚨 BREAKING: A developer on GitHub just built a tool that turns any GitHub repo into an interactive knowledge graph and open sourced it for free.

It's called GitNexus. Think of it as a visual X-ray of your codebase but with an AI agent you can actually talk to.

No server. No subscription. No enterprise sales call.

Here's what it does inside your browser:

→ Parses your entire GitHub repo or ZIP file in seconds
→ Builds a live interactive knowledge graph with D3.js
→ Maps every function, class, import, and call relationship
→ Runs a 4-pass AST pipeline: structure → parsing → imports → call graph
→ Stores everything in an embedded KuzuDB graph database
→ Lets you query your codebase in plain English with an AI agent

Here's the wildest part:

It uses Web Workers to parallelize parsing across threads so a massive monorepo doesn't freeze your tab.

The Graph RAG agent traverses real graph relationships using Cypher queries not embeddings, not vector search. Actual graph logic.

Ask it things like "What functions call this module?" or "Find all classes that inherit from X" and it traces the answer through the graph.

This is the kind of code intelligence tool enterprise teams pay thousands per month for.

It runs entirely in your browser.

Works with TypeScript, JavaScript, and Python.

100% Open Source. MIT License.

Repo: https://t.co/RzIoLR2vAe

[Shachar Rosenman ✈️ NeurIPS @SRosenamn]

@yanaiela Congrats Yanai!! 👑

[Aviya Maimon @AviyaMaimon]

🚨 New paper alert! 🚨
We propose an IQ Test for LLMs — a new way to evaluate models that goes beyond benchmarks and uncovers their core skills.

Think: 🧠🤖 psychometrics for LLMs.
👇
(1/6)

[Yuhui Zhang @Zhang_Yu_hui]

🔍 Vision language models are getting better - but how do we evaluate them reliably? Introducing AutoConverter: transforming open-ended VQA into challenging multiple-choice questions!

Key findings:

1️⃣ Current open-ended VQA eval methods are flawed: rule-based metrics correlate poorly with true performance (0.09 on VQAv2), while model-based eval has reproducibility issues (updates in GPT-4o versions constantly increase scores by 6% on MMVet).

2️⃣ To address this challenge, we propose AutoConverter, an agentic framework that automatically converts open-ended VQA to multiple-choice questions. It generates distractors matching/exceeding human difficulty, with only 3% of generated questions incorrect.

3️⃣ Using AutoConverter, we built VMCBench: 9,018 multiple-choice questions from 20 datasets testing 33 VLMs in a unified format!

🎯 Our goal: Make VLM evaluation more reliable, efficient & scalable

https://t.co/BSnOW8QmYr

Joint work with a really fantastic team: @hhhhh2033528 (co-lead) @leoliuym @XiaohanWang96 @jmhb0 @elaine__sui @ChenyuW64562111 @AkliluJosiah2 @Ale9806_ @anjiangw advised by @lschmidt3 @yeung_levy!

[Zhiqiu Lin @ZhiqiuLin]

🚀 Make Vision Matter in Visual-Question-Answering (VQA)!

Introducing NaturalBench, a vision-centric VQA benchmark (NeurIPS'24) that challenges vision-language models with pairs of simple questions about natural imagery. 🌍📸

Here’s what we found after testing 53 models (GPT-4o, Llama3.2, Qwen2VL, Molmo, and more):

1️⃣ All models struggle: They perform only 10-20% above chance, while human accuracy exceeds 90%! This shows that models still struggle with natural images and simple questions that humans answer easily—what we call natural adversarial samples.

2️⃣ Models appear strong in previous benchmarks like MME/ScienceQA by exploiting their strong language bias. However, even a blind ChatGPT (without vision) can outperform vision models on these benchmarks.

3️⃣ Debiasing is crucial: Most models prefer "Yes" far more than "No"; correcting this bias can nearly double performance, even for GPT-4o.

📄 Paper: https://t.co/Ei3b7P3bF6
🌐 Arxiv: https://t.co/NRKs0G5NTJ

Work led by CMU & UW with @Jeande_d, @baiqil0203, @zixianma02, @simi_97k, and co-advised by @RanjayKrishna, @gneubig, and @RamananDeva.

[Ori Ernst @oriern1]

🚨 Happy to announce our latest research “The Power of Summary-Source Alignments", accepted in #ACL2024nlp findings!

Work with @obspp18, @lovodkin93, Sharon Adar, @mohitban47, @JacobGoldberge1, Ran Levy, Ido Dagan
@AmazonScience @uncnlp @biunlp

https://t.co/cB5GINrckP

(1/n)

[Shauli Ravfogel @ravfogel]

Happy to announce our latest research “Representation Surgery: Theory and Practice of Affine Steering", accepted at ICML! A joint work with @roeeaharoni @jonherzig @ryandcotterell @ponguru and @shashwat_s19, done during an internship at @GoogleAI.(1/8) https://t.co/jOwQZq4mkS

[Shaltiel @SShmidman]

🚀 Dicta-LM 2.0 - מודל שפה גנרטיבי גדול בעברית, פתוח, וחזק! 🎉

דמו של גרסת הצ'אט: https://t.co/0ExuK2bjJP

אתר החדש של דיקטה עבור תרגום מכונה אנגל��ת<->עברית שמבוסס על המודל: https://t.co/et82hslf00

קישור לבלוג המלא: https://t.co/OmOlJjv2gJ

🧵>>

[Shachar Rosenman ✈️ NeurIPS @SRosenamn]

@zeevyanay @CohenDaphna @OritTwitt @Yoav__Zehavi @idokius אשמח גם בבקשה

[Vered Shwartz @VeredShwartz]

I just published Tips for Writing NLP Papers https://t.co/u9zbHdsMC5

I wrote it for my students so I don't have to sound like a broken record (and edit papers for the same issues over and over again 😃). But some of you might find it useful too.

[Xiao Xu @looperxx27]

Check out our #ACL2023NLP Oral Paper — ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning

📄 Paper: https://t.co/WACzqmb8Zk
🔎 Code: https://t.co/N7qDa5uPdy
⭐ Slides & More: https://t.co/Sm0gFolay1

[Aviv Weinstein @ACL @AvivWn]

📢 New work 🚀

With @yoavgo

🔗Preprint: https://t.co/D7DIRU4rPk
🔗Demo: https://t.co/xoRLUkCnB1

Many syntax-based NLP systems handle verbs, but not many handle deverbal nouns. We suggest a *fully unsupervised* approach to transfer nominal structures to verbal ones, using BERT.

[Shauli Ravfogel @ravfogel]

What guarantees do we get from linearly erasing information (e.g., gender information) from neural representations? I'm happy to share our @aclmeeting paper which studies this question in depth. A joint work with @yoavgo and @ryandcotterell. https://t.co/SHBS0YdO19 (1/8)

[Avi Caciularu @clu_avi]

Meet 🏘️QAmden🏘️ your new effective general multi-document model📄📄📄

In our #ACL2023 paper, we propose a new and effective QA-based multi-document pre-training method, specializing in multi-document inputs!

w/ @mattthemathman @JacobGoldberge1 Ido Dagan @armancohan

1/n

[(((ل()(ل() 'yoav))))👾 @yoavgo]

In this work, Amir (and @ravfogel, and another author who cannot reveal themselves before paper acceptance) shows that diversity is much more important than quantity for model training.

https://t.co/XXXVhYQlrE

[Alon Jacovi @alon_jacovi]

Worried about test data being used in training?
The LLM world is going through a data contamination crisis.
Here's us trying to do something about it:

Paper: https://t.co/GoAJvjY2EI
Blog: https://t.co/ixuOPZ6jlw
w\ @clu_avi @omerNLP @yoavgo

[(((ل()(ل() 'yoav))))👾 @yoavgo]

I was puzzled for a while as to why we need RL for LM training, rather than just using supervised instruct tuning. I now have a convincing argument, which is also reflected in a recent talk by @johnschulman2 . I summarize it in this post:

https://t.co/DQD1wgyjg3

[Xiao Xu @looperxx27]

Thanks to @julsimon for building two BridgeTower @Gradio demos!

Image-Text Matching: https://t.co/kvBODUmPSS
Video Frame Retrieval:
https://t.co/8Bm0GiPJJ1

Below shows the detailed thread on BridgeTower, a simple yet effective open-source multi-modal understanding model.

[Shachar Rosenman ✈️ NeurIPS @SRosenamn]

We’re presenting our work at #AAAI23 this week.

This work presents a new approach for integrating visual & textual representations in a vision-language models.
Enjoyable collaboration with @MSFTResearch.

[Xiao Xu @looperxx27]

Check out our #AAAI23 Oral Paper — BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning

📄 Paper: https://t.co/ONPsKU8Zvs
🔎 Code: https://t.co/zUJ5vuHbVv
⭐ Slides & More: https://t.co/Sm0gFolay1

#AAAI23 #AAAI #AI #ArtificialIntelligence

[Hila Gonen @hila_gonen]

Why not all Prompts are Created Equal?

Check out our preprint "Demystifying Prompts in Language Models via Perplexity Estimation"

Joint work with @sriniiyer88 @TerraBlvns @nlpnoah @LukeZettlemoyer 🧵

paper: https://t.co/TLL8uN3ZZ0