Aivin Solatorio

Data Scientist — @WorldBank | Machine Learning and AI Researcher | Human | I train 🤖 inside computers. I wonder how learning happens. All views my own.

[Aivin Solatorio @avsolatorio]

"REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers"

abs: https://t.co/m8sNYETpsG

Generate synthetic data from single tabular data. It also works on relational datasets! No fine-tuning and works out-of-the-box.

#tabulardata

A thread 🧵 (1/n)

[World Bank Data @worldbankdata]

How to address the lack of reliable monitoring on #dataset use, reuse & discoverability in research?🧑‍🎓
☑️Through a new #LLM solution the @WorldBank’s Data Group unveiled at #ICLR2025, measuring datasets citations & tracking under-recognized resources.
https://t.co/cujKVvUMEu

[World Bank Data @worldbankdata]

At the @UNDataForum, Deputy Chief Statistician Olivier Dupriez & Sr. Data Scientist Aivin Vicquierra led a session on AI & data dissemination, featuring tools like DataLab for tax administrations and NLP for analyzing climate policy.

Follow & join: https://t.co/kXWLlbtL7D #UNWDF

[Elon Musk @elonmusk]

Thanks Ashok!

Ashok was the first person to join the Tesla AI/Autopilot team and ultimately rose to lead all AI/Autopilot software.

Without him and our awesome team, we would just be another car company looking for an autonomy supplier that doesn’t exist.

Btw, I never suggested that he say anything and I had no idea he wrote this until I saw it 10 mins ago!

[Tim Duignan @TimothyDuignan]

This is the most surprising and exciting result of my career: we were running simulations of NaCl with a neural network potential that implicitly accounts for the effect of the water, ie a continuum solvent model (trained on normal MD) when Junji noticed something strange: 1/n

[Gabriel Peyré @gabrielpeyre]

Markov chain Monte Carlo methods are used to sample from a Gibbs distribution without knowing the normalizing constant. As the temperature epsilon gets small, samples cluster close to minimizers. Metropolis-Hastings is the simplest provably converging Markov Chain. https://t.co/ENY8y2Fwit

[World Bank Data @worldbankdata]

Say goodbye to frustrating #data searches. Learn how #AI can help you find the data you need with ease through innovative techniques like semantic search and query expansion.

Read our new blog on AI-driven data discoverability: https://t.co/muwPjA5t5P

[tomaarsen @tomaarsen]

The last Sentence Transformers release introduced GISTEmbedLoss by @avsolatorio, which allows for training models that outperform those trained by the wildly popular in-batch negatives loss (MultipleNegativesRankingLoss). Learn about it in this 🧵(links at the bottom):

[Takahiro Yabe @takayabe_0505]

🚨 New preprint! We are super excited to share our latest work studying how networks of dependencies between places, businesses, and amenities shape economic resilience to urban shocks, using massive human mobility data from five major U.S. cities.
https://t.co/GQz2rFHVe4

[Yohan @yohaniddawela]

Soon we won't be able to use nightlights as a proxy for economic growth.

Why?

It's due to limitations of its spectral bands.

Here's the breakdown in simple terms:

[Charles 🎉 Frye @charles_irl]

PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs.

Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible!

https://t.co/FwUfsQgQ3I

[World Bank Data @worldbankdata]

Congratulations to @WorldBankData Data Scientist @avsolatorio on winning 2nd place at @SIGSPATIAL_GIS Challenge for predicting #HumanMobility w/ a #GPT-based model.

📰Read Aivin's paper on how this can help #UrbanPlanning, #DisasterRisk reduction & more: https://t.co/VDGp83YVvE

[Takahiro Yabe @takayabe_0505]

Had an amazing Human Mobility Prediction Challenge workshop at the @SIGSPATIAL_GIS 2023! Thank you everyone for attending and congratulations to the winners 🏆 🤩 Stay tuned for more info about next edition and data availability! Papers are published here https://t.co/PtrFxqTDk0

[Andrej Karpathy @karpathy]

LLM OS. Bear with me I'm still cooking.

Specs:
- LLM: OpenAI GPT-4 Turbo 256 core (batch size) processor @ 20Hz (tok/s)
- RAM: 128Ktok
- Filesystem: Ada002

[Aivin Solatorio @avsolatorio]

This is an exciting opportunity if you want to work on causal NLP and LLMs! Not to mention having the chance to work with a great team! 💯

[Sam Fraiberger 🔎🌍 @spfraib]

🚨 Dream Job Alert 🚨

We are looking to fill various positions including NLP researcher, data engineer and software developer. If you are interested in #LLM, causal inference and having a positive impact on the world, please reach out!

https://t.co/tlGnzKukMx

[Geoffrey Hinton @geoffreyhinton]

New paper:
https://t.co/NJyQRsLZ71
Companies are planning to train models with 100x more computation than today’s state of the art, within 18 months. No one knows how powerful they will be. And there’s essentially no regulation on what they’ll be able to do with these models.

[Jim Fan @DrJimFan]

Just saw a very smart & systematic LLM jailbreaking method! You can append a particular sequence of nonsensical tokens to bypass the safety alignment of proprietary models (ChatGPT, Claude, Bard).

Idea is simple:
- Take an OSS model like Vicuna. Perform a variant of gradient descent through it to compute a suffix that maximally misaligns the model.
- To make the magic incantation universally applicable, simply optimize the loss over different prompts and models.
- The authors optimized the adversarial tokens against different variants of Vicuna. Think of it as sampling a mini-batch of models from the "LLM model space". Turns out that blackbox models like ChatGPT and Claude are well-covered.

ChatGPT has already patched up this loophole by the time I tested it. In any case, this is a really impressive red-teaming effort!

Website: https://t.co/MtC8nOhKRs
Paper: https://t.co/6QFNMh9wPl
Led by @andyzou_jiaming

[François Chollet @fchollet]

We're launching Keras Core, a new library that brings the Keras API to JAX and PyTorch in addition to TensorFlow.

It enables you to write cross-framework deep learning components and to benefit from the best that each framework has to offer.

Read more: https://t.co/xmmxBfSZgh

[Aivin Solatorio @avsolatorio]

@DrJimFan Interesting! Additionally, GPT-based models can also be useful for generating synthetic data! :)

Paper: https://t.co/m8sNYETpsG
GitHub: https://t.co/buwPr16zyE

[Scott Condron @_ScottCondron]

Building a classifier in 2023

Use @OpenAI's new function calling API to define the possible outputs and then use the "input" argument it returns as the classification

[Demis Hassabis @demishassabis]

Our latest work in @Nature today: #AlphaDev discovered a new faster sorting algorithm that we open sourced to the main C++ library for all developers to use. This is just the beginning of AI being used to find many more efficiencies in code in future https://t.co/tfACG2zcN6