Oren Melamud

CTO at @getanyword
Previously, NLP Research Scientist at IBM Research

[Mike Lewis @ml_perception]

New paper in Science today on playing the classic negotiation game "Diplomacy" at a human level, by connecting language models with strategic reasoning! Our agent engages in intense and lengthy dialogues to persuade other players to follow its plans. This was really hard! 1/5

[Shachar Mirkin @shacharmirkin]

Not from Twitter, but I've also been part of large-scale layoffs, so I'm looking for a new machine learning / data science position, esp. in NLP

Lots of experience in both academic & industry settings

I live in France so I'm after jobs that can be done from here

please retweet

[Royi Rassin @RoyiRassin]

Apparently DALL-E 2 couldn’t pick the appropriate word-sense for “bass”, and just settled on using both senses. I find it surprising #dalle2

[Janelle Shane @JanelleCShane]

Stunning transcript proving that GPT-3 may be secretly a squirrel.
GPT-3 wrote the text in green, completly unedited!

[Sam Altman @sama]

Have an idea for DALL·E? Reply with a caption. I'll generate 20 or so!

[Oren Melamud @orenmelamud]

I wrote a blog post, trying to give some insights into the incredible recent advances in NLP with language models, and then share some practical tips, based on my industrial experience with the latest and (literally) greatest at @getanyword.
https://t.co/eZ6QdtDhm9

[Oren Melamud @orenmelamud]

Great overview on the latest and greatest in LM fine-tuning. I'm especially excited about the parameter-efficient methods. This seems to be super useful https://t.co/qCsBhGHgff. I also liked the work on Prefix-Tuning that was missing from the summary https://t.co/jVErem9c8G.

[Sebastian Ruder @seb_ruder]

Recent Advances in Language Model Fine-tuning

New blog post that takes a closer look at fine-tuning, the most common way large pre-trained language models are used in practice.

https://t.co/A5KYoq5zuw

[Greg Brockman @gdb]

DALL-E — our new neural network for generating images from text:

https://t.co/HMGw12LPDd

[Oren Melamud @orenmelamud]

@GabiStanovsky @CseHuji מזל טוב!

[Anna Rumshisky @arumshisky]

Our much-anticipated BERTology primer is out on arxiv: https://t.co/8Nmp51nOSF. Why and how does BERT work? What does it learn, and where is it stored? We review 40+ recent papers in search of answers and give our view on future directions (with #OlgaKovaleva & @annargrs).

[Yann LeCun @ylecun]

How to train a model with 10^11 parameters without running out of GPU memory?
Use DeepSpeed from Microsoft Research!
It's PyTorch compatible.
It partitions the network onto multiple processors automatically and efficiently.

[Chip Huyen @chipro]

I analyzed compensation & level details of 19k tech workers to find answers to:

1. How long does it take for SWEs to reach a certain level?
2. Compensations across jobs/levels?
3. Do women get paid less than men in tech?
4. Is there a deadline for SWEs?

https://t.co/8u9rN1nKRu

[Sebastian Ruder @seb_ruder]

10 ML & NLP Research Highlights of 2019

New blog post on ten ML and NLP research directions that I found exciting and impactful in 2019.
https://t.co/wChQ5qnxKd

[Anyword @getanyword]

We've created a powerful product that uses AI to enhance social media posts. We're hosting a focus group for social media managers on November 22nd in NYC. We'll ask you a bit about your workflows and give you exclusive first access. Sign up: https://t.co/xmhYHBriCO

[Oren Melamud @orenmelamud]

I'm hiring an Applied Scientist with experience in #nlp and #ml for my team in @GoKeywee (can be either NYC or TLV). If you want to make a difference in a small dynamic company, let me know!
https://t.co/gbknG4YhPw

[Oren Melamud @orenmelamud]

https://t.co/Le4Yb13bwz

[(((ل()(ل() 'yoav))))👾 @yoavgo]

Very excited and proud to officially announce the opening of AI2 Israel!
https://t.co/WUQRVNOtra

many thanks to @etzioni and the entire team for making this happen.

[Oren Melamud @orenmelamud]

Work with @ekshoonyame on automatically generating synthetic clinical notes to overcome privacy considerations that limit our ability to publicly share this kind of data.

[Andrej Karpathy @karpathy]

New blog post: "A Recipe for Training Neural Networks" https://t.co/5lBy4J77aS a collection of attempted advice for training neural nets with a focus on how to structure that process over time

[Sebastian Ruder @seb_ruder]

A new bigger, better language model by @OpenAI:
- Scaled-up version of their Transformer (10x params)
- Trained on 10x more curated data (40 GB of Reddit out links w/ >2 karma)
- SOTA on many LM-like tasks
- Discuss potential for malicious use
https://t.co/4hnXa8DsJx