Dániel Simig

MTS @Cohere, ex @AIatMeta. Hobbies include: bending embedding spaces, traveling.

[Cohere @cohere]

Introducing our latest breakthrough in AI search and retrieval: Rerank 4!

It’s the most advanced set of reranking models on the market, with best-in-class performance across search relevance, speed, deployment flexibility, multilingual support, and domain-specific understanding.

[Cohere @cohere]

Introducing Embed 4: our latest state-of-the-art multimodal embedding model that enables enterprises to securely add powerful search and retrieval capabilities to their agentic AI applications!

[Ivan Zhang @1vnzh]

our Superquerydocumentvectoralignment team have made exciting breakthroughs to share soon

more generally, we believe superintelligence (for enterprise) will arrive sometime this decade

[Dániel Simig @simigd]

Join me at this upcoming DSPy meetup! We'll be showcasing how to leverage the power of @cohere's cutting-edge models, the magic of end-to-end DSPy, and the infrastructure provided by @arizeai and @weaviate_io to build robust solutions for complex tasks.

https://t.co/2CaP28cDcf

[Aidan Gomez @aidangomez]

Introducing Rerank 3! Our latest model focused on powering much more complex and accurate search.

It's the fastest, cheapest, and highest performance reranker that exists. We're really excited to see how this model influences RAG applications and search stacks.

[Aidan Gomez @aidangomez]

⌘R+

Welcoming Command R+, our latest model focused on scalability, RAG, and Tool Use. Like last time, we're releasing the weights for research use, we hope they're useful to everyone! https://t.co/HgESxxEYlK

[Aidan Gomez @aidangomez]

⌘-R

Introducing Command-R, a model focused on scalability, RAG, and Tool Use. We've also released the weights for research use, we hope they're useful to the community!

https://t.co/0Q4Uj6zFjd

[Jay Alammar @JayAlammar]

What I'd like to do is, say:
- Give an LLM access to neurips papers
- Ask it a question, say, "how to efficiently train LLMs?"
- Get an answer with relevant NeurIPS 2023 results (and links to the papers)

[you can do that now with the NeurIPS connector + RAG on https://t.co/hB9TnHtOw0]

[Dániel Simig @simigd]

At NeurIPS this week! Find me around the @cohere booth (we are hiring!!) or by the D4 (https://t.co/gk68kquOgl) and MEGABYTE (https://t.co/NTMXtnp9ym) posters.

[Dániel Simig @simigd]

If you care about improving the web crawl data used for LLM pre-training, our latest paper with @kushal_tirumala is for you! Very informative thread and paper link below👇

[Andrej Karpathy @karpathy]

Promising. Everyone should hope that we can throw away tokenization in LLMs. Doing so naively creates (byte-level) sequences that are too long, so the devil is in the details.

Tokenization means that LLMs are not actually fully end-to-end. There is a whole separate stage with its own training and inference, and additional libraries. It complicates the ingest of additional modalities. Tokenization also has many subtle sharp edges. Few examples:

That "trailing whitespace" error you've potentially seen in Playground? If you end your (text completion API) prompt with space you are surprisingly creating a big domain gap, a likely source of many bugs:
https://t.co/f2PBaw2iA8

Tokenization is why GPTs are bad at a number of very simple spelling / character manipulation tasks, e.g.:
https://t.co/XR3d5g4uwp

Tokenization creates attack surfaces, e.g. SolidGoldMagikarp, where some tokens are much more common during the training of tokenizer than they are during the training of the GPT, feeding unoptimized activations into processing at test time:
https://t.co/y72eaIeRrP

The list goes on, TLDR everyone should hope that tokenization could be thrown away. Maybe even more importantly, we may find general-purpose strategies for multi-scale training in the process.

[Ari Morcos @arimorcos]

Web-scale data has driven the incredible progress in AI but do we really need all that data?

We introduce SemDeDup, an exceedingly simple method to remove semantic duplicates in web data which can reduce the LAION dataset (& train time) by 2x w/ minimal performance loss.

🧵👇

[AK @_akhaliq]

SemDeDup: Data-efficient learning at web-scale through semantic deduplication

Analyzing a subset of LAION, show that SemDeDup can remove 50% of the data with minimal performance loss, effectively halving training time

abs: https://t.co/xM1Y3LWB6q

[Cohere @cohere]

@forai_ml (2/12) OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Authors: Srinivasan Iyer, @VictoriaLinML, Ramakanth Pasunuru, @tbmihaylov, @simigd, Ping Yu, et al.

[AI at Meta @AIatMeta]

Announcing OPT-IML: a new language model from Meta AI with 175B parameters, fine-tuned on 2,000 language tasks — openly available soon under a noncommercial license for research use cases.

Research paper & more details on GitHub ⬇️

[Fabio Petroni @Fabio_Petroni]

Super excited to finally release our latest work where we show that AI can help humans in verifying Wikipedia claims. Everything is open-sourced: data, models, code and our new Sphere retrieval white-box engine. A step forward in general trustworthiness of information online 🚀

[Dániel Simig @simigd]

On my way to #ACL2022. First in person conference post-COVID, but more importantly my first conference that I come to with a poster to present! Come say hi in Poster Session 2 and check out our work: https://t.co/WbqFY3WN8R

[Dániel Simig @simigd]

Hi Twitter! 👋 Thought this would be a good occasion for my very first tweet: Proud to have (briefly) contributed to this effort, it’s been quite an experience! Thanks goes to @suchenzang and @stephenroller for great mentorship.

[AI at Meta @AIatMeta]

Today Meta AI is sharing OPT-175B, the first 175-billion-parameter language model to be made available to the broader AI research community. OPT-175B can generate creative text on a vast range of topics. Learn more & request access: https://t.co/3rTMPms1vq