lumentis ai

helping businesses integrate generative AI solutions

[lumentis ai @lumentisai]

🧠 Welcome! For now, we will focus on discuss and explore the world of AI solutions and research. Stay connected as we dive into exciting insights and future announcements. 🌐🚀 #AI #LumentisAI #Research #Innovation

[OcciGlot @occiglot]

Today, we are announcing Occiglot!

A large-scale collaborative research collective focusing on open-source European LLMs.

We invite anybody working on multilingual datasets, benchmarks, or models to get in touch/join our discord.

https://t.co/OcT7DNM4Ky

[lumentis ai @lumentisai]

Amazing work 👏

[AK @_akhaliq]

LongNet: Scaling Transformers to 1,000,000,000 Tokens

paper page: https://t.co/ccBoPzphPr

Scaling sequence length has become a critical demand in the era of large language models. However, existing methods struggle with either computational complexity or model expressivity, rendering the maximum sequence length restricted. In this work, we introduce LongNet, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences. Specifically, we propose dilated attention, which expands the attentive field exponentially as the distance grows. LongNet has significant advantages: 1) it has a linear computation complexity and a logarithm dependency between tokens; 2) it can be served as a distributed trainer for extremely long sequences; 3) its dilated attention is a drop-in replacement for standard attention, which can be seamlessly integrated with the existing Transformer-based optimization. Experiments results demonstrate that LongNet yields strong performance on both long-sequence modeling and general language tasks. Our work opens up new possibilities for modeling very long sequences, e.g., treating a whole corpus or even the entire Internet as a sequence.

[AK @_akhaliq]

LongNet: Scaling Transformers to 1,000,000,000 Tokens

paper page: https://t.co/ccBoPzphPr

Scaling sequence length has become a critical demand in the era of large language models. However, existing methods struggle with either computational complexity or model expressivity, rendering the maximum sequence length restricted. In this work, we introduce LongNet, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences. Specifically, we propose dilated attention, which expands the attentive field exponentially as the distance grows. LongNet has significant advantages: 1) it has a linear computation complexity and a logarithm dependency between tokens; 2) it can be served as a distributed trainer for extremely long sequences; 3) its dilated attention is a drop-in replacement for standard attention, which can be seamlessly integrated with the existing Transformer-based optimization. Experiments results demonstrate that LongNet yields strong performance on both long-sequence modeling and general language tasks. Our work opens up new possibilities for modeling very long sequences, e.g., treating a whole corpus or even the entire Internet as a sequence.

[Shunyu Yao @ShunyuYao12]

Still use ⛓️Chain-of-Thought (CoT) for all your prompting? May be underutilizing LLM capabilities🤠

Introducing 🌲Tree-of-Thought (ToT), a framework to unleash complex & general problem solving with LLMs, through a deliberate ‘System 2’ tree search.

https://t.co/V6hjbUNjbt

[SambaNova @SambaNovaAI]

Today we are excited to introduce and open-source BLOOMChat a multilingual chat LLM. Built on top of the BLOOM model (@BigscienceW), we further train the model on conversational data from @togethercompute @databricks @laion_ai @huggingface. Some interesting observations: (1/6)

[lumentis ai @lumentisai]

Long context models are incredible!
Just had a chance to test Anthropics 100k model (25x ChatGPT) and it is amazing.
Summirized a whole project planning and even was capable of suggesting optmizations to the General approach!
Really looking forward seeing more models like this

[Steven Hoi @stevenhoi]

Introducing 🔥CodeT5+🔥, a new family of open-source code LLMs for both code understanding and generation, achieved new SoTA code generation performance on HumanEval, surpassing all the open-source code LLMs.

Paper: https://t.co/apxl03WvNc
Code: https://t.co/nHTaIGIEmm

(1/n)

[Hugging Face @huggingface]

The first RNN in transformers! 🤯
Announcing the integration of RWKV models in transformers with @BlinkDL_AI and RWKV community!
RWKV is an attention free model that combines the best from RNNs and transformers.
Learn more about the model in this blogpost: https://t.co/0FQmsaRVZw

[Jim Fan @DrJimFan]

Also “ChatGPT is old news. ChatGPT has been dethroned. Here’s something INSANE that is totally a trivial wrapper around ChatGPT”

Is it just me or do you also see such contents promoted way more than before 😅

[lumentis ai @lumentisai]

After the Strorywriter Model from MosaicML the big ones are catching up! I have underestimated the speed they can alter models to higher token lengths.
The impact of higher token lengths is pretty mind blowing, when thinking of an AI knowing your whole codebase and extending it!

[Anthropic @AnthropicAI]

Introducing 100K Context Windows! We’ve expanded Claude’s context window to 100,000 tokens of text, corresponding to around 75K words. Submit hundreds of pages of materials for Claude to digest and analyze. Conversations with Claude can go on for hours or days.

[lumentis ai @lumentisai]

Amazing and ambitious! Very welcomed!

[Ai2 @allen_ai]

Today we're thrilled to announce our new undertaking to collaboratively build the best open language model in the world: AI2 OLMo.

Uniquely open, 70B parameters, coming early 2024 – join us!

https://t.co/9lQ2KYVC0v

[CREAIT @CREAIT_OFFICIAL]

Life can be boring...
But with AI we can bring our imagination to life now ❤️
@lumentisai

[lumentis ai @lumentisai]

A Game-Changer for AI Integration in Business Operations https://t.co/Bw41paF4BW

[lumentis ai @lumentisai]

2M Tokens context length would be great to have for sure...

The future of context in LLMs https://t.co/DiuTaQQHJf

[lumentis ai @lumentisai]

New Blogpost:
The Intriguing Potential and Limitations of AutoGPT and BabyAGI https://t.co/FzuRtruepb

[lumentis ai @lumentisai]

Hello Twitter!
The journey goes on, this time with a new amazing concept! 🚀

Looking forward releasing more information in the coming months, stay tuned! 🥳🥳🥳