Kensu

I just came to say: 'Hi #Xland!'

[Brais Moure @MoureDev]

Aquí está la 2ᵃ edición de mi libro: "Git & GitHub desde cero"

¿Lo celebramos REGALANDO 10?
✅ Sígueme
❤️ Dale like
🔁 Haz RT

El viernes hago el sorteo.
Mil gracias por todo el apoyo durante este año!

[AK @_akhaliq]

Microsoft presents LongRoPE

Extending LLM Context Window Beyond 2 Million Tokens

Large context window is a desirable feature in large language models (LLMs). However, due to high fine-tuning costs, scarcity of long texts, and catastrophic values introduced by new token positions, current extended context windows are limited to around 128k tokens. This paper introduces LongRoPE that, for the first time, extends the context window of pre-trained LLMs to an impressive 2048k tokens, with up to only 1k fine-tuning steps at within 256k training lengths, while maintaining performance at the original short context window. This is achieved by three key innovations: (i) we identify and exploit two forms of non-uniformities in positional interpolation through an efficient search, providing a better initialization for fine-tuning and enabling an 8x extension in non-fine-tuning scenarios; (ii) we introduce a progressive extension strategy that first fine-tunes a 256k length LLM and then conducts a second positional interpolation on the fine-tuned extended LLM to achieve a 2048k context window; (iii) we readjust LongRoPE on 8k length to recover the short context window performance. Extensive experiments on LLaMA2 and Mistral across various tasks demonstrate the effectiveness of our method. Models extended via LongRoPE retain the original architecture with minor modifications to the positional embedding, and can reuse most pre-existing optimizations.

[Kensu @K3n5u]

Al fin y al cabo todo es para ... mantener girando la rueda de la producción en un mundo cada vez más artificial...

[LM Studio @lmstudio]

Be TheBloke you want to see in the world

gemma-2b-it-q4_k_m.gguf
gemma-2b-it-q8_0.gguf

Available to download & run in LM Studio (v0.2.15+)

https://t.co/oiOp7l01Mh

[abhishek @abhi1thakur]

Google announced gemma today, and yes, its already supported in your favorite no-code finetuning tool: 🤗 AutoTrain 🚀

[Andrej Karpathy @karpathy]

New (2h13m 😅) lecture: "Let's build the GPT Tokenizer"

Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set, training algorithm (Byte Pair Encoding), and after training implement two functions: encode() from strings to tokens, and decode() back from tokens to strings. In this lecture we build from scratch the Tokenizer used in the GPT series from OpenAI.

[Sam Altman @sama]

we believe the world needs more ai infrastructure--fab capacity, energy, datacenters, etc--than people are currently planning to build.

building massive-scale ai infrastructure, and a resilient supply chain, is crucial to economic competitiveness.

openai will try to help!

[Kensu @K3n5u]

@LucioMSP X2

[Lior Alexander @LiorOnAI]

Roboflow supervision is the open-source swiss army knife for everything Computer Vision.

It lets you implement detection, classification, segmentation, annotation to any video.

This latest update adds advanced video analytics: Trackers, Zones, Annotators, and much more.

[Suhail Kakar @SuhailKakar]

@sama sam is back!!

[Sam Altman @sama]

the mission continues

[Satya Nadella @satyanadella]

Copilot will be the new UI for both the world's knowledge and your organization's knowledge, but most importantly, it will be your agent that helps you act on that knowledge. Here are highlights from my keynote today at #MSIgnite.

[Manfred @getmanfred]

¿Por qué nadie quiere a #Java y #PHP? ¿Son malos lenguajes? 💔

Requieren más «verbosidad» y su uso está generalizado en sectores como banca, pero para muchos no sería la primera elección al empezar un proyecto nuevo. 🤷

Si quieres conocer más datos: https://t.co/ng1VX9NRVf

[Manfred @getmanfred]

Tarde de manta, palomitas y... DOCUMENTAL DE RUBY ON RAILS.

Sí, como has leído. 😌

Nosotros también hemos flipado, pero va a tocar verlo para comentar. 🍿

https://t.co/lPuqTvOTOQ

[Kensu @K3n5u]

Mola! https://t.co/noQrvkMJdJ

[Kensu @K3n5u]

😅

[Marko Denic @denicmarko]

Backend developers after writing CSS for 20 minutes:

[Brais Moure @MoureDev]

Así funcionan las principales peticiones HTTP.

→ GET: Recupera un recurso del servidor.
→ PUT: Actualiza o crea un recurso.
→ POST: Crea un nuevo recurso.
→ DELETE: Elimina un recurso.
→ PATCH: Aplica modificaciones parciales.
→ HEAD: Obtiene los encabezados del GET.
→ CONNECT: Establece un túnel de comunicación.
→ OPTIONS: Obtiene las opciones de comunicación.
→ TRACE: Usado para diagnóstico y depuración.

Gracias a @bytebytego por esta pasada de diagrama.

[Kensu @K3n5u]

@MoureDev all access

[Kensu @K3n5u]

@MoureDev ... quisiera creer que esto es cierto :-/

[Thomas Dohmke @ashtom]

GitHub Copilot Chat, GA in just a few weeks. #GitHubUniverse