RWKV

AI model built by the community, for everyone in this world

Part of the Linux Foundation, Apache 2 licensed
An RNN scaled to 14B params with GPT-level of perf

[RWKV @RWKV_AI]

#RWKV is One Dev's Journey to Dethrone Transformers

The largest RNN ever (up to 14B). Parallelizable. Fast inference & training. Quantizable. Low vram usage.
3+ years of hard work

https://t.co/ebRmfDrQbD

Created by @BlinkDL_AI
Computation sponsored by @StabilityAI @AiEleuther

[RWKV @RWKV_AI]

Wrapping up: #RWKV was created by @BlinkDL_AI
as a project at @AIEleuther
- and is now being hosted by @LFAIDataFdn

The RWKV wiki can be found at: https://t.co/ebRmfDso1b Our discord can be found at: https://t.co/veH8lO4Kf6

Give the models a try, drop by our discord

[RWKV @RWKV_AI]

Announcing a flock of RWKV models all arriving with apache2 licensing: available today

First up, the strongest and largest linear model to date: QRWKV6-32B-Instruct-Preview

Surpassing previous state space & RWKV models matching transformer performance with much lower inference

[RWKV @RWKV_AI]

Overall we are excited by the progress we look forward to
- the step jump in RWKV7-based attention
- and the upcoming converted 70B models

Once properly tuned, and trained, it will serve as a full drop-in replacement for the vast majority of AI workloads today 🪿

[RWKV @RWKV_AI]

@danofer We will accept donations to our foundation in cash or GPU time ❤️

(the project is properly attributed in the legal txt file in the folder)

[RWKV @RWKV_AI]

RWKV.cpp - is now being deployed to half a billion systems worldwide

Making it one of the world's most widely deployed, truly open-source (apache2) AI solutions out there

As it now ships with every Windows 11 system

[PicoCreator - AI builder @ ✈️🌉 @picocreator]

🚀 RWKV.cpp AI system - is being deployed to 0.5 billion installs globally

Making it one of the world's most deployed, truly open-source (apache2) AI solutions out there

As it ships with every Windows 11 system today!!

Our group's code install count, went from ~100k -> 0.5B

[RWKV @RWKV_AI]

RWKV.cpp project can be found here:
https://t.co/XI1dXqlSzd

Our RWKV wiki can be found at:
https://t.co/ebRmfDso1b

Our discord can be found at: https://t.co/veH8lO4Kf6

Give the models a try, drop by our discord, and have fun!

[RWKV @RWKV_AI]

Energy cost - Is critical for a device that aims for long battery life!

Variety of language support - is critical for shipping a product to the whole world (beyond English or Chinese)

We will eagerly keep watch on this development

https://t.co/ZLAepum3OU

[RWKV @RWKV_AI]

Wrapping up: #RWKV was created by @BlinkDL_AI as a project at @AIEleuther - and is now being hosted by @LFAIDataFdn

The RWKV wiki can be found at:
https://t.co/ebRmfDrQbD

Our discord can be found at: https://t.co/veH8lO4cpy

Give the model a try, drop by our discord

[RWKV @RWKV_AI]

The RWKV v6 Finch lines of models are here
Scaling from 1.6B all the way to 14B

Pushing the boundary for an Attention-free transformer, and Multi-lingual models.

Cleanly licensedm Apache 2, under
@linuxfoundation

Find out more from the writeup here: https://t.co/30VbPbbfCm

[RWKV @RWKV_AI]

As previously covered, this model is an upcycled model from our v5 - with an additional 1.4T tokens trained, for the conversion process.

The RWKV v6 Finch paper covering the architecture can be found here: https://t.co/9aCLn0yplJ

[Joanne Chen @joannezchen]

Is AI's insatiable appetite for compute holding back innovation?

I recently talked with Eugene Cheah about @RWKV_AI. Eugene and the open-source RWKV community are working to democratize AI by addressing some of the key limitations of transformers.