Well, that's it!
While ENTP might not replace decoder models for now, it offers a promising direction if we overcome current computational limitations.
Check out our full paper on Arxiv for details—we’d love your feedback!
https://t.co/mAEM2aT2vV
🚀 Excited to share our work on Encoder-only Next Token Prediction (ENTP)!
While most successful LLMs are decoder-based, we asked: Can encoder-only TFs be used for next-token prediction?
Yes!
Moreover, ENTP might be better than decoder-only models!!! 😎