Q Blocks

Decentralised computing platform to help ML teams get 10x more affordable computing

[Q Blocks @blocks_q]

@nivi @naval Here's our demonstration of 90% savings for Transcription and Translation using Whisper.

Optimised Whisper API, running on decentralised GPU network

With further optimisation, we are looking at 95% savings.
At moderate scale this means:

$50,000 vs $1M
https://t.co/ccaswMfSAK

[Q Blocks @blocks_q]

Wow. This world is moving at lightspeed and love how @monsterapis always comes on top of the latest innovation. Their GPT is insane.

[Brian Roemmele @BrianRoemmele]

We have successfully fine tuned LLaMA 3 using the @monsterapis GPT instruct tune method.

Overnight we have built 17 unique models and are testing now.

Early results?

Very robust and quite extraordinary.

No other 8B model can get close to these outputs.

I am floored.

[Q Blocks @blocks_q]

LORA powered Fine tuning is revolutionary

[Santiago @svpino]

LoRA is a genius idea.

To understand the fine-tuning of Large Language Models, you must understand how LoRA works.

By the end of this post, you'll know everything important about how it works.

Large Language Models are good generalists, but they have little specialization. We train them in many different tasks, so they know a bit about everything but not enough about anything.

Think of a kid who can play three different sports at a high level. While he can be proficient across the board, he won't get a scholarship unless he specializes. That's how the kid can reach his full potential.

We can do the same with these large models. We can train them to solve a particular task and nothing else.

We call this process "fine-tuning." We start with everything the model knows and adjust its knowledge to help it improve on the task we care about.

Fine-tuning is revolutionary, but it's not free.

Fine-tuning a large model takes time, care, and lots of money. Many companies can't afford the process. Some can't pay for the hardware. Some can't hire people who know how to do it. Most companies can't do either.

That's where LoRA comes in.

We realized we could approximate a large matrix of parameters using the product of two smaller matrices. There was a lot of wasted space within these large models. What would happen if we find a new, more optimal representation?

Did you ever buy a map at a gas station? Giant pages showing every small road, path, and lake around you. They were exhaustive but hard to navigate. These are like parameters in a large model.

LoRA turns a gas station map into a cartoon treasure map. Every useless parameter is gone. Only two roads, a palm tree, and a cross pointing at the treasure. We don't need to fine-tune the entire model anymore. We can only focus on the small treasure map that LoRA gives us.

It's a mind-blowing trick.

We can train the small approximation matrices from LoRA instead of fine-tuning the entire model. LoRA is cheaper, faster, and uses less memory and storage space.

You can also merge the approximation matrices with the model during deployment time. They work like simple adapters. You load up the one you need to solve a problem and use a different one for the next task.

Then, we have QLoRA, which makes the process much more efficient by adding 4-bit quantization. QLoRA deserves its own separate post.

The team at @monsterapis has created an efficient no-code LoRA/QLoRA-powered LLM fine-tuner.

What they do is pretty smart:

They automatically configure your GPU environment and fine-tuning pipeline for your specific model. For example, if you want to fine-tune Mixtral 8x7B on a smaller GPU, they will automatically use QLoRA to keep your costs down and prevent memory issues.

The @monsterapis platform specializes in no-code LoRA-powered fine-tuning. It's the fastest and most affordable offering for fine-tuning models in the market. They sponsored me and gave me 10,000 free credits for anyone who uses the code "SANTIAGO" in their dashboard:

https://t.co/DBsB9EXgYR

If you want to read their latest updates, get free credits and special offers, join their Discord server: https://t.co/oecoxEec14

TL;DR:

• Traditional fine-tuning trains the entire model. It requires a complex setup, higher memory, and expensive hardware.

• LoRA: Trains a small portion of the model. It's faster, requires much less memory, and affordable hardware.

• QLoRA: Much more efficient than LoRA, but it requires a more complex setup.

• No-code fine-tuning with LoRA/QLoRA: The best of both worlds. Low cost and easy setup.

[Csaba Kissi @csaba_kissi]

Here's a new LLM deployment solution that enables serving multiple LLMs & Lora adapters as an API endpoint powered by our robust GPU Cloud.

Ready for a test drive?

Let's deploy the Mixtral 8x7b Chat model with GPTQ 4bit quantization on a 48GB GPU. 🚀🧵

[Q Blocks @blocks_q]

@chaseleantj It's surprising to see that SDXL is able to deliver at par results. Built these using @monsterapis
same prompt.

[Neo AI @withneo]

@lazyiitian @Gaurav_vij137 @tunguz @blocks_q Your wish is our command

[Q Blocks @blocks_q]

Now, you can fine tune Llama 2 on our massive network of GPUs without writing a single line of code

[Santiago @svpino]

How to fine-tune Llama 2 without writing a single line of code.

I taught Llama 2 to classify the sentiment of movie reviews. Setting everything up took me 10 minutes. The fine-tuning process lasted 6 hours.

If you aren't familiar with the term "fine-tuning," it’s the process we use to teach a model how to solve a specific task. Large Language Models have general knowledge but struggle to solve particular problems.

Fortunately, we can fine-tune these models and make them very good at solving specific tasks. In this example, that task is determining how much a person liked a movie based on its review.

Unfortunately, fine-tuning a model is a complex, expensive process. It takes a lot of time, effort, and GPU computing. It's also hard to find experienced people who know how to do it.

The team @monsterapis built the first platform that offers no-code fine-tuning of open-source models, which changes everything.

That’s the platform I’m using here.

Here is what you need to do:

1. Sign up here: https://t.co/HaeAxyM3be, and use the code SANTIAGO during sign-up to get 5,000 free API credits.

2. Go to the FineTuning option and select Llama 2 7B.

3. Select your task. I'm using "Text Classification" since we want to classify movie reviews.

4. The last step is to select your dataset. I used the IMDb dataset from HuggingFace.

It took a bit under 6 hours to finalize the fine-tuning process. The attached image corresponds to the training loss over the first few steps. I spent 14,000 credits in the process, equivalent to $14.00.

That’s one of the @monsterapis’ advantages: Besides not dealing with code, complexity, or hardware, their pricing is very competitive, thanks to their decentralized GPU platform.

Here's an article that provides a step-by-step guide for fine-tuning Llama 2: https://t.co/KpF7kl7i27

You can also join @monsterapis’ Discord server for the latest updates, free credits, and special offers: https://t.co/oecoxEec14

Thanks to the team @monsterapis for partnering with me on this post.

[Santiago @svpino]

8 billion lives will change because of AI.

But those who think large models are the end-game don't know what's happening.

The real game-changer coming after every industry is the applications we are building right now.

[Q Blocks @blocks_q]

🚨 FREE A100 GPU for AI🚨
Get a chance to access Free A100 GPU instance on Q Blocks GPU platform for 1 week.
Apply for your business here:
https://t.co/pw6TkUvyx6

#a100 #nvidia #GPU #cloud #machinevision #LLM #GPT4 #Chatgpt #GPT4All #text2video

[SomosNLP @SomosNLP_]

"Una sorpresa" = 10000 API credits para todas las personas participantes y una suscripción anual para el equipo ganador 🎉🎉🎉

Thank you so much @Gaurav_vij137 and @saurabhvij137 (@blocks_q) 🤩

[Q Blocks @blocks_q]

Folks in Speech Recognition would love this:
90% savings

#whisper #MachineLearning

[Q Blocks @blocks_q]

And the Hackathon kicks in

[SomosNLP @SomosNLP_]

Una forma inteligente de entrenar, ajustar y desplegar tus modelos de ML

⚙️ El fundador de @blocks_q, empresa patrocinadora de las GPU VMs del #HackathonSomosNLP, nos explicará cómo ofrecen máquinas a precios tan competitivos y cómo hacer uso de ellas.

➡️https://t.co/j8OzIGzonn

[Q Blocks @blocks_q]

@singh_sequoia Some of them are saving significant amount of capital by using GPUs on decentralised networks like us.

Once the AWS credits expire, the pain kicks in.
We are here to help.

Please feel free to connect with any portfolio building in AI.

[Q Blocks @blocks_q]

@nivi @naval Here's our demonstration of 90% savings for Transcription and Translation using Whisper.

Optimised Whisper API, running on decentralised GPU network

With further optimisation, we are looking at 95% savings.
At moderate scale this means:

$50,000 vs $1M
https://t.co/ccaswMfSAK

[Q Blocks @blocks_q]

@nivi @naval Already saved more than $2M to our users:
ML developers training and deploying some really powerful models.

Now imagine this:
100s of open sources LLMs running on infinitely scalable GPUs all across the globe abstracted from X Boxes & Playstations

That's what we are building

[Q Blocks @blocks_q]

Now edit images with easy to use Instruct-pix2pix API

[Gaurav Vij @Gaurav_vij137]

Don't let your #AI research ideas stop due to the cost of computing. We at @blocks_q are enabling upto 10x cost efficient computing access for ML researchers and teams using #decentralized computing approach.
More on: https://t.co/2PYZtyqXk6

#GPU #deeplearning #ML #tech

[Q Blocks @blocks_q]

The future is here: Decentralized compute for Machine Learning, delivered through simple to use APIs

[elvis @omarsar0]

The future of computing is here & it’s exciting!

If you're a generative AI dev, try Monster API and take your projects to the next level.

https://t.co/TEfCkqpk36

If you sign up now, you get
- 5000 free credits (using GitHub)
- 2500 free credits (using another signup method)

[Q Blocks @blocks_q]

If you are in Generative AI, you can't afford to miss this:

[saurabh vij @saurabhvij137]

We are live

[Q Blocks @blocks_q]

@pmarca This is getting scary

[Q Blocks @blocks_q]

@Ilozue716 @levelsio Yes.