GottaDo

Is it what it is?

[GottaDo @got_2_do]

No one is stupid enough to compare India and Finland prices.

Most likely this Sumit wanted to take this opportunity to show he has been to a foreign country.

Wishing you best for the life friend.

[Sumit @sumitsaurabh]

I have been charged 2500 INR for a 4 minute , 1 km ride in Helsinki .
What’s the point ? Should I share the invoice ?

[GottaDo @got_2_do]

Tiktok, UC Browser and etc are now accessible in India.

[GottaDo @got_2_do]

I guess everything doesn't have to mean something everytime. Sometimes it is just passing a phase and learnings/no learning for future.

[GottaDo @got_2_do]

@stoneycodes videos starts with excellent points about how to consume content.

Below video is good for peps who are afraid with data structure algo.

Starts with good easy examples to make one comfortable to try further ques themselves before taking solutions from the video.

[Sumit Behal @sumitkbehal]

Gen-Z with lower salaries are consistently being fooled by route to 1 Crore by doing SIPs of 2000 INR for 50 years

They will stop getting fooled the day they realize 1 Crore isn't a lot of money

I highly recommend them to dream big, avoid noise, work hard and make money

[GottaDo @got_2_do]

@Vajrapani4 Neighbors think something is wrong - 100% true.

[Pranav Mehta @i_pranavmehta]

In the current age you can't really blame your bg, ancestral wealth or luck for your current situation. You have a laptop and internet you have enough to decide your own fate!

[GottaDo @got_2_do]

@Vajrapani4 😂

Anyway Praveen deserves good credit, he made large number of people interested about Indian temples. Loved his work of exploring temples.

[GottaDo @got_2_do]

@Vajrapani4 It 'might' be true for people who put years on mehnat but same can be seen for people inheriting or getting jobs due to some easy jugaad.

I think it is like, ataa hua chiz kisko bura lagta hai, aane do.

[Andrej Karpathy @karpathy]

Reading a tweet is a bit like downloading an (attacker-controlled) executable that you instantly run on your brain. Each one elicits emotions, suggests knowledge, nudges world-view.

In the future it might feel surprising that we allowed direct, untrusted information to brain.

[GottaDo @got_2_do]

@sourab_m @abhi1thakur Because Grok didn't start with thought of 'open source' and also no funding were taken from third party.

[GottaDo @got_2_do]

@JustAnkurBagchi I have seen the responses to this on sub.

No one is wrong here, people and opinions change about everything over time, one can become either one over timing. Judging either would be wrong.

[GottaDo @got_2_do]

Unnecessary hate for startup here:

These models are going to help n number of Indian orgs as they deal with customers from different language backgrounds.

I had my exp with Meta, OpenAI model for text and speech, they are nowhere to be put in production for most cases.

[Archie Sengupta @archiexzzz]

i could have used that $50M for solving more india centric problems using AI.

we have to stop copying US companies and make it 'India based' ffs. Real innovation neither gets recognition nor funding in this country.

[GottaDo @got_2_do]

@archiexzzz I don't know the number of users your company deals with as you think is useless.

Sharing my personal exp: working with crores of customers, most of them have comfortable with regional languages, ChatGPT was barely helpful due limited understanding.

[GottaDo @got_2_do]

@embedchain seems to be interesting OS project.

Seems it can be used by anyone without even having depth knowledge of tech around LLMs.

Task for me - can I deploy it 100% locally offline on simple T4 16 GB machine.

Further would try to learn more and contribute to project.

[GottaDo @got_2_do]

@svpino Two things:

1. Copilot has improved a lot, 2 years back when I used it it was meh and didn't use it till last month. Recent Copilot is very helpful and intelligent like it is totally new product.

2. MoE etc are very recent example to say there is no slow down in AI.

[GottaDo @got_2_do]

When started these were highly confusing, thanks Rohan!!

[Rohan Paul @rohanpaul_ai]

🚀 What is GGML or GGUF in the world of Large Language Models ? 🚀

GGUF / GGML are file formats for quantized models

GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML.

Basically, GGUF (i.e. "GPT-Generated Unified Format"), previously GGML, is a quantization method that allows users to use the CPU to run an LLM but also offload some of its layers to the GPU for a speed up.

📌 GGML is a C++ Tensor library designed for machine learning, facilitating the running of LLMs either on a CPU alone or in tandem with a GPU.

💡 GGUF (new)

💡 GGML (old)

Llama.cpp has dropped support for the GGML format and now only supports GGUF

------------

* GGUF contains all the metadata it needs in the model file (no need for other files like tokenizer_config.json) except the prompt template

* llama.cpp has a script to convert *.safetensors model files into *.gguf

* Transformers & Llama.cpp support both CPU, GPU and MPU inference

Being compiled in C++, with GGUF the inference is multithreaded.

↪️ GGML format recently changed to GGUF which is designed to be extensible, so that new features shouldn’t break compatibility with existing models. It also centralizes all the metadata in one file, such as special tokens, RoPE scaling parameters, etc. In short, it answers a few historical pain points and should be future-proof.

----------------

📌 GGUF (GGML) vs GPTQ

▶️ GPTQ is not the same quantization format as GGUF/GGML. They are different approaches with different codebases but have borrowed ideas from each other.

▶️ GPTQ is a post-training quantziation method to compress LLMs, like GPT. GPTQ compresses GPT models by reducing the number of bits needed to store each weight in the model, from 32 bits down to just 3-4 bits.

▶️ GPTQ analyzes each layer of the model separately and approximating the weights in a way that preserves the overall accuracy.

▶️ Quantizes the weights of the model layer-by-layer to 4 bits instead of 16 bits, this reduces the needed memory by 4x.

▶️ Achieves same latency as fp16 model, but 4x less memory usage, sometimes faster due to custom kernels, e.g. Exllama

----------------------------

▶️ There's also the bits and bytes library, which quantizes on the fly (to 8-bit or 4-bit) and is related to QLoRA. This is also knows as dynamic quantization

▶️ And there's some other formats like AWQ: Activation-aware Weight Quantization - which is a quantization method similar to GPTQ. There are several differences between AWQ and GPTQ as methods but the most important one is that AWQ assumes that not all weights are equally important for an LLM’s performance. For AWQ, best to use the vLLM package

[GottaDo @got_2_do]

@abacaj I am able to do this on 3.5 Turbo API with some strict prompt.

[GottaDo @got_2_do]

@abhi1thakur I like how it picked अनुयायी for follower.

Definitely would like to be part of this.🚀

[GottaDo @got_2_do]

@sirbayes A random explanation I can think is - training data might be acquired with all the reference links on article and then again reference article were parsed for entire context.

Eventually, text(articles here) with high backlink(SEO) were visited many times.