Jose Maria Quevedo

100% Programado en assembler

[Athletic Club @AthleticClub]

❤️ Así ha sido recibido De Marcos en su rueda de prensa de despedida en Lezama.

#MilaEskerDeMarcos #AthleticClub 🦁

[Euskobyte Elkartea EA2RKE @euskobyte]

Te proponemos darle uso a ese Walky que lleva mucho tiempo en el cajón, sin encenderse.

https://t.co/JJYxhPjOE2
#Radioaficionados

[Alex Cheema @alexocheema]

Had to stack up 8 Mac Minis to get it running.

~5 tok/sec for now.

First time running inference on 8 Mac Minis - performance can be improved a lot (theoretical limit is >10 tok/sec on this setup).

[Jose Maria Quevedo @motionlabs]

@elonmusk 👏👏👏

[Euskobyte Elkartea EA2RKE @euskobyte]

Pondremos en el aire indicativo especial con motivo del contacto escolares de Indautxuko Eskola con astronautas de la @Space_Station
Stay tuned on all Ham-Bands. @ARISS_Intl @ARISSFR @IK1SLD_ARISS #Radioaficionados #hamr #hamradio

[INNOLAB Bilbao @InnolabBilbao]

‼️ Tenemos una cita con @chemaalonso, Chief Digital Officer de @Telefonica, y su equipo.

➡️ #OpenSession Programa en #InteligenciaArtificial para Expertos en #Ciberseguridad de @deusto.

🗓️ 16 de mayo
🕐 De 16:00h a 19:00h
✍️: https://t.co/7DTELLSOwh

#IA #IAG

[DX World.net @DX_World]

Red zones in the map show where shortwave radio signals are being absorbed. Inside the Arctic Circle, frequencies below 30 MHz are almost completely blacked out, a nuisance for polar aviators and ham radio operators.

What's causing this? Protons accelerated by solar flares in the magnetic canopy of AR3664 are following the Parker Spiral back to Earth. Think of it as a magnetic superhighway. The arriving particles are funneled by our planet's magnetic field toward the poles where they ionize the atmosphere and interfere with the normal transmission of shortwave radio.

This polar cap absorption event could last for days, especially if it is boosted by more flares from AR3664.

https://t.co/jzyMYNbEIj

[Ryan Els @RyanEls4]

Types of personalities 🥛🤓

[David Marugán @RadioHacking]

📢 Pues acabo de decidir sortear esta Retekess TR-112 de mi búnker. Si quieres participar tienes que hacer lo siguiente (solo hasta mañana 30/06/2024 a las 23:59):
✅Sigue esta cuenta @RadioHacking
✅Haz RT de este tweet
⚠️Solo asumo el coste de envío en España.
¡Suerte! 😉📻

[mylo📡〰 @mylo_calibre]

absolutely killer read for all you rf fluffs out there
made my flight bearable

[Andrew Elbert Wilson @FPGA_Zealot]

Got Jupyter-lab and pyrtlsdr added to the Agilex 7 HPS yocto image. Now we got a nice GUI to demo all the different FPGA features. I wonder how Quartus's DMA IP works.

[PyQuant News 🐍 @pyquantnews]

New to Pandas?

Here's a cheat sheet you can download (free):

[The Deep View @theDeepView]

We spend all week researching the top AI tools so you don't have to...

Get insights, tools, and news for free when you join our community of 150,000+ readers.

Click the image below to subscribe for free 👇

[Osakidetza @osakidetzaEJGV]

🏅🏥 @santamarinaospi recibe en Madrid el premio ‘Institución digital del año’ por su contribución al bienestar a través de la tecnología ➕ https://t.co/LfW5ssJXkY

[hacker.house @hackerfantastic]

MS-DOS v4.0 source code released by @Microsoft https://t.co/tdR52FyZ8G

[Andrew Elbert Wilson @FPGA_Zealot]

Hack Pack + FPGA = ?

[Rohan Paul @rohanpaul_ai]

What is GGUF and the new K-quants - The basics ❓

📌 GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML.

GGUF has the same type of layout as GGML, with metadata and tensor data in a single file, but in addition is also designed to be backwards-compatible. The key difference is that previously instead of a list of values for the hyperparameters, the new file format uses a key-value lookup tables which accomodate shifting values.

Basically, GGUF (i.e. "GPT-Generated Unified Format"), previously GGML, is a quantization method that allows users to use the CPU to run an LLM but also offload some of its layers to the GPU for a speed up.

📌 GGML is a C++ Tensor library designed for machine learning, facilitating the running of LLMs either on a CPU alone or in tandem with a GPU.

💡 GGUF (new)

💡 GGML (old)

Llama.cpp has dropped support for the GGML format and now only supports GGUF

------------

* GGUF contains all the metadata it needs in the model file (no need for other files like tokenizer_config.json) except the prompt template

* llama.cpp has a script to convert *.safetensors model files into *.gguf

* Transformers & Llama.cpp support both CPU, GPU and MPU inference

Being compiled in C++, with GGUF the inference is multithreaded.

↪️ GGML format recently changed to GGUF which is designed to be extensible, so that new features shouldn’t break compatibility with existing models. It also centralizes all the metadata in one file, such as special tokens, RoPE scaling parameters, etc. In short, it answers a few historical pain points and should be future-proof.

----------------

📌 GGUF (GGML) vs GPTQ

▶️ GPTQ is not the same quantization format as GGUF/GGML. They are different approaches with different codebases but have borrowed ideas from each other.

▶️ GPTQ is a post-training quantziation method to compress LLMs, like GPT. GPTQ compresses GPT models by reducing the number of bits needed to store each weight in the model, from 32 bits down to just 3-4 bits.

▶️ GPTQ, a one-shot weight quantization method, harnesses approximate second-order information to achieve highly accurate and efficient quantization.

▶️ GPTQ analyzes each layer of the model separately and approximating the weights in a way that preserves the overall accuracy.

▶️ Quantizes the weights of the model layer-by-layer to 4 bits instead of 16 bits, this reduces the needed memory by 4x.

▶️ Achieves same latency as fp16 model, but 4x less memory usage, sometimes faster due to custom kernels, e.g. Exllama

-------------

K-quants (models will be identified as "q3_K_S" or "q3_K_L" and so on )

The k-quant system uses value representations of different bit widths depending on the chosen quant method. First, the model’s weights are divided into blocks of 32: with each block having a scaling factor based on the largest weight value, i.e., the highest gradient magnitude.

Depending on the selected quant-method, the most important weights are quantized to a higher-precision data type, while the rest are assigned to a lower-precision type. For example, the q2_k quant method converts the largest weights to 4-bit integers and the remaining weights to 2-bit. Alternatively, however, the q5_0 and q8_0 quant methods convert all weights to 5-bit and 8-bit integer representations respectively.

Examples of model named with the `k-quant` method

q2_K => Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors.

q3_K_S => Uses GGML_TYPE_Q3_K for all tensors

q3_K_M => Uses GGML_TYPE_Q4_K for the `attention.wv`, `attention.wo`, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K

q3_K_L => Uses GGML_TYPE_Q5_K for the `attention.wv`, `attention.wo`, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K

- **`attention.wv` (Value Weight Matrix)**: In the context of the transformer's attention, input embeddings are transformed into three different vectors: key (K), query (Q), and value (V) vectors. The `attention.wv` refers to the weights that linearly transform input embeddings into value vectors. These value vectors are then used, along with the output from the key and query vectors, to compute the attention scores that determine how much each part of the input should be attended to in producing the output.

- **`attention.wo` (Output Weight Matrix)**: After computing the attention scores and using them to create a weighted combination of the value vectors, the result is passed through this final linear transformation represented by `attention.wo`. This transformation is applied to the aggregated output of the attention mechanism before it is sent to subsequent layers or for further processing within the same layer. Essentially, `attention.wo` shapes the output of the attention mechanism to ensure it is suitable for the model's next steps.

-------------

▶️ The bitsandbytes library quantizes on the fly (to 8-bit or 4-bit) which is also knows as dynamic quantization

▶️ And there's some other formats like AWQ: Activation-aware Weight Quantization - which is a quantization method similar to GPTQ. It protects salient weights by observing activations rather than the weights themselves. AWQ achieves excellent quantization performance, especially for instruction-tuned LMs and multi-modal LMs.

There are several differences between AWQ and GPTQ as methods but the most important one is that AWQ assumes that not all weights are equally important for an LLM’s performance. For AWQ, best to use the vLLM package

[rtl-sdr.com @rtlsdrblog]

Gypsum: A Software-Defined GPS Receiver written in Python + A Writeup on How it Was Made https://t.co/xMwUHFfAmr

[Troll Football @TrollFootball]

VAR 😂

https://t.co/twUdApOdQQ