johnsonhk88 @johnsonhk88 - Twitter Profile

20 days ago

Took a while, but here it is, a mega write-up on language models for text classification and, of course, Jev. It's basically a visual guide to RNNs, CNNs, transformers, and calibration, with hands-on experiments on accuracy and efficiency.

rasbt's tweet photo. Took a while, but here it is, a mega write-up on language models for text classification and, of course, Jev.

It's basically a visual guide to RNNs, CNNs, transformers, and calibration, with hands-on experiments on accuracy and efficiency. https://t.co/fVHpuKvu5o

78

2K

325

2K

116K

johnsonhk88 retweeted

Dan Kornas

@DanKornas

about 2 months ago

Long-context LLMs hit a memory wall fast. This package targets the KV cache. TurboQuant-GPU is a Python package for compressing KV caches during LLM inference on NVIDIA GPUs. It helps you reduce KV-cache memory use by applying random orthogonal rotation and quantization to Keys and Values, with cuTile kernels where available and an automatic PyTorch fallback. Key features: • 5.02x KV-cache compression – the README reports this compression ratio for LLM inference • Separate Key and Value paths – Keys use 2-bit MSE quantization plus 1-bit QJL bias correction; Values use 3-bit MSE quantization • Fused compression kernel – compresses K and V in one launch per attention head • GPU-aware auto-tuning – benchmarks cuTile versus PyTorch and 2-bit versus 3-bit options for your GPU • Notebook examples – includes a quickstart plus kernel profiling and quality-comparison notebooks It’s open-source (MIT license). Link in the reply 👇

DanKornas's tweet photo. Long-context LLMs hit a memory wall fast. This package targets the KV cache.

TurboQuant-GPU is a Python package for compressing KV caches during LLM inference on NVIDIA GPUs.

It helps you reduce KV-cache memory use by applying random orthogonal rotation and quantization to Keys and Values, with cuTile kernels where available and an automatic PyTorch fallback.

Key features:

• 5.02x KV-cache compression – the README reports this compression ratio for LLM inference
• Separate Key and Value paths – Keys use 2-bit MSE quantization plus 1-bit QJL bias correction; Values use 3-bit MSE quantization
• Fused compression kernel – compresses K and V in one launch per attention head
• GPU-aware auto-tuning – benchmarks cuTile versus PyTorch and 2-bit versus 3-bit options for your GPU
• Notebook examples – includes a quickstart plus kernel profiling and quality-comparison notebooks

It’s open-source (MIT license).

Link in the reply 👇

7

79

15

84

4K

johnsonhk88 retweeted

Dan Kornas

@DanKornas

3 months ago

LLM behavior changes too easily to rely on a few manual spot checks. DeepEval is an open-source Python framework for evaluating LLM applications such as agents, RAG pipelines, and chatbots. It helps you turn quality checks into repeatable tests by combining Pytest-style test cases with ready-to-use evaluation metrics. Key features: • Ready-made metrics – evaluate answer relevancy, faithfulness, hallucination, agent task completion, and more. • Pytest-style workflow – define test cases and run them from the CLI with `deepeval test run`. • End-to-end and component evals – assess the full application or individual traced components. • Flexible evaluation models – use your chosen LLM, statistical methods, or local NLP models, depending on the metric. • Dataset and CI support – generate single- or multi-turn synthetic datasets and run evaluations in any CI/CD environment. It’s open-source (Apache 2.0 license). Link in the reply 👇

DanKornas's tweet photo. LLM behavior changes too easily to rely on a few manual spot checks.

DeepEval is an open-source Python framework for evaluating LLM applications such as agents, RAG pipelines, and chatbots.

It helps you turn quality checks into repeatable tests by combining Pytest-style test cases with ready-to-use evaluation metrics.

Key features:
• Ready-made metrics – evaluate answer relevancy, faithfulness, hallucination, agent task completion, and more.
• Pytest-style workflow – define test cases and run them from the CLI with `deepeval test run`.
• End-to-end and component evals – assess the full application or individual traced components.
• Flexible evaluation models – use your chosen LLM, statistical methods, or local NLP models, depending on the metric.
• Dataset and CI support – generate single- or multi-turn synthetic datasets and run evaluations in any CI/CD environment.

It’s open-source (Apache 2.0 license).

Link in the reply 👇

2

43

8

58

4K

johnsonhk88 retweeted

Kirk Borne

@KirkDBorne

3 months ago

Deep Learning with PyTorch — Step-by-Step Beginner's Guides (3 volumes) Vol.1 (Fundamentals): https://t.co/68j8rntZg7 Vol.2 (Computer Vision): https://t.co/kmTLjjmhk8 Vol.3 (Sequences and NLP): https://t.co/rhNr0RL8g7 ——— #MachineLearning #ML #AI #DataScience #DataScientist