踢裆视频收藏夹

热爱踢裆，更多视频：

[踢裆视频收藏夹 @yanboliang]

她斜倚沙发，粉高跟轻点他腿间，忽地一踢，他痛得颤栗，鞋飞落一旁。她鞋尖追着踢，他跪地求饶，她只冷眼不睬，直到他滚下沙发。 #女王 #足控 #调教 #疼痛游戏 #高跟

[踢裆视频收藏夹 @yanboliang]

跪在桌边，臀线绷紧，呼吸都放轻了。她的鞋跟点地，一声声近得像在数你的心跳。电流未启，凉意先抵上来，你不自觉塌下腰�� #女S #男M #跪姿 #调教 #高跟鞋

[踢裆视频收藏夹 @yanboliang]

高跟鞋踩在腿间，他疼得蜷起身子，额头抵着地面直抽气。她没停，鞋尖挑起他的下巴，垂眼看他眼眶发红地求饶。脚底碾过他胸口，慢慢往下压，像在丈量他能忍到哪一步。 #女S #男M #踩踏 #调教

[踢裆视频收藏夹 @yanboliang]

她慵懒地伏在沙发边，粉色高跟轻轻碾过���，每一下都带着不容抗拒的力道。你颤抖着求饶，她却只低笑，鞋尖精准地落下，直到你滚落在地，臣服于她的节奏。 #女王 #调教 #足控 #掌控

[Horace He @cHHillee]

Fun fact, this was actually the first novel mask implemented in FlexAttention from someone not working on FlexAttention!

[Sebastian Raschka @rasbt]

PyTorch has a new attention function for LLMs called FlexAttention that supports various attention variants.
I just took it for a spin and compared it to other implementations of multi-head attention, and it seems to work great in combination with torch.compile.
However, if you are not compiling your models (e.g., due to compatibility issues), it may be better not to use it for multi-head attention.

[Horace He @cHHillee]

For too long, users have lived under the software lottery tyranny of fused attention implementations.

No longer.

Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch.
https://t.co/IXeUS6AkrY
1/10

[Horace He @cHHillee]

Grok-1 support in gpt-fast at faster(?) than anyone else has reported so far. 75 tok/s for a 300B+ parameter model on an 8xA100 node.

If I understand correctly, ColossalAI reported 15 seconds to generate 100 tokens. gpt-fast takes 4.2 seconds to generate *400* tokens.

[Horace He @cHHillee]

@yanboliang just landed a significant improvement in mixtral perf (https://t.co/Xm83xIBXvP).

For one A100 + int8, we go from 56 tok/s to 98 tok/s!

That's about 64% MBU, not too bad.

Essentially, we had one of the weight matrices transposed the wrong way :)

[Horace He @cHHillee]

With the new release of Gemma-2B, I thought I'd see how torch.compile performs.

Gemma 2B for a single prompt runs at 144 tokens/s on a V100, a 4x increase over the uncompiled HF version.

We're working with @huggingface to upstream these improvements too!

[Horace He @cHHillee]

Announcing it 2 months after the work was done, but gpt-fast now supports Mixtral + MoE models!

Featuring:
- faster decoding than any (non-Groq) API endpoint, at up to 220 tok/s/user.
- no custom kernels
- int8/TP
- still simple!

How do we do it? Well, torch.compile :)
(1/5)

[踢裆视频收藏夹 @yanboliang]

@0xMogluc @soumithchintala We run on 8 A100-80GB GPUs, so the int8 8x7B model can fit into 1 GPU and got 56 tokens/sec.

[Soumith Chintala @soumithchintala]

gpt-fast now supports mixtral-8x7B, in addition to gpt/llama.
1000 lines of simple pytorch code blazing it out!
https://t.co/crXdcNy0uv

[Tianqi Chen @tqchenml]

Looking forward to #SysML19 (https://t.co/Srzp7w4Tdw)! Full-stack software-hardware systems + ML will collectively drive future intelligent systems.

[Matei Zaharia @matei_zaharia]

The #SysML 2019 Demo Track is now open for submissions: https://t.co/wViJ5E5PaD. We're excited to see your demos of research systems relevant to ML.

[François Chollet @fchollet]

As a developer, your job isn't about how much code you write, it's about how effectively you solve problems.

The best code is code you don't write & maintain -- because you're using off-the-shelf solutions, or because you're picking solutions that involve fewer subproblems.

[Google AI @GoogleAI]

Learn more about the research behind Smart Compose, a new feature in Gmail that uses machine learning to interactively offer sentence completion suggestions as you type, allowing you to draft emails faster → https://t.co/1cyrOPQsvT

[踢裆视频收藏夹 @yanboliang]

@holdenkarau @ApacheSpark @flySFO @DataWorksSummit Which flight do you take?

[Jimmy Lin @lintool]

This paper suggests that Airbnb runs R in prod to serve predictive models. Can someone confirm or refute? https://t.co/ApL32N8D4F

[Matei Zaharia @matei_zaharia]

The new research group I'm part of at Stanford, DAWN, is building infrastructure for usable machine learning: https://t.co/M9SCkr3WS8