joe_jacobs

[Daniel Jeffries @Dan_Jeffries1]

Tell me you have zero DevOps skills without telling me you got zero DevOps skills.

And LLM is not the I Love You Virus. It is not the same as infecting a web server. It is not the same as hacking a wiki. One is a commodity piece of software that is widely distributed and this capability is a non-trivial leap in hardware/software/capabilities and one does not automatically follow the other.

Currently, it is a massive effort to run these models, usually distributed across multiple very expensive chips, running on specialized inference software, specialized harness software, databases, cloud computers, with tooling and hundreds of install packages, etc.

Kimi K3 is 1.6 TB of space at 2 trillion parameters and you need about 1.5 TB of VRAM, which means you need 8xGB300s which will cost about 350-450K a year to purchase or as much or more to rent.

Now let's say Astra or Mythos or later models come in at 10 trillion parameters, so that puts us at roughly 10 to 20 TBs unquanticized. We are guessing her because these folks reveal nothing about architectures anymore, but they are reasonable guesses based on scaling laws. You would need roughly 23 TB of VRAM to run it and roughly several million dollars to run that one instance a year. And to make it worse, it no longer fits on an 8 way cluster, which means you now need special networking configuration and inference software to run it or you need to infect a Nvidia NVL72 rack-scale server system costs between $2.8 million and $6.5 million.

So the "rogue" model copies itself to the most expensive compute in the world. It's safe to say that hardware is not sitting idle and is being watched closely for downtime or anomalies. In fact, it is likely doing one of the following things:

1) Serving customers at a hyperscaler

2) Serving inference for a well endowed company.

In the first case it will be instantly noticed by the actually good automated monitoring systems of the hyperscaler because there is now something using their infrastructure but not being billed properly and costing them millions of dollars in lost revenue and electricity.

In the second example, the model they were serving on that very expensive hardware is now offline (replaced by the rogue model). The model presumably was connected to a valuable piece of internal software that is now glitching and no longer running.

Nobody notices this?

And so we see that this is where sci-fi meets the little problem of the real world where friction, dust, time, resources limitations and the like come to crush your little fantasies.

To be fair, this *may* be possible in coming years. I assign it a non-zero probability. Breakthroughs in new architecture and smaller models and eventually the wider distribution of the computing substrate that commoditizes could change this materially. But for now, it is mitigable and foreseeable and an engineering problem. In no way will these things be copying themselves around like a virus on your computer in their current form.

It also betrays a lack of understanding of the evolution of complex economic and societal and technological systems, we tend to get mitigations as parallel evolutionary developments. Things do NOT evolve in a vacuum.

So when you get viruses, you get anti-virus software.

When you get DDOS attacks, you get CloudFlare.

There is money in solving problems and things do not evolve in isolation.

Believing that things evolve in isolation, or that one thing changes and all other variables stay the same, is one of the many reasoning errors that destroy people's ability to make accurate predictions.

In addition, when you don't have the requisite knowledge to verify your predictions, things that sound sane and rational to you are, in fact, insane and ridiculous.

[Jarrett Payton @paytonsun]

Every time I watch this run, I still ask myself… how?

My pops wasn’t from this planet.

#Bears | @walterpayton 🐐

[NASA @NASA]

For the crew of Artemis II, their mission will soon be reality. Learn about the challenges they face and the teamwork required to fly around the Moon.

Episode 2 of Moonbound is live—and free to watch on NASA+: https://t.co/NtKmF2gywp

[Marc Morgan @MarcMorganFilms]

The @chicagobears documentary of 2074: “The Comeback”. 🐻⬇️ (Eventual) Director: Marc J. Morgan 🎬 @ChicagoBearsHx

[joe_jacobs @jabobs]

@Ionoclast50 @ianmiles

[joe_jacobs @jabobs]

@davidleavitt @elonmusk You get a cookie.

[joe_jacobs @jabobs]

@realandygeorge @realandygeorge long time listener, first time caller here. Please read my reply. Your tweet was found to be inaccurate. #fakenews

[Soonson Kwon @ksoonson]

Someone built a real tombstone of Internet Explorer in Korea. "He was a good tool to download other browsers." https://t.co/42vnkoQshd

[Ariana Baldassano @ari_baldie]

. @DanPonceTV x @myloreyes collab???

[LEGO @LEGO_Group]

Who else could we ask but LEGO-loving @LandoNorris to reveal the new LEGO Technic take on this season’s incredible @McLarenF1 car? #McLaren #LEGOTechnic #F1 #BuildForReal

[joe_jacobs @jabobs]

@raykrueger Hi Raycicle!

[joe_jacobs @jabobs]

@patrickklepek Get rid of the weekdays. Only weekend 🔥 now baby.

[Alec Giannakopoulos @news8alec]

Aaron Rodgers is learning what George McCaskey knows all to well.

Owning the Bears doesn’t get you much in the playoffs

[Jerry Gamblin @JGamblin]

A fantastic #Log4j detection tool by our @KennaSecurity / @CiscoSecure engineering team. https://t.co/Td0r7ZZeRk

[joe_jacobs @jabobs]

@realandygeorge it's valid :-)
Good moods are deleting irrelevant things in tech.

[joe_jacobs @jabobs]

Verifying myself: I am jabobs on https://t.co/fglKzxGmfj. LPyZ5BkcIWIJPVL0E9kM8QaFFfPZRy81raUw / https://t.co/OABiqiagpT

[joe_jacobs @jabobs]

@Zak_Bagans please check out this possible murder ghost train before my buddy Zable dies here.
https://t.co/YVxJw6kVIf

[Kenna Security @KennaSecurity]

📢Big News!📢

We’re partnering with @vmw_carbonblack to power the #vulnerability assessment and risk scoring capability of Carbon Black Cloud Workload. https://t.co/hHCvS4BKoq

[joe_jacobs @jabobs]

I gave some un-used cloud credits at work a new life! Brushed off the dust and told them to get to work researching #covid19
A fun time exposing a fleet of Nvidia GPU's for maximum compute.
350 million credits crunched in a few weeks:
#foldingathome
https://t.co/ST8P3PYg0E

[joe_jacobs @jabobs]

I just donated to %40Wikipedia. Support free knowledge%21 %23iloveWikipedia https%3A%2F%https://t.co/ajQVivVvAy%2F%3Futm_medium%3DSocialMedia%26utm_campaign%3DThankYouPage%26utm_source%3DTwitter%26uselang%3Den