Jpkha

[Brendan Bycroft @BrendanBycroft]

Project #2: LLM Visualization

So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)

[Lee Robinson @leerob]

So, @vercel reverted all edge rendering back to Node.js 😬

Wanted to correct the record here as it's something I've advocated for in the past, and share what I've learned since then.

Also, the "edge" naming has been a bit confusing, so let's clear that up here as well.

What is edge, anyway?

We have too many products with Edge 😅

First off, Vercel has an Edge Network. A request comes in for your site. The edge region can go grab some HTML or JS for your site and quickly return it. Great.

Then, sometimes, you have routing rules. When that request comes in, redirect it, proxy it somewhere else, or add headers before continuing (e.g. Middleware). You don't need to go all the way back to the origin. This is also great.

Sometimes those routing or configuration rules need to be read from a file. We have "Edge Config" which is like reading from a JSON file in every region. Again, still good.

Now, what about rendering my application with edge compute?

Not as great.

But Lee, you said...

Yeah, I gotta admit, this one fooled me. It sounded great at the start – running close to the visitor makes sense.

If I constrain my site to use this limited runtime, and make the tradeoff for access to the Node.js package ecosystem, I can (hypothetically) get better performance.

This hasn't worked for two reasons.

First, and most obvious, is that your compute needs to be close to your database. Most data is not globally replicated. So running compute in many regions, which all connect to a us-east database, made no sense.

I believe many folks understood this. But then, maybe this limited runtime was better even if I only run it close to my database?

Well, that was wrong too. We tested it extensively with @v0. Using the Node.js runtime with 1 vCPU (our "Standard" performance) option for Vercel Functions had faster startups 🤯

Are there other reasons for edge rendering?

If Node.js functions can be faster, especially with Node v20, what about cost?

I expected an edge compute pricing model, which is based on only paying for when your compute runs (and never I/O), would always be cheaper.

Turns out that was also wrong. It can be cheaper sometimes! But not for all workloads, so some nuance applied here.

Same story for security. I wanted edge compute to work, but customers kept telling me they wanted a private connection to other services (i.e. Vercel Secure Compute), or had concerns (or no interest) in doing global data replication (e.g. data sovereignty).

So now what?

It's a good lesson for me to be careful with naming / nuance going forward.

Also, networks have gotten really fast. We saw with @v0 that it was faster to do SSR + streaming with Node.js than edge rendering.

Now we're running @v0 with Partial Prerendering to see if this is better or worse (it's still experimental).

The summary of PPR is: user makes a request, the edge region starts streaming back the HTML for the fast initial visual, and then your compute still runs in us-east or wherever close to your data.

So far this has been about 50% faster time to first byte than SSR + streaming. Pretty amazing.

I'll be sharing more on Vercel Functions improvements soon, as well 🦀

[Astro @astrodotbuild]

🧑‍🚀⭐ Calling all aspiring astronauts—Astro 3.0 touches down next week. We're giving away 10 of these official Astro hats to celebrate!

Retweet for your chance to win.

[Nate Moore is on 🦋 @n_moore]

Trying something new... I would love your support to dedicate more time and effort to my side projects!

Please consider sponsoring me on GitHub
https://t.co/CreVK2vLsU

[Bob Ziroll @bobziroll]

It's @scrimba course launch day 🚀 My new React Router 6 course is now available 100% FOR FREE on Scrimba!

Because we're excited, we're giving away a free year of Scrimba Pro to a ☘️ lucky ☘️ someone who retweets THIS initial tweet. Go ahead, I'll wait ⏱️🙂

About the course 👇

[Easlo @heyeaslo]

What startup has the best landing page?

[Pickbeam @Pickbeam]

On vous présente Le DayBrief 🎉🤩
Votre rendez-vous actus web à retrouver tous les mardis !

Vous y retrouverez :
- Un résumé des infos web de la semaine
- Des outils et ressources
- Des tips et astuces

Hésite pas à claquer ton like et commenter 😉

#ledaybrief N°1

Thread 👇

[Siddharth Dayalwal ⏳ @siddharth_hacks]

Woohoo, kicking off the onboarding call with the all-new batch of Awesome #StoryblokAmbassadors 🧑🏻‍🚀🚀 @storyblok 🥑

[Pickbeam @Pickbeam]

📣 Changement important 📣
A partir de demain, nos émissions matinales sur Twitch commenceront à 08h30 et non plus 08h00 afin qu'un maximum d'entre vous puisse suivre les actus en direct. Merci à toutes celles et ceux qui nous ont fait remonté l'info 🙏🙏

[Theo - t3.gg @theo]

What are the best (technical) blog posts and articles you've read this year?

[Pickbeam @Pickbeam]

@BastiUi en interview live sur @TwitchFR pour débattre de l'utilisation de l'#IntelligenceArtificielle dans le #design 🤖🤖

RDV le vendredi 16 déc. à 13h sur notre chaîne Twitch pour une interview live animée par @antho_parle_web

🫶 Likes et RT appréciés

[Pickbeam @Pickbeam]

🔥 @BastiUi EN INTERVIEW 🔥

RDV le vendredi 16 déc. à 13h sur notre chaîne Twitch pour une interview live animée par @antho_parle_web

Basti reviendra sur son parcours de designer/influenceur et nous débattrons sur l'utilisation des IA dans le #design

🫶 Likes et RT appréciés

[Pickbeam @Pickbeam]

⏰ A quelle heure préférez-vous démarrer les matinales ?

[L'ÉQUIPE de Greg @lequipedegreg]

❄️ Retour du biathlon demain pour les courses de pré-saison ❄️

⏰ 9h15 sur @lachainelequipe !

⛷️ Pour l'occasion, @AliciaDauby3 vous fait gagner la toute nouvelle tenue de l'Équipe de France de biathlon !

@FedFranceSki

RT + follow

#EDG

[Pickbeam @Pickbeam]

Thread et vidéo résumée de l'actualité #web vue ensemble ce matin sur notre chaîne @TwitchFR 👇👇

1/8

[Simplement Antho @simplementAnth0]

Lancement officiel les amis ! Tout est expliqué dans la vidéo : hâte d'avoir vos retours, j'espère que ce projet vous plaira 🔥🔥

[Pickbeam @Pickbeam]

On lance Dayclic, le nouveau média des métiers du #web ! On détaille tout dans cette vidéo👇

📌Liens vers nos plateformes en commentaire

Merci à @BastiUi, grand gourou du DASAGN, pour l'accompagnement au début de ce projet 🫶

#webdev #Webdesign #ecommerce #seo #jamstack

[Pickbeam @Pickbeam]

Hello world ! Lancement officiel de la communauté web Dayclic avec un premier live ce matin sur Twitch (dispo en replay). Merci à celles et ceux qui étaient présents pour cette grande première 🥰

#jamstack #javascript #webdesign #html #css #headlesscms #react #reactjs #cms

[Storyblok @storyblok]

🎟TICKET GIVEAWAY 🎟 https://t.co/VhIHJANDaz

6 tickets for @cityjsconf Athens 🇬🇷 are ready to be sent out to you! 🤩

What do you need to do to get one?
✅ follow @storyblok
🔄 retweet this tweet

Good luck everyone!🍀