Sudhanshu Garg

lurker.

[Zhaofeng Wu @ ACL @zhaofeng_wu]

Introducing ><former

Most transformers are rectangles◻️: every layer has the same width

But is that optimal?🤔

We propose variable-width transformers that have different widths across layers, improving loss while cutting compute & KV cache size 🧵

[Sudhanshu Garg @sud_garg]

@luke_wood_ml one more, this thread is one of the "other" resources i was talking about https://t.co/qRu2lvjOip

[Sudhanshu Garg @sud_garg]

@luke_wood_ml yes, but thats what you do in regular gradient descent. Lets say there are no batches, and you have N training examples, then the loss is 1/N * (per example loss). We don't sum the loss across all the training examples and then use that to update the gradients.

[Sudhanshu Garg @sud_garg]

@OafishBlowfish yeah no idea.

[Sudhanshu Garg @sud_garg]

@AlaskaAir have flight credits that were booked for June 2021, and unable to use them due to covid travel ban. Email says expiry on 04/2022 but was told they wouldn't expire. Any help here ?

[Sudhanshu Garg @sud_garg]

@RecParkSF spotery is down today morning (May 2), no reservations possible!

[Sudhanshu Garg @sud_garg]

#seared #chefsAreAwesome

[Sudhanshu Garg @sud_garg]

Just watched a fantastically enacted play at the sf playhouse #seared

[Sudhanshu Garg @sud_garg]

Tweeting from twitter hq! Its awesome!

[Sudhanshu Garg @sud_garg]

revisiting pointers never gets old. A great explanation at https://t.co/cK4RNRdgvb and https://t.co/LQzJBlscI2 #mycodeschool

[Suresh Raina🇮🇳 @ImRaina]

Don't give up till the end Don't celebrate before you win! #IndvsBan Wat a game 👌👍✌️

[Sudhanshu Garg @sud_garg]

@Honda like I said earlier, nothing for doctorates though.

[Sudhanshu Garg @sud_garg]

@Honda This grad program is useless, doctorates are excluded, and they invest considerable time and effort.

[Harsha Bhogle @bhogleharsha]

Terrible shot from Maxwell. Terrible effort from Sohail. Pakistan fielders need to line up and apologise to Wahab Riaz.

[Vidhu Pal @raahilmahajan96]

Overcast at napier. I hope rain stays away. Windies should not miss out because of rain.
#WIvsUAE #CWC15

[Sudhanshu Garg @sud_garg]

#QZ5801, hopefully the unusual route was the safer one. #airasia http://t.co/l0oT3UiNBH

[Sudhanshu Garg @sud_garg]

@cruhari cruY. Wheres the gift.

[UberFacts @UberFacts]

Near-sightedness later in life is a sign of higher education.

[Sudhanshu Garg @sud_garg]

@BCCI #ThankYouSachin Thanks for all the goosebumps over the many years.. #NowCanIHaveThatAutograph