We as humans, need a strong foundation just like a building does.
Civil engineering doctoral student Jorge Orozco-Herrera research auger cast piles, deep foundations made by drilling and pumping concrete.
Want to hear more? Read about it: https://t.co/iyWzgd3WMM
1) DeepSeek r1 is real with important nuances. Most important is the fact that r1 is so much cheaper and more efficient to inference than o1, not from the $6m training figure. r1 costs 93% less to *use* than o1 per each API, can be run locally on a high end work station and does not seem to have hit any rate limits which is wild. Simple math is that every 1b active parameters requires 1 gb of RAM in FP8, so r1 requires 37 gb of RAM. Batching massively lowers costs and more compute increases tokens/second so still advantages to inference in the cloud. Would also note that there are true geopolitical dynamics at play here and I don’t think it is a coincidence that this came out right after “Stargate.” RIP, $500 billion - we hardly even knew you.
Real: 1) It is/was the #1 download in the relevant App Store category. Obviously ahead of ChatGPT; something neither Gemini nor Claude was able to accomplish. 2) It is comparable to o1 from a quality perspective although lags o3. 3) There were real algorithmic breakthroughs that led to it being dramatically more efficient both to train and inference. Training in FP8, MLA and multi-token prediction are significant. 4) It is easy to verify that the r1 training run only cost $6m. While this is literally true, it is also *deeply* misleading. 5) Even their hardware architecture is novel and I will note that they use PCI-Express for scale up.
Nuance: 1) The $6m does not include “costs associated with prior research and ablation experiments on architectures, algorithms and data” per the technical paper. “Other than that Mrs. Lincoln, how was the play?” This means that it is possible to train an r1 quality model with a $6m run *if* a lab has already spent hundreds of millions of dollars on prior research and has access to much larger clusters. Deepseek obviously has way more than 2048 H800s; one of their earlier papers referenced a cluster of 10k A100s. An equivalently smart team can’t just spin up a 2000 GPU cluster and train r1 from scratch with $6m. Roughly 20% of Nvidia’s revenue goes through Singapore. 20% of Nvidia’s GPUs are probably not in Singapore despite their best efforts. 2) There was a lot of distillation - i.e. it is unlikely they could have trained this without unhindered access to GPT-4o and o1. As @altcap pointed out to me yesterday, kinda funny to restrict access to leading edge GPUs and not do anything about China’s ability to distill leading edge American models - obviously defeats the purpose of the export restrictions. Why buy the cow when you can get the milk for free?
A big transition from above normal to below normal temperatures coming for most of the US in the coming days. What will this mean for you? Check out https://t.co/VyWINDkBnn for the latest.
Don’t forget to join us tomorrow for the 2024 Cross-USA Lecture @UCF . We will be hosting Dr. Kyle Rollins from @BYU in a hybrid seminar at 3 p.m. EST! #GoKnights
Goodbye Vancouver and #GeoCongress2024. I presented my research on pile driving-induced ground deformations, met new friends, and learned about amazing research ideas.
Thanks to my advisor Luis Arboleda and my colleagues @UCFCECE
P.S: I won @TMatthewEvans Hot Sauce Challenge!
Mark your calendars! We are thrilled to host the 2024 Cross-USA Lecturer Professor Kyle Rollings from
@BYU . Join us on March 7th at @UCF
by registering through this link: https://t.co/NcbBjAhgrl…
Or join our Zoom Meeting (ID: 950 4086 03358)
What a great feeling to be posted by your department 😅😅.
Thanks to my advisor (Dr. Luis Arboleda) and all of our research group members for their support!
Congratulations to civil engineering student Jorge Orozco-Herrera! He is one of 3⃣ students who received the Thomas J. Wysockey Civil Engineering Scholarship this year! https://t.co/sPG1nQyE9f @UCFCECS
Every day I am in awe of international students. Doing a PhD in your native language and home country is hard. Doing it in a non-native language, in a different country and different culture is so much harder. You are not stupid. You are legends.
Hurricane #Lee Advisory 6: Lee Strengthens Into a Hurricane. Likely to Intensify Into an Extremely Dangerous Major Hurricane By Early Saturday. https://t.co/tW4KeGe9uJ
I started a live #Python#code lecture series, '#DataScience Basics in Python', with linked well-documented codes and recorded walk-throughs so you can follow along.
Let me know what you think? I welcome suggestions for additional lectures. On #YouTube @ https://t.co/AWRa4mK6ZI ∀!
Excellent work done by our grad students at #geocongress2023 representing the amazing research performed at @ucf See you all next year at #geocongress2024.
Congrats to our graduate students and past graduates who presented their research on the ongoing FDOT-funded research projects at the 2022 FDOT GRIP meeting! 🥳 #ucf#geotech#civilengineers
Thank you so much Mr. David Wilshaw from Ground Stifness Surveys LLC for performing Advanced Continuous Surface Wave (ACSW) tests with us at UCF Arboretum 🏗 #ucf#civilengineering#geoinstitute#geotech#engineers
Thank you very much Mr. Richard Tedder from @Geosyntec for your great seminar on “Permitting landfills in karst areas of Florida”! 🚧🗣 Hope to see you all in our next seminar! 🏗 #ucf#civilengineers#geotech#engineer