I'm launching a free data engineering boot camp on YouTube on 11/15!
It will be a new data engineering video every day from 11/15 until 12/31! I'm excited to share my hard work with you in a way that is accessible!
The launch video is here:
https://t.co/djB9bmxNDR
Please share this with your friends! I believe in each and every one of you!
There are so many different roles in the data domain. Deciding which role is best for you depends on your preferences.
One dimension you can look at these roles is what percent is spent building infrastructure versus digging into the data to find issues.
I personally love #dataengineering because Iโm a builder at heart but love diagnosing things with SQL.
Iโve seen myself gravitating both directions towards #softwareengineering and #analyticsengineering depending on what Iโm currently frustrated with.
Do you like building stuff? Do you like finding root causes of business issues? This core differentiator between these roles might help you understand where in the data value you chain you want to sit!
We're big tennis fans here. For all the Rafa fans, here's Rafael Nadal's heartfelt retirement speech translated into English.
What a legend, @RafaelNadal ๐พ๐
The original gig is in Spanish:
๐๐ถ๐ด ๐๐ฎ๐๐ฎ ๐ฒ ๐๐ป๐๐ฒ๐น๐ถ๐ด๐ฒ๐ป๐ฐ๐ถ๐ฎ ๐๐ฟ๐๐ถ๐ณ๐ถ๐ฐ๐ถ๐ฎ๐น
https://t.co/S3wimCPl5y
Crawl4AI is an open-source web-crawling and data-extraction tool built to integrate well with LLM and AI applications.
It has a comprehensive list of features ranging from simultaneous URL crawling to advanced extraction strategies based on LLMs.
And most importantly, choose the model based on what you need.
๐ o1 is perfect for complex algorithms, coding, and planning.
๐ GPT-4 is better for quick answers and working with text and images.
โ ๏ธ ๐๐ณ ๐๐ผ๐'๐ฟ๐ฒ ๐๐๐ถ๐ป๐ด ๐ผ๐ญ ๐น๐ถ๐ธ๐ฒ ๐๐ฃ๐ง-๐ฐ, you're wasting its potential.
Let me tell you how to fix that...
๐๐ฒ๐ฒ๐ฝ ๐ถ๐ป ๐บ๐ถ๐ป๐ฑ:
โ Simplify. Short, direct prompts work best.
โ Don't use "chain-of-thought". o1 already does the reasoning without being told.
โ Limit context in RAG. Less is more.
๐ If you have complex challenges that require more analysis, try the o1 model and let me know how it goes.
And if you have more info about o1... drop the post in the comments, and we can make a thematic megathread ๐๐
โ ๏ธ But beware, o1 isn't for everything.
If you're looking to generate or edit text, you won't notice much difference compared to GPT-4o. However, if you're dealing with mathematical or analytical problemsโฆ that's where o1 really shines. ๐
Haha we've all been there. I stumbled by this tweet earlier today and tried to write a little utility that auto-generates git commit message based on the git diff of staged changes. Gist:
https://t.co/1SbQsHSNwK
So just typing `gcm` (short for git commit -m) auto-generates a one-line commit message, lets you to accept, edit, regenerate or cancel. Might be fun to experiment with.
Uses the excellent `llm` CLI util from @simonw
https://t.co/LnHeCSfiHc