These might be the best guides on:
- Prompt Engineering
- Building Agents
- AI integration strategies
- Working with AI
So much free value by OpenAI, Anthropic, and Google.
All the links below.
Everyone's expecting a reborn Siri at WWDC today. Well, Apple already published a paper on it that disclosed way more details than what we expect from Apple. It's called "Ferret-UI", a multimodal vision-language model that understands icons, widgets, and text on iOS mobile screen, and reasons about their spatial relationships and functional meanings.
Example questions you can ask Ferret-UI:
- Provide a summary of this screenshot;
- For the interactive element [bbox], provide a phrase that best describes its functionality;
- Predict whether the UI element [bbox] is tappable.
With strong screen understanding, it's not hard to add action output to the model and make it a full-fledged on-device assistant.
The paper even talks about details of the dataset and iOS UI benchmark construction. Extraordinary openness from Apple! They are truly redefining their AI research branch.
The paper was silently released with little PR fanfare in April. You still have enough time to warm up before WWDC: https://t.co/gVv9vK5oJz
Marketing speak to terms you already know:
semantic index -> embeddings
app intents -> function calling
on device language model -> 3B fine tuned LLM w/ included LoRA adapters
on device image model -> diffusion model w/ included LoRA adapters
orchestration -> Siri
Neural Engine -> Apple's GPU
It takes 2GB of RAM and 60GB of storage to simulate ~300 neurons.
If we scale that linearly (eew) to other organisms:
Fruit Fly (~100K neurons):
0.7 TB of RAM
20 TB of storage
Mouse (~75M neurons):
500 TB of RAM
14,900 TB of storage
Human (~86B neurons):
570,000 TB of RAM
17 million TB of storage
So Apple has introduced a new system called “Private Cloud Compute” that allows your phone to offload complex (typically AI) tasks to specialized secure devices in the cloud. I’m still trying to work out what I think about this. So here’s a thread. 1/
✨ AWS Roadmap ✨
We just published a new interactive guide for learning AWS. We are also planning on running an experiment with a task based roadmap for this one, so stay tuned for that as well.
Check it out 👇
https://t.co/KG5UgzOj0H
Rome was not built in a day, but grep was (sort of) 😎
The origin story behind the creation of grep utility is fascinating.
The co-creator of the UNIX operating system, Ken Thompson, developed grep 'overnight'.
Actually, he had a personal tool for searching for text in files.
His department head Doug McIlroy came to him and said, "You know it would be really great if we could look for things in files".
"I'll think about it overnight", said Thompson.
He went back home and modified the code in his tool to fix bugs. Took him an hour at most.
The next day, he presented it to McIlroy and he exclaimed,
"This is exactly what I wanted"
And the rest is history.
If you are wondering why the utility is called grep and not search, there is perfectly good logic behind it 👇
1/ Blind to Bias
Be mindful of potential biases in AI-generated content.
Request ChatGPT to identify any biases and provide a checklist you can use to critically evaluate the output.
The iPhone 14 Pro's BIGGEST issue with its 48MP camera is that it forces you to use ProRaw which takes up 50-80MB of storage PER PHOTO.
Thankfully, @falcon283 FIXED this issue with an easy-to-use Shortcut!
Install Link: https://t.co/KidoNbsij4
Let me explain how it works! 🧵
If you want to charge your #iPhone12 with MagSafe, Apple recommends using certain AC adapters, and they don't need to come from #Apple, despite what you may have heard. Here's what's really going on. https://t.co/niccv6AkX5
@printrbot Having a basic printer design that can be iteratively improved without replacing the whole printer would be great. I like prusa approach that the printer parts can be upgraded to next version without replacing the whole thing
Over the course of your career, you'll encounter people who use strategic sycophancy in order to level themselves up, rather than, say, doing great work.
Some will hit their inevitable ceiling early, but others will actually continue to thrive.
What if security "shifting left" really turns out to be software developers "shifting right" to also own their own security?
Why do we assume that after-market add-on products deployed by separate organizational silos are the best way to improve security?