We've added a new system prompts release notes section to our docs. We're going to log changes we make to the default system prompts on Claude dot ai and our mobile apps. (The system prompt does not affect the API.)
I am not a lawyer, and this is not legal advice, but if you are operating a business that knows it may be committing antitrust violations, don't put a 😀in a marketing deck after writing the words "That could be considered price collusion, and it’s illegal"
This is not to defend Runway or undercut @404mediaco's great reporting at all, but *all* of the big models were trained on data scraped from YouTube and other sites without permission. All of them! This was not considered a secret until a few years ago, when the companies lawyered up.
I wish AI companies talked about this more candidly, and tried to persuade people it was both legal and ethical to do. But they probably won't, so I guess we have to hear about it from leaks and lawsuits. https://t.co/MAhX7g530E
It just blows my mind how much companies keep self-owning because they think they risk anything being ‘too slow’ in adopting AI. All the fast AI implementations have been bad. Google answers. MS Recall. This Figma AI thing.
Take your time to do it right the first time.
Figma AI looks rather heavily trained on existing apps.
This is a "weather app" using the new Make Designs feature and the results are basically Apple's Weather app (left). Tried three times, same results.
It’s weird that Google gets raked over the coals, when OpenAI often gets a pass for the same phenomenon. I’m not sure why. Because Google is a trusted source? Because fewer people use Bing or GPT4 with retrieval? Or is Gemini that much more prone to hallucinations?
Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.
For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.
When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.
Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:
Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.
Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.
This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.
@neoliberalhell I understand the reaction. But anecdotally, in my community in the Bay Area there are more people who have died from k than opioids. And there are others who have developed liver and urinary problems from heavy k but are alive. I know it sounds absurd but it's possible.
As mentioned in my All-In Talk on regulatory capture, the companies that have raised $Bs in AI are using that money to fund a regulatory push that would "pull up the ladder" on competition that is fast on their heels. Pay attention. Read below from @AndrewYNg. #stopAIcapture
My greatest fear for the future of AI is if overhyped risks (such as human extinction) lets tech lobbyists get enacted stifling regulations that suppress open-source and crush innovation.
Read more in our Halloween special issue of the Batch: https://t.co/CNqP2TYBLM
@social_geek7 Friendly advice: Your profile pic and cover photo reinforce harmful stereotypes in AI. The humanoid outstretched hand (which is also visually white) and electronic brain don't accurately describe AI. This site is an excellent resource! https://t.co/8eMnWLMKEg