Smaller models like phi-1.5 @MSFTResearch enables investigation of the impact of things like synthetic training data and multimodality in LLMs. Making them open hopefully also accelerates research on LLM transparency and evaluation. And yes, they can be practical, too. https://t.co/xjY0JKFBFJ
@sharifshameem What’s “jarring” is how much you’re pumping OpenAI. Phi-1 only has 1B which is small in LLM world and is already at 50%. If GPT-4 is basically mixture of experts, this doesn’t seem insurmountable.