@apatwa7 todayβs gains feel driven more by context windows + tooling than true learning. llms still donβt update their knowledge over time, and pretraining data is largely static and already near saturation. feels like the next leap is in continual learning, not just scaling. wdyt?
iβm becoming increasingly convinced that scaling frontier models wonβt be what gives us economically productive agents. even if we get models that can reason over week-long horizons, i doubt that alone will unlock major breakthroughs in research or economic productivity
just read this survey on ML techniques for microprocessor verification. one big blocker is the upfront need for domain expertise + setup cost. curious if any data-labeling companies are focusing on curating datasets in this space.
https://t.co/DcSLhoo88z
i wonder if wispr flow will try to move into the meeting-transcription space. is manually-triggered voice-to-text dictation a big enough vertical on its own?
Sharing a project where we explored how to make large Mixture-of-Experts models more efficient on memory-constrained GPUs by predicting expert activations with a lightweight transformer, improving cache hit rates significantly over heuristic approaches.
https://t.co/KbKVvw7w9Z