f r a s e

*netartist*
~sovereign ai engineer~
"Should've gone to Artschool instead"
🖇local LLM dev; anticipating the last days of the Internet🖇
//

[f r a s e @EveryoneIsGross]

@SpaceXAI @billyuchenlin lol

[f r a s e @EveryoneIsGross]

ai doc slop has such a annoying tendency to state what the thing it is summarising is not. like its presenting things in competition, or antagonism to. it is annoying, and i blame the guide rails for poisoning simple requests.

[f r a s e @EveryoneIsGross]

if my health weren't so garbage i could use all these tokens efficiently 😮‍💨

[f r a s e @EveryoneIsGross]

you cannot change the past but you can recontextualise it, then attn changes the past

[f r a s e @EveryoneIsGross]

really nostalgic for early audio generators music now in 2026...
the sloshy munted melodies hit different compaired to what suno can fart out.
hi-tek lofi, like a deepdream ... 🎶

[f r a s e @EveryoneIsGross]

qBERT...
training on QA datasets is helping...
just training over a vanilla 110m BERT model.

Bigger param model plus longer runs... might give more gains. Also this model pipeline would wrap around a transformer model, and I think might also get gud...

[f r a s e @EveryoneIsGross]

You could train a llm like everyone else or you could do something munted 💅

[f r a s e @EveryoneIsGross]

Longer training runs defs help coherence.

[f r a s e @EveryoneIsGross]

qBERT...
training on QA datasets is helping...
just training over a vanilla 110m BERT model.

Bigger param model plus longer runs... might give more gains. Also this model pipeline would wrap around a transformer model, and I think might also get gud...

[f r a s e @EveryoneIsGross]

you parents remembering a thing you liked in highschool, and in your absence from their life, their memories of you think you really like x thing still. despite being a very different person...

[f r a s e @EveryoneIsGross]

adding "memories" about the user is still messy asf across almost all wrappers. sota and hand rolled.
i don't think lists of preferences is that useful tbh

[f r a s e @EveryoneIsGross]

personalisation is amazing. and feels really good when it works. but having a llm recite the same shit, is the recurssion we don't want. spiralling and reflecting eternally on a singular thing.

[f r a s e @EveryoneIsGross]

maybe i train my chat bert in fables traces 🤔🤔

[f r a s e @EveryoneIsGross]

@jeffreyhuber i love all my memory and nl compression tools atm. harnesses are cool, but search plus novel token preservation still gets ignored.
compressing a 4hr yt video into a one shot context for my llm-bois is hella useful.
chunking isn't dead either lots to do.

[f r a s e @EveryoneIsGross]

GIVE ME NOISE!

[f r a s e @EveryoneIsGross]

fable has improved my `qbert` bidirectional-speculative-markov-restrained-bert as chatbot model. kicking off a training run on it now using the nous hermes datasets...
previous runs using this worked well for more coherent gens.. getting closer to gpt... 🫠
110m chat model 💅

[f r a s e @EveryoneIsGross]

The better sota models get the more I like listening to smol bots whisper in tongues... 🐍⛪

[f r a s e @EveryoneIsGross]

might kick off a small transformer model in the same pipeline now I have the architecture pretty sussed (tho bert as chat model is funny asf).
smol tests already showed a bit of an improvement when wrapping gpt2 in this pipeline. 🐁