I made a creamy vegetable soup today but my mistake was to add potato and cauliflower in it too. I thought I was so smart, making a hearty bowl of soup but was it the most custardy type Cerelac tasting potato dish. Would not recommend. Gotta make new soup recipe to redeem myself.
@tweetsbyjatin@shepuchibhaji Oh but it's not just that is it? She has to be locked away in a room for 5 days just because she is believed to be impure. She will be treated as a shadow in her own house and u don't get it because u have never lived it. Good for u
New update, Had to optimize my tokenizer because it was hella slow while training, and I was not going to wait 20+ hours for 350 million token to be extracted and saved to disk. Nuuh. Did basic heap and linked list switch up in the algorithm. https://t.co/ERiLG4kyTe real nice doc
tested the new tokenizer on a basic 50mb slice from tinystories dataset and after the round trip was succesfull, tried to train it on approx 2gb dataset on kaggle, hit some issues, used parallel workers and chunks for training to not hit memory limit on kaggle. Done and dusted