This is a baby GPT with two tokens 0/1 and context length of 3, viewing it as a finite state markov chain. It was trained on the sequence "111101111011110" for 50 iterations. The parameters and the architecture of the Transformer modifies the probabilities on the arrows.
E.g. we can see that:
- state 101 deterministically transitions to 011 in the training data, so the probability of that transition becomes higher (79%). Not near 100% because we only did 50 steps of optimization.
- state 111 goes to 111 and 110 with 50% probability each, which the model almost learns (45%, 55%).
- states like 000 are never encountered during training, but have relatively sharp transition probabilities, e.g. 73% of going to 001. This is a consequence of inductive biases in the Transformer. One might imagine wanting this to be 50%, except in a real deployment almost every input sequence is unique, not present in the training data verbatim.
Not really sure where I was going with this :D, I think it's interesting to train/study tiny GPTs because it becomes tractable to visualize and get an intuitive sense of the entire dynamical system. Play with here: https://t.co/8jdceMLpqy
@intrepid_user@MonetSupply Actually banks create money out of nowhere. When they create a loan (asset) they pay it into a current account at their own bank hence creating a liability. That way they can just create money expanding the balance sheet on both sides. When the loan is repaid money is destroyed.
@TaschaLabs How’d you identify these groups?? Really curious, been writing a macro paper on crypto adoption and have struggled to find good sources on user groups. This looks awesome!!
@ribbonfinance
add eth as a currency options in the portfolio summary. What do you guys think? Would love to easily see how much yield I’ve made in eth cause its impressive!! Awesome product btw keep it up
Learning more about ZK rollups & why they matter. Recommend these to get started:
> 101 on computational integrity & overview of rollups @EliBenSasson: https://t.co/jHKL6a8uGI
> ZK-Rollups: https://t.co/an0esJ886a
> 101 on data availability @musalbas:
https://t.co/ifDwhdAZjU
@scott_lew_is Have to agree with this. This is the reason why Tether wmi. Oracles are no where near able to be included in smart contracts imo. The beauty of Dai was it’s reliance on smart contracts to successfully soft peg a currency even when assets such as eth plummeted. Huge mistake!
Interesting how crypto communities are congregating on Discord, which has traditionally been used for gaming. The intersection between gaming and crypto will be huge. (Reminder: it's also intersecting with art, finance, identity, governance, etc...Crypto is eating the world.)