this is definitely going to be less coherent of a post, it's more of a journal of what i've been doing, rather than something to share
also technically this is a part 2, i started learning about a year back but gave up
for the past couple months i've been trying to get back into figuring out ai, sparked by being unable to find a good model for my hardware (a GTX 1060)
i've kept myself as motivated as i could, though it has been hard
my first experiment of all of this was an attempt at speculative decoding at an architecture level, which wasn't a complete failure but it kept converging on apologizing over and over
and then i tried doing harness development, started using models closer to SOTA
and, i took a few shots at LSTMs, i still am taking shots at them but god they're expensive, probably the most i've learned so far was from that
i ended up biting the bullet recently and set myself up a Jupyter notebook based on nanoGPT for messing with them, maybe i'll get somewhere
this really was just a big wall of yap, wasn't it
today i learned that people read these too