Over the last month and a half, I have been learning AI.
AI is super interesting, and I have always been intimidated by the idea of learning it. But it is surprisingly approachable if the right people are teaching it!
Andrej Karpathy is an expert in AI. He has done research at Stanford and the University of Toronto and contributed significantly to the advancement of AI. He has worked at OpenAI and Tesla as an AI/ML engineer. He also has a YouTube series where he walks you through deploying simple Bigram AI models to the current industry standard Transformer models.
The spelled-out intro to neural networks and backpropagation: building micrograd
I recommend this series to anyone who has had experience with Python. The Pytorch library is used throughout the series, which I prefer over Google’s TensorFlow, as Pytorch does not hold your hand as much and forces you to start understanding the concepts behind AI. After going through this series, I started to grasp the concepts of AI (even without any formal linear algebra or calculus education).
I learn mainly from trying things out for myself, so I enjoy using the “Blackbox” method for understanding complex concepts in AI and competitive programming. Blackboxing is the practice of understanding the output based on the input but only sometimes fully understanding how it came to the production. You might think this is an unproductive practice, but I completely stand by it. Blackboxing allows me to work with concepts that might take me months to fully grasp. Whether I Blackbox the idea or not, it will still take me months to fully understand it, so why not put it to use in the meantime?
<aside> ⚠️ I recognize there are ethical implications when using AI to replicate someone else's work but, I am not good enough to make anything half decent so… (as you’ll probably see)
</aside>
I was instantly interested in seeing whether I could create an AI to generate lyrics, so I modified the Transformer model and trained it off MF DOOM’s entire discography.
Here were the results:
could twith these stramming again
Something ghalls, them things doesnt from a key
And once of the Mind like King Cobie, Cmilver did him
Fuckers up acting like em then t you know and tell me him you?
Theyre want to with the touch of highs like long disard his pich his one stip me?
But along through em like through pash the trapting protic
When the one two dream, this whats the hone thing it my bass bust