I have a strange surplus and lack of knowledge when it comes to deep learning. I have made some small models, studied backpropagation math, and played with my fair share of LLMs. I even had the opportunity to work with LLM applications professionally. However, there is plenty for me to learn still and I feel as if there are large gaps in what I know. So what better way to fix this than working on small models? I'll attempt to evaluate, improve, and explore deep learning and ML methods in this repo. This is going to be fun!
Now for some clarity of purpose for the project. I want to make what I am going to call "lessons", despite the fact that I am teaching no one but myself. I will be clearly writing out either my thought process or descriptions of what I know. Now that I am writing this I think it would also be good to add sections to each lesson that add questions for future work sessions. This will be an excellent jumping off point.
As for tools. I am going to use pytorch for the majority of the time. There is no accounting for the future but it is what I use in my current job. I will probably switch to jupyter lab at some point although at the time of writing I prefer using VS code with python. Force of habit, but I might as well switch to the standard. Another thing worth mentioning. This repo is focused on my learning. Similar to a calculator, that lets you skip some menial tasks in order to speed up the process, I will use AI assistance to write code. I find that they are useful tools in helping me find paths I didn't think about or as a better manual page. The goal here is for me to learn math and process. To me, worrying about details of [[How vs Why and What |HOW]] to do x thing I know is possible with pytorch is not the point... yet. There is something to be said for the temptation for an AI model to replace thought however. So now that I am writing this out I think I should add a restriction of some kind. Here is what I'll do, every time I want to ask it a question I have to give myself 5 minutes of time to either think or read. No instant answers here, I'm looking to become smarter.
One more note concerning tools, and this is to you who may read this apart from myself. I am writing these notes in obsidian so there may be some words surrounded by square braces. These are obsidian links and I will not be including everything I might link to. Some of the times I might remove the links so that this repo is more clear. I am doing a strange mix of writing for myself and publicly after all. Never tried this before. Chances though I won't use the feature often as the whole point of this project is to write what I know. Self teaching requires thinking and writing forces the activity. So if I don't write something out chances are I am being lazy.