wrote an essay on how LLM inference works. I spent about a week trying to build a clear understanding, and this is my distilled version of it. It covers the full inference journey: embeddings, attention, KV caching, quantization, and more.
I am no expert, but if you are looking for a starting point, this might help. Give it a read.