2024-09-20-understanding-abi-1.md

title	date	layout	categories	tags	giscus_comments	authors
How to run LLM on a laptop?	2024-09-20	post	linux	world linux ABI	true	name url affiliations Linda Joe Thadeus https://lindathadeus.github.io name SuSE name NITK name url affiliations ChatGPT https://openai.com name OpenAI

How to run a LLM in a laptop?

{% highlight python %} Step 1: Get a laptop with >16 GB RAM and core i7 processor (GPU too is good, but I did not use it, It took 800% from cpu, 10% from Memory 4GB, for a HelloWorld) Step 2: Get Permission to LLAMA LLM usage from meta (Waiting Time 24 - 48 Hours, frankly unnecessary) Step 3: Install HuggingFace-hub Step 4: Download any LLAMA model variant from HF using huggingface-cli (7B) Step 5: Install ctransformers Step 6: Run the script (that you create) that opens the downloaded LLM file and use prompts on it (Check any HF LLAMA model for this) {% endhighlight %}

References

relevant post.

Did it work for you?

---

What do you think? Let me know your thoughts in the comments below.