2024 llama.cpp 7840u: Difference between revisions

Revision as of 14:38, 13 April 2024

@@ Line 1: / Line 1: @@
-I briefly had a Macbook M3 Max with 64GB. It was pretty good at running local LLMs, but couldn't stand the ergonomics and not being able to run Linux, so returned it.
+{{SH_Triple|Lata Pada|Source|http://en.wikipedia.org/wiki/Lata_Pada}}
+{{SH_Add|5520|20100710233945|User:David Mason}}
-I picked up a Thinkpad P16s with an AMD 7840 to give Linux hardware a chance to catch up with Apple silicon. It's an amazing computer for the price, and can run LLMs. Here's how I set up llama.cpp to use ROCm.
-Install ROCm, set an env variable for the 780m: <code>export HSA_OVERRIDE_GFX_VERSION=11.0.0</code>
-clone llama.cpp and compile it:
-<code>make -j16 LLAMA_HIPBLAS=1 LLAMA_HIP_UMA=1 AMDGPU_TARGETS=gfx1030 DLLAMA_HIP_UMA=
-ON</code>
-run it like this:
-<code>./main -m /home/vid/jan/models/mistral-ins-7b-q4/mistral-7b-instruct-v0.2.Q4_K_M
-.gguf -p "example code for a lit Web Component that reverses a string" -n 50 -e
--ngl 16  -n -1</code>
-{{Blikied|April 13 12, 2024}}