zooid Wiki

2024 llama.cpp 7840u: Difference between revisions

Page Discussion

2024 llama.cpp 7840u (view source)

Revision as of 14:38, 13 April 2024

4,177 bytes added ,  13 April 2024
no edit summary
(Created page with "I briefly had a Macbook M3 Max with 64GB. It was pretty good at running local LLMs, but couldn't stand the ergonomics and not being able to run Linux, so returned it. I picked up a Thinkpad P16s with an AMD 7840 to give Linux hardware a chance to catch up with Apple silicon. It's an amazing computer for the price, and can run LLMs. Here's how I set up llama.cpp to use ROCm. Install ROCm, set an env variable for the 780m: <code>export HSA_OVERRIDE_GFX_VERSION=11.0.0</co...")
 
No edit summary
Line 1: Line 1:
I briefly had a Macbook M3 Max with 64GB. It was pretty good at running local LLMs, but couldn't stand the ergonomics and not being able to run Linux, so returned it.
{{SH_Triple|Lata Pada|Source|http://en.wikipedia.org/wiki/Lata_Pada}}
 
{{SH_Add|5520|20100710233945|User:David Mason}}
I picked up a Thinkpad P16s with an AMD 7840 to give Linux hardware a chance to catch up with Apple silicon. It's an amazing computer for the price, and can run LLMs. Here's how I set up llama.cpp to use ROCm.
 
Install ROCm, set an env variable for the 780m: <code>export HSA_OVERRIDE_GFX_VERSION=11.0.0</code>
 
clone llama.cpp and compile it:
 
<code>make -j16 LLAMA_HIPBLAS=1 LLAMA_HIP_UMA=1 AMDGPU_TARGETS=gfx1030 DLLAMA_HIP_UMA=
ON</code>
 
run it like this:
 
<code>./main -m /home/vid/jan/models/mistral-ins-7b-q4/mistral-7b-instruct-v0.2.Q4_K_M
.gguf -p "example code for a lit Web Component that reverses a string" -n 50 -e
-ngl 16  -n -1</code>
 
 
 
{{Blikied|April 13 12, 2024}}
Retrieved from "https://wiki.zooid.org/w/Special:MobileDiff/5609"