{"id":441,"date":"2026-03-19T03:48:18","date_gmt":"2026-03-19T03:48:18","guid":{"rendered":"https:\/\/nocoffei.com\/?p=441"},"modified":"2026-03-19T03:48:18","modified_gmt":"2026-03-19T03:48:18","slug":"amds-npu-working-on-fedora-linux","status":"publish","type":"post","link":"https:\/\/nocoffei.com\/?p=441","title":{"rendered":"AMD&#8217;s NPU working on Fedora Linux"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Fedora does not currently have prebuilt packages for the brand-spanking-new (like, within the last week) completed kernel + userspace NPU stack for Linux. It revolves around the xdna-driver for the kernel driver, XRT for the userspace libraries, FastFlowLM for the userspace program that does the actual running of LLMs, and finally Lemonade for the orchestrator and frontend which can serve ollama and OpenAI compatible endpoints for every AMD backend out there. (This also makes running models on the GPU with a ROCm backend <em>very<\/em> easy, a super welcome move from AMDs\u2026 Troubled previous years.)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As far as I know, I&#8217;m the first person to get everything built from source, all integrated together and working on Fedora. I&#8217;m running this on my 128GB Framework Desktop, which, amusingly, has appreciated $700 and counting since I got my first-batch preorder in August.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My feelings about LLMs are ambivalent. Personally, I suspect I won&#8217;t use this very much; it&#8217;s more useful for the researchers who I&#8217;m serving LLMs to for their NLP analysis. Hopefully, the extra compute will speed up parallel processing on their datasets on top of the GPU working as hard as it can.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, I wanted this all set up and working because the NPU is a <em>fascinating<\/em> piece of computer architecture, and I&#8217;m hoping to do some very silly things with it soon.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Considering how brand new all of this is, I&#8217;m mostly posting this for posterity, as well as hopefully a guide someone smarter than me can use to get everything packaged up with a nice bow tie for end users. I&#8217;ll update the post when that day arrives; until then, good luck, and have fun! It shouldn&#8217;t take more than half an hour now that I&#8217;ve laid out all the steps for you.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>git clone --recursive https:\/\/github.com\/amd\/xdna-driver.git\ngit clone --recursive https:\/\/github.com\/FastFlowLM\/FastFlowLM.git\n\nOpen this file:\nxdna-driver\/xrt\/src\/runtime_src\/tools\/scripgs\/xrtdeps.sh\nFind the FD_LIST variable. Install all of those packages. DO NOT execute the script, it doesn't work.\n\nif you need rocm\u2026 There's a collision between the OpenCL package it needs and the one XRT needs.. I just gave up and removed it.\n\npackage OpenCL-ICD-Loader-3.0.6-4.20241023git5907ac1.fc43.x86_64 from fedora conflicts with ocl-icd provided by ocl-icd-2.3.4-1.fc43.x86_64 from fedora\n\nBuild the kernel driver:\ncd xdna-driver\/build\n.\/build.sh -release -j 32\n\nBuild the userspace libs:\ncd ..\/xrt\/build\/\n.\/build.sh -npu -opt -noctest -j 32\n\nThe following will install all the userspace libs you just built, as well as the kernel driver you built a second ago:\nsudo dnf install Release\/*.rpm ..\/..\/build\/Release\/xrt_plugin.2.23.0_43-x86_64-amdxdna.rpm\n\nGet the dependencies for FastFlowLM:\n(Note that you can also use ffmpeg-devel with RPMFusion enabled due to Fedora's silly patent issues. Also, you should probably install Rust and Cargo through Rustup instead.)\nsudo dnf install ninja-build ffmpeg-free-devel fftw-devel rust cargo\n\nConfigure, build, and install as described in the README: https:\/\/github.com\/FastFlowLM\/FastFlowLM?tab=readme-ov-file\n\nSet memory limits for the NPU device:\n\nsudo tee \/etc\/security\/limits.d\/99-amdxdna.conf > \/dev\/null &lt;&lt; 'EOF'\n\nsoft memlock unlimited\n\nhard memlock unlimited\nEOF\n\nMake sure everything can see the XRT libraries:\n\nsudo tee \/etc\/ld.so.conf.d\/flm.conf > \/dev\/null &lt;&lt; 'EOF'\n\/opt\/fastflowlm\/lib\n\/opt\/xilinx\/xrt\/lib64\nEOF\n\nsudo ldconfig\n\nReboot\n\nRun this to make sure everything works:\n\nflm validate\n\nInstall lemonade per here: https:\/\/lemonade-server.ai\/install_options.html#linux\nLemonade should pick up on the presence of flm and make NPU models available to download and run. Enjoy!<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A week-one guide to a bleeding edge stack.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-441","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts\/441","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=441"}],"version-history":[{"count":1,"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts\/441\/revisions"}],"predecessor-version":[{"id":442,"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts\/441\/revisions\/442"}],"wp:attachment":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=441"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=441"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=441"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}