{"id":342,"date":"2024-12-29T06:02:55","date_gmt":"2024-12-29T06:02:55","guid":{"rendered":"https:\/\/nocoffei.com\/?p=342"},"modified":"2024-12-30T04:40:25","modified_gmt":"2024-12-30T04:40:25","slug":"a-new-framework-13-and-fedora-kinoite-bazzite","status":"publish","type":"post","link":"https:\/\/nocoffei.com\/?p=342","title":{"rendered":"A New Framework 13 and Fedora Kinoite\/Bazzite"},"content":{"rendered":"\n<p><strong>This post is being actively updated. Check back every day or two! I&#8217;ll keep updating it until I run out of interesting things to say and it becomes &#8220;yet another computer&#8221;.<\/strong><\/p>\n\n\n\n<p>I purchased myself a nice used Framework 13 AMD for Christmas, after my poor old ThinkPad had an unfortunate run-in with severe hardware failure.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>That&#8217;s a fun story on its own, perhaps I should write it up as well.<\/p>\n<\/blockquote>\n\n\n\n<p>This is more of a &#8220;journal&#8221;-y post, wherein I&#8217;ll write my observations as they come, being that on top of the new hardware I was also convinced to try an entirely new style of OS, the immutable Fedora Kinoite (or Silverblue). The previous owner, who I&#8217;m good friends with, used Kinoite during his time with it, and was a fan:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>[Luke, 12\/26\/2024, 12:33PM]<\/p>\n\n\n\n<p>Upside is I never had my system break and I could leave it alone for months and open it and it just worked<\/p>\n\n\n\n<p>It&#8217;s the most stable distro I&#8217;ve used I&#8217;ll give it that<\/p>\n\n\n\n<p>Not to say everything was smooth and easy but once it was working it stayed that way<\/p>\n<\/blockquote>\n\n\n\n<p>As someone who spent 2024 on a horrible Windows install due to abhorrent bugginess with late-stage KDE 5 and Arch&#8217;s ongoing game of Russian Roulette every time you install a package (will your kernel panic after <code>pacman -Syu<\/code>, as is required every time you install something? Who knows!) the qualities of an immutable OS are extremely attractive: rock-solid stability and the ability to go back if AMDs graphics engineers screw up again.<\/p>\n\n\n\n<p>It was worth a shot. So how well does all this work in 2024? And at the same time, how does the Framework feel and behave?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Comments on hardware<\/h2>\n\n\n\n<p>I&#8217;ve been using the hardware for a day now, and some things immediately come to mind.<\/p>\n\n\n\n<p>This is a Ryzen 7 7840U, with the integrated Radeon 780M graphics. (More horsepower than a Steam Deck&#8211; juicy!) 32GB of RAM is installed, and a 2TB SSD is also included.<\/p>\n\n\n\n<p>This thing feels SNAPPY and not just due to the fact that the Windows install I spent most of the year on took literal seconds to Alt+Tab between windows. It&#8217;s able to open the most bloated of Electron apps (Signal) in mere seconds, graphical effects are smooth and gorgeous, browser pages load instantly&#8230; It&#8217;s hard to say how much of this is due to the new CPU and how much is due to KDE Plasma 6&#8217;s innovations like <a href=\"https:\/\/pointieststick.com\/2024\/05\/25\/this-week-in-kde-triple-buffering-and-other-sources-of-amazingness\/\">triple buffering<\/a>, but the end result is a machine that leaves any other I&#8217;ve used in the dust!<\/p>\n\n\n\n<p>The 1st-generation matte screen is pretty and easy to look at, while power consumption isn&#8217;t off the charts like my last 4K 14&#8243; display. Of course, it&#8217;s comfortably viewed at 150% scale, so all the usual misery with fractional scaling on Linux is present. Somehow, in 2024, this is still broken, and it&#8217;s so bad that Framework made a new panel with an <a href=\"https:\/\/frame.work\/blog\/framework-laptop-13-deep-dive-creating-a-custom-high-resolution\">express design goal to work around this mess in hardware<\/a>.<\/p>\n\n\n\n<p>The input devices are a massive downgrade from my beloved ThinkPads, but I knew this would be the case no matter what I chose as <a href=\"https:\/\/www.notebookcheck.net\/Lenovo-ThinkPad-T14-G3-review-Business-laptop-is-better-with-AMD-Ryzen-Pro.657465.0.html\">Lenovo gave up on making good keyboards some years ago<\/a>. I&#8217;m coming from a ThinkPad T14 Gen 2 AMD, the last unit made before the changes. <\/p>\n\n\n\n<p>The keyboard is&#8230; Fine. I will need to find accommodations for the missing PgUp, PgDown, Home and End keys, which I&#8217;ve come to love and rely on. <\/p>\n\n\n\n<p>The trackpad is OK, about on par with the glass trackpad replacement I put in the T14, and there is <em>no TrackPoint \ud83d\ude41<\/em> I am going to miss this dearly, especially without a touchscreen. I will likely find myself using a wireless mouse much more often. It may, in fact, be a deal-breaker for the hardware in the long run. I&#8217;ve come to be totally dependent on it in the last 5 years of using ThinkPads.<\/p>\n\n\n\n<p>The Arch Wiki is, as always, incredibly helpful, and the <a href=\"https:\/\/wiki.archlinux.org\/title\/Framework_Laptop_13_(AMD_Ryzen_7040_Series)\">page for this laptop<\/a> pointed out two amazing facts:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Notebookcheck provides ICC color calibration profiles for this laptop (and many others), and<\/li>\n\n\n\n<li>Framework provides a DSP profile that can be applied in userspace.<\/li>\n<\/ul>\n\n\n\n<p>DSP has been making some waves with Marcans long-running work on making <a href=\"https:\/\/github.com\/AsahiLinux\/docs\/wiki\/SW:Speakers\">speakers sound amazing on Asahi Linux<\/a>. Although the Apple Silicon speakers are especially fancypants and require a new ground-up implementation, it seems there is still a tremendous deal you can do with basic DSP. It turns out installing this is as easy as installing the linked EasyEffects, setting it to run on boot in the preferences, and importing the profile. That&#8217;s it! It was so easy I was certain I&#8217;d missed a step somewhere. And boy does it work.<\/p>\n\n\n\n<p>These speakers are already probably the loudest of any laptop I&#8217;ve owned, and applying the DSP profile makes them sound even brighter and more powerful. You can very clearly hear the difference as I enable and disable the profile in the below snippet of Katamari on the Rocks:<\/p>\n\n\n\n<p><em>I&#8217;ll actually put it in whenever I get around to it \ud83d\ude1b<\/em><\/p>\n\n\n\n<p>I&#8217;ve never heard of a manufacturer caring this much about Linux support. Framework officially supports Ubuntu and Fedora, and has gone to great lengths to make sure that everything works on their hardware out of the box.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Fedora Kinoite and woe<\/h2>\n\n\n\n<p>This is the &#8220;journal entry&#8221; section of this post. It&#8217;s half me figuring out a new style of OS and half running headfirst into the usual breakage and half-working components of the Linux desktop as I have a tendency to do.<\/p>\n\n\n\n<p><strong>12\/26\/2024, 12:30PM:<\/strong><\/p>\n\n\n\n<p>The laptop arrives. I grab a flash drive and install Fedora Kinoite. <\/p>\n\n\n\n<p>&#8220;Hmm,&#8221; I say, &#8220;instead of transferring my files through a flash drive like usual, I bet I could network share this stuff somehow. And wouldn&#8217;t it be cool if I could even do that over Thunderbolt? Both of these computers have it.&#8221;<\/p>\n\n\n\n<p>Plugging a good USB-C cable into both ends opens up a new <code>thunderbolt0<\/code> network interface, but that&#8217;s where the idea falls apart. How do I send the files? Well, I suppose I could use KDEs built in file sharing and&#8230; <\/p>\n\n\n\n<p><strong>12\/26\/2024, 1:30PM:<\/strong><\/p>\n\n\n\n<p>KDEs file manager, Dolphin, has a nice built-in &#8220;Share&#8221; section of the properties tab of a folder. The idea is to quickly and easily network share anything you&#8217;d like.<\/p>\n\n\n\n<p>As far as I can tell, this does not work at all and there is no clear way to make it work.<\/p>\n\n\n\n<p>Both on Fedora Kinoite and Fedora Workstation (which I tested for comparison) there are two reboots worth of error messages, complaining that there are missing configurations needed to get Samba working. Fair enough, except the auto-configurations it offers to do for you don&#8217;t work on Kinoite and you have to configure them yourself as <a href=\"https:\/\/docs.fedoraproject.org\/en-US\/fedora-silverblue\/troubleshooting\/#_unable_to_add_user_to_group\">rpm-ostree does not treat user groups with respect<\/a>.<\/p>\n\n\n\n<p>Once you get past that, AND once you manually start the SMB service, as far as I can tell, what happens next is that you&#8217;re presented with an interface that <em>says<\/em> it works, it doesn&#8217;t, actually:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"649\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-1024x649.png\" alt=\"\" class=\"wp-image-343\" style=\"width:700px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-1024x649.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-300x190.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-768x487.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-1536x974.png 1536w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241227_044801-2048x1298.png 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>If you try to connect from localhost, it gives you an authentication dialog that will reject anything you put in. If you try to connect from another machine, the connection will be refused.<\/p>\n\n\n\n<p>I told an IT buddy about this and he said:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>[nocoffei, 12\/26\/2024 5:55PM]<\/p>\n\n\n\n<p>Smb on KDE is completely broken<\/p>\n\n\n\n<p>[Travis, 12\/26\/2024 5:55PM]<\/p>\n\n\n\n<p>I mean, that&#8217;s generally true<\/p>\n\n\n\n<p>I&#8217;ve never gotten an SMB share to actually work<\/p>\n<\/blockquote>\n\n\n\n<p>I give up and copy the files via a flash drive.<\/p>\n\n\n\n<p><strong>12\/26\/2024, 2:30PM<\/strong><\/p>\n\n\n\n<p>I run to the store and buy a Torx T5 screwdriver, as I didn&#8217;t think to bring one with me on travel for the holidays. I open the machine up. Indeed, the internals are very sexy. However, aside from wanting a few more Expansion Cards, this computer is already configured with the highest specs I want, so there&#8217;s not much to do here other than marvel at the engineering.<\/p>\n\n\n\n<p><strong>12\/26\/2024, 3:30PM<\/strong><\/p>\n\n\n\n<p>The usual config of a new KDE install continues. There are plenty of little nooks and crannies I vaguely remember I need to set up.<\/p>\n\n\n\n<p><strong>12\/26\/2024, 4:30PM<\/strong><\/p>\n\n\n\n<p>How do I install that ICC color profile? I spend an hour messing with the brand new Color Management page in KDE settings. The import button on this page is broken, I presume because it naively tries to copy to <code>\/usr\/share<\/code> which is read-only on Kinoite:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"598\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10-1024x598.png\" alt=\"\" class=\"wp-image-344\" style=\"width:600px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10-1024x598.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10-300x175.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10-768x449.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10-1536x898.png 1536w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-10.png 1682w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>It should be copying to <code>\/usr\/local\/share<\/code>, which also works for <code>colord<\/code>, but this isn&#8217;t what I want anyway. I want to apply the profile to the whole screen, which can easily be done from the Display section.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"598\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11-1024x598.png\" alt=\"\" class=\"wp-image-345\" style=\"width:600px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11-1024x598.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11-300x175.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11-768x449.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11-1536x898.png 1536w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-11.png 1682w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>I find out that all GTK4 apps have <a href=\"https:\/\/blogs.kde.org\/2024\/10\/09\/cursor-size-problems-in-wayland-explained\/#problems-you-might-still-encounter-today\">broken cursor sizes<\/a>. <a href=\"https:\/\/bugs.archlinux.org\/task\/76018\">Again<\/a>. The last time these guys broke this (on GTK3), they <a href=\"https:\/\/gitlab.gnome.org\/GNOME\/gtk\/-\/merge_requests\/5329\">marked as wontfix<\/a>, and then eventually fixed it about a year and a half later, forcing me to hold back GTK on an increasingly ancient version. <\/p>\n\n\n\n<p>Apparently the GTK folks are <a href=\"https:\/\/gitlab.gnome.org\/GNOME\/gtk\/-\/merge_requests\/7760#note_2302270\">not interested in a backport<\/a>, so we will have to wait until a new GTK release to fix this. Again. <\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"497\" height=\"882\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-13.png\" alt=\"\" class=\"wp-image-347\" style=\"width:230px;height:auto\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-13.png 497w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-13-169x300.png 169w\" sizes=\"auto, (max-width: 497px) 100vw, 497px\" \/><\/figure>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>At least this time it&#8217;s tolerable, I guess. The last breakage was actually unusable, making a cursor bigger than the width of my finger. <\/p>\n<\/blockquote>\n\n\n\n<p><strong>12\/26\/2024, 8PM:<\/strong><\/p>\n\n\n\n<p>It takes me well over two hours to figure out how the hell I&#8217;m supposed to get hardware video decoding working in Firefox. Fedora removes this for legal reasons at the cost of making its distro one extra installation more annoying for Workstation uses, and <strong>dramatically<\/strong> more irritating for its read-only filesystem variant.<\/p>\n\n\n\n<p>After hours of investigation, I find <a href=\"https:\/\/davejansen.com\/fedora-silverblue-hardware-accelerated-h264-video-playback\/\">this page<\/a> which describes a very aggressive manipulation of the install to yank out all the crap Fedora puts it there and replace it with normal-ass ffmpeg. After all that hassle, it <em>looks<\/em> like I got H264 decoding working in Firefox.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"557\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-12-1024x557.png\" alt=\"\" class=\"wp-image-346\" style=\"width:500px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-12-1024x557.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-12-300x163.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-12-768x418.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-12.png 1062w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><em>This can be found under about:support, but it only loads after you try to play something.<\/em><\/figcaption><\/figure>\n\n\n\n<p><code>vainfo<\/code> isn&#8217;t present, somehow, so I can&#8217;t verify this, and at this point I&#8217;m so irritated by the whole thing I want to try another approach.<\/p>\n\n\n\n<p><strong>12\/27\/2024, 5AM:<\/strong><\/p>\n\n\n\n<p>Frustrated by Silverblues lack of sane defaults for integrated graphics, I decide to try <a href=\"https:\/\/bazzite.gg\/\">Bazzite<\/a>, which is a downstream copy of Silverblue with lots and lots and lots of optimized defaults for gaming. I had vaguely seen it before, and was hot on the heels with my recent amazing experiences with the <a href=\"https:\/\/nocoffei.com\/?p=275\">Steam Deck and EmuDeck<\/a>. Community folks have really come together around the recent &#8220;open gaming handheld&#8221; market, and I suspected it would be the same way here!<\/p>\n\n\n\n<p>First, I read up on <code>rpm-ostree<\/code>s fundamentals a bit, since I didn&#8217;t <em>understand<\/em> what the system is doing. After reading some <a href=\"https:\/\/docs.fedoraproject.org\/en-US\/fedora-silverblue\/technical-information\/\">Fedora documentation<\/a> and this old, but still relevant <a href=\"https:\/\/github.com\/vmware\/photon\/wiki\/Photon-RPM-OSTree%3A-a-simple-guide\">mini-book<\/a> on the subject, I felt better about continuing.<\/p>\n\n\n\n<p>Anticipating an OS reinstall, I was <em>very<\/em> pleasantly surprised to find out that I could not only keep my current install, I could &#8220;snapshot&#8221; my current install (what they call a <a href=\"https:\/\/docs.fedoraproject.org\/en-US\/fedora-silverblue\/faq\/#_how_can_i_upgrade_my_system_to_the_next_major_version_for_instance_rawhide_or_an_upcoming_fedora_release_branch_while_keeping_my_current_deployment\">&#8220;pin&#8221;<\/a>) and simply rebase the ostree base onto Bazzite. This is an <a href=\"https:\/\/docs.bazzite.gg\/General\/Installation_Guide\/alternate-install-guide\/\">officially supported and documented installation method<\/a>!<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>The actual rebase command steps appear only once you&#8217;ve clicked through their &#8220;download&#8221; page. It&#8217;s a website trick that makes it look fancy, but harder to link to.<\/p>\n<\/blockquote>\n\n\n\n<p><strong>12\/27\/2024, 8AM:<\/strong><\/p>\n\n\n\n<p>We&#8217;re live, baby.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"611\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-14-1024x611.png\" alt=\"\" class=\"wp-image-348\" style=\"width:500px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-14-1024x611.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-14-300x179.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-14-768x458.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-14.png 1474w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Bazzite gives a ton of neat basics preinstalled that I haven&#8217;t fully explored yet. The only &#8220;breaking&#8221; changes are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Firefox is now a Flatpak, not a binary. Move your files and it works.<\/li>\n\n\n\n<li>Steam is now a binary, not a Flatpak. Move your files and it works&#8230; BUT it deleted the one game I had installed for some reason&#8230;?<\/li>\n<\/ul>\n\n\n\n<p>I spend the rest of my time at the computer today generally mucking around with more configs and setting up my computer the way I like it.<\/p>\n\n\n\n<p><strong>12\/28\/2024, 10AM<\/strong>:<\/p>\n\n\n\n<p>Marvel Rivals runs! Lowest settings at 720P and it&#8217;s a smooth 60FPS.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-1024x683.png\" alt=\"\" class=\"wp-image-351\" style=\"width:600px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-1024x683.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-300x200.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-768x512.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-1536x1024.png 1536w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/image-15-2048x1365.png 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><em>This game looks fun! Got MVP my second game, so I must be doing something right.<\/em><\/figcaption><\/figure>\n\n\n\n<p>Even got MangoHUD working so I can watch the GPU get hammered. It&#8217;d be nice to see the breakdown of power consumption between CPU and GPU like on the Steam Deck, but I haven&#8217;t figured that out yet.<\/p>\n\n\n\n<p><strong>12\/28\/2024, 5PM:<\/strong><\/p>\n\n\n\n<p>This thing has a pretty decent iGPU hooked up to it, clearly. I wonder, can I get an LLM running? It has 32GB of &#8220;V&#8221;RAM hooked up to it&#8230;<\/p>\n\n\n\n<p>I expect that it&#8217;ll just be a waste of my time, considering the state of AMDs ROCm platform, but apparently I&#8217;m underestimating my rabid peers in the self-hosting LLM community.<\/p>\n\n\n\n<details class=\"wp-block-details is-layout-flow wp-block-details-is-layout-flow\"><summary>LLM text output of llama3.2 3B running<\/summary>\n<pre class=\"wp-block-code\"><code>$ podman exec -it ollama ollama run llama3.2\npulling manifest \npulling dde5aa3fc5ff... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f 2.0 GB                         \npulling 966de95ca8a6... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f 1.4 KB                         \npulling fcc5a6bec9da... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f 7.7 KB                         \npulling a70ff7e570d9... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f 6.0 KB                         \npulling 56bb8bd477a5... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f   96 B                         \npulling 34bb5ab01051... 100% \u2595\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258f  561 B                         \nverifying sha256 digest \nwriting manifest \nsuccess \n&gt;&gt;&gt; what is your name?\nI don't have a personal name, but I'm an AI designed to assist and \ncommunicate with users. You can call me \"Assistant\" or \"AI\" if you prefer. \nSome people also refer to me as \"Chatbot\" or simply \"the AI.\" How can I \nhelp you today?\n\n&gt;&gt;&gt; what is your quest?\nI don't have a personal quest in the classical sense, but my purpose is to \nassist and provide value to users like you. My primary goals are:\n\n1. **Knowledge Sharing**: To provide accurate and up-to-date information \non a wide range of topics.\n2. **Conversation and Understanding**: To engage in natural-sounding \nconversations, ask questions, and understand user intent.\n3. **Learning and Improvement**: To learn from user interactions, improve \nmy language understanding, and refine my responses.\n\nIn a broader sense, my \"quest\" is to help people like you find answers, \ngain insights, and solve problems more efficiently. I'm here to provide \nassistance, answer questions, and offer suggestions whenever possible.\n\nWhat's your quest? Do you have a specific question or topic you'd like to \ndiscuss?\n\n&gt;&gt;&gt; what is the airspeed velocity of an unladen swallow?\nA classic reference to Monty Python's Flying Circus!\n\nThe airspeed velocity of an unladen swallow is a topic of much debate and \nhumor. In reality, there is no definitive answer, as it depends on various \nfactors such as:\n\n1. Species: Different species of swallows have varying wingbeat speeds.\n2. Condition: The bird's health, weight, and aerodynamic condition can \naffect its airspeed velocity.\n3. Altitude and wind: Weather conditions, altitude, and wind resistance \nall impact the bird's speed.\n\nHowever, for the sake of humor and entertainment, let's consider some \nestimates:\n\n* According to a 1991 study on swallow flight patterns, the average \ncruising speed of a barn swallow (Hirundo rustica) is around 35-45 km\/h \n(22-28 mph).\n* Another estimate suggests that a European swallow (Hirundo rustica \neuropaea) can reach speeds of up to 50-60 km\/h (31-37 mph).\n\nBut remember, these are just rough estimates and not exactly what the \nMonty Python sketch is asking for!\n\nTo answer your question in a more tongue-in-cheek manner: \"What do you \nmean? An African or European swallow?\"\n\nWould you like to discuss more Monty Python trivia or perhaps explore some \nfascinating facts about swallows?\n<\/code><\/pre>\n<\/details>\n\n\n\n<p>Here&#8217;s the magic command I use to one-click run ollama:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>podman run --pull newer --detach --security-opt label=type:container_runtime_t --replace --device \/dev\/kfd --device \/dev\/dri -v ollama:\/root\/.ollama -p 11434:11434 -e HSA_OVERRIDE_GFX_VERSION=11.0.0 --name ollama ollama\/ollama:rocm<\/code><\/pre>\n\n\n\n<p>The command is sourced from here. I have to add the <code>HSA_OVERRIDE<\/code> environment variable to force ROCm to use the unsupported iGPU.<\/p>\n\n\n\n<p>But&#8230; Not all is perfect. ollama is only detecting the &#8220;dedicated VRAM&#8221; which I allocated in the VBIOS to be 4GB, and offloading the rest to the CPU:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>time=2024-12-29T03:27:47.796Z level=INFO source=types.go:131 msg=\"inference compute\" id=0 library=rocm variant=\"\" compute=gfx1103 driver=0.0 name=1002:15bf total=\"4.0 GiB\" available=\"2.3 GiB\"\n...\nllm_load_tensors: offloading 18 repeating layers to GPU\nllm_load_tensors: offloaded 18\/29 layers to GPU\nllm_load_tensors:   CPU_Mapped model buffer size =  1918.35 MiB\nllm_load_tensors:        ROCm0 model buffer size =  1041.26 MiB<\/code><\/pre>\n\n\n\n<p>Can I fix that and get ollama to properly recognize the entirety of the unified memory available?<\/p>\n\n\n\n<p>Yes, there&#8217;s an open, unmerged pull request out for it, and there&#8217;s even an <em>unbelievably<\/em> <a href=\"https:\/\/blog.machinezoo.com\/Running_Ollama_on_AMD_iGPU\">convenient blog post<\/a> describing <em>exactly<\/em> how to compile it and use it!!<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"807\" src=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234-1024x807.png\" alt=\"\" class=\"wp-image-349\" style=\"width:600px\" srcset=\"https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234-1024x807.png 1024w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234-300x236.png 300w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234-768x605.png 768w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234-1536x1211.png 1536w, https:\/\/nocoffei.com\/wp-content\/uploads\/2024\/12\/Screenshot_20241228_220234.png 1658w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><em>Putting these Zen 4 cores to work&#8230;<\/em><\/figcaption><\/figure>\n\n\n\n<p>I had this <a href=\"https:\/\/github.com\/89luca89\/distrobox\/issues\/1640#issuecomment-2564619320\">really weird issue<\/a> when running the container that went away after removing <code>--pull=always<\/code> from the command. No clue why but it works. I also had to re-add the security permissions stuff from the first command, otherwise it fails to pass through. Here&#8217;s what that looks like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ podman run -d     --name ollama     --replace      --restart=always     --stop-signal=SIGKILL     -p 127.0.0.1:11434:11434     -v ollama:\/root\/.ollama     -e OLLAMA_MAX_LOADED_MODELS=1     -e OLLAMA_NUM_PARALLEL=1     --device \/dev\/dri     --device \/dev\/kfd     -e HSA_OVERRIDE_GFX_VERSION=11.0.0  --security-opt label=type:container_runtime_t   localhost\/ollama-gtt:latest<\/code><\/pre>\n\n\n\n<p>And we can see that it manages to allocate plenty of VRAM!<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>time=2024-12-29T05:27:45.938Z level=INFO source=types.go:124 msg=\"inference compute\" id=0 library=rocm variant=\"\" compute=gfx1103 driver=0.0 name=1002:15bf total=\"13.6 GiB\" available=\"13.4 GiB\"\n...\nllm_load_tensors: offloading 28 repeating layers to GPU\nllm_load_tensors: offloading non-repeating layers to GPU\nllm_load_tensors: offloaded 29\/29 layers to GPU\nllm_load_tensors:      ROCm0 buffer size =  1918.36 MiB\nllm_load_tensors:        CPU buffer size =   308.23 MiB\n<\/code><\/pre>\n\n\n\n<p>That&#8217;s still not quite all of my free RAM; I&#8217;ll need to drive that value up some if I want to start playing with some of the bigger models. Let&#8217;s ask something&#8230;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ podman exec -it ollama ollama run llama3.2\n&gt;&gt;&gt; Show me what you got!<\/code><\/pre>\n\n\n\n<p>The screen turns black for a moment, then comes up again, and the container logs show&#8230;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>HW Exception by GPU node-1 (Agent handle: 0x7fa6e50b64f0) reason :GPU Hang<\/code><\/pre>\n\n\n\n<p>Rats. At least it didn&#8217;t crash the entire system.<\/p>\n\n\n\n<p>I tried again and it worked:<\/p>\n\n\n\n<details class=\"wp-block-details is-layout-flow wp-block-details-is-layout-flow\"><summary>LLM text output of llama3.2 3B running, again<\/summary>\n<pre class=\"wp-block-code\"><code>$ podman exec -it ollama ollama run llama3.2\n&gt;&gt;&gt; What's up doc?\nNot much, just hanging out! What's on your mind? Want to chat about something in particular or just shoot the breeze? I'm all \nears (or rather, all text).\n\n&gt;&gt;&gt; Show me what you got!\nBring it on!\n\nHere are a few fun facts and tidbits:\n\n1. **Did you know** that there's a species of jellyfish that's immortal? The Turritopsis dohrnii, also known as the \"immortal \njellyfish,\" can transform its body into a younger state through a process called transdifferentiation.\n2. **Fun fact**: The shortest war in history was between Britain and Zanzibar on August 27, 1896, and lasted only 38 minutes. \nZanzibar surrendered after just 12 minutes of fighting, and the remaining 26 minutes were spent on ceasefire negotiations!\n3. **Random trivia**: The longest word in the English language that can be typed using only the left hand on a standard QWERTY \nkeyboard is \"stewardesses\".\n4. **Did you know** that there's a fungus that turns ants into \"zombies\"? Ophiocordyceps unilateralis infects ants and takes \ncontrol of their bodies, causing them to climb to high locations before producing spores!\n\nWant more? I've got plenty where those came from!<\/code><\/pre>\n<\/details>\n\n\n\n<p>If things look fast enough, I might have to buy 64GB of RAM and see what Llama 3.3 70B can do on here!<\/p>\n\n\n\n<p><strong>12\/29\/2024, 10AM:<\/strong><\/p>\n\n\n\n<p>Increasing the amount of available RAM to the GPU is easy. Just set this kernel parameter:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>rpm-ostree kargs --append=amdgpu.gttsize=26000<\/code><\/pre>\n\n\n\n<p>At this point I was able to run any model that could fit into 25GB of VRAM on the GPU, so I took this <a href=\"https:\/\/github.com\/aidatatools\/ollama-benchmark\">llm-benchmark thingy<\/a> out for a spin. I had to write a wrapper script around ollama to pass through the command line arguments into a container, and simply retry if the GPU hung (which it does, every so often, when loading new models). Here are the results, in tokens per second:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\n{\n    \"mistral:7b\": \"17.15\",\n    \"llama3.1:8b\": \"10.84\",\n    \"phi3:3.8b\": \"24.39\",\n    \"qwen2:7b\": \"12.12\",\n    \"gemma2:9b\": \"11.62\",\n    \"llava:7b\": \"17.40\",\n    \"llava:13b\": \"9.62\",\n    \"uuid\": \"0f96d8fc-0390-5f84-bd51-3595976f0b2d\",\n    \"ollama_version\": \"0.0.0\"\n}\n{\n    \"system\": \"Linux\",\n    \"memory\": 27.205055236816406,\n    \"cpu\": \"AMD Ryzen 7 7840U w\/ Radeon  780M Graphics\",\n    \"gpu\": \"Phoenix1\",\n    \"os_version\": \"Bazzite 41 (FROM Fedora Kinoite)\",\n    \"system_name\": \"Linux\",\n    \"uuid\": \"0f96d8fc-0390-5f84-bd51-3595976f0b2d\"\n}\n<\/code><\/pre>\n\n\n\n<p>For comparison, here&#8217;s the same results on the CPU only&#8230;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n    \"mistral:7b\": \"11.99\",\n    \"llama3.1:8b\": \"10.81\",\n    \"phi3:3.8b\": \"19.74\",\n    \"qwen2:7b\": \"12.07\",\n    \"gemma2:9b\": \"8.62\",\n    \"llava:7b\": \"12.71\",\n    \"llava:13b\": \"6.93\",\n    \"uuid\": \"0f96d8fc-0390-5f84-bd51-3595976f0b2d\",\n    \"ollama_version\": \"0.0.0\"\n}\n{\n    \"system\": \"Linux\",\n    \"memory\": 27.205055236816406,\n    \"cpu\": \"AMD Ryzen 7 7840U w\/ Radeon  780M Graphics\",\n    \"gpu\": \"Phoenix1\",\n    \"os_version\": \"Bazzite 41 (FROM Fedora Kinoite)\",\n    \"system_name\": \"Linux\",\n    \"uuid\": \"0f96d8fc-0390-5f84-bd51-3595976f0b2d\"\n}<\/code><\/pre>\n\n\n\n<p>And on a relatives Apple M2 16GB laptop I borrowed.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n    \"mistral:7b\": \"19.69\",\n    \"llama3.1:8b\": \"16.61\",\n    \"phi3:3.8b\": \"29.21\",\n    \"qwen2:7b\": \"17.88\",\n    \"gemma2:9b\": \"11.95\",\n    \"llava:7b\": \"15.89\",\n    \"llava:13b\": \"10.86\",\n    \"uuid\": \"b05184b1-6e81-54b1-b1fa-f2ac23784229\",\n    \"ollama_version\": \"0.5.4\"\n}\n{\n    \"system\": \"Darwin\",\n    \"memory\": 16.0,\n    \"cpu\": \"Apple M2\",\n    \"gpu\": \"Apple M2\",\n    \"os_version\": \"macOS 14.5 (23F79)\",\n    \"system_name\": \"macOS\",\n    \"model\": \"Mac14,2\",\n    \"uuid\": \"b05184b1-6e81-54b1-b1fa-f2ac23784229\"\n}<\/code><\/pre>\n\n\n\n<p>So really not bad at all against a comparably-priced Mac. Of course, Apple doesn&#8217;t make you muck around with ROCm and setting kernel parameters and the like&#8230; But they also solder and glue their entire computers shut.<\/p>\n\n\n\n<p>According to <a href=\"https:\/\/redlib.morbinti.me\/r\/LocalLLaMA\/comments\/1g8mamt\/cheap_70b_run_with_amd_apuintel_igpu\/#popup\">this Reddit post<\/a> my dreams of running Llama 3.3 70B with this machine are not feasible, since memory bandwidth limitations only grow more severe the bigger the model gets. Shame. I really want a way to get this running locally.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post is being actively updated. Check back every day or two! I&#8217;ll keep updating it until I run out of interesting things to say and it becomes &#8220;yet another computer&#8221;. I purchased myself a nice used Framework 13 AMD for Christmas, after my poor old ThinkPad had an unfortunate run-in with severe hardware failure. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-342","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts\/342","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=342"}],"version-history":[{"count":0,"href":"https:\/\/nocoffei.com\/index.php?rest_route=\/wp\/v2\/posts\/342\/revisions"}],"wp:attachment":[{"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=342"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=342"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nocoffei.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=342"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}