This post is being actively updated. Check back every day or two! I’ll keep updating it until I run out of interesting things to say and it becomes “yet another computer”.

I purchased myself a nice used Framework 13 AMD for Christmas, after my poor old ThinkPad had an unfortunate run-in with severe hardware failure.

That’s a fun story on its own, perhaps I should write it up as well.

This is more of a “journal”-y post, wherein I’ll write my observations as they come, being that on top of the new hardware I was also convinced to try an entirely new style of OS, the immutable Fedora Kinoite (or Silverblue). The previous owner, who I’m good friends with, used Kinoite during his time with it, and was a fan:

[Luke, 12/26/2024, 12:33PM]

Upside is I never had my system break and I could leave it alone for months and open it and it just worked

It’s the most stable distro I’ve used I’ll give it that

Not to say everything was smooth and easy but once it was working it stayed that way

As someone who spent 2024 on a horrible Windows install due to abhorrent bugginess with late-stage KDE 5 and Arch’s ongoing game of Russian Roulette every time you install a package (will your kernel panic after pacman -Syu, as is required every time you install something? Who knows!) the qualities of an immutable OS are extremely attractive: rock-solid stability and the ability to go back if AMDs graphics engineers screw up again.

It was worth a shot. So how well does all this work in 2024? And at the same time, how does the Framework feel and behave?

Comments on hardware

I’ve been using the hardware for a day now, and some things immediately come to mind.

This is a Ryzen 7 7840U, with the integrated Radeon 780M graphics. (More horsepower than a Steam Deck– juicy!) 32GB of RAM is installed, and a 2TB SSD is also included.

This thing feels SNAPPY and not just due to the fact that the Windows install I spent most of the year on took literal seconds to Alt+Tab between windows. It’s able to open the most bloated of Electron apps (Signal) in mere seconds, graphical effects are smooth and gorgeous, browser pages load instantly… It’s hard to say how much of this is due to the new CPU and how much is due to KDE Plasma 6’s innovations like triple buffering, but the end result is a machine that leaves any other I’ve used in the dust!

The 1st-generation matte screen is pretty and easy to look at, while power consumption isn’t off the charts like my last 4K 14″ display. Of course, it’s comfortably viewed at 150% scale, so all the usual misery with fractional scaling on Linux is present. Somehow, in 2024, this is still broken, and it’s so bad that Framework made a new panel with an express design goal to work around this mess in hardware.

The input devices are a massive downgrade from my beloved ThinkPads, but I knew this would be the case no matter what I chose as Lenovo gave up on making good keyboards some years ago. I’m coming from a ThinkPad T14 Gen 2 AMD, the last unit made before the changes.

The keyboard is… Fine. I will need to find accommodations for the missing PgUp, PgDown, Home and End keys, which I’ve come to love and rely on.

The trackpad is OK, about on par with the glass trackpad replacement I put in the T14, and there is no TrackPoint 🙁 I am going to miss this dearly, especially without a touchscreen. I will likely find myself using a wireless mouse much more often. It may, in fact, be a deal-breaker for the hardware in the long run. I’ve come to be totally dependent on it in the last 5 years of using ThinkPads.

The Arch Wiki is, as always, incredibly helpful, and the page for this laptop pointed out two amazing facts:

• Notebookcheck provides ICC color calibration profiles for this laptop (and many others), and
• Framework provides a DSP profile that can be applied in userspace.

DSP has been making some waves with Marcans long-running work on making speakers sound amazing on Asahi Linux. Although the Apple Silicon speakers are especially fancypants and require a new ground-up implementation, it seems there is still a tremendous deal you can do with basic DSP. It turns out installing this is as easy as installing the linked EasyEffects, setting it to run on boot in the preferences, and importing the profile. That’s it! It was so easy I was certain I’d missed a step somewhere. And boy does it work.

These speakers are already probably the loudest of any laptop I’ve owned, and applying the DSP profile makes them sound even brighter and more powerful. You can very clearly hear the difference as I enable and disable the profile in the below snippet of Katamari on the Rocks:

I’ll actually put it in whenever I get around to it 😛

I’ve never heard of a manufacturer caring this much about Linux support. Framework officially supports Ubuntu and Fedora, and has gone to great lengths to make sure that everything works on their hardware out of the box.

Fedora Kinoite and woe

This is the “journal entry” section of this post. It’s half me figuring out a new style of OS and half running headfirst into the usual breakage and half-working components of the Linux desktop as I have a tendency to do.

12/26/2024, 12:30PM:

The laptop arrives. I grab a flash drive and install Fedora Kinoite.

“Hmm,” I say, “instead of transferring my files through a flash drive like usual, I bet I could network share this stuff somehow. And wouldn’t it be cool if I could even do that over Thunderbolt? Both of these computers have it.”

Plugging a good USB-C cable into both ends opens up a new thunderbolt0 network interface, but that’s where the idea falls apart. How do I send the files? Well, I suppose I could use KDEs built in file sharing and…

12/26/2024, 1:30PM:

KDEs file manager, Dolphin, has a nice built-in “Share” section of the properties tab of a folder. The idea is to quickly and easily network share anything you’d like.

As far as I can tell, this does not work at all and there is no clear way to make it work.

Both on Fedora Kinoite and Fedora Workstation (which I tested for comparison) there are two reboots worth of error messages, complaining that there are missing configurations needed to get Samba working. Fair enough, except the auto-configurations it offers to do for you don’t work on Kinoite and you have to configure them yourself as rpm-ostree does not treat user groups with respect.

Once you get past that, AND once you manually start the SMB service, as far as I can tell, what happens next is that you’re presented with an interface that says it works, it doesn’t, actually:

If you try to connect from localhost, it gives you an authentication dialog that will reject anything you put in. If you try to connect from another machine, the connection will be refused.

I told an IT buddy about this and he said:

[nocoffei, 12/26/2024 5:55PM]

Smb on KDE is completely broken

[Travis, 12/26/2024 5:55PM]

I mean, that’s generally true

I’ve never gotten an SMB share to actually work

I give up and copy the files via a flash drive.

12/26/2024, 2:30PM

I run to the store and buy a Torx T5 screwdriver, as I didn’t think to bring one with me on travel for the holidays. I open the machine up. Indeed, the internals are very sexy. However, aside from wanting a few more Expansion Cards, this computer is already configured with the highest specs I want, so there’s not much to do here other than marvel at the engineering.

12/26/2024, 3:30PM

The usual config of a new KDE install continues. There are plenty of little nooks and crannies I vaguely remember I need to set up.

12/26/2024, 4:30PM

How do I install that ICC color profile? I spend an hour messing with the brand new Color Management page in KDE settings. The import button on this page is broken, I presume because it naively tries to copy to /usr/share which is read-only on Kinoite:

It should be copying to /usr/local/share, which also works for colord, but this isn’t what I want anyway. I want to apply the profile to the whole screen, which can easily be done from the Display section.

I find out that all GTK4 apps have broken cursor sizes. Again. The last time these guys broke this (on GTK3), they marked as wontfix, and then eventually fixed it about a year and a half later, forcing me to hold back GTK on an increasingly ancient version.

Apparently the GTK folks are not interested in a backport, so we will have to wait until a new GTK release to fix this. Again.

At least this time it’s tolerable, I guess. The last breakage was actually unusable, making a cursor bigger than the width of my finger.

12/26/2024, 8PM:

It takes me well over two hours to figure out how the hell I’m supposed to get hardware video decoding working in Firefox. Fedora removes this for legal reasons at the cost of making its distro one extra installation more annoying for Workstation uses, and dramatically more irritating for its read-only filesystem variant.

After hours of investigation, I find this page which describes a very aggressive manipulation of the install to yank out all the crap Fedora puts it there and replace it with normal-ass ffmpeg. After all that hassle, it looks like I got H264 decoding working in Firefox.

vainfo isn’t present, somehow, so I can’t verify this, and at this point I’m so irritated by the whole thing I want to try another approach.

12/27/2024, 5AM:

Frustrated by Silverblues lack of sane defaults for integrated graphics, I decide to try Bazzite, which is a downstream copy of Silverblue with lots and lots and lots of optimized defaults for gaming. I had vaguely seen it before, and was hot on the heels with my recent amazing experiences with the Steam Deck and EmuDeck. Community folks have really come together around the recent “open gaming handheld” market, and I suspected it would be the same way here!

First, I read up on rpm-ostrees fundamentals a bit, since I didn’t understand what the system is doing. After reading some Fedora documentation and this old, but still relevant mini-book on the subject, I felt better about continuing.

Anticipating an OS reinstall, I was very pleasantly surprised to find out that I could not only keep my current install, I could “snapshot” my current install (what they call a “pin”) and simply rebase the ostree base onto Bazzite. This is an officially supported and documented installation method!

The actual rebase command steps appear only once you’ve clicked through their “download” page. It’s a website trick that makes it look fancy, but harder to link to.

12/27/2024, 8AM:

We’re live, baby.

Bazzite gives a ton of neat basics preinstalled that I haven’t fully explored yet. The only “breaking” changes are:

• Firefox is now a Flatpak, not a binary. Move your files and it works.
• Steam is now a binary, not a Flatpak. Move your files and it works… BUT it deleted the one game I had installed for some reason…?

I spend the rest of my time at the computer today generally mucking around with more configs and setting up my computer the way I like it.

12/28/2024, 10AM:

Marvel Rivals runs! Lowest settings at 720P and it’s a smooth 60FPS.

Even got MangoHUD working so I can watch the GPU get hammered. It’d be nice to see the breakdown of power consumption between CPU and GPU like on the Steam Deck, but I haven’t figured that out yet.

12/28/2024, 5PM:

This thing has a pretty decent iGPU hooked up to it, clearly. I wonder, can I get an LLM running? It has 32GB of “V”RAM hooked up to it…

I expect that it’ll just be a waste of my time, considering the state of AMDs ROCm platform, but apparently I’m underestimating my rabid peers in the self-hosting LLM community.

$ podman exec -it ollama ollama run llama3.2
pulling manifest 
pulling dde5aa3fc5ff... 100% ▕████████████████▏ 2.0 GB                         
pulling 966de95ca8a6... 100% ▕████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da... 100% ▕████████████████▏ 7.7 KB                         
pulling a70ff7e570d9... 100% ▕████████████████▏ 6.0 KB                         
pulling 56bb8bd477a5... 100% ▕████████████████▏   96 B                         
pulling 34bb5ab01051... 100% ▕████████████████▏  561 B                         
verifying sha256 digest 
writing manifest 
success 
>>> what is your name?
I don't have a personal name, but I'm an AI designed to assist and 
communicate with users. You can call me "Assistant" or "AI" if you prefer. 
Some people also refer to me as "Chatbot" or simply "the AI." How can I 
help you today?

>>> what is your quest?
I don't have a personal quest in the classical sense, but my purpose is to 
assist and provide value to users like you. My primary goals are:

1. **Knowledge Sharing**: To provide accurate and up-to-date information 
on a wide range of topics.
2. **Conversation and Understanding**: To engage in natural-sounding 
conversations, ask questions, and understand user intent.
3. **Learning and Improvement**: To learn from user interactions, improve 
my language understanding, and refine my responses.

In a broader sense, my "quest" is to help people like you find answers, 
gain insights, and solve problems more efficiently. I'm here to provide 
assistance, answer questions, and offer suggestions whenever possible.

What's your quest? Do you have a specific question or topic you'd like to 
discuss?

>>> what is the airspeed velocity of an unladen swallow?
A classic reference to Monty Python's Flying Circus!

The airspeed velocity of an unladen swallow is a topic of much debate and 
humor. In reality, there is no definitive answer, as it depends on various 
factors such as:

1. Species: Different species of swallows have varying wingbeat speeds.
2. Condition: The bird's health, weight, and aerodynamic condition can 
affect its airspeed velocity.
3. Altitude and wind: Weather conditions, altitude, and wind resistance 
all impact the bird's speed.

However, for the sake of humor and entertainment, let's consider some 
estimates:

* According to a 1991 study on swallow flight patterns, the average 
cruising speed of a barn swallow (Hirundo rustica) is around 35-45 km/h 
(22-28 mph).
* Another estimate suggests that a European swallow (Hirundo rustica 
europaea) can reach speeds of up to 50-60 km/h (31-37 mph).

But remember, these are just rough estimates and not exactly what the 
Monty Python sketch is asking for!

To answer your question in a more tongue-in-cheek manner: "What do you 
mean? An African or European swallow?"

Would you like to discuss more Monty Python trivia or perhaps explore some 
fascinating facts about swallows?

Here’s the magic command I use to one-click run ollama:

podman run --pull newer --detach --security-opt label=type:container_runtime_t --replace --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 -e HSA_OVERRIDE_GFX_VERSION=11.0.0 --name ollama ollama/ollama:rocm

The command is sourced from here. I have to add the HSA_OVERRIDE environment variable to force ROCm to use the unsupported iGPU.

But… Not all is perfect. ollama is only detecting the “dedicated VRAM” which I allocated in the VBIOS to be 4GB, and offloading the rest to the CPU:

time=2024-12-29T03:27:47.796Z level=INFO source=types.go:131 msg="inference compute" id=0 library=rocm variant="" compute=gfx1103 driver=0.0 name=1002:15bf total="4.0 GiB" available="2.3 GiB"
...
llm_load_tensors: offloading 18 repeating layers to GPU
llm_load_tensors: offloaded 18/29 layers to GPU
llm_load_tensors:   CPU_Mapped model buffer size =  1918.35 MiB
llm_load_tensors:        ROCm0 model buffer size =  1041.26 MiB

Can I fix that and get ollama to properly recognize the entirety of the unified memory available?

Yes, there’s an open, unmerged pull request out for it, and there’s even an unbelievably convenient blog post describing exactly how to compile it and use it!!

I had this really weird issue when running the container that went away after removing --pull=always from the command. No clue why but it works. I also had to re-add the security permissions stuff from the first command, otherwise it fails to pass through. Here’s what that looks like:

$ podman run -d     --name ollama     --replace      --restart=always     --stop-signal=SIGKILL     -p 127.0.0.1:11434:11434     -v ollama:/root/.ollama     -e OLLAMA_MAX_LOADED_MODELS=1     -e OLLAMA_NUM_PARALLEL=1     --device /dev/dri     --device /dev/kfd     -e HSA_OVERRIDE_GFX_VERSION=11.0.0  --security-opt label=type:container_runtime_t   localhost/ollama-gtt:latest

And we can see that it manages to allocate plenty of VRAM!

time=2024-12-29T05:27:45.938Z level=INFO source=types.go:124 msg="inference compute" id=0 library=rocm variant="" compute=gfx1103 driver=0.0 name=1002:15bf total="13.6 GiB" available="13.4 GiB"
...
llm_load_tensors: offloading 28 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 29/29 layers to GPU
llm_load_tensors:      ROCm0 buffer size =  1918.36 MiB
llm_load_tensors:        CPU buffer size =   308.23 MiB

That’s still not quite all of my free RAM; I’ll need to drive that value up some if I want to start playing with some of the bigger models. Let’s ask something…

$ podman exec -it ollama ollama run llama3.2
>>> Show me what you got!

The screen turns black for a moment, then comes up again, and the container logs show…

HW Exception by GPU node-1 (Agent handle: 0x7fa6e50b64f0) reason :GPU Hang

Rats. At least it didn’t crash the entire system.

I tried again and it worked:

$ podman exec -it ollama ollama run llama3.2
>>> What's up doc?
Not much, just hanging out! What's on your mind? Want to chat about something in particular or just shoot the breeze? I'm all 
ears (or rather, all text).

>>> Show me what you got!
Bring it on!

Here are a few fun facts and tidbits:

1. **Did you know** that there's a species of jellyfish that's immortal? The Turritopsis dohrnii, also known as the "immortal 
jellyfish," can transform its body into a younger state through a process called transdifferentiation.
2. **Fun fact**: The shortest war in history was between Britain and Zanzibar on August 27, 1896, and lasted only 38 minutes. 
Zanzibar surrendered after just 12 minutes of fighting, and the remaining 26 minutes were spent on ceasefire negotiations!
3. **Random trivia**: The longest word in the English language that can be typed using only the left hand on a standard QWERTY 
keyboard is "stewardesses".
4. **Did you know** that there's a fungus that turns ants into "zombies"? Ophiocordyceps unilateralis infects ants and takes 
control of their bodies, causing them to climb to high locations before producing spores!

Want more? I've got plenty where those came from!

If things look fast enough, I might have to buy 64GB of RAM and see what Llama 3.3 70B can do on here!

12/29/2024, 10AM:

Increasing the amount of available RAM to the GPU is easy. Just set this kernel parameter:

rpm-ostree kargs --append=amdgpu.gttsize=26000

At this point I was able to run any model that could fit into 25GB of VRAM on the GPU, so I took this llm-benchmark thingy out for a spin. I had to write a wrapper script around ollama to pass through the command line arguments into a container, and simply retry if the GPU hung (which it does, every so often, when loading new models). Here are the results, in tokens per second:

{
    "mistral:7b": "17.15",
    "llama3.1:8b": "10.84",
    "phi3:3.8b": "24.39",
    "qwen2:7b": "12.12",
    "gemma2:9b": "11.62",
    "llava:7b": "17.40",
    "llava:13b": "9.62",
    "uuid": "0f96d8fc-0390-5f84-bd51-3595976f0b2d",
    "ollama_version": "0.0.0"
}
{
    "system": "Linux",
    "memory": 27.205055236816406,
    "cpu": "AMD Ryzen 7 7840U w/ Radeon  780M Graphics",
    "gpu": "Phoenix1",
    "os_version": "Bazzite 41 (FROM Fedora Kinoite)",
    "system_name": "Linux",
    "uuid": "0f96d8fc-0390-5f84-bd51-3595976f0b2d"
}

For comparison, here’s the same results on the CPU only…

{
    "mistral:7b": "11.99",
    "llama3.1:8b": "10.81",
    "phi3:3.8b": "19.74",
    "qwen2:7b": "12.07",
    "gemma2:9b": "8.62",
    "llava:7b": "12.71",
    "llava:13b": "6.93",
    "uuid": "0f96d8fc-0390-5f84-bd51-3595976f0b2d",
    "ollama_version": "0.0.0"
}
{
    "system": "Linux",
    "memory": 27.205055236816406,
    "cpu": "AMD Ryzen 7 7840U w/ Radeon  780M Graphics",
    "gpu": "Phoenix1",
    "os_version": "Bazzite 41 (FROM Fedora Kinoite)",
    "system_name": "Linux",
    "uuid": "0f96d8fc-0390-5f84-bd51-3595976f0b2d"
}

And on a relatives Apple M2 16GB laptop I borrowed.

{
    "mistral:7b": "19.69",
    "llama3.1:8b": "16.61",
    "phi3:3.8b": "29.21",
    "qwen2:7b": "17.88",
    "gemma2:9b": "11.95",
    "llava:7b": "15.89",
    "llava:13b": "10.86",
    "uuid": "b05184b1-6e81-54b1-b1fa-f2ac23784229",
    "ollama_version": "0.5.4"
}
{
    "system": "Darwin",
    "memory": 16.0,
    "cpu": "Apple M2",
    "gpu": "Apple M2",
    "os_version": "macOS 14.5 (23F79)",
    "system_name": "macOS",
    "model": "Mac14,2",
    "uuid": "b05184b1-6e81-54b1-b1fa-f2ac23784229"
}

So really not bad at all against a comparably-priced Mac. Of course, Apple doesn’t make you muck around with ROCm and setting kernel parameters and the like… But they also solder and glue their entire computers shut.

According to this Reddit post my dreams of running Llama 3.3 70B with this machine are not feasible, since memory bandwidth limitations only grow more severe the bigger the model gets. Shame. I really want a way to get this running locally.