Sleeves-Up AI

M2 Max is the sweet spot for local inference

Apple Silicon's unified memory makes it refreshingly easy to run LLMs locally. However, there's one significant caveat — M3 memory bandwidth is worse than previous generation M2s in all but the highest-end configuration. And based on llama.cpp benchmarks, this does make a difference. To get the best value-for-money, try to get your hands on a used M2 Max or Ultra.

Also, a way to allocate more memory for GPU use.

Llama Notebook

Everyone should try creating their own AI chat UI — it forces you to deeply think about where and how you want to employ AI in your life.

My current approach aims to "break out of the browser" and achieve persistence across apps and contexts.

• Keyboard shortcut to summon windows
• Direct access to the clipboard (and eventually files) for sharable context
• Not yet another electron app

AI-generated code isn't great for maintainability, so the process has been very 2 forward, 1 back. But there's no escaping the narrative appeal of using AI, to build the tool I use to run AI.

Armchair Speculations

AI Safety, in a bumper sticker

Everyone going faster than me is a maniac. Everyone slower is an idiot.

The prediction game

From various calls with technical friends, the general outlook is:

• In the medium-term, software will become cheaper to create and maintain.
• In the medium-term, human nature and societal norms will remain unchanged.
• In the short-term, Epistemology is going to fracture. 🥲

It's gonna be a bumpy ride.

Links and Listening

A detailed and useful guide about prompting.

Claude Opus and Claude Opus walk into a bar...

Interesting non-chat UI concepts for AI.

I still don't understand Assembly Theory, but it certainly has my attention.