mastodon
4.7.0-beta.1
Slowly working towards the inference engine I want on Apple Silicon:
- MTLSafeTensors: mmap safetensors into MTLBuffers (https://github.com/finnvoor/MTLSafeTensors)
- Popcorn: a collection of optimized Metal kernels (https://github.com/finnvoor/Popcorn)
anyone need a Metal-accelerated ascii camera preview in their terminal? no?
Same energy
Hugging Face's AnyLanguageModel is an amazing way to make provider-agnostic agents in Swift.
Here's a library to use your Claude/Codex subscription with AnyLanguageModel: https://github.com/finnvoor/OAuthLanguageModels
Metal 4 has some really cool looking new features, but pretty much every time I've tried to use one I've gotten either vague errors or kernel panics.
Okay this is *crazy*
An agent running entirely on iPad* that can control any app
you can just copy things
(computer use in pi)
@schwa@mastodon.social damn they took all of GitHub down, they really don’t want you using pi
@schwa@mastodon.social they've employed some advanced security practices but after lots of hard work and reverse engineering we've cracked the code https://gist.github.com/finnvoor/25ca1758cbb358eb7c716976ce85861c
I should really release my bezel framing app someday 🤔
@schwa@mastodon.social they've made a world-ending new model but still the best they can do is string-matching the system prompt to block requests, I think we'll be fine 😉
@schwa@mastodon.social I wanted a minimal solution so I still get all the same system prompt/etc of pi which seems to work well
if you want to keep using Claude Pro/Max in pi 🤫
https://gist.github.com/finnvoor/25ca1758cbb358eb7c716976ce85861c
I'm fascinated by this section in an Apple paper about how they're using ASTC to compress models to 4 bit, then using the hardware decode to decompress with no overhead. I don't understand how ASTC could ever be even remotely close to 4bit quantization in terms of NRMSE though…
Made a lil tui agent entirely in Swift, only dependency is swift-cmark for markdown parsing
🌐 Still a little rough around the edges, but Navi is an agent that lives in Safari and helps you research and chat with your tabs, à la Dia, Comet, or Claude in Chrome.
Sign in with ChatGPT Plus/Pro or Claude Pro/Max, open source, and available on iOS, iPadOS, and macOS.
this ain’t it
Made a lil agent that lives in Safari on iOS and macOS 🧚
Kind of crazy that in a few months my job has gone from writing code to simply sending slack messages
✨ Run ModernBERT using CoreML