Post #1753232
2026-04-01 21:37 UTC
@Lydie How much RAM do you reckon is needed for Local LLM? I saw you were offloading the models partially to system RAM. Is there any rule of thumb what works best?
Also probably good to factor in offloading CLIP/VAE to RAM for image generation…
Replies (1)
-
@Lydie@tech.lgbt 2026-04-02 12:53
@lobstermane Short answer: the more the better. Run the size model that fits with what you have. But the smaller the model, the worse the performance as an agent.