Post #1725155

@CrabbyIT I am not normally a fan of developing in virtual environments but this strikes me as a potentially valid use case. Discourage running LLM tools directly on developers' real hardware; instead, have a template VM that you can spin up, with all the tools pre-installed (make it easy and low-friction!), and then sandbox those virtual workstations at L2 (VLAN) / L3 (subnet) on the network. I am not sure if I would trust purpose-built LLM sandboxing tools currently, and LiteLLM in particular concentrates valuable API keys in a way that seems risky. I'd want per-user API keys, I think. The controls come at the border between the LLM workstation network and the broader corporate network, with firewalls and reverse proxies to control hits to internal APIs and other resources. Basically, they are locked in a dev/sandbox network and code gets promoted "out" via a PR process with human review before it can touch actual production systems.