Post #3222298
2026-01-07 15:23 UTC
@osma@sigmoid.social that’s honestly also something I had worried about.. ultimately we’d need something for the #DH that takes into account both inputs and outputs…
Replies (1)
-
@osma@sigmoid.social 2026-01-07 15:31
@sarahalang@fedihum.org OpenAI style API endpoints like the one you used report both the number of input and output tokens in their responses, so it shouldn't be hard to use those counts to estimate energy use. Or you could use a tokenizer library (e.g. tiktoken or transformers) to count the tokens before sending them to the API. Obviously there are limitations in estimating the energy spent per token (as you described), but I don't think getting the prompt/input token count should be the hard part here.