Post #3222299

@sarahalang@fedihum.org OpenAI style API endpoints like the one you used report both the number of input and output tokens in their responses, so it shouldn't be hard to use those counts to estimate energy use. Or you could use a tokenizer library (e.g. tiktoken or transformers) to count the tokens before sending them to the API. Obviously there are limitations in estimating the energy spent per token (as you described), but I don't think getting the prompt/input token count should be the hard part here.