Track Token Usage

Short Course Q&A Multi AI Agent Systems with crewAI
1

Hi,

I was wondering how one can track the total number of tokens (prompt + completion tokens) for a Multi-Agent Crew.

Can anyone share the code for tracking token usage of a Multi-Agent Crew with crewAI?

Something equivalent to

from langchain_community.callbacks import get_openai_callback

with get_openai_callback() as cb:
prompt_tokens = cb.prompt_tokens
completion_tokens = cb.completion_tokens

Thanks

2

Hello, I had the same problem. Looking at some github issues I found a solution :
Tokens count · Issue #93 · joaomdmoura/crewAI

Here is my function :

def calculate_costs(crew_usage_metrics, model_input_price, model_output_price, unit_of_tokens):
    """
    Calculate the costs based on crew usage metrics and token pricing.

    Parameters:
    crew_usage_metrics (dict): A dictionary containing the usage metrics with the keys:
        - 'total_tokens': Total number of tokens used.
        - 'prompt_tokens': Number of tokens used for input prompts.
        - 'completion_tokens': Number of tokens used for completions.
        - 'successful_requests': Number of successful requests made.
    model_input_price (float): Cost per unit of input tokens.
    model_output_price (float): Cost per unit of output tokens.
    unit_of_tokens (int): The number of tokens per unit cost (e.g., per 1000 tokens).

    Returns:
    dict: A dictionary with the calculated costs:
        - 'total_cost': Total cost of the usage.
        - 'input_cost': Cost of the input tokens.
        - 'output_cost': Cost of the output tokens.
    """
    
    prompt_tokens = crew_usage_metrics.get('prompt_tokens')
    completion_tokens = crew_usage_metrics.get('completion_tokens')
    
    input_cost = (prompt_tokens / unit_of_tokens) * model_input_price
    output_cost = (completion_tokens / unit_of_tokens) * model_output_price
    total_cost = input_cost + output_cost
    
    return {
        'total_cost': total_cost,
        'input_cost': input_cost,
        'output_cost': output_cost
    }

You have access to crew metrics like so :

→ <your_crew_object>.usage_metrics

Wish it will help you :slight_smile: