HELP ME SET UP THE RAG_TOOL IN Crew ai

Short Course Q&A ACP: Agent Communication Protocol
,
1

Hello Everyone !
I am Learning ACP Agent Communication Protocal ! i Ran into error like** Litellm,** i tried with gemini, huggingface and other model still i am facing error with** litellm provider , sometimes it might be api issue, sometimes it might be embedding issue, i am so clueless about this litellm . **

here is my code please help me setup this rag tool using hugging face api and opensource models, any guidence also would be much helpful and most needed in this stage for me.

!pip install crewai crewai-tools !pip install transformers langchain-huggingface sentence-transformers

This i have implemented, by referring to the embedder site

from langchain_huggingface import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(
model_name=“sentence-transformers/all-MiniLM-L6-v2”
)

yaml_content = “”"
llm:
provider: huggingface
config:
model: ‘huggingface/deepseek-ai/DeepSeek-R1’
api_key : RAG_HF_key
temperature: 0.7
max_tokens: 2048
top_p: 0.5
stream: false

embedder:
provider: huggingface
config:
model: ‘sentence-transformers/all-mpnet-base-v2’
model_kwargs:
trust_remote_code: True # Only use if you trust your embedder
“”"

file_path = “config.yaml”

with open(file_path, “w”) as f:
f.write(yaml_content)

print(f"Configuration saved to {file_path}")

from embedchain import App

load embedding model configuration from config.yaml file

app = App.from_config(config_path=“config.yaml”)
from crewai_tools import RagTool

config = {
“llm”: {
“provider”: “huggingface”,
“config”: {
“model”: ‘huggingface/mistralai/Mistral-7B-Instruct-v0.1’,
“api_key”: ‘RAG_HF_key’,
“temperature”: 0.7,
“max_tokens”: 2048,
“top_p”: 0.5,
“stream”: False
}
},
“embedding_model”: {
“provider”: “huggingface”,
“config”: {
“model”: “sentence-transformers/all-mpnet-base-v2”
}
}
}

rag_tool = RagTool(config=config)

rag_tool.add(“GitHub - ericyangyu/PPO-for-Beginners: A simple and well styled PPO implementation. Based on my Medium series: https://medium.com/@eyyu/coding-ppo-from-scratch-with-pytorch-part-1-4-613dfc1b14c8.”, data_type=“web_page”)
rag_tool.add(“Proximal Policy Optimization (PPO)”, data_type=“web_page”)

This was worked , rag but integrating to llm and agent is not working

Inserting batches in chromadb: 100%|██████████| 1/1 [00:13<00:00

#====================== Setting crew ai agent and task ===================
llm_free = LLM(
model=“huggingface/microsoft/DialoGPT-medium”, # Works with HF Inference API
api_key=RAG_HF_key,
temperature=0.7,
max_tokens=1024
)

#========================= Agent =============
from crewai import Task

task1 = Task(
description=“Read and analyze articles and research papers on Proximal Policy Optimization (PPO) for fine-tuning LLMs. Then, help the user implement PPO from scratch with reasoning and comments, using libraries where appropriate. Keep the code under 300 lines.”,
expected_output=“A well-commented PPO implementation for LLM fine-tuning, under 300 lines, with clear reasoning steps.”,
agent=researcher1,
tools=[rag_tool]
)

from crewai import Task

task1 = Task(
description=“Read and analyze articles and research papers on Proximal Policy Optimization (PPO) for fine-tuning LLMs. Then, help the user implement PPO from scratch with reasoning and comments, using libraries where appropriate. Keep the code under 300 lines.”,
expected_output=“A well-commented PPO implementation for LLM fine-tuning, under 300 lines, with clear reasoning steps.”,
agent=researcher1,
tools=[rag_tool]
)

crew = Crew(
agents=[researcher1],
tasks =[task1],
tools = [rag_tool],
verbose = True
)

result = crew.kickoff()
print(“Task Description:”, task1.description)
print(“Output:\n”, result)

#=================ERROR =================================
APIError: litellm.APIError: HuggingfaceException - {“error”:“Template error: syntax error: After the optional system message, conversation roles must alternate user/assistant/user/assistant/… (in :11)”,“error_type”:“template_error”}

==================== PLEASE HELP ME HOW TO SET UP THIS, TEACH ============

** THANK YOU **

1 Like
2

Dear @Sasi_Kiran_Royal,

Please refrain from sharing any part of the code publicly or over the community channels.

Kindly send me the complete notebook along with the required configuration files via direct message. I will review the implementation in detail and get back to you with a suitable solution.