Large model to deal with longer sentences

I am trying to use LLM to answer questions from a list of URLs I share. For this I tried using hugging face models in sentence transformers- all-MiniLM-L6-v2 and all-MiniLM-L12-v2. I get same indexing error while I ask my query in both the models:
“token indices sequence length is longer than the specified maximum sequence length for this model (1761>1024). Running this sequence through the model will result in indexing errors”
Which model I should use? Do we have a large model to deal with this sequence length? Any help will be really appreciated.

To address this issue, you can consider using a model with a larger maximum sequence length. For example, you could use a model such as all-mpnet-base-v2 or all-mpnet-base-v3, which have a maximum sequence length of 2048. These models are better suited for handling longer input sequences.

By using a model with a larger maximum sequence length, you can process longer input sequences without encountering indexing errors.

2 Likes

Thank you for the reply!

I tried using ‘all-mpnet-base-v2’ but I am still encountering the same error. Its a small tutorial. Please find the code snippet below.

import langchain
from langchain.llms import GooglePalm
from langchain.chains import RetrievalQAWithSourcesChain
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import UnstructuredURLLoader
from langchain.vectorstores import FAISS
from langchain.embeddings import HuggingFaceEmbeddings

langchain.debug=True
googlepalm_key="my_api_key"
llm=GooglePalm(google_api_key=googlepalm_key, temperature=0.1)

loader=UnstructuredURLLoader(urls=["https://www.moneycontrol.com/news/business/markets/wall-street-rises-as-tesla-soars-on-ai-optimism-11351111.html",
                                   "https://www.moneycontrol.com/news/business/tata-motors-launches-punch-icng-price-starts-at-rs-7-1-lakh-11098751.html"
                                  ])
data=loader.load()

text_splitter=RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=50)
docs=text_splitter.split_documents(data)
embeddings=HuggingFaceEmbeddings(model_name='sentence-transformers/all-mpnet-base-v2')
faiss = FAISS.from_documents(docs, embeddings)
chain=RetrievalQAWithSourcesChain.from_llm(llm=llm, retriever=faiss.as_retriever())
query="What is the price of Tiago iCNG?"
chain({"question": query}, return_only_outputs=True)

I am not sure where the problem is. Any help would be greatly appreciated.