Hi All,
Unable to resolve this error -
ValueError Traceback (most recent call last)
Cell In[16], line 2
1 # Output is a list of indices
----> 2 bm25_retrieve("What are the recent news about GDP?")
ValueError: k of 5 is larger than the number of available scores, which is 3 (corpus size should be larger than top-k). Please set with a smaller k or increase the size of corpus.
I have tried all the solutions suggested in the other posts. I think the part of course has been changed but not sure on this. Thanks!
The error says corpus size is 3 but you’re asking for top 5. That means your BM25_RETRIEVER got re-indexed with wrong data somewhere inside your function.
The most common cause: the starter code has a comment # Index the tokenized chunks with the retriever. If you’re calling BM25_RETRIEVER.index(tokenized_query) there, you’re overwriting the full corpus index with just the query tokens (which is only a few words, hence “3 available scores”). The corpus was already indexed in an earlier cell, so you either:
- Skip that indexing line entirely, or
- Index
TOKENIZED_DATA instead of tokenized_query
Also double-check you’re passing the k parameter as a keyword argument: BM25_RETRIEVER.retrieve(tokenized_query, k=top_k), not as a positional arg.
After fixing, do Kernel → Restart & Clear Output and run all cells from the top so the global state is clean.
@kharade.navin
my suspect based on your error log is to check if you converted the documents into indices in the list recalled as result
Check the below code line instruction 
Convert the retrieved documents into their corresponding indices in the results list
top_k_indices=here are you using corpus ? or you are using corpus.index to retrieve documents in the list.
Regards
Dr. Deepti