Assignment 2: Implementing Retriever Functions in a RAG System C1M2_Assignment

Course Q&A Retrieval Augmented Generation
, , ,
1

Hi All,

Unable to resolve this error -

ValueError                                Traceback (most recent call last)
Cell In[16], line 2
      1 # Output is a list of indices
----> 2 bm25_retrieve("What are the recent news about GDP?")


ValueError: k of 5 is larger than the number of available scores, which is 3 (corpus size should be larger than top-k). Please set with a smaller k or increase the size of corpus.

I have tried all the solutions suggested in the other posts. I think the part of course has been changed but not sure on this. Thanks!

2

The error says corpus size is 3 but you’re asking for top 5. That means your BM25_RETRIEVER got re-indexed with wrong data somewhere inside your function.

The most common cause: the starter code has a comment # Index the tokenized chunks with the retriever. If you’re calling BM25_RETRIEVER.index(tokenized_query) there, you’re overwriting the full corpus index with just the query tokens (which is only a few words, hence “3 available scores”). The corpus was already indexed in an earlier cell, so you either:

  • Skip that indexing line entirely, or
  • Index TOKENIZED_DATA instead of tokenized_query

Also double-check you’re passing the k parameter as a keyword argument: BM25_RETRIEVER.retrieve(tokenized_query, k=top_k), not as a positional arg.

After fixing, do Kernel → Restart & Clear Output and run all cells from the top so the global state is clean.

3

@kharade.navin

my suspect based on your error log is to check if you converted the documents into indices in the list recalled as result

Check the below code line instruction :backhand_index_pointing_down:

Convert the retrieved documents into their corresponding indices in the results list
top_k_indices=here are you using corpus ? or you are using corpus.index to retrieve documents in the list.

Regards
Dr. Deepti

4

Hi Both,

Thanks for looking into this. Have tried both the approach and also passing the ‘tokenized_data’. However still not get through. In step ‘Output is a list of indices’, have observed no such output - Expected output [752, 673, 289, 626, 43]. And in step ‘Test your function!’, observed - Failed test case: output has wrong type.
Expected: <class ‘list’>
Got: <class ‘NoneType’>

Please check and help. Thanks!

5

That NoneType error is the giveaway, your function isn’t returning anything. The test harness calls bm25_retrieve(...) and gets None back, which means you’re missing a return statement at the end of the function (or it’s inside a branch that never executes).

Two things to check, in order:

  1. Make sure you return the final list of indices. The function should end with returning top_k_indices (or whatever you named that variable). If there’s no return, Python defaults to None, which is exactly the error you’re seeing.

  2. The original “k of 5 > corpus size of 3” error: this happens when BM25_RETRIEVER gets re-indexed inside your function with the wrong data (e.g., the query tokens). The corpus should already be indexed in an earlier cell. Inside bm25_retrieve, you only need to retrieve, not index. If there’s a comment about indexing, make sure you’re indexing TOKENIZED_DATA (the full corpus), not tokenized_query.

After fixing, do Kernel → Restart & Clear Output and run all cells from the top so the global state is clean.

If you’re still stuck after this, send your code via DM and I can take a closer look.