How to use GenerationConfig

My query is : how do we use Generation config like temperature, top_p, top_k, num_beams to influence the summary. Since this needs the model to be available in HuggingFace repo, which model do we use from the repo for gpt3 ?

Secondly, quite often Business wants to summarize the text as well as inputs from plots and charts. Is this possible by GPT-3 ? Please explain.

To utilize these parameters, you would typically use them when calling the generate method of a HuggingFace Transformer model. Here’s an example of how you might do this in Python code:

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Initialize the tokenizer and model
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")

# Define your prompt
prompt = "Once upon a time"

# Encode the prompt and prepare the inputs
inputs = tokenizer.encode(prompt, return_tensors='pt')

# Generate the output
output = model.generate(

# Decode the output
output_text = tokenizer.decode(output[0])


In this example:

  • max_length is the maximum length of the generated text.
  • temperature is set to 0.7, making the output somewhat random but reasonably focused.
  • top_p is set to 0.8, meaning the model will only consider the most probable tokens whose cumulative probability exceeds 0.8.
  • top_k is set to 50, so the model only considers the top 50 most probable tokens at each step.
  • num_beams is set to 5, meaning the model will keep track of the five most probable hypotheses at each step.
  • do_sample is set to True, meaning the model will sample the output instead of just picking the most probable token at each step, which can lead to more diverse results.

Please note that these parameters can significantly affect the model’s output, and you may need to experiment with different values to get the desired results. Also, the generate function has many more parameters that you can use to control the output, so I recommend checking out the HuggingFace documentation for more information.

  1. Using Generation Config:
    The parameters like temperature, top_p, top_k, and num_beams control the text generation process of language models like GPT-3.

    • temperature: This parameter controls the randomness in the model’s output. A higher value (closer to 1) makes the outcome more random, while a lower weight (closer to 0) makes it more deterministic.

    • top_p (also known as nucleus sampling): This parameter limits the token selection pool to the most probable tokens whose cumulative probability exceeds a certain threshold.

    • top_k: This parameter limits the token selection pool to the top k most probable tokens.

    • num_beams: This parameter is used for beam search, a search algorithm that considers multiple hypotheses at each step and keeps the top k most probable ideas.

    As for the model, you can use any GPT-3 variant available in the HuggingFace Model Hub. The specific model to use depends on your particular requirements and resources.

  2. Summarizing Text and Inputs from Plots and Charts:
    GPT-3 is a text-based model that primarily works with text data. It can summarize text effectively, but it needs to inherently understand visual data like plots and charts.

    However, if you can convert the information from the plots and charts into a textual description, GPT-3 can include that information in its summary. For example, you could describe a chart as “a bar chart showing sales increasing over the last four quarters,” GPT-3 can use this description in its summary.

    For more advanced visual data handling, you should use computer vision models (to interpret the plots and charts) and language models (to generate the summary). This is a more complex task and goes beyond the capabilities of GPT-3 alone.

1 Like