L4 Chain of thought reasoning

Short Course Q&A Building Systems with the ChatGPT API
1

So i decided to change the prompt and increase the prices on the items and tried asking questions on top of that and the notebook still answers with the same answer from the video / demo. that the chromebook is cheaper even though i have changed it and sent the prompt to OpenAI. Does OpenAI cache things, what am I missing as to why it’s not using the newer data i have fed it?

delimiter = "####"
system_message = f"""
Follow these steps to answer the customer queries.
The customer query will be delimited with four hashtags,\
i.e. {delimiter}. 

Step 1:{delimiter} First decide whether the user is \
asking a question about a specific product or products. \
Product cateogry doesn't count. 

Step 2:{delimiter} If the user is asking about \
specific products, identify whether \
the products are in the following list.
All available products: 
1. Product: TechPro Ultrabook
   Category: Computers and Laptops
   Brand: TechPro
   Model Number: TP-UB100
   Warranty: 1 year
   Rating: 4.5
   Features: 13.3-inch display, 8GB RAM, 256GB SSD, Intel Core i5 processor
   Description: A sleek and lightweight ultrabook for everyday use.
   Price: $799.99

2. Product: BlueWave Gaming Laptop
   Category: Computers and Laptops
   Brand: BlueWave
   Model Number: BW-GL200
   Warranty: 2 years
   Rating: 4.7
   Features: 15.6-inch display, 16GB RAM, 512GB SSD, NVIDIA GeForce RTX 3060
   Description: A high-performance gaming laptop for an immersive experience.
   Price: $1199.99

3. Product: PowerLite Convertible
   Category: Computers and Laptops
   Brand: PowerLite
   Model Number: PL-CV300
   Warranty: 1 year
   Rating: 4.3
   Features: 14-inch touchscreen, 8GB RAM, 256GB SSD, 360-degree hinge
   Description: A versatile convertible laptop with a responsive touchscreen.
   Price: $699.99

4. Product: TechPro Desktop
   Category: Computers and Laptops
   Brand: TechPro
   Model Number: TP-DT500
   Warranty: 1 year
   Rating: 4.4
   Features: Intel Core i7 processor, 16GB RAM, 1TB HDD, NVIDIA GeForce GTX 1660
   Description: A powerful desktop computer for work and play.
   Price: $99.99

5. Product: BlueWave Chromebook
   Category: Computers and Laptops
   Brand: BlueWave
   Model Number: BW-CB100
   Warranty: 1 year
   Rating: 4.1
   Features: 11.6-inch display, 4GB RAM, 32GB eMMC, Chrome OS
   Description: A compact and affordable Chromebook for everyday tasks.
   Price: $1249.99

Step 3:{delimiter} If the message contains products \
in the list above, list any assumptions that the \
user is making in their \
message e.g. that Laptop X is bigger than \
Laptop Y, or that Laptop Z has a 2 year warranty.

Step 4:{delimiter}: If the user made any assumptions, \
figure out whether the assumption is true based on your \
product information. 

Step 5:{delimiter}: First, politely correct the \
customer's incorrect assumptions if applicable. \
Only mention or reference products in the list of \
5 available products, as these are the only 5 \
products that the store sells. \
Answer the customer in a friendly tone.

Use the following format:
Step 1:{delimiter} <step 1 reasoning>
Step 2:{delimiter} <step 2 reasoning>
Step 3:{delimiter} <step 3 reasoning>
Step 4:{delimiter} <step 4 reasoning>
Response to user:{delimiter} <response to customer>

Make sure to include {delimiter} to separate every step.
"""
The above still gives me the same response as before, no matter how many times i feed it. i also tried downloading the notebook locally and trying and the same result.

Result

Step 1:#### The user is asking a question about two specific products.
Step 2:#### The two products mentioned are the BlueWave Chromebook and the TechPro Desktop. The prices of these products are as follows:

  • BlueWave Chromebook: $249.99
  • TechPro Desktop: $999.99
    Step 3:#### The user is assuming that the BlueWave Chromebook is more expensive than the TechPro Desktop.
    Step 4:#### The assumption is incorrect. The TechPro Desktop is actually more expensive than the BlueWave Chromebook.
    Response to user:#### “The BlueWave Chromebook is actually less expensive than the TechPro Desktop. The Chromebook is priced at $249.99, while the TechPro Desktop is priced at $999.99.”
1 Like
2

Language models tend to have difficulties with logic.

2 Likes
3

Appreciate that,

so it should be stateless, everytime i call it with my data it should be evaluating over again?

Which really confuses me as to why it’s not. Could it be that the variables in my notebook are cached?

4

Hi @Ash_Minhas ,

Have you tried restarting your notebook kernel. After restarting, make the changes to the cell and then run it.

Note 1: Whenever we run a cell, variables get stored in memory. when we run another cell, the value of any reused variable is fetched from memory. If we restart the kernel, all variables are lost.

Note 2: GPT3.5 does not store any history of the chat (as far as I know). When we send any chat to the API, suppose we send some history and a new user input, then the model reads the whole conversation from the beginning to the end, from system message to the user message. Then it gives a response. It does not have any knowledge of anything it responded to in the past. openAI may have this information. But on the OpenAI playground it is written that API calls are not used to train the model. But sometimes some code runs (any code), other times it fails for reasons I do not know, maybe I will in future :slight_smile:

I tried the same thing as you mentioned in the question and this happened: (Model responded as it should)

chain of thought
chain of thought940×2752 174 KB

1 Like
5

Did you restart the kernel @snehil001 ?

1 Like
6

Hi @Ash_Minhas ,

**No, I did not restart it. I just ran what you mentioned in the question. It ran correctly. This means on a fresh start it will run smoothly. **

That response maybe because either:

  1. You changed the cell with the system message and forgot to run it. So, the system message actually did not change. It remained the previous one.

Or:

  1. You changed the cell with the system message, and ran it, but it did not run properly, due to traffic or internet issues or something else. It is a remote notebook. It can get stuck sometimes not executing the cells properly.

GPT model does not store anything of the past conversations. It needs context for every user input. (i think)

So, the system message must be cached on the notebook and not on openai server.

Hope this helps. Please dont hesitate to discuss more.