One shot/multi prompt engineering intuition

I’m trying to build up intuition on how providing one shot examples helps make better completions.

If the LLM is just predicting the next word successively. When you provide in context examples, it will shift the activations (or activate additional neurons) at inference time. This creates an example that more closely statistically aligns as an answer to the augmented prompt, given the LLM’s universe of stored knowledge.

Is that about right?

I think this is more about probability used to select one of the outputs of the LLM model rather than change the weights or activate more neurons! This prompt engineering is not an additional training process of the LLM model.

I understand that in-context learning does not change the weights.

Is it however, effectively shifting the activations to different parts of the network?

The input has changed, so it should right?

My thought is, that the augmented prompt changes the input such that it better activates a part of the LLM that represents knowledge from some of the billions of training examples that look most similar to the desired completion.