Human like Memory

AI Discussions
1

Hi,

I wanted to ask a fundamental question. Humans, except savants, cannot entirely reproduce a large data (essay, book, article etc.) by heart. Instead, we have the tendency to reframe and make it short, more a like summary, and produce it.

AI machine on the hand, it can reproduce it as a whole. Instead what if we create a LLM that stores only the summary or the gist of the data with losing meaning of the data, so that you will not have more token in the corpus. That way, we can have a human like memory, just the important information will be stored.

How do you do that? Maybe, every time an answer is produced, an agent can summarize it plus original data and store only the modified data (summary of answer + original data).

What are your thoughts on this? Can we make it better?

Edit: In the implementation of LangChain sort of a thread like architecture, we keep the history of the conversation. The history is kept till either the number of tokens runs out or the history is truncated. Though we don’t not know how the model works and translated into, we can make you of trick like these. But I don’t know, how it will translate to.

2

as far as I know we don’t know yet how exactly we pass from abstractions as mere synaptic impulses with a voltage range to a concept.

In the case of deep learning some models already make information compression in the input and also the weights of then neurons or convolutions for example synthesize the spatial/dimensional information creating a model/equation that represent a group of possibilities of occurrence for a phenomena that allow us to generalize and predict an outcome but even so we don’t know yet with precision and without of enough examples how to extrapolate those abstractions generally.

LeCune I think propose kind of a temporal self processing past autgenerated outputs searching for this concept.

3

Yes! I agree. In the implementation of LangChain sort of a thread like architecture, we keep the history of the conversation. The history is kept till either the number of tokens runs out or the history is truncated. Though we don’t not know how the model works and translated into, we can make you of trick like these. But I don’t know, how it will translate to.