So.. I've been running my own archive of conversations, voice recordings, documents, and etc for AI agents (mostly Claude Code) for past 6 month.
Currently, when i ask or it answers something it runs hook that checks previous conversations, lists up potential related conversations.
The intention is not only to retreive relative memories, but also to be able to connect 2 or more conversations that looks irrelevant but extract some kind of pattern or insight from it. In other words, I want my model to partially go off on a tangent, in a good way...
For example; Let's say I bring up topic on investing on OKLO as a potential 10x company. So I say I will keep it small but don't cut loss, only buy more from time to time. I want model to be able to bring up the memory that I also regularly buy things from my few favorite small restuarants just to keep them alive. Maybe connect those 2, and be able to tell that in perspective, I may have a consistant view on spending money. I even have few conversations on that exact topic as well. It just doesn't use similar keywords.
I feel like maybe the obstacle is that model can't come up with the direction that the conversation could develop to a broader(or deeper) topic on my philosophy on capital? I'm not sure.
These are the list I've tried: 1. Personalized PageRank 2. Hebbian edge reinforcement(between opened conversation) 3. character n-gram TF-IDF similarity 4.contrastive embedding trained on my conversation 5. query expansion over a word co-occurrence graph 6. matching on shared verbs 7. serendipity slot 8. simple agentic search 9. abstracting each message into a domain-free sentence 10. keword search + LLM judgement 11. BGE-M3
All these can't quite achieve to the point I describe. It certainly works well getting related memory.
The search should be fast, light, because reading the history itself takes time, so can't really afford long search time on top of that. I run this on multi device, so it should be portable friendly.
Any advice or idea on this approach? public version of the repo: https://github.com/milez770/madeleine
Comments URL: https://news.ycombinator.com/item?id=50361973
Points: 1
# Comments: 0
