AI & ML
Peeling the Matryoshka: Extracting Ontological Structure from Retrieval Embeddings
arXiv cs.IR0 views
arXiv:2610.41785v1 Announce Type: cross Abstract: Taxonomy construction is essential for knowledge-intensive NLP yet remains costly: clustering-based methods (i) need careful per-corpus tuning and yield unstable hierarchies, while LLM-based pipelines (ii) depend on expensive task-specific training that does not transfer across domains. We report an empirical finding that bears on both: in Matryoshka Representation Learning (MRL) embeddings, where each leading prefix is itself a usable, coarser vector, prefix length aligns with the coarse-to-fine granularity of gold taxonomies; shuffled- and reverse-dimension controls isolate the alignment to the learned early prefix dimensions. The pattern holds across three architecturally distinct MRL models. To test whether this regularity is usable, we introduce PEEL (Peeling Embeddings to Extract Latent ontologies): a minimal probe over these frozen embeddings, trained once with an order-embedding objective that represents hierarchy as vector containment; a minimum spanning arborescence then decodes taxonomic DAGs. Using only a concept vocabulary (no documents), PEEL recovers taxonomic DAGs with high structural fidelity on two OLLM benchmarks, transferring to a new domain with no target-edge supervision. The end-to-end OLLM pipeline, which jointly discovers concepts and induces structure from document corpora, is reported as a cross-task reference. Pre-trained retrieval embeddings encode a decodable granularity gradient; a minimal probe renders it as explicit parent-child (is-a) structure.
Source:https://arxiv.org/abs/2610.41785PreviousEnhancing Knowledge Entity Relation Extraction in Abstracts and Full-text Content of Academic Articles Using Large Language ModelsNextTime Will Tell: Harness Time to Denoise Collaborative Filtering
Related posts
Camera Shutter Count on Used Bodies: What the Number Means and What a Sample File Can Leak
0dev.to Beginners
Survivorship bias: testing on today's winners flatters every strategy
0dev.to Beginners
I checked all 40 outbound links on our site, and the dead one is inside our privacy policy
0dev.to Testing
Definition of Ready vs Done: The Two Checklists That Stop Half‑Baked Stories From Entering or Leaving a Sprint
0dev.to Testing
Hydra · Coeziune OM–IA — the complete DOI registry (2026-11-05): 20 published works
0dev.to OpenSource
Pamyo: a séance for your codebase, where a local Gemma 4 may only quote the developers who left
0dev.to Git