Paul Jorion's Blog

Tag: CHE (Contextual Hyper-Embedding)

  • Comparison between Google S2R, Pribor’s Combinatorial Magic and Pribor’s CHE (Contextual Hyper-Embedding)

    Comparison between Google S2R, Pribor’s Combinatorial Magic and Pribor’s CHE (Contextual Hyper-Embedding) This document presents the characteristics, divergences and synergies between three approaches: Google S2R, Pribor’s Combinatorial Magic and Pribor’s CHE (Contextual Hyper-Embedding). 1. Google’s S2R * “S2R” means Speech-to-Retrieval. It is a recent voice search architecture that Google is deploying, which bypasses the explicit…

  • PRIBOR: CHE — Contextual Hyper-Embedding (uint8)

    A more economical alternative to classical attention CHE (Contextual Hyper-Embedding, uint8) offers a radical gain in efficiency compared with the standard attention mechanisms of large language models. Similar methods have appeared in recent research, but none reach CHE’s level of economy. 1 · Memory efficiency Standard attention: float16 / float32 matrices → 700 to 4000…