MultiHashFormer Represents Language Model Tokens With Multiple Hashes
University of Sheffield researchers introduced MultiHashFormer in arXiv:2606.28057. Independent hash functions produce a signature for each token. An encoder compresses that signature into a latent vector, and a Transformer decoder predicts the next token's signature. The researchers reported consistent gains over standard Transformer language models on multiple benchmarks at 100M, 1B, and 3B parameters. The report did not specify weight availability or a commercial license.
Visit original source