SenseNova releases SenseNova-Loop-9B-Preview with shared-weight layers
The Apache 2.0 model accepts text and images, generates text, and runs 24 decoder layers twice with shared weights, alongside two-stream manifold-constrained hyper-connections (mHC). The model card describes 32 unique decoder layers, 56 layer operations, and training configurations of up to 262,144 tokens.
Visit original source