RealFeedApp ERNIE-5.5-Flash-Base: 280B total, 14B active per token
The sparse MoE model features 14 billion activated parameters per token and supports a 1 million-token context for fine-tuning.
- Field
- training language models
- What they did
- Baidu released the base version of the language model ERNIE 5.5 Flash Base, which has 280 billion parameters and was trained from scratch on 33.5 trillion tokens.
- Why it matters
- This model provides developers with a ready starting point for further training, fine-tuning, and research in various domains without the need to start training from scratch.
#ernie#baidu#llm#mixture-of-experts#sparse-moe#context-window
