Huawei announced today that it has officially open-sourced and launched the 505-billion-parameter Pangu openPangu-2.0-Pro model (model weights and basic inference code), along with its technical report.
openPangu is Huawei's open-source AI model brand, dedicated to providing the industry with best-practice references for making effective use of Ascend through native Ascend training and inference technologies.
openPangu-2.0-Pro is a large-scale mixture-of-experts (MoE) language model trained on Ascend NPUs, with approximately 505B parameters and approximately 18B activated parameters per token. The model supports a context length of 512k and was trained on approximately 34T tokens of data. During the post-training stage, it underwent unified fast-and-slow fine-tuning (SFT) and multiple specialized reinforcement learning (RL) processes, and achieved unified capabilities through online policy distillation (OPD).
openPangu-2.0-Pro has received a comprehensive architectural upgrade:
Attention architecture: It retains the efficient MLA and adopts an independently layered DSA+SWA hybrid architecture, with a layer ratio of 1:2. The SWA layers handle local-window modeling, while the DSA layers perform sparse global aggregation, significantly reducing the computational, memory, and memory-access overhead of long-sequence inference while maintaining accuracy.
Topology architecture: Traditional residual connections have been upgraded to a four-stream mHC architecture, improving representational diversity and generalization capabilities.
Self-speculative module: A three-head MTP architecture is used to predict three additional tokens at a time, significantly improving the model's inference speed.
Muon optimizer: The Muon optimizer is used during training to achieve faster convergence.
In the keynote speech at Huawei Developer Conference HDC 2026 in June this year, Yu Chengdong, Huawei's executive director, chairman of the Product Investment Review Board, and chairman of the Terminal BG, announced that the openPangu 2.0 project would gradually open-source seven major components starting June 30, including the newly open-sourced pre-training code, post-training code, and training operators.
On June 30, the 92B-parameter openPangu-2.0-Flash model was officially open-sourced and launched.
Regarding the total of only 505B parameters in openPangu-2.0-Pro, Yu Chengdong previously explained that substantial computing power has been supporting the needs of other domestic companies, leaving Huawei with a very limited amount for its own use. In addition, AI computing costs are extremely high, and Huawei is focusing more on improving latency and throughput.
Components related to the openPangu-2.0 model have been gradually released on open-source platforms. The open-source repository is:
https://ai.gitcode.com/ascend-tribe/openPangu-2.0-Pro