I-Ant Group namuhla ivule umthombo wemodeli ye-Ling-3.0-tiny ku-Hugging Face. Le modeli iyisakhiwo esilula se-MoE sokucabanga esixubile, esinamapharamitha angu-7.9B esewonke, kanti i-Token ngayinye ivula amapharamitha angu-1.3B. Ithimba elisemthethweni linikeza izinguqulo zezisindo ze-BF16, FP8 ne-INT4 ukuze zihambisane nezidingo zokufakwa ezinkundleni ezihlukahlukene.
Ngokwethulwa kwayo, i-Ling-3.0-tiny igxile ekucabangeni ngezindleko eziphansi. Isebenzisa isakhiwo esisebenza kahle esixubile somugqa, ishintshanisa i-KDA (Kimi Delta Attention) ne-MLA (Multi-Head Latent Attention, ukunaka okufihlekile okunamakhanda amaningi) ngesilinganiso esingu-3:1. Isebenzisa inethiwekhi ye-FFN ye-sparse MoE enochwepheshe abangu-128 abaqondiswayo (routed experts), ukuze ilinganise ikhono lokucubungula umongo nezindleko zokubala.
Le modeli isekela imodi yokuphendula ngokushesha nemodi yokucabanga enezinyathelo eziningi, futhi yakhelwe ukufakwa endaweni. Ithimba elisemthethweni seliyiqinisekisile kumakhompyutha afana ne-NVIDIA DGX Spark, i-Apple Silicon MacBook ne-Mac mini. Ku-MacBook ene-M4 Pro, ingafinyelela esivinini sokucabanga esingaba ngu-86-90 Token/s. Ngobude bomongo obungu-8K, ukusetshenziswa kwememori okuphezulu kungaba ngu-8.34 GiB.
Inkomba:
inclusionAI/Ling-3.0-tiny · Hugging Face