Qalisa i-Naive-N0.5-Flash endaweni yangakini usebenzisa i-Transformers

Yini i-Naive-N0.5-Flash?

I-Naive-N0.5-Flash iyimodeli ye-Mixture-of-Experts enesisindo esivulekile, eyakhelwe ukubhala ikhodi kanye nocwaningo nentuthuko ye-AI. Iqukethe amapharamitha angu-309 billion esewonke, kanti angu-15.5 billion awo ayasebenza, futhi isekela iwindi lomongo lendabuko elingafika kuma-token ayizigidi ezingu-1.

Izisindo zemodeli nekhodi yokwenza inference zikhishwe ngaphansi kwelayisensi ye-MIT. Ungayithola imodeli ku-Hugging Face, kanti iphrojekthi yomthombo iku-GitHub.

Izici Ezisemqoka

• Umongo wama-token ayizigidi ezingu-1: Imodeli isebenzisa inhlanganisela ye-Sliding-Window Attention (SWA) ne-DeepSeek Sparse Attention (DSA), ngaphandle kwezingqimba ze-full attention.
• Ukusebenza kahle kwe-MoE: Nakuba imodeli inamapharamitha angu-309B esewonke, angu-15.5B kuphela asebenzayo ku-token ngayinye.
• Igxile ekubhaleni ikhodi nase-R&D ye-AI: Imodeli yaqeqeshelwa ukuthuthukisa amakhono obunjiniyela besoftware, ocwaningweni, nasekuthuthukiseni amasistimu.
• Inference ye-FP8: I-checkpoint ekhishiwe isekela i-inference enembe ngokuxubile kuma-GPU e-NVIDIA akwazi i-FP8.
• I-inference esebenza ngesivinini esikhulu: Uhlelo lwe-inference lwe-NaiveRT luhlanganisa i-mega-kernel fusion, i-Programmatic Dependent Launch, kanye ne-speculative decoding. I-README ibika ukuthi kufinyelelwa kuma-token angu-50 ngomzuzwana ngomsebenzisi ngamunye kumodi ye-Standard, kanye nama-token afika ku-2,000 ngomzuzwana kumodi ye-Ultrafast.

Qonda Idizayini Ye-Hybrid Attention

I-Naive-N0.5-Flash isebenzisa izingqimba ze-transformer ezingu-48: izingqimba ze-SWA ezingu-39 nezingqimba ze-DSA ezingu-9. I-SWA ikhawulela i-local attention ewindini lama-token angu-128, kanti i-DSA isebenzisa i-indexer ukukala wonke umlando bese ikhetha ama-token angu-2,048 aphezulu ukuze asetshenziswe yi-backbone attention.

Imodeli ihlelwe yaba amamojula ayisishiyagalombili, ngalinye linezingqimba eziyisithupha. Imojula evamile iqukethe izingqimba ezinhlanu ze-SWA ezilandelwa ungqimba olulodwa lwe-DSA, kanti ungqimba lokuqala lwemojula yokuqala nalo luthathelwe indawo yi-DSA. I-DSA isebenzisa i-grouped-query attention enamaqembu amane e-key-value, eyaziwa nge-GQA4, kanti i-indexer yayo inamakhanda e-query angu-16.

Le dizayini igcina iningi lokubala kwe-attention lisendaweni, kuyilapho ivumela ulwazi olukhethiwe oluvela kumongo omude ukuthi luthinte ukukhiqizwa. I-cache ephelele ye-key-value iyagcinwa, kodwa i-backbone inaka kuphela ama-token e-DSA akhethiwe.

Izidingo Zehadiwe Nokufakwa

I-Naive-N0.5-Flash idinga ama-GPU e-NVIDIA akwazi i-FP8. Izisindo zemodeli zithatha cishe u-315 GB, futhi kufanele uvumele inkumbulo eyengeziwe ye-GPU ukuze kwenziwe i-inference.

I-README inikeza isiqalo esisheshayo esisekelwe ku-Transformers. Faka inguqulo yakamuva ye-Transformers ene-PyTorch nama-kernel adingekayo:

pip install "transformers[torch,kernels]>=5.17.0"

Qalisa Imodeli nge-Transformers

Ngemva kokufaka okuncikile nokuqinisekisa ukuthi izinsiza ze-GPU ezidingekayo ziyatholakala, layisha i-checkpoint ye-FP8 ku-Hugging Face. Isibonelo esilandelayo sakha i-tokenizer, silayisha imodeli, sihlela umlayezo womsebenzisi, bese sikhiqiza impendulo:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NaiveAI/Naive-N0.5-Flash-FP8"
tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello!"}],
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=2048,
)
response = tokenizer.decode(
    output[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(response)

Indlela Isibonelo Esisebenza Ngayo

1. Khetha i-checkpoint: NaiveAI/Naive-N0.5-Flash-FP8 ikhomba imodeli ye-FP8 eku-Hugging Face.
2. Layisha i-tokenizer: trust_remote_code=True inika amandla ikhodi yangokwezifiso ye-Transformers ekugcinwa kwemodeli.
3. Layisha imodeli: dtype="auto" ivumela i-Transformers ukuthi isebenzise uhlobo lwedatha olufanele lwe-checkpoint, kanti device_map="auto" ibeka izingxenye zemodeli kumadivayisi atholakalayo.
4. Hlela ingxoxo: apply_chat_template iguqula umlayezo womsebenzisi ube yifomethi elindelwe imodeli futhi yengeze i-generation prompt.
5. Khiqiza umbhalo: max_new_tokens=2048 ikhawulela inani lama-token amasha azokhiqizwa.
6. Decode impendulo kuphela: Ingxenye eqala ngemva kobude bokufaka isusa i-prompt yoqobo ngaphambi kokuphrinta umbhalo okhiqiziwe.

Amasethingi Okukhiqiza Anconyiwe

Ekusetshenzisweni okuvamile, iphrojekthi incoma ukusetha temperature=1.0 kanye no-top_p=0.95. Lawa magugu awafakiwe esibonelweni esincane se-README, kodwa ungawadlulisela ku-generate uma ufuna ukwenza indlela yokukhetha ibe sobala:

output = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=1.0,
    top_p=0.95,
)

Sebenzisa inani elifanele le-max_new_tokens emsebenzini wakho. Imikhawulo emikhulu ingakhiqiza izimpendulo ezinde futhi idinge isikhathi esengeziwe sokukhiqiza nenkumbulo eyengeziwe.

Okucatshangelwayo Ngokusetshenziswa Okuthuthukile

Sebenzisa Umongo Omude Ngendlela Ehlosekile

Iwindi lomongo lama-token ayizigidi ezingu-1 lisekelwa ngokwemvelo ngesakhiwo esihlanganisa i-SWA ne-DSA. Kumakhowudi amakhulu, izinto zocwaningo, noma imisebenzi enamafayela amaningi, nikeza umongo ofanele ngendlela ehlelekile futhi ugcine imiyalelo icacile. Lesi sakhiwo siklanyelwe ukugcina imininingwane yasendaweni nge-SWA kuyilapho sikhetha umlando obalulekile nge-DSA.

Hlela Umthamo We-GPU Ngaphambi Kokulayisha

I-checkpoint ithatha cishe u-315 GB ngaphambi kokubala inkumbulo eyengeziwe edingekayo ngesikhathi sokwenza inference. Qinisekisa ukuthi ukusethwa kwe-NVIDIA GPU yakho esekela i-FP8 kunomthamo owanele ngaphambi kokuqala imodeli. Ukubekwa okuzenzakalelayo kwamadivayisi nge-device_map="auto" kungasabalalisa imodeli kumadivayisi atholakalayo, kodwa akunciphisi inkumbulo ephelele edingwa yi-checkpoint kanye ne-runtime.

Cabangela i-Inference Elungiselelwe

I-README ichaza i-NaiveRT njengohlelo lwephrojekthi lwe-inference olulungiselelwe. Amasu alo ahlanganisa ukuhlanganiswa kwama-mega-kernel, i-Programmatic Dependent Launch, kanye ne-speculative decoding. Uma udinga umthamo omkhulu wokucubungula odlula ukusethwa okuyisisekelo kwe-Transformers, bheka ibhulogi yezobuchwepheshe yephrojekthi ukuze ufunde nge-case study ye-NaiveRT nemininingwane yokuyisebenzisa.

Buyekeza Umongo Wezilinganiso Zokusebenza

Ukuhlolwa okushicilelwe kuhlanganisa ukubhala ikhodi, ubunjiniyela besoftware, imisebenzi esekelwa ama-agent, ucwaningo lwe-AI, kanye nokwenza kahle kwezinhlelo. Imiphumela ngokuvamile yahlolwa kusetshenziswa i-Claude Code 2.1.207, iwindi lomongo elingu-1M-token, i-temperature engu-1.0, i-top-p engu-0.95, namathuluzi ayisisekelo e-file I/O ne-Bash ngaphandle kwalapho kuphawulwe ngenye indlela. Phatha ukuqhathaniswa kwezilinganiso zokusebenza njengokuncike kuma-harness nasezinhlelweni zokuhlola ezishiwo.

Ilayisensi Nezinsiza Ezengeziwe

I-Naive-N0.5-Flash ikhishwa ngaphansi kwe-MIT License. Ukuze uthole ukukhishwa kwamamodeli namaphrojekthi ahlobene, vakashela inhlangano ye-NaiveAI ku-GitHub noma iphrofayela ye-NaiveAI ku-Hugging Face. Ulwazi lobuchwepheshe mayelana nesakhiwo, inqubo yokuqeqesha, kanye nokwenza kahle kwe-inference luyatholakala kubhulogi yezobuchwepheshe yephrojekthi.

Isiphetho

I-Naive-N0.5-Flash ihlanganisa isakhiwo se-MoE esinama-309B parameters nama-15.5B active parameters, i-hybrid SWA–DSA attention, kanye newindi lomongo elingu-1M-token elakhelwe ngaphakathi. Ukuze uyizame endaweni yangakini, faka i-Transformers nama-kernel adingekayo, sebenzisa i-NVIDIA hardware esekela i-FP8 enesikhala esingaba ngu-315 GB sama-weight, layisha i-Hugging Face checkpoint, bese ukhiqiza izimpendulo usebenzisa izilungiselelo ezinconyiwe zokusampula.