PixelRAG Uyasebenza

I-PixelRAG iwuhlaka lwe-RAG olugxile ekuboneni, olukhishwe njenge-open source yi-SkyLab/BAIR yase-Berkeley. Esikhundleni sendlela evamile yokukhipha amakhasi ewebhu abe umbhalo omsulwa bese kuwasesha, uhlaka lusebenzisa isiphequluli ukuguqula amakhasi ewebhu nama-PDF abe amathayela ezithombe-skrini, bese lisebenzisa ama-vector okubuyisa olwazi abonakalayo e-Qwen3-VL acushwe nge-LoRA, ukuze imodeli ifunde ngokuqondile amatafula, amashadi nokwakheka kwekhasi esithombeni, ixazulule inkinga yama-RAG avamile alahlekelwa amatafula nokwakheka kwamakhasi.

Uk-rendera amakhasi (pixelshot): I-Chromium engenasikrini isetshenziselwa uku-rendera amakhasi ewebhu nama-PDF abe amathayela ezithombe-skrini, kugcinwe ngokuphelele amatafula, amashadi nokwakheka kwekhasi; ku-Windows/macOS kubizwa ngokuzenzakalelayo i-Chrome yesistimu, futhi kusekelwa ukuxutshwa kwama-URL namafayela asendaweni.
Ukusesha kwe-semantic okubonakalayo: Izithombe-skrini zifakwa kuma-vector, kusekelwe ukusesha ngesithombe nangombhalo; ngokuzenzakalela kusetshenziswa i-FAISS yasendaweni, kanti ekusetshenzisweni okukhulu kungashintshelwa ku-Qdrant (isekela ukucindezelwa kwe-quantization, ukugcinwa kudiski nokwabelana phakathi kwezinsiza eziningi).
Ukufunda ngokuqondile kuma-pixel: Izithombe-skrini ezitholakele zifakwa ngokuqondile njengezithombe ku-VLM ukuze iphendule imibuzo, ngaphandle kokuguqulwa kube umbhalo kuyo yonke inqubo.
Insizakalo yokusesha ephethwe: I-API engenaso isidingo sokhiye kanye ne-demo yewebhu, enenkomba eyakhiwe kusengaphambili yamakhasi e-Wikipedia ayizigidi ezingu-8.28, ukuze ukusesha kuqale ngokushesha.
Ukwakha inkomba yakho: Iqoqo eliphelele lemiyalo ethi pixelrag index / embed / build-index / serve likuvumela ukwakha izizindalwazi zama-PDF namasayithi akho; i-Mac (Apple Silicon) ingasebenza ngosayizi omncane.
Ukuhlanganiswa ne-Agent: I-plugin ye-Claude Code ethi pixelbrowse ayidingi i-MCP, futhi ivumela u-Claude ukuthi “abuke” amakhasi ewebhu ngezithombe-skrini esikhundleni sokugwinya i-HTML engaphelele.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lezingxoxo lamaphrojekthi e-AI open source
Isendlalelo sokumelwa — idokhumenti iyisithombe: I-Chromium engenasikrini isetshenziselwa ukuthatha izithombe-skrini zamakhasi ewebhu, ukuguqula ama-PDF abe yizithombe ikhasi nekhasi, bese iwahlukanisa abe amathayela, ukuze ukuhleleka kwemigqa namakholomu yamathebula, amashadi, ama-infographic nezinye izakhi ezibonakalayo kugcinwe njengoba kunjalo.
Isendlalelo sokusesha — ama-embedding ama-vector abonakalayo: Kusetshenziswa i-Qwen3-VL-Embedding-2B, ecushwe nge-LoRA kusetshenziswa idatha yokufunda ngokuqhathanisa, ukuze izithombe-skrini ziseshwe nge-semantic; umbuzo uthola “leyo ngxenye yesithombe” kunokuthola “leso sigaba sombhalo”; ama-vector agcinwa ku-FAISS yasendaweni, kanti ekusetshenzisweni okukhulu kungashintshelwa ku-Qdrant.
Isendlalelo sokukhiqiza — i-VLM ifunda isithombe bese iphendula: Amathayela ezithombe-skrini atholakele adluliselwa ngokuqondile njengokufakwa kwama-pixel kumodeli yolimi nokubona, yona efunda izinombolo nokwakheka kwekhasi esithombeni ngaphandle kokuguqulwa kombhalo ophakathi kuyo yonke inqubo.
Ubukhulu nokusebenza kahle: Ipayipi lokuqala le-RAG elimboza yonke i-Wikipedia lisebenzisa izithombe-skrini; ukumelwa ngama-pixel kuletha nendlela entsha yokwandisa ukusebenza kahle — ukwehlisa ukulungiswa kwesithombe-skrini kunganciphisa izindleko zama-token kuze kube izikhathi ezingu-3 ngaphandle kokwehla kokunemba, futhi ekuhlolweni kweqa ngokuphelele isisekelo se-text RAG, ngokuthuthuka okufika ku-18.1%.

Ukufaka: Qalisa i-pip install pixelrag ukuze uthole amathuluzi ayisisekelo (kudingeka i-Python 3.10+).
Uk-rendera ikhasi: Qalisa i-pixelshot https://example.com -o ./tiles ukuze uguqule ikhasi lewebhu noma i-PDF libe amathayela ezithombe-skrini.
Ukuhlola ukusesha okuphethwe: Akukho ukucushwa okudingekayo; thumela ngokuqondile isicelo se-POST ku-https://api.pixelrag.ai/search ukuze useshe inkomba yamakhasi e-Wikipedia ayizigidi ezingu-8.28, noma uvule i-pixelrag.ai esipheqululini ukuze uyizame ku-inthanethi.
Ukwakha inkomba yakho: Bhala ukucushwa kwe-pixelrag.yaml (ucacise indlela yamadokhumenti, imodeli ye-embedding kanye nefolda yokukhipha), bese uqala i-pixelrag index build ukwakha inkomba. Ngemva kwalokho, sebenzisa i-pixelrag serve --index-dir ./my_index --port 30001 ukuze uqale insizakalo yokusesha (i-Mac Apple Silicon ingasebenza ngosayizi omncane; inkomba ephelele ye-Wikipedia cishe ingu-217GB futhi idinga amandla e-GPU).
Ukuxhuma ne-Claude Code: Ngemva kokuqalisa i-uv tool install pixelrag, faka i-plugin ye-pixelbrowse (claude plugin marketplace add StarTrail-org/PixelRAG kanye ne-claude plugin install pixelbrowse@pixelrag-plugins), ukuze u-Claude “abuke” amakhasi ewebhu ngezithombe-skrini ngaphandle kwe-MCP.
Ukuqeqeshwa okuthuthukile: Uma udinga ukuzakhela imodeli ye-embedding, ngena kufolda ye-train/ (iphrojekthi ezimele ye-uv) bese ulandela i-README ukuze wenze i-fine-tuning, noma usebenzise kabusha izisindo ze-LoRA nedatha yokuqeqesha ekhishwe njenge-open source ngokusemthethweni.

Ulwazi alulahleki: Izithombe-skrini zisetshenziswa esikhundleni sokukhipha umbhalo, kugcinwe ngokuphelele imigqa namakholomu yamathebula, amashadi nokwakheka kwekhasi, kuxazululwe ngokuphelele inkinga yama-RAG avamile abonakala enempendulo kodwa alahlekelwe ulwazi ngemva kokukhipha umbhalo.
Ukusebenza kungcono: Iphepha locwaningo libonisa ukuthi idlula ngokuphelele isisekelo se-text RAG; nasemisebenzini yombhalo omsulwa efana ne-NQ ne-SimpleQA isebenza kangcono, kanti kuma-benchmark e-multimodal nawe-Agentic ukuthuthuka kufika ku-18.1%.
Izindleko ziphansi: Ukumelwa ngama-pixel kuletha indlela ekhethekile yokwandisa ukusebenza kahle; ukwehlisa ukulungiswa kwesithombe-skrini kunganciphisa izindleko zama-token kuze kube izikhathi ezingu-3 ngaphandle kokwehla kokunemba.
Ipipeline ilula: Isusa izinqubo eziyinkimbinkimbi zokuhlaziya, ukuhlanza nokuhlukanisa i-HTML; i-rendera isithombe-skrini bese usifaka enkombeni, kunciphise kakhulu umsebenzi wobunjiniyela.
Ubukhulu buqinisekisiwe: Ipayipi lokuqala le-RAG elimboza yonke i-Wikipedia ngezithombe-skrini, elinenkomba ephethwe engenaso isidingo sokhiye yamakhasi ayizigidi ezingu-8.28 ukuze uyizame ngokushesha.

Iwebhusayithi esemthethweni yephrojekthi: https://pixelrag.ai/
Indawo ye-GitHub: https://github.com/StarTrail-org/PixelRAG
Iphepha lobuchwepheshe le-arXiv: https://arxiv.org/pdf/2606.28344

Imibuzo nezimpendulo ngemibiko yezezimali namatafula edatha: Ifunda ngokuqondile izinombolo emakhasini emibiko yezezimali nasematafuleni ezibalo, igweme ukushintsha kokuhleleka kwemigqa namakholomu okubangelwa ukuhlaziya umbhalo.
Ukusesha ama-dashboard namashadi: Imiphumela yokusesha ihlanganisa izindawo zezithombe-skrini ezinamashadi namaphaneli e-KPI, ukuze imodeli ifunde isithombe bese iphendula.
Ama-infographic namadokhumenti anezincazelo ezibonakalayo: Okuqukethwe okufana nama-infographic namashadi okugeleza, okungakwazi ukumbozwa yizinkomba zombhalo ezivamile, kungaseshwa.
Amakhasi emikhiqizo anezinhlelo zamakholomu amaningi: Amakhasi okuqhathanisa imikhiqizo namakhasi ezindaba anokwakheka okuyinkimbinkimbi agcina isakhiwo sawo sokuqala lapho eseshwa.
Ukufunda amakhasi ewebhu nge-Agent: I-plugin ye-pixelbrowse ye-Claude Code ivumela i-Agent ukuthi “ibuke” amakhasi ewebhu ngezithombe-skrini esikhundleni sokugwinya i-HTML engaphelele.