Indlela Yokunikeza Ama-ejenti Okubhala Ikhodi Ukuhlaziya Umsindo Nokwakha Umsindo nge-Earshot

Lokho Okwenziwa yi-Earshot

I-Earshot iyithuluzi lendawo lokuhlaziya umsindo nokukhiqiza imisindo, elakhelwe ama-ejenti okubhala ikhodi e-AI. Iguqula umsindo ube ubufakazi obuhlelekile: izincazelo ngolimi olujwayelekile, amaphuzu afanisa alinganisiwe, imicimbi enezikhathi, izilinganiso eziqondile, amashidi omsindo abonakalayo, nokuhlaziywa kwamafilimu okwethulwa komkhiqizo.

Ingaphendula imibuzo efana nokuthi umphumela uzwakala yini njengokucindezelwa okujulile kokhiye womshini, ukuthi umcimbi othile wenzeka kuphi ekurekhodweni okude, ukuthi imisindo emibili engakhethwa ihluke kanjani, noma ukuthi ingoma ihambisana yini nokusikeka kwesithombe. Ingakhiqiza nemiphumela yomsindo ngekhodi kusetshenziswa i-earshot forge.

I-Earshot isebenza endaweni ku-CPU futhi itholakala ngesixhumi somugqa womyalo, i-API ye-Python, kanye neseva ye-MCP yamathuluzi afana ne-Claude Code. Ngemva kokulandwa kwemodeli kokuqala, izici zayo ezijwayelekile zokuhlaziya zingasebenza ngaphandle kwe-inthanethi.

Amandla Abalulekile

• Chaza imisindo: i-describe ibuyisa amathegi alinganisiwe, izincazelo ezihlukile, amalebula e-AudioSet, kanye nezilinganiso.
• Qondanisa futhi uhlele abangenele: i-match ihlola ukuthi ifayela elilodwa lihambisana kanjani nencazelo, kanti i-rank ihlela amafayela amaningi ngokuya ngombuzo.
• Thola imicimbi: i-find ikhomba izikhathi ezihambisana kakhulu ekurekhodweni okude futhi ibike izikhathi zokuqala nezokugcina ezilungele ukunqunywa.
• Qhathanisa umsindo: i-compare isesha okurekhodiwe okuyizithenjwa, kanti i-ab ichaza umehluko phakathi kwemisindo emibili noma izingxenye ezisikiwe.
• Hlola ubungokoqobo: i-realness ilinganisela ukuthi umsindo ufana yini nokurekhodwa kwangempela noma okukhiqizwe ngobungcweti, kunomsindo ohlanganiswe ngekhodi.
• Hlaziya izingoma: i-timeline ikhomba imicimbi, izigaba, amazinga, ama-beat, nokuvumelanisa nokusikeka kwevidiyo noma amafreyimu ezinkomba.
• Dala imibiko ebonakalayo: i-sheet ikhiqiza i-PNG enezichasiselo equkethe amafreyimu evidiyo, imicimbi, izigaba, i-waveform, i-spectrogram, umsindo, nezimpawu zokuvumelanisa.
• Hlaziya amafilimu okwethulwa: i-critique iqhathanisa ifilimu nephrofayela eyakhiwe ngamafilimu ayisikhombisa okwethulwa akhiqizwe ngobungcweti futhi iphakamise ukulungiswa okunamazwibela esikhathi.
• Yenza izilinganiso eziqondile: i-measure ibika izici ezifana ne-LUFS, i-true peak, ububanzi bomsindo, i-envelope, i-spectrum, i-tempo, kanye ne-key ngaphandle kokusebenzisa amamodeli.
• Hlanganisa imiphumela: i-forge inikezela imisindo ngekhodi futhi isesha amapharamitha eresiphi isebenzisa abahlaziyi be-Earshot nezilinganiso ezingasebenzisi amamodeli.

Faka i-Earshot

Izidingo Zokuqala

Udinga i-Python 3.10 noma entsha, kanye ne-ffmpeg ne-ffprobe zombili zitholakale endleleni yesistimu yakho. I-Earshot isebenzisa i-CPU kuphela futhi ayisebenzisi i-GPU.

Dala indawo

git clone https://github.com/Leonxlnx/earshot
cd earshot
python -m venv .venv
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install -e ".[mcp,llm,dev]"
.venv/bin/python -m earshot doctor --load

Kwi-Windows, shintsha i-.venv/bin/python ibe yi-.venv/Scripts/python.

Umsebenzi wokuqala osebenzisa imodeli ulanda cishe u-1.4 GB uye ku-.cache/hf ngaphakathi kwereposithri, ngaphandle uma ulungiselela i-EARSHOT_HOME. I-Earshot isebenzisa imodeli ye-CLAP laion/larger_clap_general kuma-embedding ombhalo nomsindo, kanye ne-MIT/ast-finetuned-audioset-10-10-0.4593 ekufakeni amathegi e-AudioSet.

Qalisa umyalo wokuxilonga noma nini lapho udinga ukuqinisekisa okuncikile, amafayela edatha, amamodeli, ukunemba, noma izilungiselelo ze-CPU:

.venv/bin/python -m earshot doctor --load

Lungisa izilungiselelo ze-CPU ne-cache

• I-EARSHOT_THREADS ilawula inani lama-thread e-Torch CPU.
• I-EARSHOT_HOME ishintsha impande ye-cache.
• I-EARSHOT_PRECISION=fp32 iphoqa i-FP32. Okuzenzakalelayo yi-BF16 kuma-CPU anokusekelwa kwendabuko kuphela, bese kuba yi-FP32 kwenye indawo.
• I-EARSHOT_OFFLINE=1 inika amandla ukusebenza ngaphandle kwe-inthanethi ngemva kokuba amamodeli eselandiwe.

Ukusetshenziswa Okuyisisekelo Komugqa Womyalo

Chaza umsindo

Qala nge-describe uma ufuna incazelo emfushane yefayela ongalijwayele:

earshot describe enter.ogg

Isifinyezo singafaka ubude besikhathi, ukuma kwe-transient, imikhakha yomsindo engenzeka, ama-z-score alinganisiwe, izinga le-peak, ukukhanya, kanye nezincazelo ezihlukile. Faka i---json ukuze uthole umphumela ophelele ohlelekile noma i---out result.json ukuze uwugcine.

Hlola incazelo ethile

Sebenzisa i-match uma usuwazi umsindo owufunayo. Izincazelo ezihlukile zisiza ukuhlukanisa okuhlosiwe kokungadidaniswa nakho:

earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch keyboard" "a mouse click"

Umphumela uhlanganisa ukufana kwe-cosine, i-z-score elinganiselwe ngesilulumagama, kanye nesinqumo esifana ne-strong, plausible, weak, noma no match.

Hlela amafayela amaningi angakhethwa

Ukuze ukhethe okufanele kakhulu eqoqweni, dlulisa incazelo ulandelise ngezindlela zamafayela:

earshot rank "a fast crisp air whoosh" sounds/one.wav sounds/two.wav sounds/three.wav

Lokhu kuyasiza uma i-ejenti ilande noma ikhiqize imiphumela eminingi engaba khona futhi idinga ukuhleleka okusekelwe ebufakazini.

Thola umcimbi ngaphakathi kokurekhodwa okude

Umyalo we-find usesha ngaphakathi kokurekhodiwe futhi ubuyise izikhathi zokusika amasampuli:

earshot find keys.ogg "a single mechanical key press" --top 4

Umphumela ngamunye uqukethe isikhathi sokuqala, isikhathi sokuphela, ubude besikhathi, amaphuzu, kanye nesinqumo. Lezi zitempampu zesikhathi zingasetshenziswa ukukhipha umcimbi ohlanzekile emthonjeni omkhulu.

Linganisa ngaphandle kokuhumusha kwemodeli

Sebenzisa measure uma udinga izakhiwo zomsindo eziphindaphindekayo esikhundleni sezahlulelo ze-semantic:

earshot measure soundtrack.wav --json

Lokhu kubika ulwazi oluqondile oluhlanganisa ukuphakama komsindo, i-true peak, ububanzi bokuphakama komsindo, i-envelope, i-spectrum, i-tempo, kanye ne-key.

Hlaziya i-Soundtrack Yefilimu

Dala umugqa wesikhathi wokulalela

Sebenzisa timeline ukuze udale umbhalo wokulalela wesekhondi nesekhondi. Ukunikeza ividiyo kuvumela i-Earshot ukuthi ihlole ukuvumelanisa nokusikeka kwezithombe kanye namafreyimu ezinkomba:

earshot timeline soundtrack.wav --video film.mp4 --json

Umphumela ungafaka imicimbi etholiwe, izigaba, amazinga, ukuphakama komsindo, igridi yamabhithi, kanye nobufakazi bokuvumelanisa komsindo nesithombe.

Dala ishidi lomsindo elinezichasiselo

Ishidi lomsindo lihlanganisa izingqimba eziyinhloko zokuhlaziya zibe isithombe esisodwa esingahlolwa i-ejenti ebonakalayo:

earshot sheet soundtrack.wav --video film.mp4 -o out.png

I-PNG ekhiqiziwe ingahlanganisa amafreyimu amelelayo, imicimbi ebhalwe amalebula, izigaba, i-waveform, i-spectrogram, ukuphakama komsindo, amabhithi, kanye nezimpawu zokuvumelanisa.

Khiqiza ukugxekwa kwefilimu yokwethulwa

Qalisa i-critique ukuze uqhathanise ifilimu nephrofayela efakiwe ye-Earshot yamafilimu ayisikhombisa ochwepheshe okwethulwa:

earshot critique film.mp4

Umbiko unikeza isinqumo, ukhomba amamethrikhi angaphakathi noma angaphandle kobubanzi bephrofayela, futhi ubeka phambili ukulungiswa okunamathempampu esikhathi. Kubheke lokhu njengokuqhathanisa nesitayela esithile sefilimu yokwethulwa, hhayi njengesilinganiso somhlaba wonke sekhwalithi.

Ungakha futhi iphrofayela ngamafilimu owathandayo:

earshot profile reference-one.mp4 reference-two.mp4 reference-three.mp4 -o custom-profile.json

Sebenzisa i-Earshot nge-Python

Isixhumi esibonakalayo se-Python sigwema ukuqalisa inqubo entsha ekusebenzeni ngakunye. Lokhu kuyasiza ngoba izingcingo ze-CLI ngazinye zilayisha amamodeli futhi zingathatha cishe imizuzwana engu-20 kuya kwengu-40 ngaphambi kokubuya.

import earshot

earshot.describe("whoosh.wav")
earshot.find("typing.ogg", "a single isolated key press")
earshot.timeline("soundtrack.wav", video="film.mp4")

from earshot import api
api.rank("a crisp UI click", ["click-a.wav", "click-b.wav"])
api.critique("film.mp4")

I-namespace ye-earshot.api iveza imisebenzi ehlanganisa ukukala amazinga, ukugxeka, kanye nokuhlaziya ubunjalo bamafayela amaningi. Ukugcina lezi zingcingo kunqubo eyodwa ye-Python kuvumela ukuthi amamodeli alayishiwe aphinde asetshenziswe.

Qalisa i-Earshot Njengeseva ye-MCP

Kumenzeli ozokwenza izicelo eziphindaphindiwe, iseva ye-MCP ingcono kunezingcingo ze-CLI ezihlukene ngoba amamodeli ahlala elungele ukusetshenziswa ngemva kwesicelo sokuqala.

claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp

Kwelinye iklayenti elihambisana ne-MCP, engeza izilungiselelo ezifanayo ku-.mcp.json yalo:

{"mcpServers":{"earshot":{"command":"/path/to/earshot/.venv/bin/python","args":["-m","earshot","mcp"],"env":{"EARSHOT_THREADS":"3"}}}}

Emshinini ohamba kancane, qala i-Claude Code nge-MCP_TIMEOUT=60000. Ithuluzi lokuzikhethela le-llm_critique lifunda GEMINI_API_KEY endaweni. Ungafaki ukhiye wangempela we-API efayeleni lokumisa.

Khiqiza Imisindo nge-Earshot Forge

I-Forge idala imisindo ngaphandle kokushumeka umsindo orekhodiwe. Amaresiphi ayo asebenzisa izingxenye zokuhlanganisa ezisekelwe ku-physics ezifana nama-modal resonator, ama-contact excitation, umsindo ohlungiwe, igumbi elibalwe, kanye nemodeli yemakrofoni.

Qala ngokubhala uhlu lwamaresiphi afakiwe:

earshot forge

Indawo yokugcina inamaresiphi okuchofoza, imithelela, ukucindezelwa kokhiye, ama-pop, nama-whoosh. Ukusesha umsindo wokucindezelwa kokhiye usebenzisa okuqoshiwe kwangempela njengezinkomba, sebenzisa:

earshot forge keypress --refs path/to/real/keypresses/ --budget 144 --reps 2 --tone 1.5 --keep 4 --robust 8 --dir made/keypress

Ukuqondisa incazelo ethile ngenkathi uqamba ukudideka okungenzeka:

earshot forge whoosh --target "a fast crisp air whoosh" --alts "a wind gust" "a sword swing" --refs path/to/real/whooshes/ --tone 1.5 --dir made/whoosh

I-Forge ikhiqiza amakhandidethi anamapharamitha, iwalinganise ngomgxeki, igcine imiphumela enamandla, bese iwaguqula ngokusesha kokuziphendukela kwemvelo. Ukukala kwayo kubheka ukufana kwencazelo, ezinye izindlela eziqanjwe ngamagama, ubunjalo obulinganisiwe, ukusondela ezinkombeni, ukuma okukaliwe, ibhalansi ye-tonal, ukuziphatha komsila, nobude.

Okuphumayo kufaka amafayela e-WAV ahlukahlukene kanye ne-report.json equkethe amapharamitha namaphuzu amakhandidethi. Nge---robust N, amakhandidethi akhethiwe akhiqizwa futhi ngezinhlamvu ezintsha ezingahleliwe ukuze uhlole ukuthi iresiphi ihlala ithembekile yini, kunokuba iphumelele kanye kuphela.

Hlola imiphumela ye-Forge ngokucophelela

I-Forge ilungiselela ngokumelene namamodeli ayo e-Earshot, ngakho amaphuzu aphezulu emodeli awaqinisekisi ukuthi umsindo ungcono ngokombono wokuzwa.

Qhathanisa ibhalansi ye-one-third-octave, umsila, nobude bomsindo okhiqiziwe nokurekhodiwe kwezinkomba. Hlola i-spectrogram yawo eceleni kwezinkomba, hlola ukukhiqizwa kabusha okuqinile, futhi wenze ukuhlolwa komuntu okuyimpumputhe lapho ikhwalithi ibalulekile. I-README ixwayisa ngokukhethekile ukuthi imodeli yobunjalo ingakhohliswa ngamasu afana nokuhlanganisa i-modal resonator, impendulo yegumbi, kanye nomsindo ophansi.

Amathiphu Omsebenzi Athuthukile

1. Sebenzisa isixhumi esibonakalayo esifanele umsebenzi. I-CLI ilungele uphenyo olumbalwa. Sebenzisa inqubo eyodwa ye-Python noma iseva ye-MCP ekuhlaziyeni okuphindaphindiwe ukuze amamodeli ahlale elayishiwe.
2. Qala ngokubanzi, bese uhlola imibono. Sebenzisa i-describe, wakhe incazelo yalokho okuqondiswe kukho, bese usebenzisa i-match nezinye izindlela ezicacile.
3. Hlukanisa ubufakazi besemantiki nobunobufakazi obuqondile. Hlanganisa izahlulelo ezisekelwe kumamodeli ne-measure, ukuhlolwa kwebhalansi yamathoni, ukuhlola i-waveform, nama-spectrogram.
4. Londoloza okukhiphayo okuhlelekile. Sebenzisa i---json noma i---out lapho olunye uhlelo noma i-ejenti idinga ukuhlola zonke izinkambu esikhundleni sesifinyezo solimi olujwayelekile.
5. Sebenzisa okurekhodiwe kwangempela okuyisithenjwa. Ama-directory ezithenjwa athuthukisa ukuqhathanisa nokuhlolwa kwe-Forge, kodwa wena unesibopho semvelaphi namalayisensi alokho okurekhodiwe.
6. Faka amakhono e-ejenti afakiwe. Kopisha noma wenze i-symlink ye-skills/earshot ne-skills/real-sound uye ku-~/.claude/skills/ noma ku-.claude/skills/ yephrojekthi. Achaza ukukhethwa kwemiyalo, ukuhumusha izahlulelo, ukukhiqizwa kwamaculo esithombe, ukuxuba, i-QA, nokuhlolwa kwamalayisensi.

Qonda Imikhawulo

I-Earshot inamandla kakhulu ekulinganiseni umsindo, isikhathi, ukuvumelanisa nesithombe, ukufanisa imisindo eyodwa esobala, ukuklelisa abazongenela, nokuhlonza imithombo ehlanganiswe ngekhodi esobala. Ama-tag akhishwe kuma-mix aminyene, kanye nezahlulelo zohlobo lomculo nesimo sengqondo, kufanele kuthathwe njengezinkomba.

Ayikwazi ukunquma ukunambitheka, imizwa, imvelaphi, amalayisensi, noma okuqukethwe kwenkulumo. Amamodeli ayo awahloli ama-frequency angaphezu kuka-14 kHz. Isahlulelo sokuba ngokoqobo siwubufakazi kunokuba sibe yisiqinisekiso, futhi iphrofayela yokuqalisa eyakhelwe ngaphakathi imele amafilimu ayisikhombisa kuphela asetshenziselwa ukuyidala.

Bheka imibhalo yokuqinisekisa ye-repository ukuze ubone imiphumela elinganisiwe, kanye nemibhalo yemikhawulo yayo ngaphambi kokuthembela emaphuzwini ohlelweni lokukhiqiza.

Isiphetho

I-Earshot inikeza ama-ejenti okubhala ikhodi indlela esebenzayo yokuhlola nokudala umsindo esebenzisa amathuluzi e-CPU asebenza endaweni. Qala nge-describe, match, ne-find; dlulela ku-timeline, sheet, ne-critique kumafilimu aphelele; bese usebenzisa i-Forge lapho udinga imiphumela enemingcele eyenziwe ngokuphelele ngekhodi. Ukuze wenze izinqumo ezinokwethenjelwa, hlanganisa izahlulelo zamamodeli nezilinganiso, ukuqhathanisa nezithenjwa, ukuhlola okubukwayo, nokulalela komuntu.