Qwen3.8-Omni-Flash Operational

Qwen3.8-Omni-Flash is Alibaba's Qwen native multimodal model, supporting text, image, audio, and video inputs with a 1M-token context window. It focuses on Agent capabilities that take tasks "from understanding to delivery," autonomously completing end-to-end workflows such as video editing, MV creation, short-form drama translation, film commentary, and meeting-minutes execution. Compared with the previous generation, it delivers an average improvement of over 25%; API audio input prices have fallen by over 98%. It also open-sources two supporting frameworks: Qwen-MM-Plugins and Qwen-Live Harness.

Controllable audio and video captioning: Users can freely specify the subject, time range, information granularity, and output format, enabling the model to describe "what the user wants to know" rather than merely "what the model saw."
Agentic long-form audio and video understanding: Starting from a question, the model autonomously decides where to look and what to listen to, using multiple rounds of coarse-to-fine evidence gathering to locate key information. Accuracy improves while token consumption drops by approximately 45.7%.
Long-meeting understanding and execution: Native support for one-hour audio and video inputs, completing speaker diarization, transcription, identity alignment, minutes generation, and task execution end to end.
Deep audio and video research: Combines video content with user needs, automatically retrieves multimodal materials, and generates richly illustrated research reports centered on the video.
Music2MV: Understands song structure, rhythm, and emotion at a fine-grained level, outputs timestamped lyrics, and covers the complete MV creation workflow from music understanding through final-video quality inspection.
Short-form drama translation: A single instruction completes character dialogue recognition, conversational translation, voice cloning and dubbing, audio-track remixing, and quality inspection for automated localization delivery.
Long-film commentary: Given a film and a one-line request, it automatically completes full-film understanding, plot extraction, commentary writing, voiceover and music generation, editing and rendering, and quality inspection.
Video2Note: Compresses hours of video into illustrated, timestamped PDF notes and automatically reviews and iteratively corrects them.
Omni Skill Creator: Extracts SOPs from operational demonstrations or expert instruction and converts them into validated, reusable Agent Skills.
Real-time speaking practice: Jointly models pronunciation and semantics, understands accent deviations, and generates standard-pronunciation demonstrations in real time.

Follow on WeChat and reply with “Open Source” to join the AI open-source project community.
Native unified multimodal architecture: The model jointly models text, images, audio, and video within a single architecture. Cross-modal understanding is achieved through modality-aligned encoding and a unified representation space, while preserving text capabilities comparable to those of a pure-text model of the same size. Its 1M-token context window natively accommodates hours of audio and video inputs, avoiding the information fragmentation caused by segmented processing.
Agentic on-demand perception and evidence gathering: The model first scans at a coarse granularity to locate candidate segments, then retrieves related audio and visual content at a finer granularity for verification. This creates a multi-round coarse-to-fine evidence chain that concentrates computation and token budgets on truly relevant segments.
Audio/video and Agent environment coordination: The bottleneck for long-form audio/video Agents is that harnesses lack native audio/video support. The official solution therefore evolves the model and toolchain together: Qwen-MM-Plugins provides on-demand perception, tool calling, and workflow execution, integrates with mainstream Agent frameworks such as Claude Code and OpenClaw, and connects material understanding, task planning, tool execution, and result delivery into a complete pipeline.
Multi-speaker audiovisual coordinated recognition: The model jointly models visual and audio streams, using visual information, such as a person's appearance and mouth movements, to resolve references and entity ambiguities in audio. It completes speaker diarization, speech transcription, and identity alignment end to end, substantially improving recognition metrics in meetings with alternating speakers and overlapping voices.
Real-time streaming interaction architecture: Qwen3.8-Omni-Flash-Realtime receives audio and video streams through WebSocket/WebRTC and perceives and responds while input is being received, with a first-token latency of approximately 600-980ms and an audio-generation RTF of approximately 0.153. Speaking correction relies on joint pronunciation and semantic modeling; as each new round of speech arrives, the model continuously updates its judgment and distinguishes errors that affect comprehension from natural accents.

Use it directly on the web: Open the Qwen AI platform at https://www.qianwenai.com/models/qwen3.8-omni-flash, search for qwen3.8-omni-flash, and upload a video, audio file, or image to start a conversation.
API access: Apply for an Alibaba Cloud DashScope API Key and use the OpenAI-compatible interface. Pass image, audio, or video links together with a text question.
Install plugins to get it to work: Install Qwen-MM-Plugins in tools such as Claude Code or Qwen Code, then say something like “@video.mp4 turn this into a commentary video.” The AI automatically completes the entire workflow.

Unified multimodality: One model handles text, images, audio, and video, with support for a 1M-token context window and native input of one-hour videos.
From understanding to delivery: It not only understands audio and video, but also autonomously plans, calls tools, produces finished videos and documents, and completes end-to-end tasks.
Strong Agent capabilities: It leads substantially on audio/video Agent benchmarks. WildClawBench-MM improves by 36.5 points over the previous generation, while long-horizon tasks reach a high score of 69.6 on UniClawBench.
Audio capabilities surpass Gemini: It comprehensively outperforms Gemini 3.8 Flash on speech recognition across 60 languages, 39 Chinese dialects, and multi-speaker recognition metrics.
Higher efficiency and lower cost: Agentic evidence gathering reduces token consumption for long-video understanding by approximately 45.7% while improving rather than reducing accuracy.
Significantly lower prices: API audio input prices have dropped by over 98% per hour, and audio/video input prices by over 93%, greatly reducing the cost pressure of large-scale use.
Complete open-source ecosystem: Qwen-MM-Plugins and Qwen-Live Harness are fully open source and compatible with mainstream Agent frameworks such as Claude Code and OpenClaw.

GitHub repositories: Qwen-MM-Plugins: https://github.com/QwenLM/Qwen-MM-Plugins
Qwen-Live Harness: https://github.com/QwenLM/Qwen-Live-Harness

Video content production: Enter a single request to automatically complete end-to-end delivery of film commentary, MV production, or short-form drama translation and dubbing.
Meeting productivity tools: Upload a one-hour meeting recording to automatically generate minutes and action items, and even send emails or create tasks directly.
Learning and knowledge capture: Automatically compress hours of tutorial videos into PDF notes with screenshots and timestamps.
In-depth video research: Automatically retrieve text and image resources from across the web based on video content and generate richly illustrated in-depth research reports.
Real-time interactive services: Low-latency audio/video interaction scenarios such as real-time speaking practice, intelligent customer service, and sound-based navigation.