Yesterday (August 6), Composio posted on X that it had tested four AI coding agent frameworks using 30 tasks to determine which framework could better leverage the official version of the DeepSeek V4 Flash model.
According to public information reviewed by IT Home, Composio was founded in 2023 and is headquartered in San Francisco. It is positioned as an open-source AI agent integration platform whose core function is to connect large language models with various external applications, helping developers quickly build production-ready AI agents.
Composio used real-world tools such as Gmail, GitHub, Slack, and Notion to test 30 AI agent tasks, consistently using the official version of the DeepSeek V4 Flash model. The results were as follows:
Oh My Pi: 17/30
Claude Code: 16/30
Codex: 16/30
OpenCode: 14/30
The test results showed that Oh My Pi had the highest success rate (17/30), but was the slowest, taking 272 seconds per task. OpenCode had the lowest cost, requiring only $0.073 per successful task. Claude Code was the fastest, taking just 122 seconds, but was also the most expensive at $0.195 per task, despite using the fewest tool calls and generating the fewest output tokens.
