E-Commerce Bench Operational

E-Commerce Bench is a long-horizon e-commerce operations benchmark open-sourced by Alibaba's Qwen team in collaboration with the Taotian Group. E-Commerce Bench lets an LLM Agent operate an online store with RMB 100,000 in starting capital in a simulated real-world market for 365 days, covering product selection, supplier negotiation, pricing, inventory, and cash-flow management. The environment is based on real platform data and includes 576 suppliers, including fraudulent ones, 8 promotions, and 10 market events. Its deterministic core ensures reproducibility while evaluating a model's commercial operating capabilities across seven dimensions, including profitability, negotiation, fraud prevention, and long-horizon learning.

Long-horizon operations simulation: Enables an LLM Agent to continuously operate an online store in a dynamic market for 365 days with RMB 100,000 in starting capital, maximizing total assets at year-end.
Multi-store operations: Supports opening up to 4 stores simultaneously across 12 different store types for differentiated category planning.
Supplier negotiation: Conducts multiple rounds of price negotiations with 576 suppliers to secure lower procurement costs.
Fraud detection: Identifies and avoids 5 common scams from 152 fraudulent suppliers, including VIP fees, stockouts, quality downgrades, false urgency, and discount traps.
Dynamic market response: Adjusts operating strategies during 8 promotions and 10 unexpected events, including disasters and supply-chain disruptions.
Inventory and warehousing management: Manages the purchase, sale, and inventory of 6,886 SKUs while paying storage fees calculated per item per day.
Cash-flow control: Handles a three-account settlement system and 9-day payment delays to avoid being profitable-but-bankrupt.
Seven-dimensional evaluation: Scores seven independent dimensions: profitability, negotiation, fraud prevention, solvency, efficiency, execution, and long-horizon learning.

Follow us on WeChat and reply “开源” to join the AI open-source project community.
Four-layer architecture: The system is built from top to bottom with an Agent loop layer, tool layer, environment layer, and data layer, forming a closed loop of decision-making, execution, and environmental feedback.
Deterministic negotiation core: All supplier quotes, concessions, and accept/reject decisions are calculated by a fixed algorithmic core. The LLM only renders numerical decisions as natural-language dialogue. This preserves the realistic experience of multi-round bargaining while eliminating sampling randomness, preventing models from using jailbreaks or deception to obtain prices below the cost floor and ensuring reproducible results for different Agents following the same strategy.
Dynamic economic engine: User demand is not randomly sampled. It is calculated from factors including baseline category demand, price elasticity, market events, and store reputation, with reproducible randomness controlled only by the episode seed.
Three-account settlement and cash-flow pressure: Costs are deducted from the bank account on the day they occur, while sales revenue returns to the bank only after “shipping → 9-day platform escrow period → platform wallet → manual withdrawal.” This deliberately creates a working-capital gap, forcing the Agent to actively manage liquidity and avoid short-sighted behavior focused on profit without considering cash.
Long-horizon learning evaluation (AnchorRatio): Measures whether an Agent pays less over time when repeatedly purchasing the same product. A random reshuffling of the Agent's own historical quotes serves as the null hypothesis; a result significantly below 1.0 indicates that the model can continuously optimize its procurement strategy based on past negotiation experience.

Get the code: Clone the E-Commerce Bench open-source repository from GitHub to your local machine.
Configure the environment: Install the project dependencies and prepare a Python runtime environment.
Connect a model: Enter the API key and model name of the LLM to be evaluated in the configuration file.
Start the simulation: Run the main program, and the Agent will automatically begin a 365-day e-commerce operations episode.
View the results: After the simulation ends, the system automatically outputs a seven-dimensional evaluation report and a full-year asset trend curve.

Pioneering long-horizon operations evaluation: Extends LLM Agent evaluation from short-term tasks to dynamic commercial scenarios involving 365 days of continuous operations.
Real-market data driven: Its 6,886 products, 60 categories, and 576 suppliers are built from anonymized data from a real e-commerce platform.
Reproducibility through a deterministic core: Negotiation quotes and market demand are calculated by a fixed algorithmic core, while the LLM only renders the dialogue, completely eliminating random-sampling noise.
Seven-dimensional evaluation system: In addition to year-end assets, it independently assesses negotiation, fraud prevention, cash flow, efficiency, execution, and long-horizon learning.
Realistic commercial pressure simulation: Three-account delayed settlement and per-day storage fees force the Agent to manage working capital instead of pursuing only accounting profit.
Quantified fraud-prevention capabilities: Includes 152 fraudulent suppliers and 5 common scams to precisely measure a model's risk-identification and fund-protection capabilities.

Project website: https://ecbench.github.io/
GitHub repository: https://github.com/QwenLM/E-CommerceBench
arXiv technical paper: https://arxiv.org/pdf/2608.30730

Commercial capability evaluation for large models: Provides standardized cross-model comparisons of long-horizon operating capabilities for leading models such as GPT, Claude, and Qwen.
E-commerce Agent strategy iteration: Serves as a training sandbox for reinforcement learning and Agent architectures, enabling rapid validation of pricing, procurement, and inventory strategies.
Supply-chain risk-control research: Tests and optimizes supplier screening and risk-identification models in a simulated environment containing fraudulent suppliers.
Dynamic pricing algorithm validation: Uses real price-elasticity data to evaluate the revenue performance of different pricing strategies during promotions and seasonal fluctuations.
Business decision simulation training: Serves as an operations simulator for business schools or corporate training, developing managers' decision-making capabilities under cash-flow pressure and market uncertainty.