Skip to main content
AI tutorials

One API Deployment Guide: Access Multiple LLMs Through a Unified OpenAI API

Deploy One API and connect multiple LLMs through a unified OpenAI-compatible interface.

About 27 min read
One API deployment and configuration guide

What Is One API?

One API is an open-source LLM API management and relay system. It standardizes APIs from different model providers into the OpenAI API format, allowing clients to call multiple upstream channels using the same API Base and access token.

The project supports OpenAI and Azure OpenAI, Anthropic Claude, Google Gemini, Mistral, Doubao, ERNIE Bot, Qwen, iFLYTEK Spark, ChatGLM, Tencent Hunyuan, Moonshot AI, DeepSeek, Ollama, Cohere, Groq, Cloudflare Workers AI, xAI, and many other services.

Compliance notice: When using One API, you must comply with the OpenAI Terms of Use and all applicable laws and regulations. It must not be used for illegal purposes. Do not provide unregistered generative AI services to the public in China.

Core Features

  • Unified interface: Access different LLMs through an OpenAI-compatible API format.
  • Multiple channels and load balancing: Configure multiple upstream channels and automatically load-balance requests when no channel is specified.
  • Streaming output: Supports stream mode for typewriter-style responses.
  • Token management: Configure expiration times, quotas, allowed IP ranges, and accessible models.
  • Channel management: Supports bulk channel creation, model lists, channel groups, and model mapping.
  • Quotas and billing: Supports quota details, user groups, channel groups, multipliers, and quota display in US dollars.
  • Reliability: Supports automatic retries after failures, multi-node deployment, Redis caching, and database connection configuration.
  • Management extensions: Call the management API through system access tokens to extend management capabilities without modifying the source code.
  • User system: Supports registration and sign-in through email, Feishu, GitHub, WeChat Official Accounts, and other methods.
  • UI customization: Customize the system name, logo, footer, home page, About page, and theme.

Quick Deployment with Docker

Prepare the Data Directory

SQLite is suitable for quick evaluation or low-concurrency deployments. First, prepare a writable data directory where the database and logs will be stored.

mkdir -p /home/ubuntu/data/one-api

Start One API

docker run --name one-api -d --restart always -p 3000:3000 -e TZ=Asia/Shanghai -v /home/ubuntu/data/one-api:/data justsong/one-api

In -p 3000:3000, the port on the left is the host port and can be changed as needed. After startup, visit http://localhost:3000/.

The initial username is root, and the initial password is 123456.

Important: You must change the default password immediately after signing in as root for the first time.

If the Docker Hub image cannot be pulled, replace justsong/one-api with ghcr.io/songquanpeng/one-api. If the container fails to start, try adding --privileged=true as suggested in the project's issue records.

Deploy with MySQL

For higher concurrency, the README explicitly recommends setting SQL_DSN instead of continuing to use SQLite. The oneapi database must be created in advance, but the application will create its tables automatically.

docker run --name one-api -d --restart always -p 3000:3000 -e SQL_DSN="root:123456@tcp(localhost:3306)/oneapi" -e TZ=Asia/Shanghai -v /home/ubuntu/data/one-api:/data justsong/one-api

Replace the username, password, address, and port with your actual database settings. If MySQL runs on the host, you can add --network="host" as appropriate for your network environment so that the container can access the host database.

Use Docker Compose

The repository provides a Docker Compose deployment option. The current instructions use MySQL and store data in the ./data/mysql directory.

docker-compose up -d
docker-compose ps

Manual Deployment from Source

To build the project yourself, prepare Node.js, npm, and Go environments, then build the frontend and backend in sequence.

git clone https://github.com/songquanpeng/one-api.git
cd one-api/web/default
npm install
npm run build

cd ../..
go mod download
go build -ldflags "-s -w" -o one-api

After the build is complete, grant execute permission and start the service:

chmod u+x one-api
./one-api --port 3000 --log-dir ./logs

--port specifies the listening port and defaults to 3000; --log-dir specifies the log directory. You can also use --version to view the version or --help to view help information.

Complete the Basic Configuration

  1. Open the One API administration page, sign in with the root account, and change the default password.
  2. Go to the Channels page, select the appropriate provider type, and add the upstream API Key.
  3. Configure the channel's supported model list and groups according to the upstream service's capabilities.
  4. Go to the Tokens page and create an access token for clients.
  5. Configure the token's quota, validity period, allowed IP range, and accessible models as needed.

One API works out of the box. Other settings can be configured through the administration interface, environment variables, a .env file, or command-line arguments. To use a .env file, refer to .env.example in the repository and rename it to .env.

Use the OpenAI-Compatible Interface

Set the client's API Base to your One API deployment address and its API Key to the One API token you just created. The exact Base URL format depends on the client. For the official OpenAI library, the README provides the following configuration:

OPENAI_API_KEY="sk-xxxxxx"
OPENAI_API_BASE="https://<HOST>:<PORT>/v1"

Requests first reach One API, which then forwards them to OpenAI, Azure, or another model channel. For upstream services with different API formats, One API converts the request and response bodies as required for relaying.

Use with Existing Clients

Most clients that support a custom OpenAI API endpoint can connect to One API. Using ChatGPT Next Web as an example, first run the client container:

docker run --name chat-next-web -d -p 3001:3000 yidadaa/chatgpt-next-web

Then enter the One API endpoint and One API token on the client page. Avoid using port 3000, which would conflict with One API.

ChatGPT Web can also connect to One API through environment variables:

docker run --name chatgpt-web -d -p 3002:3002 -e OPENAI_API_BASE_URL=https://openai.example.com -e OPENAI_API_KEY=sk-xxx chenzhaoyu94/chatgpt-web

Replace the example domain and token with your own settings.

Advanced Configuration Tips

Specify a Channel or Enable Load Balancing

By default, requests that do not specify a channel are load-balanced across multiple available channels. A channel ID can also be appended to a token created by an administrator, forcing the request to be handled by a specific channel:

Authorization: Bearer ONE_API_KEY-CHANNEL_ID

Tokens created by regular users cannot specify a channel ID this way.

Use Model Mapping with Caution

Model mapping can redirect a model requested by the user to another model. However, the README advises against configuring model mappings unless necessary. When mapping is enabled, the request body is reconstructed rather than passed through directly, so some fields that are not yet officially supported may not be transmitted successfully.

Use Redis Caching

After REDIS_CONN_STRING is set, One API uses Redis as its cache:

REDIS_CONN_STRING=redis://default:redispw@localhost:49153

If database access latency is very low, Redis does not need to be enabled because caching may delay data synchronization. For Sentinel or cluster mode, set this variable to a list of nodes and use it together with REDIS_PASSWORD and REDIS_MASTER_NAME.

Optimize Database Connections

Database connections can be adjusted with the following environment variables:

  • SQL_MAX_IDLE_CONNS: Maximum number of idle connections; the default is 100.
  • SQL_MAX_OPEN_CONNS: Maximum number of open connections; the default is 1000.
  • SQL_CONN_MAX_LIFETIME: Maximum connection lifetime; the default is 60 minutes.
  • BATCH_UPDATE_ENABLED=true: Enables aggregated batch database updates. This may reduce excessive connection usage but delays quota updates.
  • BATCH_UPDATE_INTERVAL=5: Sets the batch update aggregation interval in seconds.

If Error 1040: Too many connections occurs, appropriately reduce SQL_MAX_OPEN_CONNS and evaluate whether to enable batch updates.

Configure Sessions, Timeouts, and Proxies

  • SESSION_SECRET: Sets a fixed session secret so that signed-in users' cookies remain valid after a restart.
  • RELAY_TIMEOUT: Sets the relay request timeout in seconds.
  • RELAY_PROXY: Routes relay requests through a specified proxy when accessing upstream APIs.
  • USER_CONTENT_REQUEST_TIMEOUT: Sets the timeout for downloading user-uploaded content.
  • USER_CONTENT_REQUEST_PROXY: Configures a download proxy for user-uploaded content such as images.

Configure Channel Monitoring

You can periodically update channel balances and test channel availability:

CHANNEL_UPDATE_FREQUENCY=1440
CHANNEL_TEST_FREQUENCY=1440
POLLING_INTERVAL=5

The first two variables are measured in minutes. POLLING_INTERVAL is measured in seconds and controls the interval between requests during batch operations.

Configure Multi-Node Deployment

A multi-node deployment cannot continue to rely on local SQLite. All nodes must connect to the same MySQL database and use consistent session settings.

  1. Set the same SESSION_SECRET on all servers.
  2. Connect all servers to the same MySQL database through SQL_DSN.
  3. Set NODE_TYPE=slave on secondary servers. If this is not set, the server acts as the primary by default.
  4. Use SYNC_FREQUENCY to synchronize configuration from the database periodically.
  5. Secondary servers can set FRONTEND_BASE_URL to redirect page requests to the primary server.
  6. You can deploy Redis on each secondary server and set REDIS_CONN_STRING to reduce database access while cached data remains valid.
  7. If the primary server has high database access latency, consider enabling Redis and periodic synchronization there as well.
SESSION_SECRET=random_string
SQL_DSN=root:123456@tcp(database:3306)/oneapi
NODE_TYPE=slave
SYNC_FREQUENCY=60
FRONTEND_BASE_URL=https://openai.example.com
REDIS_CONN_STRING=redis://default:redispw@localhost:49153

Configure Nginx and HTTPS

In production, Nginx can be used as a reverse proxy for One API. For models with long response times, increase the read timeout appropriately.

server {
    server_name openai.example.com;

    location / {
        client_max_body_size 64m;
        proxy_http_version 1.1;
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Accept-Encoding gzip;
        proxy_read_timeout 300s;
    }
}

On Ubuntu, you can use Certbot to obtain and configure HTTPS:

sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot --nginx
sudo service nginx restart

Troubleshooting

  • Insufficient quota message: In addition to the account quota, check the token's own quota limit.
  • No available channels message: Check the user group, channel group, and the list of models supported by the channel.
  • The current group's load is saturated: This usually means that an upstream channel returned a 429 response.
  • The client displays Failed to fetch: Check the endpoint, API Key, and HTTPS configuration. Browsers may block HTTP requests made from an HTTPS page.
  • Channel testing returns invalid character: The upstream service may be returning HTML instead of valid JSON. The deployment server's IP address or proxy node may also have been blocked by Cloudflare.
  • Concern about data loss after upgrading: MySQL data will not be lost during a normal upgrade. For SQLite, the data volume must be mounted correctly to persist one-api.db.

Conclusion

One API is suitable for unifying multiple model providers behind an OpenAI-compatible interface while centrally managing channels, tokens, quotas, and users. Individuals and low-concurrency deployments can start with Docker and SQLite. High-concurrency and multi-node environments should use a shared database and introduce Redis cautiously based on latency. Before putting the service into use, change the default password, configure HTTPS, persist your data, and ensure compliance with all relevant terms, laws, and regulations.

The project is open source under the MIT License, but the README requires the attribution and project link to remain in the page footer. Authorization must be obtained before removing the attribution. For more information, visit the One API GitHub repository.