Chinese AI Text Detector Tutorial: Local Inference, Model Training, and Result Validation

Project Overview

Chinese Guba AI Detector is a Chinese AI text binary classification tool. The project is based on hfl/chinese-roberta-wwm-ext and fine-tuned with the C-ReD corpus. It was previously used for research-oriented measurement of Guba posts. The current repository provides public training, inference, and testing scripts, along with related documentation.

The tool loads the model locally and does not call external APIs. Note carefully: the repository currently does not provide trained weights or tokenizer files. Users must lawfully obtain the model themselves and place it in the specified directory.

Important limitations: This is not the official C-ReD model. This fine-tuning did not use Guba posts and has not undergone human-labeled testing in the financial forum domain. Model scores cannot prove the actual identity of a text's author and should not be used as the sole basis for personal identification or content penalties.

Main Features

• Performs binary classification of AI-generated and human-written text in UTF-8 CSV files.
• Outputs the softmax score for the AI class and a threshold-based classification result.
• Reads 帖子内容文本 by default, while also supporting custom text and stock-code columns.
• Normalizes 股吧代码 to six-character text while preserving leading zeros.
• Preserves other columns in the input file without actively changing their text content.
• Reduces the risk of output-file corruption through temporary files and atomic replacement.
• Supports integrity checks, overwrite recalculation, and single-process write protection.
• Provides training, unit-test, and public-content audit scripts.

Installing the Runtime Environment

1. Prepare Python and PyTorch

The project's test environment uses Python 3.10. We recommend using an isolated virtual environment and, depending on whether you use a CPU or CUDA device, first installing the PyTorch 2.5.1 build suitable for your device according to the official PyTorch instructions.

In Windows PowerShell, go to the repository root, then create and activate a virtual environment:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements.txt

Inference can run on a CPU, but a CUDA GPU is usually faster. If you run out of GPU memory, first reduce --batch-size from its default or larger value to 8 or 4.

2. Prepare the Local Model

Place the legally obtained trained model in models/detector. The directory must contain the following files:

• model.safetensors
• config.json
• tokenizer.json
• tokenizer_config.json
• special_tokens_map.json
• vocab.txt

These weights and tokenizer files are not included in the public repository. If the directory is incomplete, the inference script will not be able to load the model correctly.

Running Your First Detection

1. Prepare the Input CSV

The input file must be a UTF-8 CSV. The default text-column name is 帖子内容文本, and the default stock-code column name is 股吧代码. The examples provided by the repository contain fictional text only; they do not contain real users, real posts, or human-versus-AI ground-truth labels.

2. Run Batch Inference

After preparing the model, run the following command from the repository root:

python scripts/predict.py --model-dir models/detector --input examples/synthetic_posts.csv --output outputs/demo.csv --batch-size 32

The command loads the local model from models/detector, reads the example CSV, and writes the detection results to outputs/demo.csv. The entire detection process does not call external APIs.

3. Understand the Output Fields

The output file preserves the original columns and adds the following fields:

• 是否AI生成: Set to 1 when the unrounded model score is greater than or equal to the threshold, and 0 otherwise; empty text remains empty.
• AI生成概率: The softmax score for the AI class. This score is not probability-calibrated and cannot be treated directly as a true probability.
• AI检测模型: The public model identifier; the local model path is not written.
• AI检测阈值: The default value is 0.7. This is a project setting, not a threshold established as optimal by a paper.
• AI检测状态: ok indicates that detection completed, while empty_text indicates that the input text was empty.

Using Custom CSV Columns

If the text column is named text and the stock-code column is named id, specify them explicitly:

python scripts/predict.py --model-dir models/detector --input input.csv --output outputs/result.csv --text-column text --stock-column id

If the input file has no stock-code column, pass an empty string:

python scripts/predict.py --model-dir models/detector --input input.csv --output outputs/result.csv --text-column text --stock-column ""

When a stock-code column exists, the program normalizes it to six-character text while preserving leading zeros. Other columns in the input are preserved as text exactly as provided.

Overwrite, Validation, and Parallel Processing

Regenerate the Complete Result

Inference results are first written to a temporary file and then atomically replace the target file after success. When the same command is run again, the program skips existing results only when the checksums for the input, model, parameters, and complete output all match.

To ignore existing results and regenerate the complete file, add --overwrite:

python scripts/predict.py --model-dir models/detector --input examples/synthetic_posts.csv --output outputs/demo.csv --batch-size 32 --overwrite

Interruptions and Parallel Tasks

After an interruption, the program recalculates from the beginning of the file rather than blindly resuming based on the number of existing rows. Only one process may write to the same output path at a time. For parallel processing, assign each process a different input file and output file.

Length Limits and Result Interpretation

By default, each text input is limited to 256 tokens, and excess content is truncated. The number of tokens is not equal to the number of Chinese characters, so this limit should not be understood as 256 Chinese characters.

Some of the original research data consisted of collected Guba post excerpts of only about 203 characters, so the detected object may be only part of the original body text. When interpreting results, also consider truncation, domain differences, and missing context.

AI生成概率 is an uncalibrated softmax score, while the default threshold 0.7 is merely a project setting. Changing the threshold changes the binary classification result, but the README does not declare any threshold to be an optimal value validated by a paper.

Training a Model with C-ReD Data

Licensing Prerequisite

Before training, you must independently verify and obtain lawful permission to use the C-ReD data. No clear license was found in the upstream repository; citing the paper does not grant authorization to use or redistribute the data. This project does not mirror C-ReD data, and the repository's MIT license does not cover C-ReD data or derived weights.

Install Training Dependencies

python -m pip install -r requirements-train.txt

Start Training

Assuming the lawfully obtained data has been placed in data/C-ReD, run:

python scripts/train.py --data-root data/C-ReD --output-dir models/detector --epochs 3 --batch-size 8 --eval-batch-size 16 --no-fp16

The training program reads CSV files whose filenames contain CReD_ and uses their text and label columns. The source-data labels are defined as 0=AI and 1=human; during training, they are converted to 0=human and 1=AI.

The training protocol and historical metrics are recorded in MODEL_CARD.md. Retraining uses complete random-seed initialization, but historical training did not explicitly set a global seed before model initialization, so bit-for-bit reproduction of the old weights cannot be guaranteed.

Testing and Release Auditing

Run all unit tests:

python -m unittest discover -s tests -v

Unit tests use fictional data and simulated predictions, so they do not require downloading a model. A smoke test with a real model can show only that the model loads and performs inference; it cannot prove that the predictions are correct.

Run the public-content audit:

python scripts/audit_release.py

The public repository is checked using a file allowlist and sensitive-information patterns. Original Guba CSV files, publisher IDs, nicknames, IP addresses, post URLs, paper panels, commercial databases, caches, logs, Notebook outputs, intermediate training checkpoints, and historical Git records should not enter the repository. See PRIVACY.md for the specific de-identification scope and docs/PUBLISHING.md for the release process.

Advanced Usage Recommendations

1. Validate the workflow with fictional data first: Confirm that the environment, column names, model directory, and output path are correct before processing your own lawful data.
2. Reduce the batch size when GPU memory is insufficient: First try --batch-size 8 or --batch-size 4.
3. Avoid shared output paths: Parallel tasks must use different output files to avoid triggering the single-process write restriction.
4. Save the run parameters: The model, input, parameters, and output checksum affect result reuse. Recheck the output after changing any of them.
5. Interpret long text cautiously: Content exceeding 256 tokens is truncated, so the model may not see the ending.
6. Do not treat scores as identity credentials: The softmax score is uncalibrated, and the project lacks human-labeled validation in the financial forum domain.
7. Confirm licensing before releasing weights: The code and documentation use the MIT license, and the base model uses Apache-2.0, but the C-ReD data and derived weights require separate authorization checks.

Conclusion

Chinese Guba AI Detector provides a locally runnable Chinese AI text-classification workflow covering CSV inference, model training, unit testing, and release auditing. In practice, you should first lawfully obtain the model or training data and treat the output as a research-oriented model score rather than conclusive evidence of a text author's identity.