MemHarness – An Agent Memory Reconstruction Framework by Shanghai AI Lab and Others Operational

MemHarness is an LLM Agent memory reconstruction framework released by Shanghai AI Lab in collaboration with Zhejiang University, Fudan University, Shanghai Jiao Tong University, and other universities. Existing memory-augmented Agents often directly inject retrieved historical experiences into the context, which can instead cause negative transfer when old experiences do not match the current state. Inspired by the reconstructive mechanisms of human memory, MemHarness inserts explicit critique and reconstruction steps between retrieval and action. It uses the current context to retain, rewrite, or discard historical experiences, and employs GRPO for end-to-end training without requiring additional human annotations.

Memory retrieval: The Agent generates a query based on the current observation and retrieves the top-k relevant historical experiences and their source states from the Milvus vector memory database.
Critique and reconstruction: A unified policy model compares the memory source state with the current state, critiques its applicability, rewrites the experience as state-aligned guidance, or determines that it is inapplicable and discards it.
Action generation: Based on the reconstructed guidance or autonomous reasoning, the Agent generates executable actions in the environment.
Experience write-back and pruning: After each round of interaction, the trajectory is summarized as an experience and written to the memory database. Semantic deduplication is performed, and low-value memories are periodically pruned according to their utility.
End-to-end training: GRPO jointly optimizes retrieval, reconstruction, and action generation, without requiring independently labeled data for the reconstruction stage.

Follow us on WeChat and reply “开源” to join the AI open-source project group
Five-stage decision-making process: MemHarness decomposes memory-guided decision-making into five consecutive stages: environment observation, experience retrieval, memory critique, contextual memory reconstruction, and action generation. This simulates the human cognitive process of retrieval–evaluation–reconstruction, replacing the static paradigm in which traditional Agents directly replay memories.
Contextual memory reconstruction mechanism: The policy model concatenates the task description, current history, retrieved experiences, and their source states into a reconstruction context. By comparing the historical source states with the current state, it identifies differences, retains transferable knowledge, rewrites mismatched content, or outputs the marker to reject the memory and fall back to autonomous reasoning.
Unified policy model architecture: The three stages—retrieval query generation, memory critique and reconstruction, and executable action generation—share the parameters of a single policy model. This eliminates the need to separately train a value network or provide supervised data for the reconstruction module, tightly coupling memory utilization and decision-making reasoning within the same model.
GRPO end-to-end training: Because reconstruction guidance has no ground-truth annotations, MemHarness uses GRPO to optimize the entire retrieval–reconstruction–action pipeline end to end. The reward consists of sparse task-result rewards and format rewards. Group-normalized advantages assign trajectory-level credit to all tokens while jointly training reasoning, reconstruction, and action-generation capabilities.

Clone the repository and create the environment: Run git clone https://github.com/KnowledgeXLab/MemHarness.git and create a Conda environment with python==3.12, then install vLLM, Flash Attention 2, and MemHarness itself in sequence.
Deploy the embedding service: Start the BGE-M3 embedding model with vllm serve BAAI/bge-m3 --port 8001 to provide vector encoding services for the Milvus memory database.
Install the target environments: Install the ALFWorld or WebShop interaction environment as required by the task, and download the corresponding game files and pretrained detector.
Cold-start SFT (optional): Run scripts/build_*_coldstart_data.py to build cold-start data, then execute scripts/cold_start_sft.sh to align the model with the interaction format and memory-summary format.
GRPO end-to-end training: Run run_scripts/train_alfworld.sh or run_scripts/train_webshop.sh. The framework will automatically start the memory vector database, perform Agent retrieval, write back experiences, and prune memories to complete GRPO training.

Reconstruction is better than replay: By explicitly inserting critique and reconstruction steps, static memory fragments are transformed into context-sensitive, state-aligned guidance, avoiding negative transfer.
A small model surpasses larger models: With a 7B-parameter scale, it surpasses Gemini-2.5-Pro by 23.1% and 39.7% on ALFWorld and WebShop, respectively.
Strong OOD robustness: It achieves an average success rate of 85.9% in out-of-distribution scenarios, significantly higher than the 76.3% achieved by the original memory replay approach.
Implicit reasoning enhancement: Even when memory is disabled during testing, the reconstruction training objective increases the base policy success rate from 76.4% to 83.0%, fundamentally strengthening the Agent’s intrinsic reasoning ability.
No additional annotations required: Reconstruction capabilities emerge naturally through end-to-end GRPO training and do not depend on manually written reconstruction supervision data.

GitHub repository: https://github.com/KnowledgeXLab/MemHarness
HuggingFace model repository: https://huggingface.co/KnowledgeXLab/MemHarness
arXiv technical paper: https://arxiv.org/pdf/2607.28272

Embodied household tasks: In home environments such as ALFWorld, the Agent can reconstruct past object-fetching and cleaning experiences and adapt to different room layouts to complete complex household tasks.
Autonomous online shopping: On e-commerce platforms such as WebShop, historical shopping experiences can be reconstructed into current product-search and filtering strategies, improving goal-oriented shopping success rates.
Intelligent customer service dialogue: After retrieving similar historical support tickets, a customer service Agent reconstructs a solution to avoid directly applying an old response and giving an irrelevant answer.
Code-assisted development: A programming Agent reconstructs past bug-fixing experiences to fit the current code context, providing precise fix recommendations rather than copying old solutions.
Scientific experiment planning: An experiment Agent reconstructs historical experiment parameters and results into optimal configuration recommendations for the current experimental conditions, reducing the cost of trial and error.