<div align="center">
<img src="omni/asset/logo.png" style="width: 50%; height: auto; box-shadow: 0 4px 18px rgba(0,0,0,0.06); display: block; margin: 0 auto;">
### MobileMem: Learning from a Year of Mobile Experiences
[](https://arxiv.org/abs/2608.13606v2)
[](https://chinaxiv.org/abs/202608.00183V1)
[](https://zjunlp.github.io/MobileMem/)
[](https://huggingface.co/datasets/zjunlp/MobileMem)
[](#license)
---
<img src="omni/asset/fig1.png" style="width:100%; height: auto; border-radius: 14px; box-shadow: 0 4px 18px rgba(0,0,0,0.06);" align=center>
**MobileMem** is a comprehensive benchmarking framework for evaluating **on-device memory systems** in realistic mobile environments.
</div>
---
## ๐ Table of Contents
- [๐ Table of Contents](#-table-of-contents)
- [๐ฅ News](#-news)
- [๐ฏ Applications](#-applications)
- [๐ Dataset Structure](#-dataset-structure)
- [๐๏ธ Project Structure](#๏ธ-project-structure)
- [๐ Getting Started](#-getting-started)
- [๐ Text Track](#-text-track)
- [๐ผ๏ธ Omni Track](#๏ธ-omni-track)
- [๐งฉ Struct Track](#-struct-track)
- [๐ Analyzing Failures with MemTrace](#-analyzing-failures-with-memtrace)
- [๐ฉ Citation](#-citation)
---
## ๐ฅ News
- **2026-09-30** โ We released a structured Chinese version of the dataset and benchmark.
- **2026-08-18** โ We have officially released our [technical report](https://arxiv.org/abs/2608.13606v2).
- **2026-08-01** โ We publicly release the MobileMem dataset.
- **2026-05-16** โ We launch the English version of the dataset and benchmark.
- **2026-05-03** โ We launch the Chinese version of the dataset and benchmark.
---
## ๐ฏ Applications
MobileMem is built from **multiple heterogeneous sources** to enable comprehensive on-device memory modeling.
<img src="omni/asset/fig2.png" style="width:100%; height: auto; border-radius: 14px; box-shadow: 0 4px 18px rgba(0,0,0,0.06);" align=center>
---
## ๐ Dataset Structure
MobileMem contains three complementary tracks:
| Track | Modality | Description |
| :--- | :--- | :--- |
| **text** | Text | Long-horizon userโassistant conversations and structured mobile-app events for evaluating textual memory systems. |
| **omni** | Text and images | Multimodal mobile interactions with screenshots and photos. |
| **struct** | Structured data | Simulated structured data from on-device applications. |
The dataset is available for download at [HuggingFace](https://huggingface.co/datasets/zjunlp/MobileMem).
---
## ๐๏ธ Project Structure
The repository is organized into three main tracks. Track-specific details remain in each subdirectory.
```bash
MobileMem/
โ
โโโ text/ # ๐ Text Track
โ โโโ README.md # Track-specific guide and dataset download
โ โโโ keme/ # ๐ ๏ธ KEME synthesis pipeline code
โ โโโ eval/ # โ๏ธ Evaluation scripts for text track
โ
โโโ omni/ # ๐ผ๏ธ Omni Track
โ โโโ README.md # Track-specific guide and dataset download
โ โโโ src/ # ๐ ๏ธ Data construction pipeline code
โ โโโ eval/ # โ๏ธ Evaluation scripts for omni track
โ
โโโ struct/ # ๐งฉ Struct Track
โ โโโ README.md # Track-specific guide and dataset download
โ โโโ construct/ # ๐ ๏ธ Example generation and review (optional)
โ โโโ eval/ # โ๏ธ Evaluation for struct track
โ
โโโ README.md # This file
```
---
## ๐ Getting Started
MobileMem offers three benchmark tracks. Choose the path that fits your needs and navigate to the corresponding resources.
For an interactive visualization of the MobileMem data, visit the [Dataset Explorer](https://github.com/zjunlp/MobileMem/tree/Dataset-Explorer) branch.
### ๐ Text Track
The textual benchmark for evaluating memory systems on long-term, knowledge-intensive mobile agent trajectories.
| Section | Description | Quick Link |
| :--- | :--- | :--- |
| **๐ฅ Data Access** | Download the synthesized KEME trajectories and QA pairs from HuggingFace. | [Link](https://huggingface.co/datasets/zjunlp/MobileMem) |
| **โ๏ธ How to Evaluate** | **Detailed evaluation guide for reproducing leaderboard results is available in the [MemBase](https://github.com/zjunlp/MemBase/tree/main/examples/evaluate_memory_systems_on_mobilemem) repository.** | [Link](https://github.com/zjunlp/MemBase/tree/main/examples/evaluate_memory_systems_on_mobilemem) |
| **๐ ๏ธ Data Construction** | Reproduce the KEME synthesis pipeline from scratch. | [Link](https://github.com/zjunlp/MobileMem/tree/main/text) |
### ๐ผ๏ธ Omni Track
The multimodal benchmark for evaluating on-device memory with realistic mobile images and dialogues.
| Section | Description | Quick Link |
| :--- | :--- | :--- |
| **๐ฅ Data Access** | Download the MobileMem-Omni dataset, including images and dialogues. | [Link](https://huggingface.co/datasets/zjunlp/MobileMem) |
| **โ๏ธ How to Evaluate** | **Detailed evaluation guide for reproducing leaderboard results is available in the [MemBase](https://github.com/zjunlp/MemBase/tree/main/examples/evaluate_memory_systems_on_mobilemem_omni) repository.** | [Link](https://github.com/zjunlp/MemBase/tree/main/examples/evaluate_memory_systems_on_mobilemem_omni) |
| **๐ ๏ธ Data Construction** | Rebuild the entire MobileMem-Omni dataset with the provided pipeline. | [Link](https://github.com/zjunlp/MobileMem/tree/main/omni) |
### ๐งฉ Struct Track
Simulated structured data from on-device applications.
| Section | Description | Quick Link |
| :--- | :--- | :--- |
| **๐ฅ Data Access** | Download Struct cases and structured app records from HuggingFace. | [Link](https://huggingface.co/datasets/zjunlp/MobileMem) |
| **โ๏ธ How to Evaluate** | Evaluate agent JSONL traces against Struct cases. | [Link](https://github.com/zjunlp/MobileMem/tree/main/struct) |
| **๐ ๏ธ Data Construction** | Example generation and review Skills. | [Link](https://github.com/zjunlp/MobileMem/tree/main/struct/construct) |
---
## ๐ Analyzing Failures with MemTrace
We recommend using **[MemTrace](https://github.com/zjunlp/MemTrace)** to perform an in-depth error analysis. MemTrace helps you visualize and diagnose where and why your memory system fails, making it easier to pinpoint areas for improvement. For an example of how to use MemTrace, please refer to the [tutorial](https://github.com/zjunlp/MemBase/tree/main/examples/trace_memory_lifecycle_with_membase) in MemBase.
---
## ๐ฉ Citation
If this work or datasets is helpful, please kindly cite as this:
```bibtex
@techreport{mobilemem,
title={MobileMem: Learning from a Year of Mobile Experiences},
author={Xinle Deng and Yida Xue and Xiangyuan Ru and Yijun Chen and Buqiang Xu and Mingjun Mao and Xinjie Liu and Haoming Xu and Shuofei Qiao and Mengru Wang and Chen Jiang and Yuchen Eleanor Jiang and Lizhong Wang and Jason Wang and Li Zeng and Haofen Wang and Guilin Qi and Huajun Chen and Ningyu Zhang},
year={2026},
institution={OPPO and OpenKG},
eprint={2608.13606},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2608.13606},
}
```