arXiv：cs.LG· Kristoffer Hjelmen Nesland (University of Stavanger, Stavanger, Norway, Solution Seeker AS, Oslo, Norway), Bjarne Grimstad (Solution Seeker AS, Oslo, Norway), Antorweep Chakravorty (University of Stavanger, Stavanger, Norway), Dan Sui (University of Stavanger, Stavanger, Norway)·· 6 小时前
用 LLM 做文本信息训练数据过滤，提升虚拟流量计性能
Text-Informed Training Data Filtering for Enhanced Virtual Flow Meter Performance
AI 导读
研究者提出一种文本信息驱动的数据过滤方法，利用 LLM 将工程师评论、运营日志等非结构化文本与表格传感器数据结合，先做上下文异常检测，再生成二值掩码过滤训练集。在合成油气数据集上，该方法用于虚拟流量计软测量回归任务时，相比仅用传感器的过滤，把下游 MAPE 从 7.1% 降至 5.4%，效果优于传统规则过滤。
正文
Abstract:In sectors such as energy, aquaculture and chemical processing, the effectiveness of both mechanistic and data-driven models is frequently limited by poor-quality training data, which obscures underlying physical laws and degrades predictive accuracy. Such quality issues commonly manifest as anomalies in the dataset, ranging from sensor faults to process upsets and incorrect operator inputs. While traditional rule-based filtering can remove a subset of the anomalies, it is unable to leverage contextual information stored as unstructured text, such as engineer comments and operational logs that explain under what conditions the data was collected. We propose a text-informed data filtering method that leverages large language models (LLMs) to integrate unstructured operational data with tabular sensor data. The method consists of two steps: (1) contextual anomaly detection by reasoning over both data modalities, and (2) generation of a binary mask that is applied to filter the training dataset for model development. To investigate the potential benefit, a steady-state soft-sensor regression model, namely a virtual flow meter that estimates well flow rates, is used as the downstream evaluation target. Experiments on a synthetic oil and gas dataset show improved model performance compared to traditional rule-based filtering. Ablation studies show that textual operator comments can provide filtering-relevant signals beyond sensor data alone, with the multimodal approach reducing downstream MAPE from 7.1% to 5.4% relative to sensor-only filtering. This work demonstrates that integrating unstructured operational knowledge with sensor data through LLMs can improve data quality for industrial machine learning applications, offering a scalable alternative to manual data curation.
| Comments: | 19 pages, 5 figures. Accepted at FAIEMA 2026 |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.19860 [stat.ML] |
| (or arXiv:2610.19860v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19860 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kristoffer Hjelmen Nesland [view email]
[v1]
Wed, 14 Oct 2026 09:24:05 UTC (184 KB)
来源：arXiv：cs.LG · arxiv.org