[Objective]With the rapid development of unmanned technologies, unmanned monitoring ships have become indispensable platforms for marine fishery monitoring and illegal fishing detection, offering advantages such as high flexibility, extended endurance, and remote operability. These platforms significantly expand monitoring scope, improve detection precision, and reduce labor costs. However, the evaluation of intelligence for water-based unmanned platforms remains in its initial stage. Existing evaluation frameworks are predominantly designed for land-based autonomous vehicles and fail to address the unique characteristics of unmanned monitoring ships, including complex marine environments, dynamic non-cooperative targets, and adversarial interactions. Current methods focus primarily on static functional descriptions or virtual environment testing, lacking the capability to reflect real-world dynamic interactions and environmental uncertainties. To address these critical limitations, this study aims to establish a comprehensive, scientific, and targeted intelligence evaluation methodology specifically tailored for unmanned monitoring ships operating in non-cooperative scenarios. [Methods]This paper proposes a multi-layered evaluation system comprising three core components. First, the UP-RAGAs (Unmanned Platform-based Retrieval-Augmented Generation Assessment) framework is constructed by reconstructing the traditional RAGAs framework into three stages—task input, platform response, and action output—thereby adapting it to real-scenario operational logic and supporting both algorithm-level software testing and system-level integrated evaluations. Second, a function mapping grading method is developed, classifying intelligence into 0—9 grades across three dimensions: autonomy (independent decision-making capability), cooperativity (multi-sensor data fusion and interoperability), and study ability (adaptive improvement through environmental interaction). Complementing this qualitative approach, an ontology capability evaluation method is proposed, involving 12 key capability indicators distributed across four perception stages: detection (optical imaging, multi-channel fusion, target detection), recognition (target identification, key component detection, behavioral intention analysis), positioning (distance measurement, coordinate transformation, track generation), and tracking (target acquisition, stable tracking, trajectory prediction). Third, to address non-cooperative scenarios characterized by dynamic target behaviors and environmental uncertainties, an innovative OODA+E (Observe-Orient-Decide-Act plus Effect) evaluation method is developed by integrating OODA loop theory with environmental and target state factors. Furthermore, a Bayesian-based multi-factor linkage model is established to dynamically quantify the influence of target movement levels, relative distances, viewing angles, and weather conditions on perception performance, enabling comprehensive assessment under dynamic adversarial conditions. [Results]Validation through a typical operational scenario demonstrates the effectiveness and feasibility of the proposed methodologies. The ontology capability evaluation successfully quantifies intelligence using normalized data across the four perception stages, revealing specific performance characteristics. The OODA+E method effectively incorporates dynamic factors, revealing that high target mobility significantly degrades tracking stability, while relative distance inversely affects detection accuracy and positioning precision. Additionally, weather condition variations impact optical imaging performance following expected probabilistic distributions. These quantitative results confirm that the proposed framework can accurately assess intelligence levels under complex, non-cooperative conditions where traditional static evaluation methods would prove inadequate, providing granular insights into factor-specific influences on system intelligence. [Conclusions]The proposed evaluation system successfully integrates static grading, quantitative capability assessment, and dynamic multi-factor analysis, effectively addressing the limitations of existing methods that are confined to cooperative scenarios and static testing environments. By realizing multi-dimensional and multi-level intelligence evaluation spanning algorithm performance to system-wide operational effectiveness, this research provides reliable technical support for the standardized application, performance optimization, and mission planning of unmanned monitoring ships in marine-related fields. The innovative incorporation of Bayesian-based multi-factor linkage analysis significantly enhances the scientific rigor and practical applicability of intelligence assessment, offering a systematic and comprehensive solution for advancing the deployment of intelligent unmanned platforms in complex maritime operations characterized by uncertainty and adversarial dynamics.
Key words
unmanned monitoring ships /
evaluation of intelligence /
non-cooperative scenarios
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1] 王耀南, 安果维, 王传成, 等. 智能无人系统技术应用与发展趋势[J]. 中国舰船研究, 2022, 17(5): 9-26. DOI: 10.19693/j.issn.1673-3185.02705. WANG Y N, AN G W, WANG C C, et al. Technology application and development trend of intelligent unmanned system[J]. Chinese Journal of Ship Research, 2022, 17(5): 9-26. DOI: 10.19693/j.issn.1673-3185.02705. (in Chinese)
[2] 许婷婷, 王颖. 基于人工智能技术的海洋渔业生态自动化监测[J]. 制造业自动化, 2020, 42(7): 153-156. XU T T, WANG Y. Automatic monitoring of marine fishery ecology based on artificial intelligence technology[J]. Manufacturing Automation, 2020, 42(7): 153-156. (in Chinese)
[3] 孟金东. 基于强化学习的多UUV围捕算法研究[D]. 哈尔滨: 哈尔滨工程大学, 2024. DOI: 10.27060/d.cnki.ghbcu.2024.002888. MENG J D. Research on multi-UUV hunting algorithms based on reinforcement learning[D]. Harbin: Harbin Engineering University, 2024. DOI: 10.27060/d.cnki.ghbcu.2024.002888. (in Chinese)
[4] 吴重远. 无人船与传统船舶在海洋监测中的协同作业模式[J]. 船舶物资与市场, 2025, 33(11): 114-116. DOI: 10.19727/j.cnki.cbwzysc.2025.11.037. WU C Y. Cooperative operation mode of unmanned ships and traditional ships in marine monitoring[J]. Marine Equipment/Materials & Marketing, 2025, 33(11): 114-116. DOI: 10.19727/j.cnki.cbwzysc.2025.11.037. (in Chinese)
[5] WIBISONO A, PIRAN M J, SONG H K, et al. A survey on unmanned underwater vehicles: Challenges, enabling technologies, and future research directions[J]. Sensors, 2023, 23(17): 7321.
[6]朱伟明. 无人系统智能水平评测方法及软件设计[D]. 南京: 东南大学, 2024. DOI: 10.27014/d.cnki.gdnau.2024.004190. ZHU W M. Intelligent level evaluation method and software development of unmanned systems[D]. Nanjing: Southeast University, 2024. DOI: 10.27014/d.cnki.gdnau.2024.004190. (in Chinese)
[7] CHANG Y P, WANG X, WANG J D, et al. A survey on evaluation of large language models[EB/OL]. (2023-07-09) [2025-09-10]. https://arxiv.org/abs/2307.03109.
[8] BOMMASANI R, LIANG P, LEE T. Holistic evaluation of language models[J]. Annals of the New York Academy of Sciences, 2023, 1525(1): 140-146. DOI: 10.1111/nyas.15007.
[9] ZHOU W C S, XU C W, GE T, et al. BERT loses patience: Fast and robust inference with early exit[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2020: 1539.
[10] CHEN M, TWOREK J, JUN H, et al. Evaluating large language models trained on code[J]. Communications of the ACM, 2022, 65(11): 109-117.
[11] Bommasani R, Liang P, Lee T. Holistic Evaluation of Language Models[J].Annals of the New York Academy of Sciences, 2023, 1525(1).DOI:10.1111/nyas.15007.
[12] 赵睿卓, 曲紫畅, 陈国英, 等. 大语言模型评估技术研究进展[J]. 数据采集与处理, 2024, 39(3): 502-523. ZHAO R Z, QU Z C, CHEN G Y, et al. Research progress in evaluation techniques for large language models[J]. Journal of Data Acquisition and Processing, 2024, 39(3): 502-523. (in Chinese)
[13] SPATHARIS C, BLEKAS K. Multiagent reinforcement learning for autonomous driving in traffic zones with unsignalized intersections[J]. Journal of Intelligent Transportation Systems, 2024, 28(1): 103-119.
[14] 国家市场监督管理总局, 国家标准化管理委员会. 汽车驾驶自动化分级: GB/T 40429—2021[S]. 北京: 中国标准出版社, 2022. State Administration for Market Regulation, National Standardization Administration. Road vehicles-Diagnostic communication over controller area network (DoCAN)- Dictionary: GB/T 40429—2021[S]. Beijing: Standards Press of China, 2022. (in Chinese)
[15] YAO H F, WANG H J, WANG Y. UUV autonomous decision-making method based on dynamic influence diagram[J]. Complexity, 2020, 2020: 8565106.
[16] Ntakolia C, Lyridis D V.A comparative study on Ant Colony Optimization algorithm approaches for solving multi-objective path planning problems in case of unmanned surface vehicles[J].Ocean Engineering, 2022, 255.DOI:10.1016/j.oceaneng.2022.111418.
[17] 王玉虎, 刘伟. 一种基于人机融合的态势认知模型[J]. 指挥与控制学报, 2023, 9(1): 76-84. WANG Y H, LIU W. A situation cognition model based on human-machine hybrid fusion[J]. Journal of Command and Control, 2023, 9(1): 76-84. (in Chinese)
[18] 王越超, 刘金国. 无人系统的自主性评价方法[J]. 科学通报, 2012, 57(15): 1290-1299. WANG Y C, LIU J G. Evaluation methods for the autonomy of unmanned systems[J]. Chinese Science Bulletin, 2012, 57(26): 3409-3418. (in Chinese)
[19]段海滨, 范彦铭, 魏晨, 等. 群体熵: 一种群体智能行为的量化分析工具[J]. 中国科学(信息科学), 2020, 50(3): 335-346. DUAN H B, FAN Y M, WEI C, et al. Swarm entropy: A quantitative analysis tool for swarm intelligence behaviors[J]. Scientia Sinica (Informationis), 2020, 50(3): 335-346. (in Chinese)
[20] 耿化品, 李易洁, 谢峰. 基于KAN的任务效能评估方法研究[J]. 无人系统技术, 2025, 8(4): 58-66. DOI: 10.19942/j.issn.2096-5915.2025.04.35. GENG H P, LI Y J, XIE F. Evaluation method of effectiveness based on Kolmogorov-Arnold networks[J]. Unmanned Systems Technology, 2025, 8(4): 58-66. DOI: 10.19942/j.issn.2096-5915.2025.04.35. (in Chinese)
[21] 杰恩斯·乌马尔拜克. 基于人工智能的个性化学习路径优化算法研究[J]. 信息系统工程, 2025(8): 130-133. WUMAERBAIKE J. Research on optimization algorithm of personalized learning path based on homo sapiens artificial intelligence[J]. China CIO News, 2025(8): 130-133. (in Chinese)
[22] SEFFERS G I. AI and attack radios teach DARPA unexpected lessons[J]. Signals, 2021(8): 75.
[23] 王晓慧, 黄刚, 丁洁, 等. 基于改进型ADRC算法的无人水面侦察艇轨迹跟踪[J]. 水下无人系统学报, 2021, 29(3): 286-292. WANG X H, HUANG G, DING J, et al. Trajectory tracking of unmanned surface reconnaissance vessel based on improved ADRC algorithm[J]. Journal of Unmanned Undersea Systems, 2021, 29(3): 286-292. (in Chinese)
[24] ZHANG T, LI Q, ZHANG C S, et al. Current trends in the development of intelligent unmanned autonomous systems[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(1): 68-85.
[25] 谢海文. 多智能体分布式协同任务分配策略设计[D]. 北京: 北方工业大学, 2025. DOI: 10.26926/d.cnki.gbfgu.2025.001060. XIE H W. Multi-agent distributed cooperative task allocation strategy design[D]. Beijing: North China University of Technology, 2025. DOI: 10.26926/d.cnki.gbfgu.2025.001060. (in Chinese)
[26] 韦韬, 朱遴, 梁世龙. 水下无人系统集群感知与协同技术发展[J]. 指挥控制与仿真, 2022, 44(5): 6-13. WEI T, ZHU L, LIANG S L. Research on perception and cooperation technologies for underwater unmanned system swarm[J]. Command Control & Simulation, 2022, 44(5): 6-13. (in Chinese)
[27] LYU L, SHEN Y, ZHANG S C. The advance of reinforcement learning and deep reinforcement learning[C]//2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA). Changchun, China: IEEE, 2022: 644-648.
[28] 沈博, 武文亮, 杨刚, 等. 基于群体OODA的无人集群系统智能评价模型及方法[J]. 航空学报, 2023, 44(14): 328003. SHEN B, WU W L, YANG G, et al. Evaluation models and methods for intelligence of unmanned swarm systems based on collective OODA loop[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(14): 328003. (in Chinese)
PDF(4752 KB)
