疲劳驾驶是造成道路交通伤害的主要原因之一，高效准确的疲劳驾驶检测是保障交通安全的重要手段。但传统单一信号在捕捉疲劳状态时存在数据采集侵入性高、数据结构复杂、难以实时预测、准确率低等局限性，融合不同模态的信息成为新的发展方向。该文通过长期实车驾驶实验采集的心电信号、车辆轨迹数据、驾驶员脸部图像数据构建基于多模态信号的疲劳驾驶检测模型。该模型包含数据处理、特征提取、特征筛选和疲劳预测等模块，通过自注意力机制捕获长时间依赖关系并输出预测结果。该模型预测疲劳状态类别的准确率最高达97.89%，整体召回率和F1分数达80%以上，表明模型具有较好的预测准确性。将同时利用3个模态数据的检测模型设置为对照组，以仅使用单个模态数据或同时使用2个模态的数据为实验组，共6组实验组。试验发现同时利用3个模态的数据的疲劳驾驶检测模型在不同指标上均取得最优结果，表明所提出的模型成功融合了不同模态的信息，具有较高的准确率和适应性，能够提升驾驶安全的保障水平。

Objective: As a primary cause of road traffic injuries, fatigued driving requires efficient and accurate detection to improve traffic safety. Traditional single-signal approaches face limitations in capturing fatigue states, including high data collection intrusiveness, complex data structures, difficulty in real-time prediction, and low accuracy. Integrating information from different modalities has emerged as a new direction for development. Methods: This study developed a multimodal fatigue detection model for drivers by using electrocardiogram signals, vehicle trajectory data, and driver facial video data collected during long-term real-vehicle driving experiments. For ECG signal processing, an improved two-step adaptive filtering method was adopted for denoising, followed by time-domain and frequency-domain analyses to extract the driver's ECG feature set. For vehicle trajectory data, the quartile method was first used to remove outliers, backward filling was applied to fill in missing values, and two-dimensional discrete wavelet analysis was then employed for data denoising. For facial image data, the LabelMe annotation tool was used to construct a personalized training dataset by manually annotating each driver's face with at least 300 images per subject. The YOLOv8 deep learning model was then fine-tuned and trained on this dataset, and the optimal model weights were saved. Next, the optimized model was used to automatically annotate the remaining unlabeled images, and the Hopenet algorithm was applied to extract head pose angles from the cropped facial regions. The model incorporated modules for data processing, feature extraction, feature selection, and fatigue prediction, utilizing a self-attention mechanism to capture long-term dependencies and generate predictive outputs. Results: The model achieved a maximum accuracy of 97.89% in predicting fatigue state categories, with overall recall and F1 scores exceeding 80%, demonstrating strong predictive accuracy. The detection model utilizing data from all three modalities served as the control group, while six experimental groups were formed using a single modality or a combination of two modalities. The experiments revealed that the fatigued driving detection model employing data from all three modalities achieved optimal performance across various metrics. Conclusions: This study demonstrates that the proposed model successfully integrates information from different modalities, exhibits high accuracy and adaptability, and enhances the assurance of driving safety. Specifically, this model outperforms all comparison models in terms of accuracy, precision, recall, and F1 score, achieving the best performance in fatigued driving detection tasks, and maintains stable detection performance even under complex data structures, diverse sources, large time spans, average-quality facial images, and individual driver differences. The model achieves a maximum accuracy of 97.89% in identifying fatigue states, indicating that it correctly learns and recognizes fatigue patterns. Furthermore, models using a single modality or a combination of two modalities yield lower evaluation metrics than the three-modality model. Moreover, two-modality models consistently outperform single-modality variants, confirming that ECG signals, trajectory data, and facial images contribute positively to accurate fatigued driving detection.