Temporal Variation of Multimodal Features of Mental Workload in Long-term Parallel Multi-task Scenarios

To address the lack of systematic investigation into the temporal characteristics and dynamic evolution of mental workload (MWL), this study aims to explore the evolution of MWL over time and its multimodal representation differences under complex multitasking environments. This study employed a two-factor experimental design (three task difficulty levels × twelve time segments) based on the multi-attribute task battery (MATB) to induce different MWL levels under a longitudinal multi-tasking. Data were collected from 24 participants, including subjective MWL ratings, task performance, and multimodal physiological measurements such as electroencephalography(EEG), electrocardiography(ECG), and eye-tracking indicators. Repeated measures ANOVA was used to analyze the changing trends of MWL characteristics over time under different task difficulties. Results showed that, as task difficulty increases, subjective MWL ratings significantly increased, and task performance significantly decreased. Saccade velocity significantly increased, whereas EEG and ECG features exhibited minor changes. With increasing task duration, subjective MWL displayed slight fluctuations, while task performance improved initially and then stabilized. Physiological indicators revealed sustained dynamic changes: during the first half of the task, δ power decreased significantly, whereas θ, α, and β power increased significantly. Heart rate increased significantly, and eye-movement features decreased significantly. In the second half, except for heart rate which decreased, the other indicators gradually stabilized. From a physiological perspective, the results revealed that MWL does not exhibit a monotonous temporal evolution, but is jointly regulated by task adaptation and cognitive resource reallocation, showing a nonlinear evolution from initial fluctuations to later dynamic equilibrium. They also revealed the complementarity of multimodal indicators across different dimensions: subjective evaluations and performance primarily reflect the static dimension of task difficulty, while physiological indicators are more sensitive to the temporal dynamic evolution driven by task duration. These findings provide a reference for the development of dynamic prediction methods of mental workload in long-term multi-task scenarios.