MAF-RL: Multi-Source Actor-Critic fusion reinforcement learning for dynamic decision systems

Abstract

Dynamic decision systems increasingly depend on the integration of heterogeneous information sources—textual, visual, contextual, and relational—to achieve adaptive and context-aware intelligence. However, existing learning frameworks often process these signals in isolation, limiting their ability to adapt decisions over time. To address this challenge, we propose MAF-RL, a Multi-Source Actor-Critic Fusion Reinforcement Learning (RL) framework that formulates sequential recommendation primarily as an RL problem and uses multi-source fusion to construct expressive state representations for the agent. The principal novelty of MAF-RL lies not in the fusion operator itself, but in its role as a decision-aware state construction mechanism. Diverse data streams—including sequential histories, textual semantics, visual representations, contextual metadata, and relational signals—are integrated into a unified RL state that is optimized end-to-end through long-horizon Actor-Critic policy learning rather than short-term prediction loss. The Actor-Critic architecture, optimized through Proximal Policy Optimization (PPO), learns dynamic policies guided by a multi-objective reward that balances immediate performance, novelty, and strategic repetition. By grounding policy learning on fused multi-source states, this formulation enables the agent to reason over multi-source evidence and adapt actions across evolving environments. Empirical evaluation on three large-scale multi-source benchmarks —MovieLens-1 M, Amazon-Books, and Yelp —demonstrates that MAF-RL consistently outperforms state-of-the-art baselines, achieving superior ranking accuracy (HR@10, NDCG@10) and a better trade-off between repetition and novelty (RR@10, Novelty@10). Overall, MAF-RL should be viewed as an RL-based sequential decision framework whose effectiveness derives from its multi-source state construction, enabling more adaptive and principled behavior in dynamic recommendation settings.