Reinforcement learning (RL) offers a data-driven alternative to model-based control, yet its reliance on online exploration is largely impractical in sensitive industrial processes, where exploratory actions can lead to critical failures. Offline RL provides a safer paradigm but still suffers from biased and imbalanced datasets that lack faulty or diverse operational trajectories, thereby degrading reliability in out-of-distribution regions. We propose GRAM-QT (Graph-based Regime-Aware Mixture-of-Q-Transformers), a regime-aware offline RL framework that performs self-supervised semantic regime partitioning. Process data are represented as directed variable graphs, and a Graph Attention Transformer encoder captures temporal and multivariate dependencies while enabling interpretable regime identification. A dedicated expert Q-Transformer is trained per regime, and control actions are produced by membership-weighted aggregation of the regime-specific Q-values, avoiding global averaging and enabling fine-grained policy learning under limited data. A smooth redistribution of membership further ensures stable operation across regime transitions. On industrial simulators (fed-batch penicillin, lithium-ion battery), GRAM-QT outperforms offline RL (CQL, IQL, TD3BC) and state-of-the-art baselines (A2PR, ReBRAC), achieving the highest yield and charging speed with the lowest variance, while remaining competitive on D4RL (Hopper, Franka-Kitchen). Component-removal confirms graph-based regime discovery, mixture routing, and OOD-aware regularization are each indispensable, and attention-based explainability supports safety-critical deployment.