Green ammonia production systems powered by intermittent renewable energy must meet periodic demand under tight unit and storage constraints. We propose Q-learning-based stochastic model predictive control, a methodology integrating a stochastic model predictive control framework with a Q-function as the terminal cost. The proposed method explicitly enforces hard constraints, effectively manages both short-term and long term disturbances, and offers significant advantages in terms of on-line computational speed. Simulation results show that the proposed method outperforms Nonlinear Model Predictive Control, Double Deep Q-Network, and Q-learning-based Model Predictive Control baselines. The proposed method achieves the lowest total cost, minimal soft constraint penalties, and eliminates both tank overflow and ammonia demand shortfall, enabling practical, real-time operation.