Proximal Policy Optimisation-driven Stock Trading Using Simple Moving Average Crossover and Higher High-Higher Low Stock Price Structure

Abstract

Due to the fluctuations and unpredictability of financial markets, a rule-based trading approach is often ineffective in an environment where conditions are constantly changing. This project is an automated stock trading system that implements the proximal policy optimization (PPO) reinforcement learning approach and signals a simple moving average (SMA) crossover to identify the trend direction and use higher high, higher low (HH, HL) stock price structure to confirm trends with greater certainty. An SMA crossover also identifies those trends that may reverse; meanwhile, an HH-HL structure confirms a bullish price continuation by detecting market microstructures and maintaining the momentum of the market microstructure. The PPO agent is trained on a customised trading environment that models trade execution realities (transaction cost model, position limits, and risk exposure). Consequently, market states are represented as normalised price trends and crossover dynamics and are reflective of structural price action features. The reward function of the agent is designed to maximise risk-adjusted returns while minimising drawdowns and overtrading. The experiment performed by the researcher based on a large sample of historical equity data (multi-year) showed that the proposed method yielded higher cumulative returns, an increased Sharpe ratio, and drawdown reduction compared to conventional SMA strategies and baseline reinforcement learning models. Moreover, the experimental data imply that the use of policy, gradient optimisation techniques combined with trend and structural awareness features, can lead to increased trading stability and adaptability to different market regimes.