Robotic systems rely on a series of connected control cycles, each with its own time limit and deadline that is crucial to the overall operation.
Researchers at Washington State University have developed an artificial-intelligence framework that automatically manages that series of cascading deadlines, which could help robots and other cyber-physical systems make safe decisions under rapidly changing circumstances. The work was tested successfully in simulations and on a UniTree Go2, a “robotic dog” available on the commercial market.
The advance could improve the reliability of AI deployed in self-driving vehicles, industrial robots, drones, and other systems where computational resources fluctuate, and delayed decisions can create unsafe conditions.
Known as Dynamic Time Reinforcement Learning (DTRL), is the first reinforcement-learning framework designed specifically for cyber-physical systems operating under dynamic timing requirements. The research was accepted to the 47th IEEE Real-Time Systems Symposium, held in December in Yokohama, Japan.
In reinforcement learning, an AI agent “learns” to improve its decision-making capacity through trial-and-error. The new framework, rather than assuming a fixed computation time for every control decision, continuously adjusts how much neural network computation is performed based on the time available before each deadline.
“Real-world robot systems rarely have identical computation time available at every control cycle,” said Mengyu Liu, assistant professor in the School of Engineering and Applied Sciences at WSU Tri-Cities. “Instead of forcing AI to always make either the fastest or the most accurate decision, DTRL intelligently adapts to the computation time available to balance the control performance and safety.”
Many existing AI controllers compress neural networks into permanently simplified models to satisfy worst-case deadlines or maintain multiple independent models that switch during execution. These approaches often sacrifice control performance, require significantly more memory, or produce inconsistent actions.
DTRL instead integrates early-exit neural networks, which make decisions as soon as they reach high confidence rather than completing a full computation, directly into reinforcement learning. That allows a single policy — or set of decision-making rules — to execute at multiple computation depths while maintaining consistent control across different timing budgets.
Unlike previous adaptive inference techniques originally developed for computer vision, DTRL introduces reinforcement-learning-specific training methods that preserve the consistency of policy across decision points.
The researchers evaluated DTRL on a Unitree Go2 robot across multiple control tasks, including pendulum stabilization, deadline-sensitive reach-avoid navigation, and quadrupedal locomotion. Across both simulation and hardware experiments, DTRL consistently achieved a stronger balance between deadline satisfaction, control performance, and action consistency than conventional approaches.
By combining adaptive neural network inference with real-time systems principles, the framework enables AI controllers to better utilize available computing resources while maintaining predictable timing behavior required for safety-critical applications.
