PhD defence of Hassan Ghraieb

– Prof. Ramon Codina, Department of Civil and Environmental Engineering, Universitat Politècnica de Catalunya, Spain

 

– Prof. Anil Anthony Bharath, Imperial College, London, UK

 

– Dr. Nissrine Akkari, SafranTech, France

 

– Dr. Anca Bleme, Sorbonne University, France

 

– Prof. Elie Hachem, CEMEF Mines Paris, France

 

– Dr. Philipe Méliga, CEMEF Mines Paris, France

 

– IR Jonathan Viquerat, CEMEF Mines Paris, France

 

 

Abstract:

 

This thesis gauges the relevance of deep reinforcement learning (DRL) techniques for the optimal control of fluid mechanical systems. Reinforcement learning (RL) is the process by which an agent learns by trial and error interactions with its environment the succession of actions maximizing its cumulative reward over time. In a deep reinforcement learning context (deep RL or DRL), the agent is a deep neural network based on the neural circuits formed by neurons in the human brain. The coupling between state-of-the-art DRL algorithms and computational fluid dynamics (CFD) solvers and their implementation in a high performance computing context make for the novelties and main objective of the thesis. The CFD resolution framework used to compute the reward provided to the DRL agent relies on the Variational Multiscale (VMS) stabilized finite element method. The latter introduces an a priori decomposition of the numerical solution into large and small-scale components, the general picture being that only the large scales and resolved at the discrete level, while the effect of the small scales is modeled after consistently derived source terms proportional to the residual of the large scale solution. Regarding the DRL algorithms, two different frameworks are considered. The first one has the agent interact only once per episode with its environment to learn the mapping from a constant input state to an optimal action (hence, single-step episodes, and by extension, single-step DRL), and is thus relevant to open loop control, where a desired output is optimized under pre-determined actuation parameters (for instance, a constant inlet velocity). The second one has the agent interact multiple time per episode to learn a more complex state-action relation (hence, multi-step DRL) and is more relevant to closed-loop control, where the output is optimized by continuously adjusting the design parameters to flow measurements. Several test-cases in two and three dimensions (both in laminar and turbulent flow regimes) are successfully tackled and presented to assess the relevance, accuracy and performance of the proposed methodologies, with particular emphasis put on drag reduction and thermal control applications. The obtained results emphasize the high potential of the DRL-CFD framework, and are expected to contribute to further progress towards improved and faster design and control of industrial fluid mechanical systems.

 

 

 

Keywords: Deep Reinforcement Learning, Neural networks, Computational fluid dynamics, Flow control, Thermal control