Open PhD position INRIA/CEMEF: Reinforcement learning for the optimal locomotion of micro-swimmers in a complex chaotic environment.

    PHD THESIS INRIA/CEMEF 2021: Reinforcement learning for the optimal locomotion of micro-swimmers in a complex chaotic environment.

    Context

    Swimming bacteria, spermatozoa, or plankton are natural examples of self-propelled, active particles. These living microorganisms have the ability to deform or alter their internal features according to their environment in order to achieve a specific goal. Finding inspiration from such adaptive behaviors is key for the design of artificial devices used in medicine for micro-surgery, targeted drug delivery or diagnosis [1].
    Many questions are still open on how these micro-swimmers optimize their displacement, in particular when they are embedded in a complex environment [2]. In practice the swimmers navigate in a fluctuating medium comprising walls and obstacles and possibly with non-Newtonian properties. They sometimes form swarms, synchronize and display collective behaviors.
    In such situations, the dynamics of the swimmers involve a number of physical effects (hydrodynamics, elasticity, contact and lubrification forces, mutual interactions), nonlinearly coupled with each other. Such a high-dimensional, often chaotic dynamics hinders the use of classical techniques from optimal control. A promising alternative is given by strategies based on machine learning [3,4].

    Global objective of work

    When flagellated micro-swimmers are immersed in a fluid flow, the undulations of their tail compete with the currents, stresses and deformations inflicted by their environment. The idea of this thesis is to couple numerical simulations of these swimmers with reinforcement learning algorithms in order to build optimal strategies combining an efficient swimming technique and smart navigation in a complex environment.

    Detailed presentation

    This PhD focuses on flagellated micro-swimmers that move by waving their tail and pushing the fluid in which they are immersed [5]. They are modelled as semi-elastic slender bodies whose curvature varies by prescribing a locomotion force. The aim is to use machine- learning techniques in order to find optimal force controls that will allow the swimmer to efficiently swim and navigate in complex media. The plan is to address simultaneously the questions of finding an efficient swimming strategy and of optimizing the swimmers navigation. The combination of these two aspects is particularly novel. This is justified by the focused applications, in which the size of swimmers is comparable to the scales on which the environment varies. Developing such aspects will require an important modelling effort and to design innovative optimization tools.
    Various environments will be considered. They are characterized by variations of the fluid flow around the swimmers that originate from the presence of walls, from non-Newtonian elastic turbulence, or from hydrodynamic interactions between several swimmers. The dynamical system formed by the swimmers and the fluid is then expected to exhibit a chaotic behavior, jeopardizing the convergence of learning algorithms. A part of the thesis consists in addressing this issue by developing original learning algorithms based on multiagent approaches.
    The proposed work is divided into three parts.
    1. The first step will consist in using existing codes to simulate the dynamics of isolated flagellated swimmers in turbulent flows, in the presence or not of boundaries, and to test various learning strategies. Particular attention will be paid to quantify the convergence of adversarial reinforcement learning methods that have shown their efficiency in time- dependent flows [6]. This will require developing appropriate statistical tools and will lead to design new procedures that improve this convergence.
    2. The second part of the PhD will study the effects of a non-Newtonian fluid rheology. This will require modifying the simulation codes and to implement visco-elastic effects in the interactions between the swimmer and the fluid [7]. The goal is to understand how this alters swimming and navigation strategies.
    3. Finally, the last step will consist in addressing optimal collective motions of micro- swimmers. This will require implementing interactions between swimmers in the code. Optimization will be achieved using Kriging methods [8]. This approach will permit emulating the swarm dynamics as a function of each individual deformation strategy from a few number of selected simulations.

    References :
    [1] Palagi & Fischer, Nature Rev. Mat. 3:113, 2018.
    [2] Eur. Phys. J. Special Topics 225 (No 11-12), 2016.
    [3] Cichos et al., Nat. Mach. Intell. 2:94, 2020.
    [4] Muiños-Landin et al., Science Robotics 6:eabd9285, 2021.
    [5] Berti, Giraldi & Prud’Homme, ESAIM Proc. Surv. 67:46, 2020.
    [6] Jaya Kumar, Verma, Bec & Pandit, Phys. Rev. E 101:043110, 2020.
    [7] Shen & Arratia, Phys. Rev. Lett. 106:208101, 2011.
    [8] Rasmussen & Williams, Gaussian processes for machine learning, MIT Press, 2006.


    Candidate profile and skills

    - Master of Science or equivalent in applied mathematics, physics, or engineering, with competences in fluid dynamics, statistics, optimization, or scientific computing
    - Basic knowledge in programming (C, C++, Python) and in data analysis
    - Rigorous, autonomous, creative

    Apply to this PhD offer

    General informations

    • Industrial field: Computational Mathematics, High Performance Computing and Data
    • Location: Valbonne - Sophia Antipolis (06), on the French Riviera, France.
    • Keywords: Statistical learning, Optimization, Active particles, Flagellated micro-swimmers, Chaotic dynamics, Turbulence.
    • Duration: 3 years from October 4, 2021

    Contacts

    • Team: CALISTO, common project-team between Inria and Cemef
    • Supervisor: Jérémie BEC

    To apply

    Please follow the procedure on:
    https://jobs.inria.fr/public/classic/en/offres/2021-03572

    > back to Open PhD positions