FLOW36: A spectral solver for phase-field based multiphase turbulence simulations on heterogeneous computing architectures

We present FLOW36, a GPU-ready solver for interface-resolved simulations of multiphase turbulence. The simulation framework relies on the coupling of direct numerical simulation of turbulence, used to describe the flow field, with a phase-field method, used to describe the shape and deformation of a deformable interface and the presence of surfactants. An additional transport equation for a passive scalar can be solved to describe heat transfer in multiphase turbulence. The governing equations are solved in a cuboid domain bounded by two walls along the wall-normal direction where no-slip, free-slip or fixed/moving wall boundary conditions can be applied, while periodicity is applied along the streamwise and spanwise directions. The numerical method relies on a pseudo-spectral approach where Fourier series (periodic directions) and Chebyshev polynomials (wall-normal direction) are used to discretize the governing equations in space. Equations are advanced in time using an implicit-explicit scheme. From a computational perspective, FLOW36 relies on a multilevel parallelism. The first level of parallelism relies on the message-passing interface (MPI). A second level of parallelism uses OpenACC directives and cuFFT libraries; this second level is used to accelerate the code execution when heterogeneous computing infrastructures are targeted. In this work, we present the numerical method and we discuss the main implementation strategies, with particular reference to the MPI and OpenACC directives and code portability, performance and maintenance strategies. FLOW36 is released open source under the GPLv3 license. Program summary: Program Title: FLOW36 CPC Library link to program files: https://doi.org/10.17632/ygcn7dsb9k.1 Developer's repository link: https://github.com/MultiphaseFlowLab/FLOW36 Licensing provisions: GPLv3 License Programming language: Modern Fortran Nature of problem: Solving the three-dimensional incompressible Navier–Stokes equations in a Cartesian domain configured for open and closed channel flows. A phase-field method is used to describe the shape and topological changes of deformable interfaces. Additional equations are included to account for the presence of surfactants, heat transfer problems and for the transport of point-wise Lagrangian particles. Solution method: The system of governing equations is advanced in time using an implicit-explicit strategy while the governing equations are discretized in space using a pseudo-spectral approach: Fourier series are employed along the homogeneous directions while Chebyshev polynomial along the wall-normal direction. A first order explicit Euler method is used to advance the equations for the Lagrangian particles motion. A two-dimensional pencil distributed MPI parallelization is implemented and OpenACC directives are used to execute the code on GPUs.