# Direct Numerical Simulation of Turbulent Channel Flow on High-Performance GPU Computing System

^{*}

^{†}

^{‡}

## Abstract

**:**

## 1. Introduction

- (i)
- the development of ejections distributed on spheric-like isosurfaces behind an initial $\Omega $-shaped vortex filament;
- (ii)
- the subsequent development of sweeps distributed on elongated isosurfaces adjacent to the external sides of hairpins’ heads and necks.

## 2. Numerical Techniques

## 3. Flow-Structure Extraction

## 4. Numerical Simulations

- (i)
- 2 Intel Xeon 5660 exa-core CPU processors (12 cores) at 2.8 GHz, with 48 GB GDDR3 RAM;
- (ii)
- 3 Nvidia C-1060 (Tesla) 240-core GPU boards (720 computing cores) at 1.3 GHz, each with 4 GB GDDR3 RAM at 102 GB/s (12 GB available);
- (iii)
- 1 Nvidia GTS-450 (GeForce) 192-core GPU board at 1804 MHz, with 1 GB GDDR5 RAM at 57.7 GB/s (mainly used for visualization);
- (iv)
- storage system including 5 Hard Drives at 7200 rpm, for a total supply of 5 TB.

## 5. Turbulence Statistics

## 6. Flow Structures

## 7. Concluding Remarks

- (i)
- the physical condition for the development and subsequent morphological evolution of a stable hairpin-like vortical structure is the occurrence of ejections distributed onto an isosurface almost equally developed along the streamwise, spanwise and normal-to-the-wall directions (a spheric-like isosurface) behind an initially connected Ω-shaped vortex filament, lying near the wall. These ejections actually constitute the physical mechanism according to which the head of the hairpin is raised upward;
- (ii)
- the physical condition for the development of a complete and persistent hairpin is the subsequent occurrence of sweeps, as distributed on elongated isosurfaces adjacent to the external sides of the neck and legs of the hairpin.

- (ii/a)
- the legs of the hairpin are stably kept near the wall;
- (ii/b)
- the right portion (leg and neck) of the hairpin is characterized by local clockwise particle rotation, the left portion (leg and neck) by counter clockwise local particle rotation.

## Author Contributions

## Conflicts of Interest

## Nomenclature

Roman symbols (upper case) | |

${A}_{ij}=\partial {u}_{i}/\partial {x}_{j}$ | velocity-gradient tensor |

${C}_{fb}=2{\tau}_{w}/\rho {u}_{b}^{2}$ | bulk-velocity friction coefficient |

$Dsc$ | discriminant of characteristic equation |

${D}_{K}$ | viscous-diffusion term of turbulent kinetic-energy transport equation |

${F}_{{u}^{\prime}},{F}_{{v}^{\prime}},{F}_{{w}^{\prime}}$ | flatness factors of velocity fluctuations |

${{F}_{{v}^{\prime}}|}_{peak}$ | peak value of ${F}_{{v}^{\prime}}$ |

$K=\overline{{{u}^{\prime}}_{i}{{u}^{\prime}}_{i}}/2$ | mean turbulent kinetic energy |

${L}_{x},{L}_{y},{L}_{z}$ | domain dimensions along x,y,z (h units) |

${L}_{x}^{+},{L}_{y}^{+},{L}_{z}^{+}$ | domain dimensions along x,y,z (wall units) |

${N}_{x},{N}_{y},{N}_{z}$ | number of grid points along x,y,z |

${N}_{tot}$ | total number of grid points |

${P}_{K}$ | production term of turbulent kinetic-energy transport equation |

$P,Q,R$ | scalar invariants of velocity-gradient tensor |

$PP,QQ$ | parameters in the grid-stretching law |

$Q2$ | second-quadrant event (ejection) |

$Q4$ | fourth-quadrant event (sweep) |

${Re}_{\tau}={u}_{\tau}h/\nu $ | friction-velocity Reynolds number |

${S}_{{u}^{\prime}},{S}_{{v}^{\prime}},{S}_{{w}^{\prime}}$ | skewness factors of velocity fluctuations |

${{S}_{{u}^{\prime}}|}_{peak}$ | peak value of ${S}_{{u}^{\prime}}$ |

${T}_{K}$ | transport term of turbulent kinetic-energy transport equation |

Roman symbols (lower case) | |

$h$ | channel half-height |

$k$ | wavenumber |

$p$ | pressure |

$t$ | time coordinate |

${t}^{+}$ | time coordinate (wall units) |

${t}_{DB}^{tot}$ | total database calculated time |

${{t}_{DB}^{+}|}_{saved}$ | actually saved database calculated time (wall units) |

${u}_{i}\left(u,v,w\right)$ | velocity components along x,y,z |

${{u}^{\prime}}_{i}\left({u}^{\prime},{v}^{\prime},{w}^{\prime}\right)$ | fluctuating-velocity components along x,y,z |

${{u}^{\prime}}_{rms},{{v}^{\prime}}_{rms},{{w}^{\prime}}_{rms}$ | rms velocity fluctuations |

${{{u}^{\prime}}_{rms}|}_{peak}$ | peak value of ${{u}^{\prime}}_{rms}$ |

$-\overline{{u}^{\prime}{v}^{\prime}}$ | Reynolds shear stress |

$-{\overline{{u}^{\prime}{v}^{\prime}}|}_{peak}$ | peak value of $-\overline{{u}^{\prime}{v}^{\prime}}$ |

${u}_{b}$ | bulk velocity |

${u}_{c}$ | centerline velocity |

${u}_{\tau}$ | friction velocity |

${x}_{i}(x,y,z)$ | Cartesian coordinates |

${x}_{i}^{+}({x}^{+},{y}^{+},{z}^{+})$ | Cartesian coordinates (wall units) |

${{y}^{+}|}_{peak}^{{{u}^{\prime}}_{rms}}$ | y-position of ${{{u}^{\prime}}_{rms}|}_{peak}$ (wall units) |

${{y}^{+}|}_{peak}^{-\overline{{u}^{\prime}{v}^{\prime}}}$ | y-position of $-{\overline{{u}^{\prime}{v}^{\prime}}|}_{peak}$ (wall units) |

${{y}^{+}|}_{peak}^{{S}_{{u}^{\prime}}}$ | y-position of ${{S}_{{u}^{\prime}}|}_{peak}$ (wall units) |

${{y}^{+}|}_{peak}^{{F}_{{v}^{\prime}}}$ | y-position of ${{F}_{{v}^{\prime}}|}_{peak}$ (wall units) |

Greek symbols (upper case) | |

$\Delta {t}^{+}$ | time resolution of calculations (wall units) |

$\Delta {x}^{+},\Delta {z}^{+}$ | space resolution of calculations along x,z (wall units) |

$\Delta {y}_{wall}^{+}$ | space resolution of calculations along y at channel wall (wall units) |

$\Delta {y}_{center}^{+}$ | space resolution of calculations along y at channel center (wall units) |

Greek symbols (lower case) | |

$\epsilon $ | average rate of dissipation of turbulent kinetic energy per unit mass |

${\epsilon}_{K}$ | dissipation term of mean turbulent kinetic-energy transport equation |

${\eta}^{+}$ | Kolmogorov space microscale (wall units) |

$\lambda $ | eigenvalue |

${\lambda}_{r}$ | real eigenvalue |

${\lambda}_{cr}$ | real part of complex eigenvalue |

${\lambda}_{ci}$ | imaginary part of complex eigenvalue |

${\left({\lambda}_{ci}\right)}_{th}$ | threshold value of swirling strength |

$\nu $ | fluid kinematic viscosity |

$\rho $ | fluid density |

${\tau}_{w}$ | mean shear stress at wall |

${\tau}_{\eta}^{+}$ | Kolmogorov time microscale (wall units) |

Acronyms | |

CPU | Central Processing Unit |

DNS | Direct Numerical Simulation (of turbulence) |

GPU | Graphic Processing Unit |

## References

- Kim, J.; Moin, P.; Moser, R. Turbulence statistics in fully developed channel flow at low Reynolds number. J. Fluid Mech.
**1987**, 177, 133–166. [Google Scholar] [CrossRef] - Lyons, S.L.; Hanratty, T.J.; McLaughlin, J.B. Large-scale computer simulation of fully developed turbulent channel flow with heath transfer. Int. J. Num. Meth. Fluids
**1991**, 13, 999–1028. [Google Scholar] [CrossRef] - Antonia, R.A.; Teitel, M.; Kim, J.; Browne, L.W.B. Low-Reynolds-number effects in a fully developed turbulent channel flow. J. Fluid Mech.
**1992**, 236, 579–605. [Google Scholar] [CrossRef] - Kasagi, N.; Tomita, Y.; Kuroda, A. Direct numerical simulation of passive scalar field in a turbulent channel flow. ASME J. Heat Transf.
**1992**, 114, 598–606. [Google Scholar] [CrossRef] - Rutledge, J.; Sleicher, C.A. Direct simulation of turbulent flow and heat transfer in a channel. Part I: Smooth walls. Int. J. Num. Meth. Fluids
**1993**, 16, 1051–1078. [Google Scholar] [CrossRef] - Moser, R.D.; Kim, J.; Mansour, N.N. Direct numerical simulation of turbulent channel flow up to $R{e}_{\tau}=590$. Phys. Fluids
**1999**, 11, 943–945. [Google Scholar] [CrossRef] - Abe, H.; Kawamura, H.; Matsuo, Y. Direct numerical simulation of a fully developed turbulent channel flow with respect to the Reynolds number dependence. ASME J. Fluids Eng.
**2001**, 123, 382–393. [Google Scholar] [CrossRef] - Iwamoto, K.; Suzuki, Y.; Kasagi, N. Reynolds number effect on wall turbulence: Toward effective feedback control. Int. J. Heat Fluid Flow
**2002**, 23, 678–689. [Google Scholar] [CrossRef] - Del Alamo, J.C.; Jiménez, J. Spectra of the very large anisotropic scales in turbulent channels. Phys. Fluids
**2003**, 15, L41–L44. [Google Scholar] [CrossRef] - Del Alamo, J.C.; Jiménez, J.; Zandonade, P.; Moser, R.D. Scaling of the energy spectra of turbulent channels. J. Fluid Mech.
**2004**, 500, 135–144. [Google Scholar] [CrossRef] - Tanahashi, M.; Kang, S.J.; Miyamoto, T.; Shiokawa, S.; Miyauchi, T. Scaling law of the fine-scale eddies in turbulent channel flows up to $R{e}_{\tau}=800$. Int. J. Heat Fluid Flow
**2004**, 25, 331–340. [Google Scholar] [CrossRef] - Iwamoto, K.; Kasagi, N.; Suzuki, Y. Direct numerical simulation of turbulent channel flow at $R{e}_{\tau}=2320$. Available online: http://www.nmri.go.jp/turbulence/PDF/symposium/FY2004/Iwamoto.pdf (accessed on 17 February 2016).
- Hoyas, S.; Jiménez, J. Scaling of the velocity fluctuations in turbulent channels up to $R{e}_{\tau}=2003$. Phys. Fluids
**2006**, 18, 011702. [Google Scholar] [CrossRef] - Hu, Z.W.; Morfey, C.L.; Sandham, N.D. Wall pressure and shear stress spectra from direct simulations of channel flow. AIAA J.
**2006**, 44, 1541–1549. [Google Scholar] [CrossRef] - Alfonsi, G.; Primavera, L. Direct numerical simulation of turbulent channel flow with mixed spectral-finite difference technique. J. Flow Visual. Image Proc.
**2007**, 14, 225–243. [Google Scholar] - Lozano-Durán, A.; Flores, O.; Jiménez, J. The three-dimensional structure of momentum transfer in turbulent channels. J. Fluid Mech.
**2012**, 694, 100–130. [Google Scholar] [CrossRef] - Lozano-Durán, A.; Jiménez, J. Effect of the computational domain on direct simulations of turbulent channels up to $R{e}_{\tau}=4200$. Phys. Fluids
**2014**, 26, 011702. [Google Scholar] [CrossRef] - Vreman, A.W.; Kuerten, J.G.M. Comparison of direct numerical simulation databases of turbulent channel flow at $R{e}_{\tau}=180$. Phys. Fluids
**2014**, 26, 015102. [Google Scholar] [CrossRef] - Vreman, A.W.; Kuerten, J.G.M. Statistics of spatial derivatives of velocity and pressure in turbulent channel flow. Phys. Fluids
**2014**, 26, 085103. [Google Scholar] [CrossRef] - Bernardini, M.; Pirozzoli, S.; Orlandi, P. Velocity statistics in turbulent channel flow up to $R{e}_{\tau}=4000$. J. Fluid Mech.
**2014**, 742, 171–191. [Google Scholar] [CrossRef] - Lee, M.; Moser, R.D. Direct numerical simulation of turbulent channel flow up to $R{e}_{\tau}=5200$. J. Fluid Mech.
**2015**, 774, 395–415. [Google Scholar] [CrossRef] - Kim, J.; Moin, P. Application of a fractional-step method to incompressible Navier-Stokes equations. J. Comput. Phys.
**1985**, 59, 308–323. [Google Scholar] [CrossRef] - Alfonsi, G.; Passoni, G.; Pancaldo, L.; Zampaglione, D. A spectral-finite difference solution of the Navier-Stokes equations in three dimensions. Int. J. Numer. Meth. Fluids
**1998**, 28, 129–142. [Google Scholar] [CrossRef] - Alfonsi, G. On direct numerical simulation of turbulent flows. Appl. Mech. Rev.
**2011**, 64, 020802. [Google Scholar] [CrossRef] - Marusic, I.; McKeon, B.J.; Monkewitz, P.A.; Nagib, H.M.; Smits, A.J.; Sreenivasan, K.R. Wall-bounded turbulent flows at high Reynolds numbers: Recent advances and key issues. Phys. Fluids
**2010**, 22, 065103. [Google Scholar] [CrossRef] - Smits, A.J.; McKeon, B.J.; Marusic, I. High-Reynolds number wall turbulence. Annu. Rev. Fluid Mech.
**2011**, 43, 353–375. [Google Scholar] [CrossRef] - Kim, J. Progress in pipe and channel flow turbulence, 1961–2011. J. Turbul.
**2012**, 13. [Google Scholar] [CrossRef] - Jiménez, J. Cascades in wall-bounded turbulence. Annu. Rev. Fluid Mech.
**2012**, 44, 27–45. [Google Scholar] [CrossRef] - Alfonsi, G.; Primavera, L. Temporal evolution of vortical structures in the wall region of turbulent channel flow. Flow Turbul. Combust.
**2009**, 83, 61–79. [Google Scholar] [CrossRef] - Zhou, J.; Adrian, R.J.; Balachandar, S.; Kendall, T.M. Mechanisms for generating coherent packets of hairpin vortices in channel flow. J. Fluid Mech.
**1999**, 387, 353–396. [Google Scholar] [CrossRef] - Le, H.; Moin, P. An improvement of fractional step methods for the incompressible Navier-Stokes equations. J. Comput. Phys.
**1991**, 92, 369–379. [Google Scholar] [CrossRef] - Passoni, G.; Alfonsi, G.; Tula, G.; Cardu, U. A wavenumber parallel computational code for the numerical integration of the Navier-Stokes equations. Parall. Comput.
**1999**, 25, 593–611. [Google Scholar] [CrossRef] - Passoni, G.; Cremonesi, P.; Alfonsi, G. Analysis and implementation of a parallelization strategy on a Navier-Stokes solver for shear flow simulations. Parall. Comput.
**2001**, 27, 1665–1685. [Google Scholar] [CrossRef] - Passoni, G.; Alfonsi, G.; Galbiati, M. Analysis of hybrid algorithms for the Navier-Stokes equations with respect to hydrodynamic stability theory. Int. J. Numer. Meth. Fluids
**2002**, 38, 1069–1089. [Google Scholar] [CrossRef] - Wallace, J.M. Twenty years of experimental and direct numerical simulation access to the velocity gradient tensor: What have we learned about turbulence? Phys. Fluids
**2009**, 21, 021301. [Google Scholar] [CrossRef] - Alfonsi, G. Coherent structures of turbulence: Methods of eduction and results. Appl. Mech. Rev.
**2006**, 59, 307–323. [Google Scholar] [CrossRef] - Alfonsi, G.; Primavera, L. The structure of turbulent boundary layers in the wall region of plane channel flow. Proc. R. Soc. A
**2007**, 463, 593–612. [Google Scholar] [CrossRef] - Alfonsi, G.; Primavera, L. On identification of vortical structures in turbulent shear flow. J. Flow Visual. Image Proc.
**2008**, 15, 201–216. [Google Scholar] - Alfonsi, G. Numerical simulations of wave-induced flow fields around large-diameter surface-piercing vertical circular cylinder. Computation
**2015**, 3, 386–426. [Google Scholar] [CrossRef] - Alfonsi, G.; Primavera, L. Determination of the threshold value of the quantity chosen for vortex representation in turbulent flow. J. Flow Visual. Image Proc.
**2009**, 16, 41–49. [Google Scholar] - Bakewell, H.P.; Lumley, J.L. Viscous sublayer and adjacent wall region in turbulent pipe flow. Phys. Fluids
**1967**, 10, 1880–1889. [Google Scholar] [CrossRef] - Alfonsi, G.; Ciliberti, S.A.; Mancini, M.; Primavera, L. GPGPU implementation of mixed spectral-finite difference computational code for the numerical integration of the three-dimensional time-dependent incompressible Navier-Stokes equations. Comput. Fluids
**2014**, 102, 237–249. [Google Scholar] [CrossRef] - Ciliberti, S.A. Coherent Structures of Turbulence in Wall-Bounded Turbulent Flows. Ph.D. Thesis, Università della Calabria, Rende, Italy, 2011. [Google Scholar]
- Dean, R.B. Reynolds number dependence of skin friction and other bulk flow variables in two-dimensional rectangular duct flow. ASME J. Fluids Eng.
**1978**, 100, 215–223. [Google Scholar] [CrossRef] - Mochizuki, S.; Nieuwstadt, F.T.M. Reynolds-number-dependence of the maximum of the streamwise velocity fluctuations in wall turbulence. Exp. Fluids
**1996**, 21, 218–226. [Google Scholar] [CrossRef] - Alfonsi, G. Analysis of streamwise velocity fluctuations in turbulent pipe flow with the use of an ultrasonic Doppler flowmeter. Flow Turbul. Combust.
**2001**, 67, 137–142. [Google Scholar] [CrossRef] - Xu, C.; Zhang, Z.; den Toonder, J.M.J.; Nieuwstadt, F.T.M. Origin of high kurtosis level in the viscous sublayer. Direct numerical simulation and experiment. Phys. Fluids
**1996**, 8, 1938–1944. [Google Scholar] [CrossRef] - Alfonsi, G. Evaluation of radial velocity fluctuations in turbulent pipe flow by means of an ultrasonic Doppler velocimeter. J. Flow Visual. Image Proc.
**2003**, 10, 155–161. [Google Scholar] [CrossRef] - Kline, S.; Reynolds, W.C.; Schraub, F.A.; Rundstadler, P.W. The structure of turbulent boundary layers. J. Fluid Mech.
**1967**, 30, 741–773. [Google Scholar] [CrossRef]

**Figure 2.**Reynolds shear stress (wall coordinates); $-\overline{{u}^{\prime}{v}^{\prime}}$: (●●●) present work, (—) data from Moser et al. [6]; (

**a**) present work at $R{e}_{\tau}=200$, data from Moser et al. [6] at $R{e}_{\tau}=180$; (

**b**) present work at $R{e}_{\tau}=400$, data from Moser et al. [6] at $R{e}_{\tau}=395$; (

**c**) present work at $R{e}_{\tau}=600$, data from Moser et al. [6] at $R{e}_{\tau}=590$.

**Figure 3.**Rms values of the velocity fluctuations (wall coordinates); ${{u}^{\prime}}_{rms}$: (●●●) present work, (—) data from Moser et al. [6]; ${{v}^{\prime}}_{rms}$: (

**♦♦♦**) present work, (— —) data from Moser et al. [6]; ${{w}^{\prime}}_{rms}$: (▲▲▲) present work, (– –) data from Moser et al. [6]: (

**a**) present work at $R{e}_{\tau}=200$, data from Moser et al. [6] at $R{e}_{\tau}=180$; (

**b**) present work at ${\mathrm{Re}}_{\tau}=400$, data from Moser et al. [6] at $R{e}_{\tau}=395$; (

**c**) present work at $R{e}_{\tau}=600$, data from Moser et al. [6] at $R{e}_{\tau}=590$.

**Figure 4.**Skewness factors of the velocity fluctuations (wall coordinates); ${S}_{{u}^{\prime}}$: (●●●) present work, (—) data from Moser et al. [6]; ${S}_{{v}^{\prime}}$: (

**♦♦♦**) present work, (— —) data from Moser et al. [6]; ${S}_{{w}^{\prime}}$: (▲▲▲) present work, (– –) data from Moser et al. [6]: (

**a**) present work at $R{e}_{\tau}=200$, data from Moser et al. [6] at $R{e}_{\tau}=180$; (

**b**) present work at $R{e}_{\tau}=400$, data from Moser et al. [6] at $R{e}_{\tau}=395$; (

**c**) present work at $R{e}_{\tau}=600$, data from Moser et al. [6] at $R{e}_{\tau}=590$.

**Figure 5.**Flatness factors of the velocity fluctuations (wall coordinates); ${F}_{{u}^{\prime}}$: (●●●) present work, (—) data from Moser et al. [6]; ${F}_{{v}^{\prime}}$: (

**♦♦♦**) present work, (— —) data from Moser et al. [6]; ${F}_{{w}^{\prime}}$: (▲▲▲) present work, (– –) data from Moser et al. [6]: (

**a**) present work at $R{e}_{\tau}=200$, data from Moser et al. [6] at $R{e}_{\tau}=180$; (

**b**) present work at $R{e}_{\tau}=400$, data from Moser et al. [6] at $R{e}_{\tau}=395$; (

**c**) present work at $R{e}_{\tau}=600$, data from Moser et al. [6] at $R{e}_{\tau}=590$.

**Figure 6.**Terms of the turbulent kinetic-energy transport equation; ${P}_{K}$: (●●●) present work, (—) data from Moser et al. [6]; ${T}_{K}$: (

**♦♦♦**) present work, (····) data from Moser et al. [6]; ${D}_{K}$: (▲▲▲) present work, (— —) data from Moser et al. [6]; ${\epsilon}_{K}$: (■■■) present work, (– –) data from Moser et al. [6]: (

**a**) present work at $R{e}_{\tau}=200$, data from Moser et al. [6] at $R{e}_{\tau}=180$; (

**b**) present work at $R{e}_{\tau}=400$, data from Moser et al. [6] at $R{e}_{\tau}=395$; (

**c**) present work at $R{e}_{\tau}=600$, data from Moser et al. [6] at $R{e}_{\tau}=590$.

**Figure 7.**Side view of vortical structures in the computing domain: (

**a**) $R{e}_{\tau}=200$; (

**b**) $R{e}_{\tau}=600$.

**Figure 8.**General view of vortical structures on both walls of computing domain at $R{e}_{\tau}=600$ (vortical structures are colored with values of local streamwise velocity, reddish indicates high values, greenish indicates low values).

**Figure 9.**General view of vortical structures and $Q2$ / $Q4$ quadrant events on lower wall of computing domain at $R{e}_{\tau}=400$ (vortical structures are shown in cyan, isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow).

**Figure 10.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=450$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 11.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=451$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 12.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=452$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 13.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=453$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 14.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=220$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 15.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=221$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 16.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=222$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

**Figure 17.**${\lambda}_{ci}-$ isosurface representation of vortical structures in conjunction with quadrant events at ${t}^{+}=223$ (isosurfaces of ejections are shown in red, isosurfaces of sweeps are shown in yellow): (

**a**) vortical structures are shown in cyan; (

**b**) vortical structures are colored with the values of ${\lambda}_{r}$ (reddish indicates stretching, bluish indicates compression).

Author(s) | Year | Numerical Technique |
---|---|---|

Kim et al. [1] | 1987 | Spectral |

Lyons et al. [2] | 1991 | Spectral |

Antonia et al. [3] | 1992 | Spectral |

Kasagi et al. [4] | 1992 | Spectral |

Rutledge and Sleicher [5] | 1993 | Spectral |

Moser et al. [6] | 1999 | Spectral |

Abe et al. [7] | 2001 | Finite Difference |

Iwamoto et al. [8] | 2002 | Spectral |

Del Alamo and Jiménez [9] | 2003 | Spectral |

Del Alamo et al. [10] | 2004 | Spectral |

Tanahashi et al. [11] | 2004 | Spectral-Finite Difference |

Iwamoto et al. [12] | 2005 | Spectral |

Hoyas and Jiménez [13] | 2006 | Spectral-Finite Difference |

Hu et al. [14] | 2006 | Spectral |

Alfonsi and Primavera [15] | 2007 | Spectral-Finite Difference |

Lozano-Durán et al. [16] | 2012 | Spectral |

Lozano-Durán and Jiménez [17] | 2014 | Spectral |

Vreman and Kuerten [18] | 2014 | Spectral |

Vreman and Kuerten [19] | 2014 | Spectral |

Bernardini et al. [20] | 2014 | Finite Difference |

Lee and Moser [21] | 2015 | Spectral |

Quantities | $R{e}_{\tau}=200$ | $R{e}_{\tau}=400$ | $R{e}_{\tau}=600$ |
---|---|---|---|

${L}_{x}$ | $4\pi h$ | $4\pi h$ | $4\pi h$ |

${L}_{y}$ | $2h$ | $2h$ | $2h$ |

${L}_{z}$ | $2\pi h$ | $2\pi h$ | $2\pi h$ |

${L}_{x}^{+}$ | 2513 | 5026 | 7540 |

${L}_{y}^{+}$ | 400 | 800 | 1200 |

${L}_{z}^{+}$ | 1256 | 2513 | 3770 |

${N}_{x}$ | 256 | 343 | 512 |

${N}_{y}$ | 181 | 321 | 451 |

${N}_{z}$ | 256 | 343 | 512 |

${N}_{tot}$ | $11.9\times {10}^{6}$ | $37.8\times {10}^{6}$ | $118.2\times {10}^{6}$ |

$\Delta {x}^{+}$ | 9.82 | 14.65 | 14.73 |

$\Delta {y}_{wall}^{+}$ | 0.25 | 0.28 | 0.30 |

$\Delta {y}_{center}^{+}$ | 3.87 | 4.36 | 4.66 |

$\Delta {z}^{+}$ | 4.91 | 7.33 | 7.36 |

${\eta}^{+}$ | 1.89 | 2.19 | 2.42 |

$\Delta {x}^{+}/{\eta}^{+}$ | 5.20 | 6.69 | 6.09 |

$\Delta {y}_{wall}^{+}/{\eta}^{+}$ | 0.13 | 0.13 | 0.12 |

$\Delta {y}_{center}^{+}/{\eta}^{+}$ | 2.05 | 1.99 | 1.93 |

$\Delta {z}^{+}/{\eta}^{+}$ | 2.6 | 3.35 | 3.04 |

$\Delta {t}^{+}$ | 0.02 | 0.04 | 0.06 |

$\Delta t$ | $1\times {10}^{-4}h/{u}_{\tau}$ | $1\times {10}^{-4}h/{u}_{\tau}$ | $1\times {10}^{-4}h/{u}_{\tau}$ |

${t}_{DB}^{tot}$ | 50·$h/{u}_{\tau}$ | 50·$h/{u}_{\tau}$ | 50·$h/{u}_{\tau}$ |

${{t}_{DB}^{+}|}_{saved}$ | 500·$\Delta {t}^{+}$ | 500·$\Delta {t}^{+}$ | 500·$\Delta {t}^{+}$ |

${\tau}_{\eta}^{+}$ | 3.56 | 4.79 | 5.87 |

$\Delta {t}^{+}/{\tau}_{\eta}^{+}$ | 0.006 | 0.008 | 0.010 |

**Table 3.**Runtime of the calculations with different computing-platform configurations (seconds per $\Delta t$).

CPU/GPU Cores | $R{e}_{\tau}=200$ | $R{e}_{\tau}=400$ | $R{e}_{\tau}=600$ |
---|---|---|---|

1 CPU/240 GPU Cores | 0.37 | 1.71 | - |

3 CPU/720 GPU Cores | - | - | 3.32 |

Quantities | $R{e}_{\tau}=200$ | $R{e}_{\tau}=400$ | $R{e}_{\tau}=600$ |
---|---|---|---|

${u}_{\tau}$ (nominal) | 200 | 400 | 600 |

${u}_{\tau}$ (present work) | 200.23 | 399.94 | 600.55 |

${u}_{b}$ (nominal, after Dean [44]) | 3390.71 | 7254.35 | 11,343.22 |

${u}_{b}$ (present work) | 3197.67 | 6966.95 | 11,106.17 |

${u}_{c}$ (nominal, after Dean [44]) | 3918.71 | 8310.35 | 12,927.22 |

${u}_{c}$ (present work) | 3706.26 | 7978.80 | 12,701.63 |

${u}_{b}/{u}_{\tau}$ (nominal, after Dean [44]) | 16.95 | 18.14 | 18.91 |

${u}_{b}/{u}_{\tau}$ (present work) | 16.97 | 17.42 | 18.49 |

${u}_{c}/{u}_{\tau}$ (nominal, after Dean [44]) | 19.59 | 20.78 | 21.55 |

${u}_{c}/{u}_{\tau}$ (present work) | 18.51 | 19.95 | 21.15 |

${u}_{c}/{u}_{b}$ (nominal, after Dean [44]) | 1.16 | 1.15 | 1.14 |

${u}_{c}/{u}_{b}$ (present work) | 1.16 | 1.14 | 1.14 |

${C}_{fb}$ (nominal, after Dean [44]) | $8.04\times {10}^{-3}$ | $6.65\times {10}^{-3}$ | $5.95\times {10}^{-3}$ |

${C}_{fb}$ (present work) | $7.86\times {10}^{-3}$ | $6.86\times {10}^{-3}$ | $6.58\times {10}^{-3}$ |

$-{\overline{{u}^{\prime}{v}^{\prime}}|}_{peak}$ (present work) | 0.739 | 0.828 | 0.866 |

$-{\overline{{u}^{\prime}{v}^{\prime}}|}_{peak}$ (from Moser et al. [6]) | 0.723 | 0.837 | 0.864 |

${{y}^{+}|}_{peak}^{-\overline{{u}^{\prime}{v}^{\prime}}}$ (present work) | 30.238 | 40.170 | 43.938 |

${{y}^{+}|}_{peak}^{-\overline{{u}^{\prime}{v}^{\prime}}}$ (from Moser et al. [6]) | 30.019 | 41.882 | 44.698 |

${{{u}^{\prime}}_{rms}|}_{peak}$ (present work) | 2.680 | 2.720 | 2.751 |

${{{u}^{\prime}}_{rms}|}_{peak}$ (from Moser et al. [6]) | 2.660 | 2.740 | 2.770 |

${{y}^{+}|}_{peak}^{{{u}^{\prime}}_{rms}}$ (present work) | 14.909 | 14.199 | 13.444 |

${{y}^{+}|}_{peak}^{{{u}^{\prime}}_{rms}}$ (from Moser et al. [6]) | 15.281 | 14.209 | 13.268 |

${{S}_{{u}^{\prime}}|}_{peak}$ (present work) | 1.003 | 1.096 | 1.141 |

${{S}_{{u}^{\prime}}|}_{peak}$ (from Moser et al. [6]) | 0.922 | 1.013 | 1.066 |

${{y}^{+}|}_{peak}^{{S}_{{u}^{\prime}}}$ (present work) | 1.315 | 1.423 | 1.504 |

${{y}^{+}|}_{peak}^{{S}_{{u}^{\prime}}}$ (from Moser et al. [6]) | 1.339 | 1.446 | 1.591 |

${{F}_{{v}^{\prime}}|}_{peak}$ (present work) | 26.679 at ${y}^{+}=0.249$ | 19.424 at ${y}^{+}=0.280$ | 20.882 at ${y}^{+}=0.298$ |

${F}_{{v}^{\prime}}|$ (from Moser et al. [6]) | 26.712 at ${y}^{+}=0.249$ | 34.757 at ${y}^{+}=0.280$ | 37.653 at ${y}^{+}=0.298$ |

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Alfonsi, G.; Ciliberti, S.A.; Mancini, M.; Primavera, L.
Direct Numerical Simulation of Turbulent Channel Flow on High-Performance GPU Computing System. *Computation* **2016**, *4*, 13.
https://doi.org/10.3390/computation4010013

**AMA Style**

Alfonsi G, Ciliberti SA, Mancini M, Primavera L.
Direct Numerical Simulation of Turbulent Channel Flow on High-Performance GPU Computing System. *Computation*. 2016; 4(1):13.
https://doi.org/10.3390/computation4010013

**Chicago/Turabian Style**

Alfonsi, Giancarlo, Stefania A. Ciliberti, Marco Mancini, and Leonardo Primavera.
2016. "Direct Numerical Simulation of Turbulent Channel Flow on High-Performance GPU Computing System" *Computation* 4, no. 1: 13.
https://doi.org/10.3390/computation4010013