[Mpi-forum] fftw and linear solver
tlk0812 at hotmail.com
Thu Aug 14 17:43:44 CDT 2014
I have a general question of implementing FFTW and KSPSolve in Petsc.
My code consists mainly two parts:
part1: calculating the values of arrays, which takes a convolution form. So I am using FFTW for high efficiency.
part2: solve a linear system Ax=b, where A is constant and b is provided from part1.
In part1, the array size is 512*512, and I found that taking np=8 gives optimal time cost ~0.025s. But I have 225 of such arrays and they can be done speparately.
In part2, the vector b is composed of the arrays calculated before, the size of b is about 50000. It takes around 1s to do KSPSolve when choosing np=8, but futher increase of np will slow down KSPSolve by a factor of 3~4.
To save time, I use MPI_Comm_split to divide MPI_COMM_WORLD into 15 sub-communicators and do FFTW in parallel. Due to the overhead in transferring data and increasing time cost in part2, I found by taking np=16, and using one processor to solve each FFTW (and solve 15 FFTW in parallel) is the fastest. The total time for part1 and part2 is about 3s.
I have tried my best to get the optimal result, but I am not sure if there is any way to improve efficiency. I would very much appreciate your comments and suggestions.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the mpi-forum