[Mpi-forum] fftw and linear solver

[LikunTan]

Hi all,

I have a general question of implementing FFTW and KSPSolve in Petsc.

My code consists mainly two parts: 
part1: calculating the values of arrays, which takes a convolution form. So I am using FFTW for high efficiency.
part2: solve a linear system Ax=b, where A is constant and b is provided from part1.

In part1, the array size is 512*512, and I found that taking np=8 gives optimal time cost ~0.025s. But I have 225 of such arrays and they can be done speparately.

In part2, the vector b is composed of the arrays calculated before, the size of b is about 50000. It takes around 1s to do KSPSolve when choosing np=8, but futher increase of np will slow down KSPSolve by a factor of 3~4.

To save time, I use MPI_Comm_split to divide MPI_COMM_WORLD into 15 sub-communicators and do FFTW in parallel. Due to the overhead in transferring data and increasing time cost in part2, I found by taking np=16, and using one processor to solve each FFTW  (and solve 15 FFTW in parallel)  is the fastest.  The total time for part1 and part2 is about 3s.

I have tried my best to get the optimal result, but I am not sure if there is any way to improve efficiency. I would very much appreciate your comments and suggestions.

Thanks,
Likun




 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-forum/attachments/20140815/581648a6/attachment.html>