Message: Re: Clarifying the correct way to run MT jobs with MPI on a cluster with multiple nodes Not Logged In (login)
 Next-in-Thread Next-in-Thread
 Next-in-Forum Next-in-Forum

Feedback Re: Clarifying the correct way to run MT jobs with MPI on a cluster with multiple nodes 

Forum: Multithreading
Re: Question Clarifying the correct way to run MT jobs with MPI on a cluster with multiple nodes (Sam Blake)
Date: 07 Apr, 2016
From: Andrea Dotti <Andrea Dotti>

Hello Sam, sorry for the long delay in answering this. It slipped out of my e-mail inbox

What you would like to do is probably the following: - Have 21 MPI ranks. - The 21 ranks should be distributed one per node - Use multi-threading to use the entire node

I suggest you to use, instead of "/run/numberOfThreads <n>" the macro command: "/run/useMaximumLogicalCores". In this way the G4 application will use always the maximum number of available cores. This should remove one parameter (n).

At this point k becomes the number of nodes you want to run on (k=21).

The tricky part is to convince PBS to submit your jobs one per node and ask the full node. I am not an expert of PBS and you should verify with your sys admins (note that sys admins can block some type of requests). But I think that:

#Give me 21 nodes and I want all 12 processors per node
#PBS -l nodes=21:ppn=12

So: i=k=21 and j=12

Suggestion, in your main, put the MPI call to get the host name so you can verify that the jobs end in all different nodes.

Andrea

 Add Message Add Message
to: "Re: Clarifying the correct way to run MT jobs with MPI on a cluster with multiple nodes"

 Subscribe Subscribe

This site runs SLAC HyperNews version 1.11-slac-98, derived from the original HyperNews