Message: Saving partial results Not Logged In (login)

None Saving partial results 

Forum: Multithreading
Date: 18 Aug, 2015
From: <herr_apa>

Hi,

I'm running a simulation that includes movement. In my GeneratePrimaries, I offset the position of the initial particle by an amount that depends on the event number. The position of the particle is x(t), and t = N/Ntot where N is the event number of the current event and Ntot is the total number of events to be simulated. The quantity of interest is the energy deposited in different parts of the target, which I record using a G4PSEnergyDeposit.

I would like to make a "movie" of how the dose distribution develops as a function of time. That is, I want to save the output of the simulation every so many events. For the sake of example, let's say I'm simulating 1000 events and I want a snapshot of the output every 50. In single-threaded mode, I could simply make an EndOfEventAction to check the current event number N, and if (N % 50) == 0 I could fetch the deposited energy from my G4THitsMap and write it to file. In multithreaded mode this approach is not possible because each thread only has access to the events it happened to process, and only one of the threads will process the event that triggers the write.

Question 1: When a thread detects that (N % 50) == 0, is it possible to somehow merge all the events processed so far across all threads so that the output can be written to file?

Another approach could be to tell Geant4 "I want to simulate 1000 events - But start the run at event 400 and stop it at event 449!" That way the time dependence and merging would happen as normal, and I could just split the run into many sub runs that start and stop at the appropriate event numbers. The output would not be cumulative for each segment of the run like with the first approach, but this could be fixed by post-processing.

Question 2: Is there a convenient way to process only a subrange of the events in a run? Maybe a check in BeginOfEventAction to skip the entire event unless it is in the desired range. Can this be done?

Suggestions on other approaches are also welcome. I might very well be missing something obvious.

// Anders

Inline Depth:
 0 0
 All All
Outline Depth:
 1 1
 2 2
 All All
Add message: (add)

Feedback Re: Saving partial results  by Andrea Dotti <Andrea Dotti>,   26 Aug, 2015
Re: None Saving partial results
Hello, sorry for the late answer.

> 
> Question 1: When a thread detects that (N % 50) == 0, is it possible to
> somehow merge all the events processed so far across all threads so that
> the output can be written to file?
> 

No this is not possible, because this would require synchronization between threads during event loop that would cause a large CPU penalty.

> 
> Question 2: Is there a convenient way to process only a subrange of the
> events in a run? Maybe a check in BeginOfEventAction to skip the entire
> event unless it is in the desired range. Can this be done?

I think you have already found the correct solution to your problem. The solution is to make separate runs. A G4UserRunAction should be used. For example: t=Nevt/Ntot + Ntot*Nrun where Nrun is the run number.

Geant4 will guarantee that G4Run are correctly merged and threads are synchronized between the runs that compose your job.

Andrea

None Re: Saving partial results  by <herr_apa>,   20 Aug, 2015
Re: None Saving partial results
Hi again,

I've worked some more on this problem, and now I have a different question: In multithreaded mode, how can I prompt a call to G4VUserActionInitialization::Build after having already called beamOn once?

And here's some context if that helps. Let's go back to the 1000 events that I wanted to split up into 50-event slices. What I do now is loop over beamOn and execute twenty runs 50 events each. The position of the initial particle is calculated by a dedicated class called ParticlePosition, that defines x(t). I added two new variables to this class: The total number of events to be simulated, Nsim (1000 in this example), and the number of events completed by previous runs, Noffset. Time then becomes t = (Noffset + N)/Nsim where N is the event number of the current event (i.e. a number between 0 and 49 in this example). Before each call to beamOn, I update Noffset. And this is where I run into trouble.

The program is structured such that an ActionInitialization is created in the main function. This ActionInitialization has a ParticlePosition variable. In ActionInitialization::Build a PrimaryGeneratorAction is created, and the ParticlePosition is passed to its constructor. Everything works fine the first time beamOn is called. For each subsequent turn in the loop, I update the Noffset in the ParticlePosition owned by the ActionInitialization and call runManager->SetUserInitialization(actionInitialization) before the next call to beamOn. Had it been a single-threaded G4RunManager, Build would have been invoked. But because it is a G4MTRunManager, only BuildForMaster is invoked. No new PrimaryGeneratorAction object is created, because that happens in Build. How can I relay the updated ParticlePosition to my PrimaryGeneratorAction? Is there a way to destroy the worker threads and then re-create them (which would invoke Build)?

Thanks in advance!

// Anders

1 None: Re: Saving partial results   (herr_apa - 25 Aug, 2015)
2 Warning: Re: Saving partial results   (Andrea Dotti - 26 Aug, 2015)
 Add Message Add Message
to: "Saving partial results"

 Subscribe Subscribe

This site runs SLAC HyperNews version 1.11-slac-98, derived from the original HyperNews