Message: Re: Memory use by ROOT histograms Not Logged In (login)
 Next-in-Thread Next-in-Thread
 Next-in-Forum Next-in-Forum

None Re: Memory use by ROOT histograms 

Forum: Analysis
Re: Question Memory use by ROOT histograms (Alexander Zyazin)
Date: 23 May, 2014
From: Andreas Pfeiffer <Andreas Pfeiffer>

Hi Alexander,

> In my application I want to store a separate 2D histogram for each
> event. Naively, I would think that if I run 10000 events, and if each
> histogram is 100x100 bins, they should consume only 8*100*100*10000=800
> MiB plus some small overhead. However, in reality Geant4 seems to
> allocate ~17 times more memory during histogram creation. In the MT
> mode, this becomes really painful as that amount of memory is allocated
> by every thread and by a master.
> 
> Here is my code:
> 
> void RunAction::BeginOfRunAction(const G4Run* run)
> {
> if(const Run* theRun = dynamic_cast<const Run*>(run))
> {
>  G4int nbEvents = theRun->GetNumberOfEventToBeProcessed();
>  G4AnalysisManager* anManager = G4AnalysisManager::Instance();  
>  anManager->SetFirstHistoId(0);
>  anManager->CreateH2("h0","h0", nbXbins, 1, nbXbins, nbYbins, 1, nbYbins);
> std::ostringstream ostr;
>   G4String histName;
>   for (G4int i = 1; i<=nbEvents; ++i)
>   {  
>       ostr<<"h"<<i;
>       histName = G4String(ostr.str()); 
>       anManager->CreateH2(histName,histName,nbXbins,1,nbXbins,nbYbins,1,nbYbins);
>       ostr.str("");
>       ostr.clear();
>   }
> }
> }
> 
> As a workaround I tried to book histograms in BeginOfEventAction(), so
> that no memory is spent on thread-local histograms that will never be
> filled, but merging of histograms booked like that is challenging. Is
> there any solution to this problem?

a histogram usually stores more than just one double for each bin (otherwise there 
would be no way to calculate for example the errors for each bin in case weights != 1
are used when filling). What kind of histogram types you are using ? Root directly ?
AIDA (if so, which implementation) ?

In the AIDA environment there are also "dynamic" histograms available ("Clouds") 
which make more efficient use of memory for histograms with a limited number of 
fill() calls as the data is stored directly and the actual 10k bins are only allocated 
when actually needed. Please see the AIDA documentation [1] for details on this.

cheers, andreas

[1] AIDA Users Guide:

 http://aida.freehep.org/doc/v3.3.0/UsersGuide/

Specifically for Clouds:

 http://aida.freehep.org/doc/v3.3.0/UsersGuide/clouds.jsp#Clouds


 [ MIME part of type application/pgp-signature stripped ]

Inline Depth:
 1 1
 All All
Outline Depth:
 1 1
 2 2
 All All
Add message: (add)

1 None: Re: Memory use by ROOT histograms   (Ivana Hrivnacova - 23 May, 2014)
(_ None: Re: Memory use by ROOT histograms   (Alexander Zyazin - 23 May, 2014)
(_ None: Re: Memory use by ROOT histograms   (Ivana Hrivnacova - 27 May, 2014)
(_ None: Re: Memory use by ROOT histograms   (Alexander Zyazin - 28 May, 2014)
 Add Message Add Message
to: "Re: Memory use by ROOT histograms"

 Subscribe Subscribe

This site runs SLAC HyperNews version 1.11-slac-98, derived from the original HyperNews


[ Geant 4 Home | Geant 4 HyperNews | Search | Request New Forum | Feedback ]