Main Class Reference

Overall flow control of the simulation. More...

#include <ParallelGravity.h>

Inheritance diagram for Main:

Public Member Functions
	Main (CkArgMsg *m)
	Main routine to start simulation.
	Main (CkMigrateMessage *m)
	Restart Main constructor.
void	niceExit ()
	entry method to cleanly shutdown. Only used for debugging.
void	setupICs ()
	Load particles into pieces.
void	initialForces ()
	Initial calculation of forces.
void	doSimulation ()
	Principal method which does all the coordination of the simulation over timesteps.
void	restart (CkCheckpointStatusMsg *msg)
	Callback to restart simulation after a checkpoint or after writing a checkpoint.
void	waitForGravity (const CkCallback &cb, double startTime, int activeRung)
	wait for gravity in the case of concurrent SPH
void	advanceBigStep (int)
	Take one base timestep of the simulation.
void	domainDecomp (int iPhase)
	Perform domain decomposition.
void	loadBalance (int iPhase)
	Perform load balance.
void	buildTree (int iPhase)
	Build Tree.
void	startGravity (const CkCallback &cbGravity, int iActiveRung, double *startTime)
	Routine to start self gravity; if gravity is not being calculated, then clear the accelerations.
void	externalForce (int iActiveRung)
	Apply external gravitational field.
void	updateuDot (int iActiveRung, const double duKick[], const double dStartTime[], int bUpdateState, int bAll)
	Update time derivative of thermal energy.
void	kick (bool bClosing, int iActiveRung, int nextMaxRung, const CkCallback &cbGravity, double gravStartTime)
	Update velocities.
void	advanceBigCollStep (int)
int	adjust (int iKickRung)
	Calculate timesteps of particles.
void	rungStats ()
	Count and print out the number of particles in each timestep bin.
void	countActive (int activeRung)
	determine number of active particles in the given rung
void	emergencyAdjust (int iRung)
	Change timesteps of particles experiencing sudden gas forces.
void	starCenterOfMass ()
	Main::starCenterOfMass Calculates the total mass and center of mass of all the star particles and saves them to all the available COOL structs.
void	logCollision (double, ColliderInfo *, int)
void	writeCollLog (const char *)
	Write collision events in buffer to log file and clear the buffer.
void	calcEnergy (double, double, const char *)
	Calculate various energy and momentum quantities, and output them to a log file.
void	getStartTime ()
	determine start time of simulation
void	getOutTimes ()
	Read in desired output times and reshifts from a file.
int	bOutTime ()
	Return true if we need to write an output.
void	writeOutput (int iStep)
	Output a snapshot.
void	outputBinary (OutputParams &params, int bParaWrite, const CkCallback &cb)
	Output a Tipsy or NChilada XDR binary float array file.
void	cbOpen (Ck::IO::FileReadyMsg *msg)
void	cbIOReady (Ck::IO::SessionReadyMsg *msg)
	Session is ready; write my data.
void	cbIOComplete (CkMessage *msg)
	All IO has completed. Close the file.
void	cbIOClosed (CkMessage *msg)
	File is closed. Update header for NChilada. Resume main program.
std::string	getNCNextOutput (OutputParams &params)
void	writeNCXML (std::string filename)
void	NCXMLattrib (ofstream *desc, std::vector< std::string > &names, std::string family)
void	updateSoft ()
	Change the softening in comoving coordinates.
void	growMass (double dTime, double dDelta)
	Slowly increase mass of a subset of particles.
void	initSph ()
	initialize SPH quantities
void	initCooling ()
	Initialize cooling constants and integration data structures.
void	initLWData ()
void	initStarLog ()
void	initHMStarLog ()
int	ReadASCII (char *extension, int nDataPerLine, double *dDataOut)
	function from PKDGRAV to read an ASCII table
void	restartGas ()
	Read in array files for complete gas information.
void	doSph (int activeRung, int bNeedDensity=1)
	Perform the SPH force calculation.
void	AGORAfeedbackPreCheck (double dTime, double dDelta, double dTimeToSF)
	This routine is called when AGORA feedback is enabled. It checks for any star particles that will have a feedback event in the next timestep and puts the neighboring gas particles onto a smaller timestep. Because the amount of energy injected is very large, particles need to be placed on a small timestep BEFORE the first force calculation is done to avoid significant integration errors.
void	FormStars (double dTime, double dDelta)
void	StellarFeedback (double dTime, double dDelta)
void	outputBlackHoles (double dTime)
	Output black hole orbit information.
void	SetSink ()
	Initial identify sinks.
void	FormSinks (double dTime, double dDelta, int iKickRung)
	Form sink particles; main routine.
void	doSinks (double dTime, double dDelta, int iKickRung)
	Process sink particles.
int	DumpFrameInit (double dTime, double dStep, int bRestart)
void	DumpFrame (double dTime, double dStep)
int	nextMaxRungIncDF (int nextMaxRung)
void	addDelParticles ()
	Coalesce all added and deleted particles and update global quantities.
void	memoryStats ()
void	memoryStatsCache ()
void	pup (PUP::er &p)
void	liveVizImagePrep (liveVizRequestMsg *msg)
void	doSIDM (double dTime, double dDelta, int activeRung)
	Main method to perform Self Interacting Dark Matter interactions.
void	restartNSIDM ()
	Read in array files for SIDM interact count, if needed.

Detailed Description

Overall flow control of the simulation.

As well as controlling the overall flow of the simulation, the constructors are the main entry points into the program. The sequence of tasks is: read the simulation parameters (Main()), read in the initial conditions (setupICs()), calculate the initial forces (initialForces()), then iterate across timesteps and write the final output (doSimulation()).

Constructor & Destructor Documentation

Main() [1/2]

Main::Main

(

CkArgMsg *

m

)

Main routine to start simulation.

This routine parses the command line and file specified parameters, and allocates the charm structures. The charm "readonly" variables are writable in this method, and are broadcast globally once this method exits. Note that the method finishes with an asynchronous call to setupICs().

Main() [2/2]

Main::Main

(

CkMigrateMessage *

m

)

Restart Main constructor.

This is only called when restarting from a checkpoint.

Member Function Documentation

addDelParticles()

void Main::addDelParticles

(

)

Coalesce all added and deleted particles and update global quantities.

This is large, but no larger than the counts message

adjust()

int Main::adjust

(

int

iKickRung

)

Calculate timesteps of particles.

Particles on the KickRung and shorter have their timesteps adjusted. Calls TreePiece::adjust().

Parameters

iKickRung

Rung (and above) about to be kicked.

advanceBigStep()

void Main::advanceBigStep

(

int

iStep

)

Take one base timestep of the simulation.

Parameters

iStep

The current step number.

This method implements the standard "Kick Drift Kick" (Quinn et al 1997) hierarchical timestepping algorithm. It assumes that the forces for the first opening kick have already been calculated.

AGORAfeedbackPreCheck()

void Main::AGORAfeedbackPreCheck	(	double	dTime,
		double	dDelta,
		double	dTimeToSF )

This routine is called when AGORA feedback is enabled. It checks for any star particles that will have a feedback event in the next timestep and puts the neighboring gas particles onto a smaller timestep. Because the amount of energy injected is very large, particles need to be placed on a small timestep BEFORE the first force calculation is done to avoid significant integration errors.

Parameters

dTime	The current simulation time (in years)
dDelta	The size of the next timestep (in years)
dTimeToSF	The time until the next star formation event (in years)

bOutTime()

int Main::bOutTime

(

)

Return true if we need to write an output.

Advances iOut attribute, therefore this can only be called once per timestep.

buildTree()

void Main::buildTree

(

int

iPhase

)

Build Tree.

Parameters

iPhase

Active rung (or phase).

cbIOClosed()

void Main::cbIOClosed

(

CkMessage *

msg

)

File is closed. Update header for NChilada. Resume main program.

Continue to next particle type

cbOpen()

void Main::cbOpen

(

Ck::IO::FileReadyMsg *

msg

)

Determine offsets for all pieces, then start the write session. This needs to be a threaded entry method.

countActive()

void Main::countActive

(

int

activeRung

)

determine number of active particles in the given rung

Calls TreePiece::countActive() and sets Main::nActiveGrav and Main::nActiveSPH.

domainDecomp()

void Main::domainDecomp

(

int

iPhase

)

Perform domain decomposition.

Parameters

Active

rung (or phase).

doSIDM()

void Main::doSIDM	(	double	dTime,
		double	dDelta,
		int	activeRung )

Main method to perform Self Interacting Dark Matter interactions.

Parameters

dTime	current simulation time
dDelta	timestep over which to calculate interactions
activeRung	timestep rung corresponding to dDelta

doSimulation()

void Main::doSimulation

(

)

Principal method which does all the coordination of the simulation over timesteps.

This routine calls advanceBigStep() for each step, logs statistics, determines if output is needed, and halts the simulation when done.

doSph()

void Main::doSph	(	int	activeRung,
		int	bNeedDensity = 1 )

Perform the SPH force calculation.

Parameters

activeRung	Timestep rung (and above) on which to perform SPH
bNeedDensity	Does the density calculation need to be done? Defaults to 1

emergencyAdjust()

void Main::emergencyAdjust

(

int

iRung

)

Change timesteps of particles experiencing sudden gas forces.

Parameters

iRung

The rung on which we are calculating forces.

For gas simulations, find particles who are are in the middle of too large a timestep and adjust their velocities to a smaller timestep.

externalForce()

void Main::externalForce

(

int

iActiveRung

)

Apply external gravitational field.

Parameters

iActiveRung

Rung on which to apply forces.

FormSinks()

void Main::FormSinks	(	double	dTime,
		double	dDelta,
		int	iKickRung )

Form sink particles; main routine.

Parameters

dTime	Current time.
dDelta	Current timestep.
iKickRung	Rung being kicked.

Calls TreePiece::formSinks.

FormStars()

void Main::FormStars	(	double	dTime,
		double	dDelta )

form stars main method

getNCNextOutput()

std::string Main::getNCNextOutput

(

OutputParams &

params

)

Return the file name of the next nchilada output to write (gas, dark or star). Returns empty if we are done. Also advances the iTypeWriting in params.

getOutTimes()

void Main::getOutTimes

(

)

Read in desired output times and reshifts from a file.

Fills in the vdOutTime vector by reading the .red file

getStartTime()

void Main::getStartTime

(

)

determine start time of simulation

Function to determine the start time of the simulation. Modifies dTime member of main.

initialForces()

void Main::initialForces

(

)

Initial calculation of forces.

This is called both when starting or restarting a run. It concludes by calling doSimulation(), the main simulation loop.

initSph()

void Main::initSph

(

)

initialize SPH quantities

Initial calculation of densities and internal energies, and cooling rates.

kick()

void Main::kick	(	bool	bClosing,
		int	iActiveRung,
		int	nextMaxRung,
		const CkCallback &	cbGravity,
		double	gravStartTime )

Update velocities.

Parameters

bClosing	Is this a closing kick?
iActiveRung	Rung (and higher) which to update
nextMaxRung	Rung of smallest timestep to be taken
cbGravity	Callback to wait for gravity (close only)
gravStartTime	Timing start for gravity

Based on bClosing the velocities of particles on rung iActiveRung through nextMaxRung are either moved to a 1/2 step based on their rung (open) or moved from the 1/2 step to the end of the step (close).

loadBalance()

void Main::loadBalance

(

int

iPhase

)

Perform load balance.

Parameters

iPhase

Active rung (or phase). -1 indicates initial LB.

memoryStats()

void Main::memoryStats

(

)

Diagnostic function to summmarize memory usage across all processors

memoryStatsCache()

void Main::memoryStatsCache

(

)

Diagnostic function to summmarize cache memory usage across all processors

outputBlackHoles()

void Main::outputBlackHoles

(

double

dTime

)

Output black hole orbit information.

Calls TreePiece::outputBlackHoles()

ReadASCII()

int Main::ReadASCII	(	char *	extension,
		int	nDataPerLine,
		double *	dDataOut )

function from PKDGRAV to read an ASCII table

Parameters

extension	Appended to outName to determine file name to read.
nDataPerLine	Number of columns in the table.
dDataOut	pointer to array in which to store the table. Note if dDataOut is NULL it just counts the number of valid input lines.

restart()

void Main::restart

(

CkCheckpointStatusMsg *

msg

)

Callback to restart simulation after a checkpoint or after writing a checkpoint.

A restart looks like we've just finished writing a checkpoint. We can tell the difference by the bIsRestarting flag set in the Main CkMigrate constructor. In that case we do some simple parameter parsing and go to InitialForces. Otherwise we return to the doSimulation() loop.

rungStats()

void Main::rungStats

(

)

Count and print out the number of particles in each timestep bin.

This routine is for information only.

setupICs()

void Main::setupICs

(

)

Load particles into pieces.

Reads the particle data in from a file. Since the full information about the run (including starting time) isn't known until the particles are loaded, this routine also completes the specification of the run details and writes out the log file entry. It concludes by calling initialForces()

starCenterOfMass()

void Main::starCenterOfMass

(

)

Main::starCenterOfMass Calculates the total mass and center of mass of all the star particles and saves them to all the available COOL structs.

Requires that cooling for planets be enabled at compile time.

startGravity()

void Main::startGravity	(	const CkCallback &	cbGravity,
		int	iActiveRung,
		double *	startTime )

Routine to start self gravity; if gravity is not being calculated, then clear the accelerations.

Parameters

cbGravity	Callback if we overlapping gravity with SPH.
iActiveRung	Rung (and higher) on which to calculate forces.
pointer	to start time

StellarFeedback()

void Main::StellarFeedback	(	double	dTime,
		double	dDelta )

Feedback main method

updateSoft()

void Main::updateSoft

(

)

Change the softening in comoving coordinates.

When compiled with -DCHANGESOFT, and bPhysicalSoft is set, the (comoving) softening is changed so that it is constant in physical units.

updateuDot()

void Main::updateuDot	(	int	iActiveRung,
		const double	duKick[],
		const double	dStartTime[],
		int	bUpdateState,
		int	bAll )

Update time derivative of thermal energy.

Parameters

iActiveRung	Rung (and higher) which to update
duKick	Array of timesteps per rung
dStartTime	Array of beginning times (simulation units) per rung
bUpdateState	Whether to update the ionization fractions
bAll	Whether to update all rungs below iActiveRung

writeOutput()

void Main::writeOutput

(

int

iStep

)

Output a snapshot.

Parameters

iStep

Timestep we are outputting, used for file name.

---

The documentation for this class was generated from the following files:

• ParallelGravity.h
• feedback.cpp
• InOutput.cpp
• ParallelGravity.cpp
• SIDM.cpp
• sinks.cpp
• Sph.cpp
• starform.cpp