Memory Usage FAQ
An Overview of Memory Issues in Hartree-Fock Calculations
There are two different algorithms that may be used by the
Ab Initio program, depending on how much memory is available.
The two algorithms are:
- DIRECT : A conservative memory method.
The 4-center integrals are calculated each time they
are needed.
- INCORE : Can take LOTS of memory but is faster,
IF there is
enough memory. All 4-center integrals are stored
in RAM for quick access. This implies that the memory
requirements grow to the fourth power of the number of basis
functions.
Spartan does a good job choosing which memory module
to use. One can override default choices by typing either the
DIRECT or INCORE keyword in the Options field in
the calculations dialogue.
Chances are that if you force INCORE when Spartan doesn't choose
this approach, the calculation will end up swapping RAM causing
a VERY LARGE
AND SIGNIFICANT slowdown! (See our discussion of
swap space for more information.)
There is no way to make the DIRECT method use memory more efficiently.
It is an all-or-nothing choice. A short table giving the memory
requirements for the INCORE method can be found
here
One can use the 'TESTMEMORY' keyword as a quick test to see which
memory model will be used. No calculation will be done, but some
memory information will be printed to the output file.
Setting Spartan's memory limits
Spartan checks the machine to see how much Memory is available.
You can set the amount of memory a job can use from within
Spartan. The place you do this is different on each platform.
- Windows: In the limits dialogue.
(Options/Monitor/Options/Limits)
- UNIX/LINUX: From the command line type
"Spartan -setup" and follow the prompts.
- Macintosh: In the preference panel.
The default memory value is also unique for each type machine:
- Windows NT/2000/XP:
The available physical memory is determined
via a system call. A minimum of 30 MB is always assumed to be
available. The amount available can vary depending on what
other applications are doing at the time. (Other tasks such
as watching a DVD, surfing the internet, or running another spartan molecule can use
significant amounts of RAM.)
- Windows 95/98: A fraction of total RAM installed is used.
(Currently, total RAM - 36 MB)
- UNIX/LINUX and Macintosh:
(Amount of Ram - 32)/(Max. number of concurrent jobs)
or 48 MB whichever is greater.
Memory Requirements for Frequency Calculations
Frequency calculations use significantly more RAM than a
geometry optimization. The more RAM the better, and
quicker the calculation of second derivatives.
Memory Requirements for Semi-Empirical Calculations
While Semi-Empirical calculations take much less memory than
Abinitio calculations, however, it can add up for very large systems.
As a rough approximation (only good to 1 digit)
memory usage in kB = 950*8*(N2)/1024
where N is the number of atoms.
If using the SM54 Cramer-Truhlar solvation method one can expect
another 9*8*(N3)/1024 kB of memory allocated
for the semi-empirical modification.
Memory Requirements for DFT Calculations
Memory is usually not a limiting factor in DFT calculations. Since we
use a direct method for the exchange calculation. The biggest arrays
are linear in the number of atoms, and quadratic with the number of
basis functions.
Memory Requirements for post HF Calculations
While more memory is always better with the advanced (post HF)
methods, the usual machine limits are the amount of available
scratch disk space.
Return to Top
Common Questions
- How much memory do I need to run Spartan?
A bare minimum is 256 MB, but 256-512 MB is a good range for modest
sized molecules. More
is always better, but after about 1 GB of RAM the
benefits are harder to come by. The biggest advantage of more
memory is that it allows Spartan to choose the faster
INCORE
methods for Hartree-Fock (HF) calculations.
The table below gives some sample memory systems and
lists the number of carbon atoms which can be calculated using the
INCORE method for the 3-21G and the 6-31G* basis sets.
Available Ram | Basis functions | 3-21G Carbons | 6-31G* Carbons |
128 MB | 107 | 11 | 7 |
512 MB | 151 | 16 | 10 |
2024 MB | 200 | 22 | 13 |
Return to Top
- My DFT frequency calculation will not run, whwat can I do?
In our current implementation of DFT analytical frequency we do
not use memory wisely.
(This is being corrected, changes will appear in future editions.)
Currently, as we run out of memory we just crash
(sometimes with a very cryptic message).
The current workaround is to
calculate the Hessian numerically using the FREQ=CD
keyword. This will run slower, but will require less
memory.
Return to Top
- I'm using the default RAM settings, but Spartan seems to
think I don't have much memory at all? (Windows
Specific)
If you don't set any of the Memory defaults the Spartan program
will examine the computer to see how much memory is available and
attempt to use it. (see above).
If you only use your machine for running Spartan this is a good
idea, but it can be misleading in many 'office' cases. For example
if you have been surfing the web, the web browser may be taking
memory that it may not really need, but helps it to run
faster. (i.e. memory which can be swapped to disk.) It is
possible to have a 512 MB machine with almost all the memory
being used by other programs. In this case Spartan will attempt
to use the minima of 32MB for scratch space. If you always want
more memory available for Spartan you should set the default
memory in the
Monitor to a number larger than 32 MB but
less than the amount of physical memory in your machine.
(See Setting Spartan's memory limits above.)
Return to Top
- Will increasing swap improve performance?
Probably not. The most likely affect will be to slow down
Spartan calculations. But if you know what you are doing, and
are careful it might help.
In the worse case scenario, you will increase your swap, and
spartan will think your machine has more RAM than it really does.
Spartan will then use this "virtual" ram. Spartan will run,
but often the ram required by Spartan will be residing on
the disk. Accessing the disk is many order of magnitudes slower
than running in Ram, and the speed of Spartan will plummet. A
similar problem can occur even if spartan is using less than the
total RAM in your system. If a spartan job and another job such
as a web browser, a java applet, or even another spartan job are
both trying to use the real RAM a thrashing of memory will occur.
There is a case where increasing swap space may help, but
since it is so easy to do something wrong be careful. If there
are some bloated applications on your machine which are running,
but you are not using them, yet you choose not to exit them, they
will take up valuable space in memory; Memory which spartan
wants to use. The best thing to do is to quit these applications.
But you can also increase swap in the hope that the OS will swap
the applications you aren't using out of memory and let Spartan
use the real RAM.
When increasing swap it is very important to set the
memory limits
of spartan to be less than real Memory in your system.
Typical modern operating systems require a bare minimum of 32-64
MB of RAM to run. So the maximum memory should be:
("Total Ram" - 32MB)/"Number of jobs running at the same time"
Return to Top
- My job failed with an error
about MEM_STATIC. What does this mean?
In DFT calculations the algorithms used partition memory
memory into two different regions. For discussion purposes
let's call them 'STATIC' and 'HEAP' memory. There are times
when the algorithm incorrectly predicts the amount of 'STATIC'
memory. When this occurs you may get an error message in the
verbose output asking for more memory. This bug
can usually be fixed by adding the MEM_STATIC=xx keyword
to the output where xx is in megabytes (MB). The default
is usually on the order of 32 MB. It is important that this
number is less than the total memory allocated in the machine.
See the discussion on setting Spartan's memory
limits.
Note: This replaces the deprecated keyword MEMORY=yy
where yy was in units of "double-words" (64-bits)
Return to Top
Wavefunction Support
Author: Phil Klunzinger
Last modified: Mon Aug 7 13:00:32 PDT 2006