Problems converging TDDFT calculations

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Problems converging TDDFT calculations
I am calculating excitation energies using TDDFT but only half of my calculations are successful.
The input files differ only in small displacements of one dihedral angle and one H atom, but do not seem to have anything to do with the failed calculations.
The errors seem to be equally shared between (see attached .out files)
"SCF has failed to converge in 100 iterations" and
"Too many rejections; Number of rejections 7
This could be related to lack of integral accuracy or a complicated electronic
structure."
Increasing the integral accuracy did not help.
Any suggestions as to how to increase the success rate of the calculations will be greatly appreciated.
André Anda  University of Copenhagen
The input files differ only in small displacements of one dihedral angle and one H atom, but do not seem to have anything to do with the failed calculations.
The errors seem to be equally shared between (see attached .out files)
"SCF has failed to converge in 100 iterations" and
"Too many rejections; Number of rejections 7
This could be related to lack of integral accuracy or a complicated electronic
structure."
Increasing the integral accuracy did not help.
Any suggestions as to how to increase the success rate of the calculations will be greatly appreciated.
André Anda  University of Copenhagen
 Attachments

 2dTyrBchla_2dTyrBchla_0_0.6.out
 (721.95 KiB) Downloaded 430 times

 2dTyrBchla_2dTyrBchla_4_0.3.out
 (242.1 KiB) Downloaded 423 times
Re: Problems converging TDDFT calculations
It looks to be a complicated electronic structure
Concerning
2dTyrBchla_2dTyrBchla_0_0.6.out
1. there is a warning  you seem to only use 1 OpenMP thread but 4 MPI nodes  this is not optimal
2. it looks to converge smmothly, but slow. I suggest that you increase the maximum number of SCF iterations with
.
*DENSOPT
.MAXIT
300
Concerning
2dTyrBchla_2dTyrBchla_4_0.3.out
I would try to use
.ARH FULL
instead of
.ARH
TK
Concerning
2dTyrBchla_2dTyrBchla_0_0.6.out
1. there is a warning  you seem to only use 1 OpenMP thread but 4 MPI nodes  this is not optimal
2. it looks to converge smmothly, but slow. I suggest that you increase the maximum number of SCF iterations with
.
*DENSOPT
.MAXIT
300
Concerning
2dTyrBchla_2dTyrBchla_4_0.3.out
I would try to use
.ARH FULL
instead of
.ARH
TK

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Re: Problems converging TDDFT calculations
Both of them now worked. Thanks a lot!
André Anda
André Anda
Re: Problems converging TDDFT calculations
Did the rate of success also increase?
Naturally you can do both for this.
*DENSOPT
.ARH FULL
.MAXIT
300
Let us know if we can be of further assistance
Naturally you can do both for this.
*DENSOPT
.ARH FULL
.MAXIT
300
Let us know if we can be of further assistance

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Re: Problems converging TDDFT calculations
Only 2 out of 99 calculations failed (due to "too many rejections..."). However, for a particular hydrogen displacement (0.6 Å, and not for 0.5 Å or 0.7 Å) it doesn't find the excited state corresponding to the optical transition we're looking for (the oscillator strengths are too weak). The ground state looks good. We use
.NEXCIT
5
but will increase this to see if that helps.
Thanks!
.NEXCIT
5
but will increase this to see if that helps.
Thanks!
 Attachments

 2dTyrBchla_2dTyrBchla_6_0.6.out
 (584.4 KiB) Downloaded 368 times
Re: Problems converging TDDFT calculations
I would be interested in figuring out why those 2 do not converge, so could you send the input?
Concerning you problems with excited state I agree that you should try to increase the number of excitation energies.
Remember that you can "RESTART" you calculation from the converged density of you previous calculation  so that you do not need to redo the SCF calculation.
*DENSOPT
.RESTART
You can also restart the calculation of excitation energies using
.RESTEXC
5
Note that (it does not look like it) if you system have a high degree of point group symmetry you may run into the problem that the particular point group symmetry was not included in the original set of trial vectors used for the excitation energies, which means it will not find the proper set of excitation energies.
TK
Concerning you problems with excited state I agree that you should try to increase the number of excitation energies.
Remember that you can "RESTART" you calculation from the converged density of you previous calculation  so that you do not need to redo the SCF calculation.
*DENSOPT
.RESTART
You can also restart the calculation of excitation energies using
.RESTEXC
5
Note that (it does not look like it) if you system have a high degree of point group symmetry you may run into the problem that the particular point group symmetry was not included in the original set of trial vectors used for the excitation energies, which means it will not find the proper set of excitation energies.
TK

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Re: Problems converging TDDFT calculations
Here are the output and input of the two failed calculations
 Attachments

 2dTyrBchla.dal
 (144 Bytes) Downloaded 385 times

 2dTyrBchla_2dTyrBchla_4_0.6.out
 (531.63 KiB) Downloaded 404 times

 2dTyrBchla_2dTyrBchla_8_0.6.out
 (397.91 KiB) Downloaded 391 times
Re: Problems converging TDDFT calculations
Thank you very much.
It looks like the calculations are very close to convergence (gradient = 2.67E05 which means the energy is converged to around 1E10 ) but fail to converge.
My thoughts would be that this is a DFT grid precision issue or an issue that needs a more careful investigation.
if you want to push these calculations through I suggest to use a "FINE" DFT grid.
*DFT INPUT
.FINE
TK
It looks like the calculations are very close to convergence (gradient = 2.67E05 which means the energy is converged to around 1E10 ) but fail to converge.
My thoughts would be that this is a DFT grid precision issue or an issue that needs a more careful investigation.
if you want to push these calculations through I suggest to use a "FINE" DFT grid.
*DFT INPUT
.FINE
TK

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Re: Problems converging TDDFT calculations
The fine grid did squeeze them through. But the problem with excitation energies that do not fit in with the rest of the configuration space still remains.
Any thoughts on what's causing this?
Thanks again.
André Anda
Any thoughts on what's causing this?
Thanks again.
André Anda
Re: Problems converging TDDFT calculations
Concerning the excitation energies I can only suggest that you
1. Converge the SCF harder. However,
.CONVDYN
TIGHT
should be tight enough
2. request more excitation energies
3. If the SCF converge to a saddle point the excitation energies would not make sense  but this should not happen with ARH FULL  in principle you can check that you have converged to a minimum, but that is very expensive  as expensive as the full SCF.
can you send the output  maybe that will tell me more
1. Converge the SCF harder. However,
.CONVDYN
TIGHT
should be tight enough
2. request more excitation energies
3. If the SCF converge to a saddle point the excitation energies would not make sense  but this should not happen with ARH FULL  in principle you can check that you have converged to a minimum, but that is very expensive  as expensive as the full SCF.
can you send the output  maybe that will tell me more

 Posts: 6
 Joined: 07 Aug 2015, 13:25
 First name(s): André
 Last name(s): Anda
 Affiliation: University of Copenhagen
 Country: Denmark
Re: Problems converging TDDFT calculations
Here is an example of the output.
Can you learn anything from it?
André
Can you learn anything from it?
André
 Attachments

 2dTyrBchla_2dTyrBchla_8_0.6_fine.out
 (444.08 KiB) Downloaded 367 times

 Posts: 7
 Joined: 11 Mar 2014, 13:41
 First name(s): Luca
 Last name(s): De Vico
 Affiliation: Copenhagen University
 Country: Denmark
Re: Problems converging TDDFT calculations
Dear LSDalton forum,
we are still having some difficulties with our TDDFT calculations. While doing a geometry scan, that is moving one or very few atoms very very little at a time, we often encounters geometries where nothing works out as in those surrounding, creating strange and unexpected artifacts.
One of the most disturbing aspects is that computed excitation energies suddenly come out mixed up. Let me explain with an example. We are interested in the first excitation energy, and we compute 5 for good measure.
Here is the output of a "good" calculation:
Excitation Transition Dipole Moments Oscillator
Energies x y z Strengths
===============================================================================
0.05813173 3.21873928 0.15735079 0.39032106 0.40837123
0.07556057 0.58977443 1.32138545 0.04746374 0.10559066
0.08414462 0.08317878 0.50510467 0.09980511 0.01525881
0.09505294 0.00398517 0.13386777 0.03063855 0.00119609
0.10261680 0.32171268 0.36315346 0.01186048 0.01611222
Here is the output from a "bad" one:
Excitation Transition Dipole Moments Oscillator
Energies x y z Strengths
===============================================================================
0.00426285 0.24270230 0.39201328 0.79239283 0.00238852
0.04182679 0.01733164 0.02886855 0.06171046 0.00013780
0.05460040 0.75428362 1.67398345 0.66631174 0.13887196
0.05769026 0.99870531 2.18183275 0.18755275 0.22279912
0.06781730 0.00222396 0.03686129 0.04607496 0.00015763
0.07504558 0.02057032 0.33897552 1.62795848 0.13836285
0.07601010 0.04845519 0.06814299 0.09954983 0.00085646
0.08729695 0.04833878 0.09489194 0.01443687 0.00067216
As you can see, we increased the number of computed excitations to 8, to try and get better numbers, but to no avail. Even 12 did not help.
The third and forth reported excitations seem to be the one we want, but for some reason split in two, as also evinced by the oscillator strength. We tried to use the .DTHR keyword for *SOLVER, with a value of even 0.03 (I know, it should be an unnecessary exaggeration, but we tried everything) but it did not have any effect.
I attached the output files. I hope you can tell me what are we doing wrong, or what could we use to get those excitation energies to behave.
Thanks, regards
Luca De Vico
we are still having some difficulties with our TDDFT calculations. While doing a geometry scan, that is moving one or very few atoms very very little at a time, we often encounters geometries where nothing works out as in those surrounding, creating strange and unexpected artifacts.
One of the most disturbing aspects is that computed excitation energies suddenly come out mixed up. Let me explain with an example. We are interested in the first excitation energy, and we compute 5 for good measure.
Here is the output of a "good" calculation:
Excitation Transition Dipole Moments Oscillator
Energies x y z Strengths
===============================================================================
0.05813173 3.21873928 0.15735079 0.39032106 0.40837123
0.07556057 0.58977443 1.32138545 0.04746374 0.10559066
0.08414462 0.08317878 0.50510467 0.09980511 0.01525881
0.09505294 0.00398517 0.13386777 0.03063855 0.00119609
0.10261680 0.32171268 0.36315346 0.01186048 0.01611222
Here is the output from a "bad" one:
Excitation Transition Dipole Moments Oscillator
Energies x y z Strengths
===============================================================================
0.00426285 0.24270230 0.39201328 0.79239283 0.00238852
0.04182679 0.01733164 0.02886855 0.06171046 0.00013780
0.05460040 0.75428362 1.67398345 0.66631174 0.13887196
0.05769026 0.99870531 2.18183275 0.18755275 0.22279912
0.06781730 0.00222396 0.03686129 0.04607496 0.00015763
0.07504558 0.02057032 0.33897552 1.62795848 0.13836285
0.07601010 0.04845519 0.06814299 0.09954983 0.00085646
0.08729695 0.04833878 0.09489194 0.01443687 0.00067216
As you can see, we increased the number of computed excitations to 8, to try and get better numbers, but to no avail. Even 12 did not help.
The third and forth reported excitations seem to be the one we want, but for some reason split in two, as also evinced by the oscillator strength. We tried to use the .DTHR keyword for *SOLVER, with a value of even 0.03 (I know, it should be an unnecessary exaggeration, but we tried everything) but it did not have any effect.
I attached the output files. I hope you can tell me what are we doing wrong, or what could we use to get those excitation energies to behave.
Thanks, regards
Luca De Vico
 Attachments

 tyr4_tyr_0_0.6.out
 LSDalton output
 (218.66 KiB) Downloaded 175 times

 tyr_0_0.6.4040651.out
 Standard output
 (32.5 KiB) Downloaded 159 times

 Posts: 519
 Joined: 15 Oct 2013, 05:37
 First name(s): Peter
 Middle name(s): Robert
 Last name(s): Taylor
 Affiliation: Aarhus University
 Country: Denmark
Re: Problems converging TDDFT calculations
I am not an LSDalton developer, nor even an expert user, although I have used the code quite a bit. But an excitation energy of 0.004 E_{h} is obviously ridiculous. One possibility is that this is a problem in Nature (that this particular geometry has some sort of resonance or nearresonant behaviour) that causes some sort of instability in the calculation. Another possibility is that the perceived numerical instability (which is clearly the issue) is intrinsic to some of the methodology as programmed (the algorithms or their implementation). A possible path forward in identifying problems might be to use a different method (this is to test the behaviour of the code, not as an alternative that would be better scientifically!). For example, if you take this problem geometry and run with a different functional, or even with straight SCF?
Best regards
Pete
P.S. Needless to say, I would defer to any of the LSDalton developers/experts who might post subsequently: this is not false modesty, I really am not an LSDalton expert!
Best regards
Pete
P.S. Needless to say, I would defer to any of the LSDalton developers/experts who might post subsequently: this is not false modesty, I really am not an LSDalton expert!

 Posts: 7
 Joined: 11 Mar 2014, 13:41
 First name(s): Luca
 Last name(s): De Vico
 Affiliation: Copenhagen University
 Country: Denmark
Re: Problems converging TDDFT calculations
Hi Pete,
Thanks for your reply. We are producing a huge number of structures, and only a limited set of them shows this strange behavior. It would seem very unphysical if there was a "Nature" cause behind. One never knows, obviously, but ... in this case I'm pretty sure there is something else. We have so many single points that we are not collecting data by hand, we use scripts which check for a reasonable enough oscillator strength before collecting the corresponding energy. That's why it took us some time before we realized that what we thought were unfinished calculations were, actually, finished but with weird results. I will give it a try to another functional, but I don't want (at this point) redo the entire set of calculations with another method, it would require too much time. What we need is a way of "patching up" the holes in the surfaces we are,otherwise, producing.
Cheers!
/Luca
Thanks for your reply. We are producing a huge number of structures, and only a limited set of them shows this strange behavior. It would seem very unphysical if there was a "Nature" cause behind. One never knows, obviously, but ... in this case I'm pretty sure there is something else. We have so many single points that we are not collecting data by hand, we use scripts which check for a reasonable enough oscillator strength before collecting the corresponding energy. That's why it took us some time before we realized that what we thought were unfinished calculations were, actually, finished but with weird results. I will give it a try to another functional, but I don't want (at this point) redo the entire set of calculations with another method, it would require too much time. What we need is a way of "patching up" the holes in the surfaces we are,otherwise, producing.
Cheers!
/Luca

 Posts: 519
 Joined: 15 Oct 2013, 05:37
 First name(s): Peter
 Middle name(s): Robert
 Last name(s): Taylor
 Affiliation: Aarhus University
 Country: Denmark
Re: Problems converging TDDFT calculations
No, I wasn't suggesting "redoing" anything, and certainly not "everything"! I was saying more that the more information that can be provided to the LSDalton mob, the more there's a chance they might be able to figure out that there's a problem. If you were to redo just a few calculations with some geometries that worked and one that went crazy, with (e.g.) a different functional,, or SCF, it might provide some information to someone seeking to find a bug and kill it. Conversely, it is essential to ensure that this isn't some sort of "freak of Nature" like an unexpected resonance (which might be an artefact of a particular functional, or basis set, for instance): it makes no sense for someone to chase a bug unless it's clear there is a bug, as opposed to "unsatisfactory results"...
Best regards
Pete
Best regards
Pete
Re: Problems converging TDDFT calculations
As far as I remember the excitation energy of 0.004 Eh could indicate an issue with the SCF convergence  that the density is not fully converged
I suggest to use
.CONVDYN
TIGHT
when doing response in general
It does look like you have problems with the SCF convergence (converging very slowly  76 iterations)
I suggest to use
**INTEGRALS
.THRESH
1.0D15
**WAVE FUNCTIONS
.DFT
PBE0
*DFT INPUT
.FINE
*DENSOPT
.ARH FULL
.CONVDYN
TIGHT
.START
TRILEVEL
.NVEC
100
.MICROVECS
40
.MAXIT
300
**RESPONS
.NEXCIT
1
*SOLVER
.NSTART
10
.CONVDYN
TIGHT
*END OF INPUT
this will only converge the first excited state (save time) but use 10 different startvectors (or some other value depending on molecule symmetry) that hopefully contain the proper symmetry to ensure the lowest excited state is the right one.
I suggest to use
.CONVDYN
TIGHT
when doing response in general
It does look like you have problems with the SCF convergence (converging very slowly  76 iterations)
I suggest to use
**INTEGRALS
.THRESH
1.0D15
**WAVE FUNCTIONS
.DFT
PBE0
*DFT INPUT
.FINE
*DENSOPT
.ARH FULL
.CONVDYN
TIGHT
.START
TRILEVEL
.NVEC
100
.MICROVECS
40
.MAXIT
300
**RESPONS
.NEXCIT
1
*SOLVER
.NSTART
10
.CONVDYN
TIGHT
*END OF INPUT
this will only converge the first excited state (save time) but use 10 different startvectors (or some other value depending on molecule symmetry) that hopefully contain the proper symmetry to ensure the lowest excited state is the right one.

 Posts: 7
 Joined: 11 Mar 2014, 13:41
 First name(s): Luca
 Last name(s): De Vico
 Affiliation: Copenhagen University
 Country: Denmark
Re: Problems converging TDDFT calculations
The SCF convergence was another issue, that we tried to solve previously. The choice of
.ARH DAVID
.CONVDYN
STANDARD
is the one that ensures we always get an SCF to converge, even if slowly. We found it after many try and errors. However, it may very well be that it is insufficient. I will try one thing at the time. First the suggested 1 excitation with 10 starting vectors. If that were not enough I will try also to change the SCF convergence, but it might create other problems.
Cheers!
/Luca
.ARH DAVID
.CONVDYN
STANDARD
is the one that ensures we always get an SCF to converge, even if slowly. We found it after many try and errors. However, it may very well be that it is insufficient. I will try one thing at the time. First the suggested 1 excitation with 10 starting vectors. If that were not enough I will try also to change the SCF convergence, but it might create other problems.
Cheers!
/Luca
Who is online
Users browsing this forum: No registered users and 1 guest