Post by SNNAP Board Admin on Nov 30, 2004 17:41:19 GMT -5
Title: The Forward Euler Method
Attachments: This posting includes several attachments:
Exponential.jpg
RC.smu
RC.ntw
RC.trt
RC.ous
RC.neu
RC.vdg
Goal: To illustrate the importance of selecting an appropriate integration time step.
In SNNAP, the ordinary differential equations (ODEs) are integrated with the forward-Euler method, which uses a fixed time step to numerically solve ODEs. The forward Euler method is simple and fast, but its accuracy and stability depend critically on the size of the integration time step, which is specified by the user.
As a general rule, the forward Euler is numerically unstable unless the time step is at least two times (2 x) smaller than smallest time constant within a model. Moreover, the integration will be numerically inaccurate unless the time step is at least ten times (10 x) smaller than the smallest time constant.
One of the Examples that is provided with SNNAP (/Examples/Numerical_Integration) illustrates how the size of the integration time step can affect accuracy and stability of a simulation, and the example discusses how to select a proper time step. The present posting presents another example of how the accuracy of the forward Euler is affected by the size of the integration time step.
The present posting presents the simulation of the voltage decay in a simple RC circuit. This is analogues to a single compartment model of a neuron with a constant membrane conductance and a membrane capacitance. The analytical solution for the voltage decay in a RC circuit is given by:
V(t) = V(0) exp(-t/tau) (Eq. 1)
where V(0) is the initial value of the voltage, tau is time constant of the RC circuit and t is time. In the present example V(0) = 1 mV and tau = 1 sec. Eq. 1 was solved and plotted using Excel (MicroSoft). A model with identical properties was developed and solved using SNNAP8. A series of simulations was conducted in which the integration time step was 100 x, 50 x, 20 x, 10 x, 5 x and 2 x smaller than tau. The results of the simulations were graphically compared to the analytical solution. The results are presented in the attached image file (Exponential.jpg).
Figure Legend: Numerical accuracy of the forward Euler method for integrating ODEs. A: Comparison between analytical and simulated solutions of the voltage decay in a RC circuit. Seven curves are plotted: the solution of Eq. 1 (dashed yellow line), and the results of six SNNAP simulations in which the integration time step was 100 x (orange line), 50 x (green line), 20 x (red line), 10 x (purple line), 5 x (blue line), and 2 x (black line) faster than the time constant (tau) of the RC circuit. The analytical solution, 100 x, 50 x, 20 x and 10 x traces are virtually indistinguishable, which indicates that the SNNAP simulations accurately reproduced the analytical solution. The accuracy of the simulation deteriorated, however, as the integration time step became too large (traces 5 x and 2 x). The segments of the traces outlined by the dashed box are expanded in Panel B. B: Expanded view of a segment of the traces illustrated in Panel A. At this higher resolution, it is possible to distinguish the individual traces. As the integration time step becomes larger, the accuracy of the SNNAP simulation decreases. A sum of squared errors analysis (not shown) indicated that the difference between the analytical and simulated solutions became significant when the integration time step was increased to 5 x smaller than tau.
Although the integration time step had an important impact on the numerical accuracy of a simulation, the accuracy was not affected by the complexity of the neural network. Increasing the number of neurons in the simulation by 1000 fold produced the exact same solutions.
The attachments to this posting also include the SNNAP input files necessary to reproduce the results presented above.
Summary: As a general rule, the user should select an integration time step that is at least ten times faster than the smallest time constant in the simulation. However, as the integration time step is reduced the speed of the simulation will slow. This is simply because more iterations are occurring as the time step becomes smaller. Thus, selecting an appropriate time step is a balancing act between accuracy, stability and speed.
Doug
snnap.uth.tmc.edu/post/pic/A0/RC.neu
snnap.uth.tmc.edu/post/pic/A0/RC.ntw
snnap.uth.tmc.edu/post/pic/A0/RC.ous
snnap.uth.tmc.edu/post/pic/A0/RC.smu
snnap.uth.tmc.edu/post/pic/A0/RC.trt
snnap.uth.tmc.edu/post/pic/A0/RC.vdg
Attachments: This posting includes several attachments:
Exponential.jpg
RC.smu
RC.ntw
RC.trt
RC.ous
RC.neu
RC.vdg
Goal: To illustrate the importance of selecting an appropriate integration time step.
In SNNAP, the ordinary differential equations (ODEs) are integrated with the forward-Euler method, which uses a fixed time step to numerically solve ODEs. The forward Euler method is simple and fast, but its accuracy and stability depend critically on the size of the integration time step, which is specified by the user.
As a general rule, the forward Euler is numerically unstable unless the time step is at least two times (2 x) smaller than smallest time constant within a model. Moreover, the integration will be numerically inaccurate unless the time step is at least ten times (10 x) smaller than the smallest time constant.
One of the Examples that is provided with SNNAP (/Examples/Numerical_Integration) illustrates how the size of the integration time step can affect accuracy and stability of a simulation, and the example discusses how to select a proper time step. The present posting presents another example of how the accuracy of the forward Euler is affected by the size of the integration time step.
The present posting presents the simulation of the voltage decay in a simple RC circuit. This is analogues to a single compartment model of a neuron with a constant membrane conductance and a membrane capacitance. The analytical solution for the voltage decay in a RC circuit is given by:
V(t) = V(0) exp(-t/tau) (Eq. 1)
where V(0) is the initial value of the voltage, tau is time constant of the RC circuit and t is time. In the present example V(0) = 1 mV and tau = 1 sec. Eq. 1 was solved and plotted using Excel (MicroSoft). A model with identical properties was developed and solved using SNNAP8. A series of simulations was conducted in which the integration time step was 100 x, 50 x, 20 x, 10 x, 5 x and 2 x smaller than tau. The results of the simulations were graphically compared to the analytical solution. The results are presented in the attached image file (Exponential.jpg).
Figure Legend: Numerical accuracy of the forward Euler method for integrating ODEs. A: Comparison between analytical and simulated solutions of the voltage decay in a RC circuit. Seven curves are plotted: the solution of Eq. 1 (dashed yellow line), and the results of six SNNAP simulations in which the integration time step was 100 x (orange line), 50 x (green line), 20 x (red line), 10 x (purple line), 5 x (blue line), and 2 x (black line) faster than the time constant (tau) of the RC circuit. The analytical solution, 100 x, 50 x, 20 x and 10 x traces are virtually indistinguishable, which indicates that the SNNAP simulations accurately reproduced the analytical solution. The accuracy of the simulation deteriorated, however, as the integration time step became too large (traces 5 x and 2 x). The segments of the traces outlined by the dashed box are expanded in Panel B. B: Expanded view of a segment of the traces illustrated in Panel A. At this higher resolution, it is possible to distinguish the individual traces. As the integration time step becomes larger, the accuracy of the SNNAP simulation decreases. A sum of squared errors analysis (not shown) indicated that the difference between the analytical and simulated solutions became significant when the integration time step was increased to 5 x smaller than tau.
Although the integration time step had an important impact on the numerical accuracy of a simulation, the accuracy was not affected by the complexity of the neural network. Increasing the number of neurons in the simulation by 1000 fold produced the exact same solutions.
The attachments to this posting also include the SNNAP input files necessary to reproduce the results presented above.
Summary: As a general rule, the user should select an integration time step that is at least ten times faster than the smallest time constant in the simulation. However, as the integration time step is reduced the speed of the simulation will slow. This is simply because more iterations are occurring as the time step becomes smaller. Thus, selecting an appropriate time step is a balancing act between accuracy, stability and speed.
Doug
snnap.uth.tmc.edu/post/pic/A0/RC.neu
snnap.uth.tmc.edu/post/pic/A0/RC.ntw
snnap.uth.tmc.edu/post/pic/A0/RC.ous
snnap.uth.tmc.edu/post/pic/A0/RC.smu
snnap.uth.tmc.edu/post/pic/A0/RC.trt
snnap.uth.tmc.edu/post/pic/A0/RC.vdg