Hello,
I am trying to access the list of approx. 13 000 000 synapses in my network. However, accessing any of the values in SynapseCollection runs longer than my whole simulation and can cause out of memory issues.
In the following snippet, just the `conns.get('weight')` line takes my computer 13 minutes to compute and uses over 5 GB of memory, even though the data I am interested in should not be larger than 50 MB.
importnest neurons= nest.Create('iaf_psc_exp', 20000) conn_dict= {'rule': 'fixed_indegree', 'indegree': 300} nest.Connect(neurons, neurons, conn_dict, syn_spec={'weight': 1.0}) conns= nest.GetConnections() conns.get('weight')
How should I efficiently access/export the `source`, `target`, `weight` and `distance` synaptic variables without unnecessary time or memory overhead?
Thank you.
Best regards,
Maurycy Miękus
Hi Maurycy,
Unfortunately, NEST does not have an efficient way of extracting single information values from large sets of connections at the moment.
What you could do at the Python level to reduce the memory overhead would be to loop over the source neurons.
The code below ran in about 3.5 minutes on my MacBook Pro and used slighly less than 500 MB (Jupyter notebook included):
The total number of connections is always available as nest.num_connections so you don’t need to calculate it to pre-size the numpy array if you want all connections.
If you frequently need to do this operation on large networks, we could consider a more efficient implementation, but that would require work a the C++ level. Would you be interested in contributing to that?
Best regards, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser Research Committee Chair, Faculty of Science and Technology Head, Department of Data Science
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: M. Maurycy Miękus miekus@em.uni-frankfurt.de Date: Wednesday, 15 October 2025 at 09:30 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] SynapseCollection.get() causes overhead Some people who received this message don't often get email from miekus@em.uni-frankfurt.de. Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
Hello,
I am trying to access the list of approx. 13 000 000 synapses in my network. However, accessing any of the values in SynapseCollection runs longer than my whole simulation and can cause out of memory issues.
In the following snippet, just the `conns.get('weight')` line takes my computer 13 minutes to compute and uses over 5 GB of memory, even though the data I am interested in should not be larger than 50 MB. import nest neurons = nest.Create('iaf_psc_exp', 20000) conn_dict = {'rule': 'fixed_indegree', 'indegree': 300} nest.Connect(neurons, neurons, conn_dict, syn_spec={'weight': 1.0}) conns = nest.GetConnections() conns.get('weight')
How should I efficiently access/export the `source`, `target`, `weight` and `distance` synaptic variables without unnecessary time or memory overhead?
Thank you.
Best regards,
Maurycy Miękus
hello, I am on the mailing list and my version of the email does not include the "code below". I am interested in adjusting the weight in mS rather than seconds in a real time simulation. If someone can get the "code below" to me or acquaint me with magic that is google mail I would appreciate it. Thank you Peter
On Wed, Oct 15, 2025 at 2:27 PM Hans Ekkehard Plesser < hans.ekkehard.plesser@nmbu.no> wrote:
Hi Maurycy,
Unfortunately, NEST does not have an efficient way of extracting single information values from large sets of connections at the moment.
What you could do at the Python level to reduce the memory overhead would be to loop over the source neurons.
The code below ran in about 3.5 minutes on my MacBook Pro and used slighly less than 500 MB (Jupyter notebook included):
The total number of connections is always available as nest.num_connections so you don’t need to calculate it to pre-size the numpy array if you want all connections.
If you frequently need to do this operation on large networks, we could consider a more efficient implementation, but that would require work a the C++ level. Would you be interested in contributing to that?
Best regards,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Research Committee Chair, Faculty of Science and Technology
Head, Department of Data Science
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser
*From: *M. Maurycy Miękus miekus@em.uni-frankfurt.de *Date: *Wednesday, 15 October 2025 at 09:30 *To: *users@nest-simulator.org users@nest-simulator.org *Subject: *[NEST Users] SynapseCollection.get() causes overhead
Some people who received this message don't often get email from miekus@em.uni-frankfurt.de. Learn why this is important https://aka.ms/LearnAboutSenderIdentification
Hello,
I am trying to access the list of approx. 13 000 000 synapses in my network. However, accessing any of the values in SynapseCollection runs longer than my whole simulation and can cause out of memory issues.
In the following snippet, just the `conns.get('weight')` line takes my computer 13 minutes to compute and uses over 5 GB of memory, even though the data I am interested in should not be larger than 50 MB.
import nest
neurons = nest.Create('iaf_psc_exp', 20000)
conn_dict = {'rule': 'fixed_indegree',
'indegree': 300}
nest.Connect(neurons, neurons, conn_dict, syn_spec={'weight': 1.0})
conns = nest.GetConnections()
conns.get('weight')
How should I efficiently access/export the `source`, `target`, `weight` and `distance` synaptic variables without unnecessary time or memory overhead?
Thank you.
Best regards,
Maurycy Miękus _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
-
Hans Ekkehard Plesser -
M. Maurycy Miękus -
Peter Mason