GetConnections is too slow- can i obtain connection ids as I connect the neurons? (nest v2.20)
Hello,
I have a serious bottleneck in terms of runtime in my simulations. I have a modular cortical network and i need to 'manually' change weights in some specific connections before i start the simulation. The modularity means that this happens inside a for loop for all populations pairs.
so, before simulation starts i have a function that goes over all population pairs and does this:
AMPA_conns = nest.GetConnections(source=pre_pop,target=post_pop, synapse_model='AMPA_synapse_longrange') # RETRIEVE ALL AMPA CONNECTIONS BETWEEN SOURCE AND TARGET POPULATION nest.SetStatus(AMPA_conns, {'weight': pattern_weight}) # CHANGE CONNECTION WEIGHT NMDA_conns = nest.GetConnections(source=pre_pop,target=post_pop, synapse_model='NMDA_synapse_longrange') # RETRIEVE ALL NMDA CONNECTIONS BETWEEN SOURCE AND TARGET POPULATION nest.SetStatus(NMDA_conns, {'weight': pattern_weight * nmda2ampa_ratio}) # CHANGE CONNECTION WEIGHT
commenting out these 4 lines reduces my runtime by 8 hours. for reference, the model consists of ~27000neurons, is created in 20 minutes (nodes generated and connections drawn) and the actual simulation runs for about 15 minutes.
so my question is: is there a way to obtain the connection ids as i draw the connections (right after the nest.Connect() function) so that i can store them in a data structure and avoid this expensive operation of searching? or perhaps it is the setstatus that makes it expensive?
*I havent yet migrated to nest v3, this concerns version 2.20. thank you for your time! angeliki
hey again!
let me know if you are aware of this issue and whether to the best of this knowledge it is resolved in nest 3.0.
thank you for your time angeliki
Hello Angeliki,
GetConnections() takes time, but eight hours compared to 20 + 15 minutes for a network of 27.000 neurons seems very slow. I would definitely suggest that you try with the newest NEST version, 3.5. NEST 2.20 is quite old by now and we have made a large number of improvements in the meantime.
NEST 3.5 allows parameterization of neurons and connections while they are created directly at the C++ level, which saves a lot of time (20 minutes to build a 27.000 neuron network seems slow). Depending on what kind of "manual" manipulation of synaptic weights you need to do, it might even be possible to include this in connection creation, so you can avoid GetConnections() altogether.
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Angeliki Papadimitriou angpap@kth.se Date: Wednesday, 23 August 2023 at 11:24 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: GetConnections is too slow- can i obtain connection ids as I connect the neurons? (nest v2.20) [Some people who received this message don't often get email from angpap@kth.se. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
hey again!
let me know if you are aware of this issue and whether to the best of this knowledge it is resolved in nest 3.0.
thank you for your time angeliki _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Thank you for the reply. I will attempt to migrate to 3.5 and see if it works faster.
i know for a fact that i can benefit from the collocated synapse class to draw both ampa and nmda connections between the same neurons. but can you elaborate a bit on the last thing you mentioned: "Depending on what kind of "manual" manipulation of synaptic weights you need to do, it might even be possible to include this in connection creation, so you can avoid GetConnections() altogether."
the scenario: i initially draw non specific recurrent connections within the global neuron population using a connection probability. The network is organized in hypercolumns that contain minicolumns. Knowing the minicolumn index of the pre and post populations i want to upregulate the weight. Can that be done without a call to getconnections?
Thank you for your time Angeliki
Hello Angeliki,
IN NEST 3.5, one can provide relatively complex expressions for connection rules, including weights, see
https://nest-simulator.readthedocs.io/en/stable/neurons/parametrization.htmlhttps://nest-simulator.readthedocs.io/en/stable/neurons/parametrization.html#parametrization
This mostly applies to networks with spatial structure, though.
If you would be willing to share more details about your network and simulation scripts, I could provide more detailed suggestions. You can send that information directly to my mail address if you do not want to make it public on the mailing list.
Best, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: Angeliki Papadimitriou angpap@kth.se Date: Thursday, 24 August 2023 at 12:05 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: GetConnections is too slow- can i obtain connection ids as I connect the neurons? (nest v2.20) [Du mottar ikke ofte e-post fra angpap@kth.se. Finn ut hvorfor dette er viktig p? https://aka.ms/LearnAboutSenderIdentification ]
Thank you for the reply. I will attempt to migrate to 3.5 and see if it works faster.
i know for a fact that i can benefit from the collocated synapse class to draw both ampa and nmda connections between the same neurons. but can you elaborate a bit on the last thing you mentioned: "Depending on what kind of "manual" manipulation of synaptic weights you need to do, it might even be possible to include this in connection creation, so you can avoid GetConnections() altogether."
the scenario: i initially draw non specific recurrent connections within the global neuron population using a connection probability. The network is organized in hypercolumns that contain minicolumns. Knowing the minicolumn index of the pre and post populations i want to upregulate the weight. Can that be done without a call to getconnections?
Thank you for your time Angeliki _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Oh I see what you mean now. There was a similar functionality in the v2.20 using the 'distribution' parametrization. This is not applicable in my case however as the network organization relies on abstract indexing and not spatial coordinates. Thank you all the same.
Best Angeliki
-
Angeliki Papadimitriou -
Hans Ekkehard Plesser