Hi Camilo,
Thanks a lot for your help ... but problem remains. Now I get:
Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((readout_signal[0], n_iter, batch_size, steps["sequence"])) #readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) TypeError: only integer scalar arrays can be converted to a scalar index
Xavier ________________________________ From: Camilo Jara Do Nascimento C.Jara-Do-Nascimento@sussex.ac.uk Sent: Wednesday, October 29, 2025 4:56 PM To: NEST User Mailing List users@nest-simulator.org; Xavier Otazu Porter xotazu@cvc.uab.cat Subject: Re: [NEST Users] MPI problem with e-prop tutorials
Hi Xavier,
The problem I see is that the readout first dimension is based on the n_out, and when using mpi to parallelizing it will use this first dimension to parallelize your instances. I quickly fixed it by changing the first dimension to be readout_signal.shape[0] since its already defined before by using the "idc" indexes.
readout_signal = readout_signal.reshape((readout_signal.shape[0], n_iter, batch_size, steps["sequence"]))
Hope this helps.
Best, Camilo Jara D. ________________________________ From: Xavier Otazu xotazu@cvc.uab.cat Sent: 29 October 2025 14:03 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] MPI problem with e-prop tutorials
Hello,
I have tried to execute one of the e-prop tutorials (eprop_supervised_regression_handwriting_bsshslm_2020.py) and get into problems when activating MPI. I do not know whether it is an e-prop C++ implementation problem or a Python tutorial code problem (in some cases, but not all of them, it looks like the last one).
I have two files (hostfile_orig_1 and hostfile_orig_2) to define in what nodes of the cluster to execute the program. The content of these files is: hostfile_orig_1: node0 slots=1 node1 slots=1
hostfile_orig_2: node0 slots=2 node0 slots=2
The 'slots' key tells how many (mpi) processes can be executed on a particular node.
Depending on the number of processes, the errors are a bit different. In all the following examples, I use "total_num_virtual_procs": 2 on line 173 of tutorial file.
Below I write the execution command and the output error.
Command: mpirun -np 1 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: With a "serial" execution like this one, everything is OK:
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 404, in <module> nest.GetConnections(nrns_rec[0], nrns_rec[1:3]).set([params_init_optimizer] * 2) File "/home/neurobit/local/nest_3.9/lib64/python3.8/site-packages/nest/lib/hl_api_types.py", line 945, in set raise TypeError("status dict must be a dict, or a list of dicts of length {}".format(self.__len__())) TypeError: status dict must be a dict, or a list of dicts of length 1
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 2 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 4 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404 and using "total_num_virtual_procs": 4 on line 173 of tutorial file.) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 493, in <module> "rec_out": get_weights(nrns_rec, nrns_out), File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 482, in get_weights conns["senders"] = np.array(conns["source"]) - np.min(conns["source"]) TypeError: tuple indices must be integers or slices, not str
In this last case, program stops and hungs.
If you want, I could submit a bug report on github.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
Dear Xavier,
I am happy to hear you are using our examples :)
The tutorials are intentionally kept simple and therefore do not support MPI. While NEST’s internal functions do support MPI, data retrieval operations (such as `get("events")` or `GetConnections()`) are executed independently on each MPI process. The variations in the error messages occur because different settings lead to different distributions of neurons across MPI processes and thus also process-specific neuron numbers and behavior.
By default, recorders store their data in memory. The simplest way to collect results is to save them to disk by adding the parameter `"record_to": "ascii"` to the recorder settings. This will write the data to files. After running the simulation, you can load the results on a single MPI process (e.g., using `if nest.Rank() == 0`), and the resulting matrix should then have the correct shape.
The line
```python nest.GetConnections(nrns_rec[0], nrns_rec[1:3]).set([params_init_optimizer] * 2) ```
is included only as a demonstration that dynamic variables can be initialized manually. In the tutorial, they are set to their default values, so if you are not changing them explicitly, you can safely comment out this line.
Further adjustments to the code may be necessary following these principles.
Hopefully this helps!
All the best, Agnes
On 2025-10-29 17:12, Xavier Otazu Porter wrote:
Hi Camilo,
Thanks a lot for your help ... but problem remains. Now I get:
Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((readout_signal[0], n_iter, batch_size, steps["sequence"])) #readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) TypeError: only integer scalar arrays can be converted to a scalar index
Xavier
*From:* Camilo Jara Do Nascimento C.Jara-Do-Nascimento@sussex.ac.uk *Sent:* Wednesday, October 29, 2025 4:56 PM *To:* NEST User Mailing List users@nest-simulator.org; Xavier Otazu Porter xotazu@cvc.uab.cat *Subject:* Re: [NEST Users] MPI problem with e-prop tutorials Hi Xavier,
The problem I see is that the readout first dimension is based on the n_out, and when using mpi to parallelizing it will use this first dimension to parallelize your instances. I quickly fixed it by changing the first dimension to be readout_signal.shape[0] since its already defined before by using the "idc" indexes.
readout_signal = readout_signal.reshape((readout_signal.shape[0], n_iter, batch_size, steps["sequence"]))
Hope this helps.
Best, Camilo Jara D.
*From:* Xavier Otazu xotazu@cvc.uab.cat *Sent:* 29 October 2025 14:03 *To:* users@nest-simulator.org users@nest-simulator.org *Subject:* [NEST Users] MPI problem with e-prop tutorials Hello,
I have tried to execute one of the e-prop tutorials (eprop_supervised_regression_handwriting_bsshslm_2020.py) and get into problems when activating MPI. I do not know whether it is an e-prop C++ implementation problem or a Python tutorial code problem (in some cases, but not all of them, it looks like the last one).
I have two files (hostfile_orig_1 and hostfile_orig_2) to define in what nodes of the cluster to execute the program. The content of these files is: hostfile_orig_1: node0 slots=1 node1 slots=1
hostfile_orig_2: node0 slots=2 node0 slots=2
The 'slots' key tells how many (mpi) processes can be executed on a particular node.
Depending on the number of processes, the errors are a bit different. In all the following examples, I use "total_num_virtual_procs": 2 on line 173 of tutorial file.
Below I write the execution command and the output error.
Command: mpirun -np 1 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: With a "serial" execution like this one, everything is OK:
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 404, in <module> nest.GetConnections(nrns_rec[0], nrns_rec[1:3]).set([params_init_optimizer] * 2) File "/home/neurobit/local/nest_3.9/lib64/python3.8/site-packages/nest/lib/hl_api_types.py", line 945, in set raise TypeError("status dict must be a dict, or a list of dicts of length {}".format(self.__len__())) TypeError: status dict must be a dict, or a list of dicts of length 1
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 2 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 4 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404 and using "total_num_virtual_procs": 4 on line 173 of tutorial file.) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 493, in <module> "rec_out": get_weights(nrns_rec, nrns_out), File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 482, in get_weights conns["senders"] = np.array(conns["source"]) - np.min(conns["source"]) TypeError: tuple indices must be integers or slices, not str
In this last case, program stops and hungs.
If you want, I could submit a bug report on github.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Center http://www.cvc.uab.cat CONFIDENTIALITY WARNING http://www.cvc.uab.es/?page_id=7475
NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Agnes,
Thanks a lot for your help! We are going to take a look at our code, specially on the retrieval and post-processing of the variables.
Xavier ________________________________ From: Agnes Korcsak-Gorzo a.korcsak-gorzo@fz-juelich.de Sent: Thursday, October 30, 2025 9:43 AM To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] Re: MPI problem with e-prop tutorials
Dear Xavier,
I am happy to hear you are using our examples :)
The tutorials are intentionally kept simple and therefore do not support MPI. While NEST’s internal functions do support MPI, data retrieval operations (such as `get("events")` or `GetConnections()`) are executed independently on each MPI process. The variations in the error messages occur because different settings lead to different distributions of neurons across MPI processes and thus also process-specific neuron numbers and behavior.
By default, recorders store their data in memory. The simplest way to collect results is to save them to disk by adding the parameter `"record_to": "ascii"` to the recorder settings. This will write the data to files. After running the simulation, you can load the results on a single MPI process (e.g., using `if nest.Rank() == 0`), and the resulting matrix should then have the correct shape.
The line
```python nest.GetConnections(nrns_rec[0], nrns_rec[1:3]).set([params_init_optimizer] * 2) ```
is included only as a demonstration that dynamic variables can be initialized manually. In the tutorial, they are set to their default values, so if you are not changing them explicitly, you can safely comment out this line.
Further adjustments to the code may be necessary following these principles.
Hopefully this helps!
All the best, Agnes
On 2025-10-29 17:12, Xavier Otazu Porter wrote:
Hi Camilo,
Thanks a lot for your help ... but problem remains. Now I get:
Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((readout_signal[0], n_iter, batch_size, steps["sequence"])) #readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) TypeError: only integer scalar arrays can be converted to a scalar index
Xavier
*From:* Camilo Jara Do Nascimento C.Jara-Do-Nascimento@sussex.ac.uk *Sent:* Wednesday, October 29, 2025 4:56 PM *To:* NEST User Mailing List users@nest-simulator.org; Xavier Otazu Porter xotazu@cvc.uab.cat *Subject:* Re: [NEST Users] MPI problem with e-prop tutorials
Hi Xavier,
The problem I see is that the readout first dimension is based on the n_out, and when using mpi to parallelizing it will use this first dimension to parallelize your instances. I quickly fixed it by changing the first dimension to be readout_signal.shape[0] since its already defined before by using the "idc" indexes.
readout_signal = readout_signal.reshape((readout_signal.shape[0], n_iter, batch_size, steps["sequence"]))
Hope this helps.
Best, Camilo Jara D.
*From:* Xavier Otazu xotazu@cvc.uab.cat *Sent:* 29 October 2025 14:03 *To:* users@nest-simulator.org users@nest-simulator.org *Subject:* [NEST Users] MPI problem with e-prop tutorials
Hello,
I have tried to execute one of the e-prop tutorials (eprop_supervised_regression_handwriting_bsshslm_2020.py) and get into problems when activating MPI. I do not know whether it is an e-prop C++ implementation problem or a Python tutorial code problem (in some cases, but not all of them, it looks like the last one).
I have two files (hostfile_orig_1 and hostfile_orig_2) to define in what nodes of the cluster to execute the program. The content of these files is: hostfile_orig_1: node0 slots=1 node1 slots=1
hostfile_orig_2: node0 slots=2 node0 slots=2
The 'slots' key tells how many (mpi) processes can be executed on a particular node.
Depending on the number of processes, the errors are a bit different. In all the following examples, I use "total_num_virtual_procs": 2 on line 173 of tutorial file.
Below I write the execution command and the output error.
Command: mpirun -np 1 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: With a "serial" execution like this one, everything is OK:
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 404, in <module> nest.GetConnections(nrns_rec[0], nrns_rec[1:3]).set([params_init_optimizer] * 2) File "/home/neurobit/local/nest_3.9/lib64/python3.8/site-packages/nest/lib/hl_api_types.py", line 945, in set raise TypeError("status dict must be a dict, or a list of dicts of length {}".format(self.__len__())) TypeError: status dict must be a dict, or a list of dicts of length 1
Command: mpirun -np 2 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 2 -hostfile hostfile_orig_1 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 545, in <module> readout_signal = readout_signal.reshape((n_out, n_iter, batch_size, steps["sequence"])) ValueError: cannot reshape array of size 364800 into shape (2,200,1,1824)
Command: mpirun -np 4 -hostfile hostfile_orig_2 python3 eprop_supervised_regression_handwriting_bsshslm_2020.py (commenting line 404 and using "total_num_virtual_procs": 4 on line 173 of tutorial file.) Output: Traceback (most recent call last): File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 493, in <module> "rec_out": get_weights(nrns_rec, nrns_out), File "eprop_supervised_regression_handwriting_bsshslm_2020.py", line 482, in get_weights conns["senders"] = np.array(conns["source"]) - np.min(conns["source"]) TypeError: tuple indices must be integers or slices, not str
In this last case, program stops and hungs.
If you want, I could submit a bug report on github.
Xavier _______________________________________________ NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
Computer Vision Center http://www.cvc.uab.cat CONFIDENTIALITY WARNING http://www.cvc.uab.es/?page_id=7475
NEST Users mailing list -- users@nest-simulator.org To unsubscribe send an email to users-leave@nest-simulator.org
-- Agnes Korcsak-Gorzo Institute for Advanced Simulation (IAS-6) Phone: +49 2461/61-4748 Computational and Systems Neuroscience Fax: +49 2461/61-9460 Jülich Research Centre, Jülich, Germany
https://www.fz-juelich.de/en/ias/ias-6
Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Dr. Stephanie Bauer (stellvertretende Vorsitzende), Prof. Dr. Ir. Pieter Jansens, Prof. Dr. Laurens Kuipers
Das Forschungszentrum Jülich stellt zurzeit auf einen neuen Zertifikatsanbieter zum digitalen Signieren von E-Mails um. Während dieser Umstellungsarbeiten kann es vorkommen, dass das DFN Community PKI Zertifikat, mit dem diese E-Mail signiert worden ist, als ungültig angezeigt wird.
Computer Vision Centerhttp://www.cvc.uab.cat CONFIDENTIALITY WARNINGhttp://www.cvc.uab.es/?page_id=7475
-
Agnes Korcsak-Gorzo -
Xavier Otazu Porter