Hans,

Here I send an even more improved version. It onlys look for node position if the node is different from the one of the previous iteration. It also considers (I do not know if it is safe -I included an sssert-) that source node positions (src_vec) and connectome are ordered by source node. It completely avoids the use of the for loop on the source node positions.

The final execution time now is 9 seconds.

Instead of sending more messages in this list, do you prefer I send all this information through the github issue?

Xavier





template < int D >
void
Layer< D >::dump_connections( std::ostream& out,
  NodeCollectionPTR node_collection,
  AbstractLayerPTR target_layer,
  const Token& syn_model )
{
  std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );

  // Dictionary with parameters for get_connections()
  DictionaryDatum conn_filter( new Dictionary );
  def( conn_filter, names::synapse_model, syn_model );
  def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );

  // Avoid setting up new array for each iteration of the loop
//  std::vector< size_t > source_array( 1 );

//  for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin();
//        src_iter != src_vec->end();
//        ++src_iter )
//  {

//    const size_t source_node_id = src_iter->second;
//    const Position< D > source_pos = src_iter->first;

//    source_array[ 0 ] = source_node_id;
//    def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) );
    def( conn_filter, names::source, NodeCollectionDatum( node_collection ) );
    ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );

    size_t previous_source_node_id = getValue< ConnectionDatum >( connectome.get( 0 ) ).get_source_node_id();
    typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin();
       Position< D > source_pos = src_iter->first;

    // Print information about all local connections for current source
    for ( size_t i = 0; i < connectome.size(); ++i )
    {
      ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) );
      const size_t source_node_id = con_id.get_source_node_id();

      // Search source_pos for source node only if it is a different node
      if(source_node_id != previous_source_node_id)
            {
//                source_pos = src_vec->begin()->first;
//
//                for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin();
//                  src_iter != src_vec->end() && source_node_id!=src_iter->second;
//                  ++src_iter, source_pos =  src_iter->first);

                  assert((src_iter++)!=src_vec->end());
                  source_pos = src_iter->first;

                  previous_source_node_id = source_node_id;
            }

//      DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(),
      DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id,
        con_id.get_target_node_id(),
        con_id.get_target_thread(),
        con_id.get_synapse_model_id(),
        con_id.get_port() );

      long target_node_id = getValue< long >( result_dict, names::target );
      double weight = getValue< double >( result_dict, names::weight );
      double delay = getValue< double >( result_dict, names::delay );

      // Print source, target, weight, delay, rports
      out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;

      Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );

      out << ' ';
      const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id );
      assert( tnode_lid >= 0 );
      tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out );
      out << '\n';
    }
//  }
}

From: Xavier Otazu <xotazu@cvc.uab.cat>
Sent: Monday, March 18, 2024 12:46 PM
To: users@nest-simulator.org <users@nest-simulator.org>
Subject: [NEST Users] Re: DumpLayerConnections() enhancement request
 
Hans,

I have modified the dump_connections() method. For a particular test code, with the "old" code it took 1m13s. With the "new" code it takes 14 seconds. I checked results are exactly the same.

In my implementation, I have to look for the source node position iterating (for every source layer node) over the src_vec (that contains all the source node positions). May be there could be an even more efficient way to find it?

This is my new code. I have not deleted the previous code (I have commented it).


template < int D >
void
Layer< D >::dump_connections( std::ostream& out,
  NodeCollectionPTR node_collection,
  AbstractLayerPTR target_layer,
  const Token& syn_model )
{
  std::vector< std::pair< Position< D >, size_t > >* src_vec = get_global_positions_vector( node_collection );

  // Dictionary with parameters for get_connections()
  DictionaryDatum conn_filter( new Dictionary );
  def( conn_filter, names::synapse_model, syn_model );
  def( conn_filter, names::target, NodeCollectionDatum( target_layer->get_node_collection() ) );

  // Avoid setting up new array for each iteration of the loop
//  std::vector< size_t > source_array( 1 );

//  for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin();
//        src_iter != src_vec->end();
//        ++src_iter )
//  {

//    const size_t source_node_id = src_iter->second;
//    const Position< D > source_pos = src_iter->first;

//    source_array[ 0 ] = source_node_id;
//    def( conn_filter, names::source, NodeCollectionDatum( NodeCollection::create( source_array ) ) );
    def( conn_filter, names::source, NodeCollectionDatum( node_collection ) );
    ArrayDatum connectome = kernel().connection_manager.get_connections( conn_filter );

    // Print information about all local connections for current source
    for ( size_t i = 0; i < connectome.size(); ++i )
    {
      ConnectionDatum con_id = getValue< ConnectionDatum >( connectome.get( i ) );
      const size_t source_node_id = con_id.get_source_node_id();

      // Search source_pos for source node
      Position< D > source_pos = src_vec->begin()->first;

      for ( typename std::vector< std::pair< Position< D >, size_t > >::iterator src_iter = src_vec->begin();
        src_iter != src_vec->end() && source_node_id!=src_iter->second;
        ++src_iter, source_pos =  src_iter->first);

//      DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( con_id.get_source_node_id(),
      DictionaryDatum result_dict = kernel().connection_manager.get_synapse_status( source_node_id,
        con_id.get_target_node_id(),
        con_id.get_target_thread(),
        con_id.get_synapse_model_id(),
        con_id.get_port() );

      long target_node_id = getValue< long >( result_dict, names::target );
      double weight = getValue< double >( result_dict, names::weight );
      double delay = getValue< double >( result_dict, names::delay );

      // Print source, target, weight, delay, rports
      out << source_node_id << ' ' << target_node_id << ' ' << weight << ' ' << delay;

      Layer< D >* tgt_layer = dynamic_cast< Layer< D >* >( target_layer.get() );

      out << ' ';
      const long tnode_lid = tgt_layer->node_collection_->get_lid( target_node_id );
      assert( tnode_lid >= 0 );
      tgt_layer->compute_displacement( source_pos, tnode_lid ).print( out );
      out << '\n';
    }
//  }
}



Hans Ekkehard Plesser wrote:
> Hi Xavier,
>
> Great! Don’t hesitate to get in touch if you have any questions. BTW, a nice compact book
> on modern C++ is Stroustrup’s A tour of C++ (https://stroustrup.com/Tour.html).
>
> Best,
> Hans Ekkehard
>
> --
>
> Prof. Dr. Hans Ekkehard Plesser
>
> Department of Data Science
> Faculty of Science and Technology
> Norwegian University of Life Sciences
> PO Box 5003, 1432 Aas, Norway
>
> Phone +47 6723 1560
> Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no>
> Home http://arken.nmbu.no/~plesser
>
>
>
> From: Xavier Otazu &lt;xotazu(a)cvc.uab.cat&gt;
> Date: Sunday, 17 March 2024 at 23:45
> To: users(a)nest-simulator.org &lt;users(a)nest-simulator.org&gt;
> Subject: [NEST Users] Re: DumpLayerConnections() enhancement request
> [Du mottar ikke ofte e-post fra xotazu(a)cvc.uab.cat. Finn ut hvorfor dette er viktig p?
> https://aka.ms/LearnAboutSenderIdentification ]
>
> Hans,
>
> I have experience with C++, but not with this modern auto reference loops (although taking
> a quick look it doesn't look like a very difficult concept). I can try to implement
> your approach. Will tell you in the next days.
>
> Xavier
> _______________________________________________
> NEST Users mailing list -- users(a)nest-simulator.org
> To unsubscribe send an email to users-leave(a)nest-simulator.org
_______________________________________________
NEST Users mailing list -- users@nest-simulator.org
To unsubscribe send an email to users-leave@nest-simulator.org

Computer Vision Center
CONFIDENTIALITY WARNING