Routing

We have seen how neighboring PEs may communicate with each other. We have seen how PEs connected together in a region may communicate region values. In this chapter we will see how a PE can send a message to any other PE. Many problems require this kind of communication. Examples include Fast Fourier Transforms, summing trees, and graph algorithms.

There are three categories of this type of communication:

  1. One-to-one (permutation)
  2. Many-to-one (combining)
  3. One-to-many (broadcast)
In one-to-one communication, each sender sends just one message and each receiver receives just one message. This type of communication is sometimes referred to as a permutation because, in a computer like the CAAPP, the result is to move the values around between the PEs. An example would be for every PE to send its message to the PE whose address is formed by doing a bit-wise complement of the sender's address.

In many-to-one communication, the objective is generally to combine the messages in some way. An example would be obtaining the count of the PEs in a region. Every PE would send the message `1'. This would be combined by summation with the message arriving at the receiver being the number of PEs in the region.

In one-to-many communication, a small set of PEs send their messages to a larger set. At one extreme would be one PE sending or broadcasting to every other PE.

The ICL supports the first two communications schemes through a very general routing facility. Each PE specifies a message and the address of the destination PE. One-to-one communication is created by insuring that the destination addresses are unique. Many-to-one communication allows combining by sum-mation or other primitive operations. The one-to-many communication is provided by operations on regions (Coterie).

One to One

The following code implements the one-to-one routing where the destination address is derived by a bit-wise complement of the sender's address.
    CharPlane permute_comp(CharPlane value)
     {
      return value.Route(~value.RowIndex(), ~value.ColIndex());
     }
The Route method sends the message (value) specified to the destination PE address. Destination addresses are given as row and column PE indexes. The PE in the upper, left-hand corner of the SIMD grid is at row 0, column 0. The destination address for the message sent by the PE at row 12, column 48 is row 65523, column 65487
(1).

Let us assume that the plane has 512 rows and 512 columns. In this case the highest address of any PE is (511,511). A message sent to (65523,65487) is sent to a non-existent PE. What happens to these messages? They are discarded.

Let us re-do the above example assuming that the plane value is exactly 512 rows by 512 columns and avoid losing any messages.

    CharPlane permute_comp(CharPlane value)
     {
      return value.Route(0x1ff & ~value.RowIndex(), 
                         0x1ff & ~value.ColIndex());
     }
Now the destination address for the message sent by the PE at (12,48) is (499,463).

A plane may be transposed by just interchanging the row and column addresses.

    value = value.Route(value.ColIndex(), value.RowIndex());
Of course, if the plane is not square, some messages will be lost. As shown in the figures below, the values in the right hand side of the plane are unknown as these PEs receive no message.

What happens if a PE receives more than one message? Only one of the messages arrives and the others are lost. Which one arrives? The ICL does not define this -- it only guarantees that no more than one will arrive!

Many to One

If you want to do a many-to-one route, you should use the combining methods. For example, let us sum up the number of PEs in a region and send this count to every PE in the region (a one-to-many broadcast).
  IntPlane region_pe_count(BitPlane w, e, n, s)
   {
    PlaneSize ps(w);
    IntPlane sum(ps);
     {CoterieWENS pattern(w,e,n,s);
      BitPlane master(ps);
      UShortPlane master_row(ps), master_col(ps);

      master = (master.Index()).RegionSelectMin();
      master_row = (master.RowIndex()).RegionBroadcast(master);
      master_col = (master.ColIndex()).RegionBroadcast(master);

      sum = 1;
      sum = sum.RoutePlus(master_row, master_col);
      sum = sum.RegionBroadcast(master);
     }
    return sum;
   }
In this example, we first select one PE to be the master of the region
(2). This PE will receive the summation of the messages sent by every PE in the region. In order for each PE to know where to send its message, it must know the master PE's address. This value is broadcast to every PE in the region. The RoutePlus method sends the message (a value of 1) from each PE to the master PE. As each message arrives, it is summed with any previous message that has arrived. This result is a count of all the PEs in a region which is then sent from the master PE to every PE in the region by the RegionBroadcast method.

Other methods exist to route and combine messages. They are RouteAnd, RouteOr, and RouteXOR. The order in which messages arrive is undefined.


Next -- Debugging & Display
ICL Table of Contents
ICL Index
ICL Home Page


Notes

  1. Row and column indexes are unsigned 16-bit values.
  2. In algorithms of this type (i.e. routing in regions), there is a definite advantage to selecting the Northern most PE in each region as the master PE. This is due to the way in which routing is implemented. See Practical Algorithms for Online Routing on SIMD Meshes by Martin Herbordt. We use a three channel modified greedy routing algorithm with reconfigurable bus. We have East, West, and North channels but no South channel. Therefore, messages, whose destination is to the South, are routed North, using the torus connection, to the South of their destination.