MIMD is a parallel computer structure composed of multiple independent processors. MIMD machines are the most common type of parallel machines in use today. In this architecture, each processor has its own copy of a program and it can work on different data streams. The programming model for MIMD resembles the way processes interact in UNIX, but the obvious difference is that they are actually executing concurrently.
One factor that can have great impact on the performance of parallel] [ systems is the interconnection of the processors. Because processors need to interact, some way to share information must be implemented. The study of the topological properties of the communication network and the way processors share common resources, as memory, generates further division in parallel computer architectural classification.
The two extreme cases for the way MIMD machines share information are shown in Figures 1 and 2. Figure 1 shows the general architecture of a shared memory MIMD machine. In this model all the processors have equal access to the memory, and the memory is seen as a single address space by every processor.
In Figure 2, we have the general model of a distributed memory MIMD. In this model each processor has its own memory. This memory may be accessed by other processors but it is clear that the interconnection network will impose different access constraints to the different processors.
Shared memory machines have the advantage of allowing the same programming model as a UNIX workstation. The main drawback of pure shared memory machines is high contention for the memory bus and the speed of dynamic RAM, which is known to have high latency. This further limits the scalability of shared memory machines.
Actually, pure systems are rare and most large shared memory machines are in fact distributed memory MIMD with special hardware for accessing memory. The reason is that the bus to memory interface becomes a bottleneck when just a few processors need to access the same shared memory limiting the scalability of the system. For this reason the largest MIMD machines have distributed memory.
To build scalable shared memory architectures, several researchers have been trying novel interconnection techniques. Early optimizations employed caches (single and double level caches greatly improved the performance of the systems by reducing the use of bus bandwidth). The drawback of this approach is the complexity associated with cache coherence in multiprocessor architectures. Several shared memory architectures in use today, such as the BBN TC2000 [19], are really distributed memory architectures with a layer (usually in hardware) that makes the communication transparent.
When more than a few tens of processors are put together, contention for the communication bus can be very high. Thus, much research is being devoted to minimizing bus contention and increasing bus utilization. Very complicated communication topologies and protocols have been developed for multicomputers. For example, the Intel Paragon [18] uses a dedicated communication processor for each set of four processors.
Figure 1: General architecture of a shared memory MIMD.
Figure 2: General architecture of a distributed memory MIMD.
The interconnection network can have a great impact on performance and
in most cases, it is the most expensive piece of hardware in a
parallel machine. The study of algorithms for parallel computers
heavily relies on the characteristics of such networks. For instance,
algorithms that need global communication will, in general, run
slower on network with large diameter 
.