SMG2000

• Experimental Testbed: Each node of our testbed has two 8-core 2.6 GHz Intel Xeon E5-2680 (Sandy Bridge) processors and 32 GB main memory. The nodes support 16x PCI Express Gen3 interfaces and are equipped with Mellanox ConnectX-3 FDR HCAs with PCI Express Gen3 interfaces. The operating system used was CentOS release 6.3.

• SMG2000 is a parallel semi-coarsening multigrid solver, which is part of the ASC Purple Benchmarks.

• InfiniBand (IB) offers many transport modes of which Reliable Connected (RC) and Unreliable Datagram (UD) Queue Pairs (QP) are two of the more popular ones. MVAPICH2 uses the RC QP for all communication by default. As part of our on-going research, we have enabled MVAPICH2 to use UD QPs exclusively. Applications can use only UD transport by simply configuring MVAPICH2 with the --enable-hybrid option and setting the environment variable MV2_USE_ONLY_UD. Please refer to the following section of the userguide for more information on running MVAPICH2 in UD only mode.

• The bar graph compares and contrast the performance of SMG2000 obtained with RC and UD QPs. We observe that as the system size increases the UD QP based design performs better than the RC QP based design.