Performance numbers of MVAPICH2 on Intel Broadwell Architecture with Mellanox ConnectX-4 (09/09/16)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.12 us 11970.41 MBps* 23838.68 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2680 2x14 @ 2.4GHz 64GB Mellanox ConnectX-4 (100Gbps) Mellanox EDR IB Switch CentOS 7.2.1511

Performance numbers of MVAPICH2 on Intel Haswell Architecture with Mellanox ConnectX-3 (09/09/16)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.09 us 6354.80 MBps* 12211.21 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB Mellanox ConnectX-3 (56Gbps) Mellanox FDR IB Switch CentOS 7.1.1503

Performance numbers of MVAPICH2 on Intel Haswell Architecture with Mellanox Connect-IB Dual Port (09/09/16)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.25 us 12622.07 MBps* 21766.20 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB Mellanox Connect-IB (2x56Gbps) Mellanox FDR IB Switch CentOS 7.1.1503

Performance numbers of MVAPICH2 on Intel Haswell Architecture with Mellanox ConnectX-4 (RoCE) (09/09/16)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.03 us 11357.40 MBps* 21050.97 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB Mellanox ConnectX-4 (RoCE) (100Gbps) Back to Back CentOS 7.1.1503

Performance numbers of MVAPICH2 on Intel Haswell Architecture with Intel Omni-Path (09/09/16)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
0.98 us 12384.35 MBps* 24293.16 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB Intel Omni-Path HFI (100Gbps) Intel Omni-Path Switch CentOS 7.1.1503

Performance numbers of MVAPICH2 on Intel Haswell Architecture Intra-Node (09/09/16)

MPI Latency Bandwidth Bidirectional Bandwidth Put Latency Get Latency Accumulate Latency Notes
Intra-Socket 0.20 us 15347.49 MBps* 27815.43 MBps* 0.05 us 0.05 us 0.08 us MV2_CPU_MAPPING=1:2
*MBps = Million Bytes per second
Inter-Socket 0.43 us 14897.48 MBps* 27658.28 MBps* 0.05 us 0.05 us 0.08 us MV2_CPU_MAPPING=1:12
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB CentOS 7.1.1503

Intra-Socket

Inter-Socket