Performance numbers of MVAPICH2 on Intel Haswell Architecture with Mellanox ConnectX-4 (07/23/18)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.05 us 12380.52 MBps* 24628.38 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory Network Adapter Switch OS Network Stack
Intel E5-2687W 2x10 @ 3.1GHz 64GB Mellanox ConnectX-4 (100Gbps) Mellanox EDR Switch CentOS 7.4.1708 Mellanox OFED 4.2-1.2.0.0

Performance numbers of MVAPICH2 on Intel Haswell Architecture with Mellanox ConnectX-4 (RoCE) (07/23/18)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.03 us 11332.69 MBps* 21118.42 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory Network Adapter Switch OS Network Stack
Intel E5-2687W 2x10 @ 3.1GHz 64GB Mellanox ConnectX-4 (RoCE) (100Gbps) Back to Back CentOS 7.4.1708 Mellanox OFED 4.2-1.2.0.0

Performance numbers of MVAPICH2 on Intel Skylake Architecture with Intel Omni-Path (07/23/18)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
0.94 us 12391.06 MBps* 24635.66 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory Network Adapter Switch OS Network Stack
Intel Xeon Platinum 8160 Processor 2x24 @ 2.1GHz 192GB Intel Omni-Path HFI (100Gbps) Intel Omni-Path Switch CentOS 7.4.1708 IFS 10.6

Performance numbers of MVAPICH2 on Intel Haswell Architecture Intra-Node (07/23/18)

MPI Latency Bandwidth Bidirectional Bandwidth Put Latency Get Latency Accumulate Latency Notes
Intra-Socket 0.22 us 15141.78 MBps* 26922.73 MBps* 0.05 us 0.05 us 0.08 us MV2_CPU_MAPPING=1:2
*MBps = Million Bytes per second
Inter-Socket 0.41 us 14432.20 MBps* 26033.54 MBps* 0.05 us 0.05 us 0.08 us MV2_CPU_MAPPING=1:11
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory OS
Intel E5-2687W 2x10 @ 3.1GHz 64GB CentOS 7.4.1708

Intra-Socket

Inter-Socket


Performance numbers of MVAPICH2 on Intel KNL Architecture with Mellanox EDR (07/23/18)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
2.22 us 12321.75 MBps* 24419.24 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory Network Adapter Switch OS Network Stack
Intel Xeon Phi 7250 1x68 @ 1.4GHz 96GB Mellanox EDR (100Gbps) Mellanox EDR Switch CentOS 7.4.1708 Mellanox OFED 4.2-1.2.0.0

Performance numbers of MVAPICH2 on Intel KNL Architecture with Intel Omni-Path (07/23/18)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
2.33 us 10721.49 MBps* 14213.64 MBps* The processes are bound to core 7 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory Network Adapter Switch OS Network Stack
Intel Xeon Phi 7250 1x68 @ 1.4GHz 96GB Intel Omni-Path HFI (100Gbps) Intel Omni-Path Switch CentOS 7.4.1708 IFS 10.6