Performance numbers of MVAPICH2 on Intel IvyBridge Architecture with Mellanox ConnectX-3 (04/03/15)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.08 us 6364.02 MBps* 12322.25 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2680 2x10 @ 2.8Ghz 64GB Mellanox ConnectX-3 (56Gbps) Mellanox FDR IB Switch RHEL 6.5

Performance numbers of MVAPICH2 on Intel IvyBridge Architecture with Mellanox Connect-IB Dual Port (04/03/15)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.15 us 12581.30 MBps* 24353.08 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2680 2x10 @ 2.8Ghz 64GB Mellanox Connect-IB (2x56Gbps) Mellanox FDR IB Switch RHEL 6.5

Performance numbers of MVAPICH2 on Intel IvyBridge Architecture with Mellanox ConnectX-3 (RoCE) (04/03/15)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
0.98 us 4569.32 MBps* 9128.90 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2680 2x10 @ 2.8Ghz 64GB Mellanox ConnectX-3 (RoCE) (40Gbps) Back to Back RHEL 6.5

Performance numbers of MVAPICH2 on Intel IvyBridge Architecture with Intel True Scale (04/03/15)

One Way Latency Unidirectional Bandwidth Bidirectional Bandwidth Notes
1.19 us 3389.55 MBps* 6308.22 MBps* The processes are bound to core 1 on both nodes
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory IB Card IB Switch OS
Intel E5-2680 2x10 @ 2.8Ghz 64GB Intel True Scale QLE7340 (40Gbps) Mellanox FDR IB Switch RHEL 6.5

Performance numbers of MVAPICH2 on Intel IvyBridge Architecture Intra-Node (04/03/15)

MPI Latency Bandwidth Bidirectional Bandwidth Put Latency Get Latency Accumulate Latency Notes
Intra-Socket .21 us 13175.20 MBps* 24462.05 MBps* 0.07 us 0.08 us 0.09 us MV2_CPU_MAPPING=1:2
*MBps = Million Bytes per second
Inter-Socket 0.49 us 12844.29 MBps* 23948.79 MBps* 0.07 us 0.08 us 0.09 us MV2_CPU_MAPPING=1:12
*MBps = Million Bytes per second

Machine Specifications

CPU Model CPU Core Info Memory OS
Intel E5-2680 2x10 @ 2.8Ghz 64GB RHEL 6.5

Intra-Socket

Inter-Socket