SGI 10-Gigabit Ethernet Network Adapter Manual de usuario Pagina 143
- Pagina / 150
- Tabla de contenidos
- MARCADORES
Valorado. / 5. Basado en revisión del cliente
Hadoop: DFS IO Write Performance
• DFS IO included in Hadoop, measures sequential access throughput
• We have two map tasks each writing to a file of increasing size (1-10GB)
• Significant improvement with IPoIB, SDP and 10GigE
• With SSD, performance improvement is almost seven or eight fold!
• SSD benefits not seen without using high-performance interconnect!
• In-line with comment on Google keynote about I/O performance
Four Data Nodes
Using HDD and SSD
0
10
20
30
40
50
60
70
80
90
1
2
3
4
5
6
7
8
9
10
Average Write Throughput (MB/sec)
File Size(GB)
1GE with HDD
IGE with SSD
IPoIB with HDD
IPoIB with SSD
SDP with HDD
SDP with SSD
10GE-TOE with HDD
10GE-TOE with SSD
CCGrid '11
143
- A Tutorial at CCGrid ’11 1
- Computing Systems 3
- Cluster Computing Environment 4
- (http://www.top500.org) 5
- Grid Computing Environment 6
- Compute cluster 8
- Cloud Computing Environments 9
- Hadoop Architecture 10
- Memcached Architecture 11
- • Hardware components 13
- • Software components 13
- • Ex: TCP/IP, UDP/IP 14
- – E.g., PCI, PCI-X 15
- – One bit per wire 15
- – Not scalable: 15
- Myrinet (1993 -) 1 Gbit/sec 16
- IB Trade Association 19
- • Network speed bottlenecks 21
- • I/O interface bottlenecks 21
- (not shown) 24
- • Fast Messages (FM) 27
- • Myricom GM 27
- IB Hardware Acceleration 28
- • Interrupt Coalescing 29
- • Jumbo Frames 29
- • Hardware Checksum Engines 29
- TOE and iWARP Accelerators 30
- 2006 (v3.0), 2008 (v3.1) 34
- PCI-Express (PCIe) 34
- 2003 (Gen1), 2007 (Gen2) 34
- IB, HSE and their Convergence 36
- Traditional Ethernet 37
- IB Overview 38
- Components: Channel Adapters 39
- • Switches: intra-subnet 40
- – Not directly addressable 41
- IB Communication Model 43
- Queue Pair Model 44
- Memory Registration 45
- Memory Protection 46
- (Send/Receive Model) 47
- InfiniBand Device 48
- Hardware ACK 48
- Hardware Protocol Offload 51
- • Buffering and Flow Control 52
- • Switching and Multicast 52
- Buffering and Flow Control 53
- Virtual Lanes 54
- Service Levels and QoS 55
- Traffic Segregation Benefits 56
- Identifiers) 57
- Switch Complex 58
- – 3D Torus (Sandia Red Sky) 59
- More on Multipathing 60
- IB Multicast Example 61
- IB Transport Services 63
- Reliability 64
- Transport Layer Capabilities 65
- Data Segmentation 66
- Transaction Ordering 67
- Message-level Flow-Control 68
- Concepts in IB Management 71
- Subnet Manager 72
- HSE Overview 74
- Differences 75
- – Multi Stream Semantics 76
- Offloaded TCP 83
- Myrinet Express (MX) 85
- Datagram Bypass Layer (DBL) 86
- • Solarflare approach: 87
- Ethernet 90
- IB Network 90
- IB Verbs 90
- Application 90
- Hardware 90
- IB Transport 91
- IB iWARP/HSE RoE RoCE 92
- IB Hardware Products 94
- Tyan Thunder S2935 Board 95
- IB Hardware Products (contd.) 96
- – Nortel Networks 97
- • Support for VPI and RoCE 98
- – www.openfabrics.org 99
- – OFED 1.6 is underway 99
- Verbs Interface 100
- (libibverbs) 100
- • Within the hardware: 101
- OpenFabrics Software Stack 102
- InfiniBand in the Top500 103
- Cray Interconnect 104
- NUMAlink 104
- SP Switch 104
- CCGrid '11 106
- • Other small clusters 107
- • Integrated Systems 107
- Other HSE Installations 108
- Presentation Overview 109
- 1/10 GigE 110
- InfiniBand 110
- Case Studies 111
- Latency (us) 112
- Message Size (bytes) 112
- Bandwidth (MBps) 113
- MVAPICH/MVAPICH2 Software 115
- One-way Latency: MPI over IB 116
- Bandwidth: MPI over IB 117
- Bandwidth: MPI over iWARP 119
- Convergent Technologies: 121
- Traditional Model 123
- Possible SDP Model 123
- InfiniBand CA 123
- SDP vs. IPoIB (IB QDR) 124
- IB on the WAN 126
- Features 127
- • Supports SONET OC-192 or 128
- “appear” local 128
- Sunnyvale 129
- 700 miles 130
- 3300 miles 4300 miles 130
- Cluster B 131
- Communication Options in Grid 132
- Interface 133
- Modern WAN 133
- Performance of Memory Based 134
- Data Transfer 134
- ADTS-64MB 135
- IPoIB-64MB 135
- Application Level Performance 136
- Memcached Design Using Verbs 139
- Memcached Get Latency 140
- Memcached Get TPS 141
- Bandwidth with C version 142
- Hadoop Sort Benchmark 145
- Concluding Remarks 147
- Funding Acknowledgments 148
- Personnel Acknowledgments 149
- Web Pointers 150
Relacionado con productos y manuales para Redes SGI 10-Gigabit Ethernet Network Adapter
Redes SGI network adapter Guía de usuario
(34 paginas)
(34 paginas)
Redes SGI Ethernet Card Guía de usuario
(52 paginas)
(52 paginas)
Redes SGI Ethernet Card Guía de usuario
(36 paginas)
(36 paginas)
© 2020, manymanuals.es. Todos los derechos reservados | 0.098 s |
Manymanuals.com
Manymanuals.de
Manymanuals.fr
Manymanuals.it
Manymanuals.pl
Manymanuals.cz
Manymanuals.es
Manymanuals-pt.com
Comentarios a estos manuales