In March 2004 the Spanish government and IBM signed an agreement to build one of the the fastest computer in Europe. In July 2006 its capacity was increased due to be the large demand of scientific projects.
MareNostrum 2 was a supercomputer based on processors PowerPC, the architecture BladeCenter, a Linux system and a Myrinet interconnection. These four technologies configure the base of the architecture and design.
See below a summary of the system:
- Peak Performance of 94.21 TFLOPS
- 10240 IBM PowerPC 970MP processors at 2.3 GHz (2560 JS21 blades)
- 20 TB of main memory
- 390 + 90 TB of disk storage
- Interconnection networks:
- Myrinet and Gigabit Ethernet
- Linux: SuSe Distribution
MareNostrum 2 had 44 racks and took up a space of 120m2.
Myrinet racks
The 2560 blade nodes JS21 are interconnected through a high speed interconnection network called Myrinet. The different nodes are interconnected via fibre optic cables.
Four of the 44 racks in MareNostrum are dedicated to network elements which allow to interconnect the different nodes connected to the Myrinet network. These four racks are located in the center of the room and each node has a fibre optic cable. The network elements connect the different cables allowing the interconnection from one point to another from the different nodes.
The total Myrinet network is connected through:
- 10 elements Clos256+256
- 2 elements Spine 1280
Storage servers
Further to the local disk of each node with a 36GB capacity, MareNostrum has 20 storage servers arranged in 7 racks. These have a total of 560 disks of 512GB and each one provide a total capacity of 280 TB external storage. These disks are working with GPFS (Global Parallel File System) which offers a global vision of the file system and also allows a parallel access.
The 2560 nodes access the disks through the Gigabit network.
Each one of the 20 storage nodes has two nodes p615 in charge of the disk requests, a controller type FAStT100 and one unit EXP100. Each node p615 Gbit connections and Ethernet 10/100 connection.
Operations rack
One of the racks is the operation rack where the system can be managed. This rack is located in the machine console.
The content of each rack is the following one:
- 1 Monitor 7316-TF3
- 2 nodes p615
- 2 HMC (consoles) 7315-CR2
- 3 remote nodes asynchronous
- 1 Chasis BCIO BladeCenter IO
- 4 Switches Cisco 3550
Gigabit switches
One of the racks of MareNostrum is dedicated to the interconnection of the Gigabit network and one part of the interconnection elements of the Ethernet 10/100 network.
The elements are the following ones:
- 1 switch Force10 E600 Gigabit Ethernet. It contains 7 slots in total, 6 of them have cards 10/100/1000 Base-T of 48 ports offering a total of 288 ports GigabitEthernet/IEEE 802.3
- 4 Switches Cisco 3550 48-port Fast Ethernet