These posts assume a brand-new deployment and attempt to cover all of the bits required to implement the solution. I'm including everything from the core layer's SFP+ downlink modules through the SFP+ ports for server access. I'm assuming that the core consists of SFP+ interfaces capable of supporting twinax cables, and that each server will provide it's own twinax cable of the appropriate length.
Each scenario supports racks housing sixteen 2U servers. Each server has a pair of SFP+ based 10Gb/s NICs and a single 100Mb/s iLO interface.
Option 1 - Top of Rack Nexus 5596
This is a 3-rack pod consisting of two Nexus 5596 switches and three Catalyst 2960s for iLO/PDU/environmental/etc...
48 servers have 960Gb/s of access bandwidth with no oversubscription inside the pod. The pod's uplink is oversubscribed by 6:1.
The 5596 management connections (Ethernet and serial) will require the installation of some structured wiring (copper) so that they can reach their management network and terminal server.
The 2960 console connections are less critical. My rule on this is: If the clients I'm supporting can't be bothered to provision more than a single NIC, then they can't be all that important. Redundancy and supportability considerations at the network layer may be compromised.
Here's the resulting bill of matierals with pricing:
|Config Set Lines|
|Line||Item / Part#/Description||List Price||Qty.||Discount(s)||Unit Price||Extended Price|
|1.0||N5K-C5596UP-FA Nexus 5596UP 2RU Chassis, 2PS, 4 Fans, 48 Fixed 10GE Ports||36,800.00||2||0%||36,800.00||73,600.00|
|1.1||CAB-C13-CBN Cabinet Jumper Power Cord, 250 VAC 10A, C14-C13 Connectors||0.00||4||0%||0.00||0.00|
|1.2||GLC-T 1000BASE-T SFP||395.00||10||0%||395.00||3,950.00|
|1.3||SFP-10G-SR 10GBASE-SR SFP Module||1,495.00||32||0%||1,495.00||47,840.00|
|1.4||SFP-H10GB-CU3M 10GBASE-CU SFP+ Cable 3 Meter||210.00||10||0%||210.00||2,100.00|
|1.5||N55-M16UP Nexus 5500 Unified Mod 16p 10GE Eth/FCoE OR 16p 8/4/2/1G FC||11,200.00||2||0%||11,200.00||22,400.00|
|1.6||N55-M16UP Nexus 5500 Unified Mod 16p 10GE Eth/FCoE OR 16p 8/4/2/1G FC||11,200.00||2||0%||11,200.00||22,400.00|
|1.7||N5KUK9-503N2.1 Nexus 5000 Base OS Software Rel 5.0(3)N2(1)||0.00||2||0%||0.00||0.00|
|1.8||DCNM-L-NXACCK9 DCNM for LAN Advanced Edition for N1/2/4/5K||0.00||2||0%||0.00||0.00|
|1.9||N55-M-BLNK Nexus 5500 Module Blank Cover||0.00||2||0%||0.00||0.00|
|1.10||N55-PAC-1100W Nexus 5500 PS, 1100W, Front to Back Airflow||0.00||4||0%||0.00||0.00|
|1.11||N5596-ACC-KIT Nexus 5596 Chassis Accessory Kit||0.00||2||0%||0.00||0.00|
|1.12||N5596UP-FAN Nexus 5596UP Fan Module||0.00||8||0%||0.00||0.00|
|2.0||WS-C2960-24TC-S Catalyst 2960 24 10/100 + 2 T/SFP LAN Lite Image||725.00||3||0%||725.00||2,175.00|
|2.1||CAB-C13-C14-AC Power cord, C13 to C14 (recessed receptacle), 10A||0.00||3||0%||0.00||0.00|
List price for this layout is $174,465, or $58,155 for each 16-server cabinet.
There is sufficient capacity on the Nexus 5596 (only 69 ports are in use) to hang a fourth cabinet from it for an additional $23,915. The price includes an expansion module for each 5596, a 2960 and a couple of GLC-T modules. That brings the per-cabinet cost down to $49,595 but raises the oversubscription ratio to 8:1.