Thursday, January 27, 2011

East - West adjacency: Why do I do this?

Every now and then I deploy a topology that includes the following requirements:
  • L3 access layer switches
  • VLANs extended across two switches

The VLAN extension requirement usually spawns from a server NIC redundancy (teaming), or from the desire to have pooled systems on the same subnet, but with diverse upstream network hardware.

The deployment winds up looking something like this:


There's a full mesh of OSPF point-to-point links in the topology including an East-West access layer adjacency on VLAN 10, which rides the same L2 aggregate link as the access VLANs.  Yesterday, I said "don't use SVIs for routing adjacency."  I confess:  I've done it myself.  Lots.

But now I'm starting to wonder why I've done it.

If all VLANs are shared between the two access switches, then what benefit do I get from the adjacency on VLAN 10?  I'm starting to think the answer might be "nothing."  Maybe I should eliminate it.

We've already got Equal Cost Multi-Path to everything else in the network, and any traffic between access VLANs will always be locally routed (and then bridged across Po10 when necessary).

Is there a case to be made for keeping this link, or should I kill it off?

8 comments:

  1. The first question to ask yourself: do you want to run transit OSPF on an access VLAN?

    If you already do that (and looking at the partial config it seems that you do), there's hardly any reason for the third VLAN, it just increases your OSPF adjacency count and the complexity of your OSPF database.

    ReplyDelete
  2. Hi Ivan,

    There's no OSPF adjacency on access VLANs.

    All interfaces are passive except for the green ones.

    ReplyDelete
  3. I tend to do exactly the same kind of configurations: OSPF session is formed on a dedicated VLAN, other SVIs are configured with passive-interface. That way I keep the infrastructure traffic separated from the end-user VLANs.

    ReplyDelete
  4. Markku,

    Yep, keeping access LANs segregated from transit networks is probably a good idea, but it isn't the heart of what I'm getting at here.

    I created a knee-jerk full-mesh of OSPF adjacencies between these four switches, but I can't really defend the design, and I might do it differently next time.

    The VLAN 10 adjacency doesn't seem to be adding anything to the environment other than extra complexity.

    My pal Tom pointed out something might be the only "advantage" introduced by this extra adjacency: Traffic routed by Access A, and which is destined for Access B's loopback interface will take the direct east/west path, rather than bouncing off a distribution switch.

    Meh.

    ReplyDelete
  5. You are right. I rarely have topologies where I can ECMP (it is a verb, yes) from layer to layer so I usually have the east-west link to enforce active-standby routing from layer to layer (rectangular topology, one vertical side is active, other side has higher OSPF costs).

    ReplyDelete
  6. Hi,
    OSPF between Access switches may be not important for User VLANs, and actually I use passive on User interfaces too. However a dedicated Vlan with L3 can be useful in case if you loose both your uplinks from the edge switch to the distribution switches (e.g. faulty card or fiber, etc) and you would like to maintain L3 communication to this Access switch from management and monitoring perspective.

    Regards,

    Seba

    ReplyDelete
  7. Hi Seba,
    You're getting to levels of detail I hadn't included in the drawing.

    Assume that each Access switch is a chassis-based thing with dual supervisors.

    The uplinks to the distribution layer consist of the first ten gig port on each supervisor, and they take diverse paths in the structured cabling.

    Po10 consists of the second ten gig port on each supervisor, and consist of 1m patch cords.

    I think the diversity angle is pretty well covered.

    It's looking more and more like I'll drop the vlan 10 adjacency in future deployments.

    Thank you for sharing your thoughts on the matter.

    ReplyDelete
  8. Chris,

    I agree. Based on the level of hardware redundancy you have, and as long as all of the user VLANs are L2 trunked between the two L3 access switches, there seems to be no benefit to the direct OSPF connection on VL10. I can't see where it would ever really be used with a single failure mode. Now if you were to add VL70 to access switch A and VL80 to access switch B and (by design) don't trunk them between the two switches, that changes things.

    ReplyDelete