VSAN Overview – Cisco Describing VSANs and Fibre Channel Zoning

VSAN Overview

It’s very common to use the analogy of Ethernet’s VLANs when explaining FC VSANs. Certainly, there are a lot of similarities between the two, but do not forget that these are features of two very different communication protocols, so there are some fundamental differences.

So, what is a VSAN, and why do you need it in the data center?

To answer this question, you have to look back at the beginning of the SANs, before Cisco introduced this feature in the FC Protocol (later adopted to some degree by Brocade as well). Traditionally, as there was the need to keep the communication within a SAN protected, under control, and reliable, the approach used was to build multiple separate physical SANs, each of them dedicated to the specific storage communication between a set of servers, running an application or a set of common applications, and the storage system or systems used only by these servers. For example, let’s assume that the servers used by the HR department are running the HR applications and databases. They will use a dedicated set of storage systems only for their data. The HR servers and the HR storage systems will be connected to each other with a physical dedicated SAN, built by Fibre Channel switches. This means that the SAN will be able to communicate only with the specified servers and storage systems, for the simple reason that only they are physically connected to the SAN. Traditionally, this is how we achieved the fabric isolation and separation between the needs of different groups, workloads, and so on. Just imagine how you would need multiple, separate, dedicated, physical SANs, servicing the needs of other departments, groups, and applications in an organization. Usually, an organization has multiple different departments and types of applications, with different functionality. Therefore, there was a lot of Fibre Channel equipment, installed and connected in separate SANs, to meet the requirements for separation and isolation between the fabrics. That was expensive. Extremely expensive! Another issue was that these Fibre Channel infrastructures had to be monitored and maintained. These operations were complicated and difficult. This was also known as a siloed approach (that is, creating multiple SAN silos), as shown in Figure 12-1

  

Figure 12-1 Traditional Siloed Approach with Separate SANs

The solution came from Cisco: the idea and functionality implemented in their version of the Fibre Channel Protocol was that the physical SAN could be virtualized, following the analogy with VLANs that the communication in an infrastructure can be separated and isolated in different groups. However, because this is the Fibre Channel Protocol, there was the need for this separation to be stricter compared to that with VLANs. With the VLANs, based on the VLAN ID the traffic is tagged with, the communication can happen only where this tag is allowed. However, at the same time, it’s very easy to allow communication between devices that communicate in different VLANs. This behavior is not acceptable in Fibre Channel communication. That’s why the VSAN isolation is implemented in the hardware of the Cisco MDS switches, at the level of the physical Fibre Channel ports. When a Fibre Channel frame enters the Cisco MDS switch through an F_Port, it is tagged with the appropriate VSAN tag in the hardware. Then it can be communicated only through ports, which belong to the same VSAN. Basically, the VSANs do exactly as their name states—they divide the physical Fibre Channel switched fabric, composed of one or more switches, into multiple SANs, but they are virtual. The initiators and the targets can communicate only with other end nodes that belong to the same VSAN. It is not allowed for devices from different VSANs to communicate to each other. Actually, there is a functionality called Inter-VSAN Routing (IVR) that can allow such a communication, but it needs to be specifically configured, and that configuration is complex. In other words, you cannot allow the communication between devices from different VSANs by accident. The traffic isolation implemented with the usage of the VSANs is a huge advantage, as it allows fewer Fibre Channel switches to be used and at the same time have the flexibility and scalability needed by the organization. Figure 12-2 shows the benefits of a VSAN through traffic isolation.

  

Figure 12-2 Virtual SANs

Each VSAN has an ID. The ID is a number in the range 1 to 4094. There are some VSAN IDs that are reserved. VSAN 1 is the default VSAN, and it can be used. It just exists by default and cannot be deleted.

There is the special VSAN 4094, which is used by the system to isolate orphaned FC ports. Orphaned FC ports on the switch are those that belonged to a VSAN, but the VSAN was deleted. These ports cannot automatically move to VSAN 1, as there might be a configuration or end devices connected to them that would cause miscommunication or loss of data. That’s why such ports are automatically put in VSAN 4094 and are isolated, which means they can’t communicate with any other ports. This leaves the range of user-assigned VSAN IDs from 2 to 4093. However, there is a little bit more to this. Table 12-1 shows which VSAN IDs can be assigned by the administrators and which are reserved.

  

Table 12-1 VSAN IDs Allocation

VSAN IDs

System Reserved

Usage

1

Yes

Default VSAN for all ports

2–4078 and 4080–4093

No

User-configurable

4079

Yes

Exchange Virtual Fabrics Protocol (EVFP)

4094

Yes

Isolated VSAN for orphaned ports

Here are some other attributes of a VSAN:

  • Name: The name must be unique for each VSAN. A string of up to 32 characters can be used for management purposes. If a name is not configured, the system takes the string VSAN and adds the number as a four-digit string. For example, the system will provide a VSAN with an ID of 11 with the name VSAN0011.
  • State: The state can be active or suspended. Active state means that the VSAN is configured and enabled, which also enables the Fibre Channel Protocol services for the VSAN. The suspended state specifies that the VSAN is created but it’s not enabled.
  • Operational state: This can be up or down.
  • Load-balancing scheme: This is the load-balancing scheme used for path selection. It can be based on the source-destination FCID (src-dst-id) or the default one, which is based on using the combination of source and destination FCIDs together with the originator exchange ID (OX ID), presented as src-id/dst-id/oxid. It means that the communication between a specific source and destination for a specific exchange (or conversation) will take the same path.

The VSAN’s goal is to mimic a SAN, and because of that most FCP services run per VSAN. There are separate per-VSAN instances running for some of the FCP services, and other services will use separate databases for each VSAN. No matter the approach, the FCP services must be configured and managed per VSAN. These services are the zone server, principal switch selection and domain ID distribution, management server, and name server.

This being the case, the SAN is not limited only to the theoretical maximum of 239 domain IDs, as each VSAN is a separate virtual SAN, and there is now a theoretical maximum of 239 domain IDs per VSAN. Each switch will have a unique domain ID in each VSAN. This domain ID is unique within the VSAN among the other switches that also communicate in the same VSAN. The principal switch selection service will select one of the switches for a VSAN to become the principal as well as control the domain ID distribution within the VSAN. So, the domain IDs are now limited to 239 per VSAN, and within the VSAN, each of the switches must have a unique domain ID. Because multiple VSANs can be created on a switch, this means there will be a separate domain ID for each of the VSANs. Can the domain ID of a specific switch have the same value for all the VSANs that exist on the switch? It can, if this is configured manually, but this doesn’t mean that this domain ID is the same! The domain IDs, even using the same value, are separate and independent between the VSANs. The other important point is that a switch can be elected to be the principal switch for one VSAN but not for another. It is not mandatory that the same switch be the principal for all the VSANs that exist on it. It can be the principal switch for one VSAN and not the principal switch for another.

The E_Ports are the Fibre Channel ports that connect the SAN switches to each other. They expand the switched fabric. The VSANs have a new E_Port mode called the TE_Port. When a port is in E_Port mode, it will allow communication only for the traffic of the VSAN to which it belongs. Also, this means that the other E_Port to which it is connected must also belong to the same VSAN. Otherwise, there will not be communication, and even though the two ports come up, they will stay isolated. If there is a need to carry traffic for multiple VSANs across the E-to-E_Port link, VSAN trunking must be enabled, and the ports will be in TE_Port mode, which stands for trunking E_Ports. The configuration must be the same on both sides of the TE_Ports, and they have to allow communication for the same VSANs. Figure 12-3 illustrates VSAN trunking.

  

Figure 12-3 VSANs Trunking

The same applies if the F_Port of the switch is connected to an NPV switch, and there is the need to carry the communication for multiple VSANs. The F_Port will become a TF_Port (trunking), and the NP_Port on the NPV switch will become TNP_Port (also trunking). The same rules apply in this situation as well; the same VSANs need to be allowed on both sides.

In summary, the VSANs have the following qualities:

  • They are equal to SANs, with their own routing, zoning, naming, and addressing.
  • They limit the unicast, multicast, and broadcast traffic.
  • They are transparent for the initiators and targets, as the membership is defined on the F_Ports.
  • An end node can belong only to a single VSAN.
  • The membership is enforced on every F_Port and E_Port through which the frames travel.