What are the advantages and disadvantages of Fiber Channel and iSCSI storage technologies? In a high storage environment of file systems such as video

What are the advantages and disadvantages of Fiber Channel and iSCSI storage technologies? In a high storage environment of file systems such as video and images, which technology is preferable

People are very loyal to both sides of this argument, so keep that in mind as people answer. Generally they've invested a lot in one or the other and have to justify that.

Here's my non-completely-unbiased opinion. I've worked primarily with FC installations, but the general advantage is lack of congestion. You have dedicated bandwidth just for your data and FC has low overhead, so more of the 4 Gb/s or 2 Gb/s or whatever your connections are rated, gets used for actual data.

Both systems essentially use the same underlying commands and transports to move data from one place to another, but with iSCSI you're moving the stuff across cables that may be used by other services and are probably only rated at 1Gb/s.

Having said that iSCSI has come a long way and there's probably a lot I don't know about it. And, of course, it's much cheaper to implement since you can use your current wiring infrastructure. Because of that it lends itself much more easily to physically large installations, where running FC cabling might be financially unrealistic.

My gut reaction is that if you're moving images around, you would probably be OK with iSCSI, but if you're dealing with HD video and you need to be able to ingest right to your storage, FC would be better for guaranteeing you the bandwidth and throughput you need not to drop frames.

Either can deliver an architecture for delivering video and images which will stress the MB/seconds limitations of your storage frames rather than the I/Os per second limitations of your frames.

Fibre Channel is the traditional method of delivering such storage to servers so it has the advantage that many people understand it and you can pull off-the-shelf designs from vendors or mix and match storage frames, SAN switches, and fibre-channel HBAs quite easily. The protocol delivers a network with low latency speeds with no possibility of collisions.

iSCSI is the newer solution. The challenge in implementing iSCSI is addressing the potential for congestion on an IP-based (and therefore collision-based) protocol. These limitations can be overcome but typical IP network teams don't manage to collision rates and latency. Therefore you may face an organizational challenge.

Nonetheless others have implemented iSCSI SANs so there is a growing body of practical knowledge available.

According to Wendy Betts who spoke at a SNIA IP Storage forum, iSCSI is less complex and less expensive. She implemented a very large iSCSI SAN to replace FC and has been quite happy with it.

More information is available at www.snia.org

FC and iSCSI are foundation technologies that enable raw block storage access. Also, if you want to build media file systems, it is better to consider a clustered NAS solution instead of block storage access with front-ending media servers.
If you are looking specifically for Rich media storage systems, you should look at services like Nirvanix that enable media delivery in the Web 2.0 world.
There are high performance Filesystem based storage systems that are specifically optimized for rich medial delivery if you want build your own infrastructure.

We have been using iSCSI for quite some time and in general things work ok and performance seems ok even though we haven't done a iSCIS vs. FC direct performance comparison ...yet.

However, there are still a few unresolved issues with the OpeniSCSI implemtation that is widely used. We found that MPIO doesn't work with all storage providers and if you have a very large number of LUN's like we have (into 1000's), the scanning time takes far too long. There are plans to fix it but it will not be until Q1 2008. With our stack, trunking doesn't work either yet which is a pain.

The other aspect which has to be monitored closely is the processor overhead to process iSCSI packets. Reports indicate that you can loose quite a bit of CPU cycles from doing that. The two possible solutions are TOE (many only do part offloading while others offload the full stack) or an iSCSI HBA. Both cost about the same as a QLogic or Emlulex FC card, so no savings there.

Fibre Channel
Pros
- It's been around for 10+Years
- High Performance
Cons
- More expensive than iSCSI
- More complex than iSCSI
- Requires Cards in your Servers, FC Cables and Switches - all which most companies don't have lying around.

iSCSI
Pros
- Based on TCP-IP - easy to understand, implement and manage.
- inexpensive to add more servers to SAN later.
Cons
- Most products can't achieve FC performance.

I would like to add that SOME iSCSI products on the market today can exceed the performance of Fibre Channel, the key is to find a solution that is ‘distributed’….think about what makes google.com so fast, it’s not one big storage system, it’s distributed…many small storage systems are faster than one big one.

The link I’m providing is a list of storage solutions that have been submitted to Microsoft for performance testing. The Lefthand Networks submission is iSCSI and beats almost every other fibre channel solution listed. The rating can be found in the subject of each link in number of users.

Good luck!