Tests of RAID adapters

[Francois Tigeot <ftigeot@xxxxxxxxxxxx>]

Hi all,

During last week, I've done some systematic hardware tests to determine which
recent RAID cards work fine with DragonFly.

I've bought an Areca ARC-1880-i adapter and thanks to a local
server-class hardware distributor, I've been able to test a grand total
of 11 different SAS or SATA RAID adapters.


The tests were done with a DragonFly 2.9/x86_64 snapshot from 2011-04-19.

Besides the various RAID adapters, the following hardware was used:

- 1x Supermicro X7SPA-H mainboard with Atom D510 CPU
- 1x 2GB DDR2 667 SODIMM
- 2x Samsung SATA 3Gb/s 160GB hard drives
- 1x SATA DVD-ROM Drive plugged on one of the motherboard's SATA ports

For the tests, all RAID volumes were configured as a mirror with the two
Samsung disks (RAID 1)


List of RAID adapters:

- Areca ARC-1200 (SATA only)
- Areca ARC-1210 (SATA only)
- Areca ARC-1222
- Areca ARC-1680-ix
- Areca ARC-1880-i
- LSI MegaRAID SAS 8204ELP
- LSI SAS 3081E-R
- LSI SAS 9211-8i
- 3Ware 9690SA-4i
- Adaptec AAR-1120SA
- Adaptec 3405


Areca ARC-1200
--------------

http://www.areca.com.tw/products/pciex1_2p_internal.htm
This is a SATA card only

The card was immediately recognized by the DragonFly snapshot. There were no
problems to install the system and to run the new installation from the RAID
volume.

dmesg extract:
  arcmsr0: <Areca SATA Host Adapter RAID controller>



Areca ARC-1210
--------------

http://www.areca.com.tw/products/pcie.htm
This is a SATA card only

This card was quite similar to the ARC-1200; as before, all went well.



Areca ARC-1222
--------------

http://www.areca.com.tw/products/pcietosas01.htm

The adapter was recognized out of the box:

  arcmsr0: Areca SAS 3G Host Adapter RAID contreoller (RAID6 capable) mem ...  irq16 at device 0.0
  ARECA RAID ADAPTER0: Driver Version 1.20.00.19 2010-11-11
  ARECA RAID ADAPTER0: FIRMWARE version v1.48 2010-07-08

A RAID volume not beeing fully initialized is not detected.
Once the initialization is complete or if "background initialization" is
selected in the adapter BIOS setup, the volume is correctly recognized:

  da0 at arcmsr0 bus0 target0 lun0
  da0: <Areca ARC-1212-VOL#000 R001> Fixed Direct Access SCSI-5 device
  da0: Serial Number 41811d2909406830
  da0: 166.666MB/s transfers (83.333 MHz, offset 32, 16bit)
  da0: Command queuing Enabled
  da0: 152587MB (312499200 512 byte sectors: 255H 63 S/t 19452C)

A linear read of the first 1.25 GB of the drive gave a speed of 127.24 MB/s,
twice that of the previous models, which lead me to believe the reading was
parallelized on both drives.



Areca ARC-1680-ix
----------------

http://www.areca.com.tw/products/pcietosas1680series.htm

As before, all went well.

The adapter recognized the volume created on the ARC-1222 and could boot
from it.



Areca ARC-1880-i
----------------

http://www.areca.com.tw/products/1880.htm

A complete installation and boot from the new system went fine.

dmesg extract:
  arcmsr0: <Areca SAS 6G Host Adapter RAID Controller (RAID6 capable)> port 0xc800-0xc8ff mem 0xfe940000-0xfe97ffff,0xfe930000-0xfe93ffff irq 11 at device 0.0 on pci1
  ARECA RAID ADAPTER0: Driver Version 1.20.00.19 2010-11-11 
  ARECA RAID ADAPTER0: FIRMWARE VERSION V1.49 2010-12-10 
  
  da0 at arcmsr0 bus 0 target 0 lun 0
  da0: <Areca ARC-1880-VOL#000 R001> Fixed Direct Access SCSI-5 device 
  da0: Serial Number 4187860270766765
  da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit)
  da0: Command Queueing Enabled
  da0: 152587MB (312499200 512 byte sectors: 255H 63S/T 19452C)




LSI MegaRAID SAS 8204ELP
------------------------

http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sas/megaraid_sas_8204elp/

Even though a RAID volume had been created in the BIOS of the card, it was
not visible from DragonFly
The individual disks were visible as da0 and da1.



LSI SAS 3081E-R
---------------

http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/

The RAID1 volume created in the BIOS of the card was visible but there were
some timeout error messages from the start:

dmesg extract:
  mpt0: <LSI Logic SAS/SATA Adapter>
  mpt0: MPI Version = 1.5.20.0
  mpt0: Capabilities: (RAID-0 RAID-1E RAID-1)
  mpt0: 1 Active Volume (2 Max)
  mpt0: 2 Hidden Drive Members (14 Max)
  ...
  mpt0: mpt_wait_req(4) timed out
  mpt0: read_cfg_header timed out
  mpt0: vol0(mpt0:0:0) mpt_refresh_raid_vol: Failed to read RAID vol Hdr(0)
  mpt0: vol0(mpt0:0:0): Settings()
  mpt0: vol0(mpt0:0:0): 0 Members:
  mpt0: vol0(mpt0:0:0): RAID0-Optimal
  (mpt0:0:1): Physical(mpt0:0:1:0),Pass-thru (mpt0:1:0:0)
  (mpt0:0:1): Online
  (mpt0:0:2): Physical(mpt0:0:2:0),Pass-thru (mpt0:1:1:0)
  (mpt0:0:2): Online
  ...
  da0 at mpt0 bus 0 target 0 lun 0
  da0: <LSILOGIC Logical Volume 3000> Fixed DirectAccess SCSI-2 device
  da0: 300.000MB/s transfers
  da0: 151634MB (310546432 512 byte sectors: 255H 63S/T 19330C)

I couldn't install DragonFly, newfs_hammer hung at 90% completion. The disks
were making strange noises, as if they were constantly seeking



LSI SAS 9211-8i
---------------

http://www.lsi.com/channel/products/hba/sas_sata_hbas/internal/lsisas92118i/

The card was not recognized at all.

It seems the mps(4) driver needs to be updated to support this model.
LSI is known to be developing a new version of the driver, which should be
committed to the FreeBSD tree soon.



3Ware 9690SA-4i
---------------

http://www.lsi.com/channel/products/raid_controllers/sata_sas/3ware_9690sa4i/

3Ware has been recently bought by LSI, but the 3Ware branded cards are still
different from the LSI ones and use different drivers.


The card was recognized instantly and DragonFly could be installed without
any problem

dmesg extract:
  3ware device driver for 9000 series storage controllers, version: 3.80.06.003
  twa0: <3ware 9000 series Storage Controller>
  twa0: INFO: (0x15: 0x1300): Controller details: Model 9690SA-4I, 128 ports,
  Firmware FH9X 4.08.00.006, BIOS BE9X 4.08.00.001
  ...
  da0: <AMCC 9690SA-4I DISK 4.08> Fixed Direct Access SCSI-5 device
  da0: 100 MB/s transfers
  da0: 152577MB (312477696 512 byte sectors: 255H 63S/T 19450C)

A little while after boot, some informational messages appeared on console:

  twa0: WARNING: (0x04: 0x002F): verify not started; unit never initialized: unit=0
  twa0: INFO: (0x04: 0x000C): Initialize started: unit=0




Adaptec AAR-1120SA
------------------

http://www.adaptec.com/en-us/products/controllers/hardware/sata/entry/aar-1220sa/
This is a SATA card

During startup, I could not enter directly the BIOS setup. After pressing
<CTRL>+<A>, I got an error message saying "not enough free memory to load
the utility!"
At the end of BIOS initialization, the setup started correctly nevertheless

Neither the card nor a disk volume was detected by DragonFly. This model may
not have a driver.



Adaptec 3405
------------

http://www.adaptec.com/en-us/products/controllers/hardware/sas/value/sas-3405/

During startup, I pressed <CTRL>+<A> to enter its BIOS setup but at the end
of the boot sequence all I got was an error message saying to hit Enter to
force the Config Utility to load.
It didn't load, and the machine froze. The controller was constantly beeping
and I had to do a hard reset to get the mainboard to start normally again.

I then plugged this card into a Supermicro X7SBL-LN2 mainboard. This time, I
could enter the BIOS setup and create a RAID1 volume

The controller was recognized by the aac(4) driver but was unfortunately
unusable:

  aacd0: <RAID 1 (Mirror)> on aac0
  aacd0: 152490MB (312299520 sectors)
  disk scheduler: set policy of aacd0 to noop
  CAM: Configuring 8 busses
  intr18 at 40001/40000 hz, livelocked limit engaged!
  **WARNING** waiting for the following device to finish configuring:
    wpt:  func=0xffffffff80277157 arg=0
  
  [last two lignes repeated 4 times]
  
  Giving up, interrupt routing is probably hosed
  no B_DEVMAGIC (bootdev=0)
  aac0: COMMAND 0xffffffe035d902a0 (TYPE 601) TIMEOUT AFTER 137 SECONDS
  
  [three almost identical lines with slight command# and type# variations]

At this point, the machine was deeply frozen and I had to do a hard reset
to get it to reboot.

It seems I'm not the only one to have trouble with Adaptec and Supermicro
mainboards:
http://www.webhostingtalk.com/archive/index.php/t-926239.html



Conclusions
-----------


I'm very impressed by Areca. All their cards worked flawlessly out of the box.
The Areca people were very cooperative, they tested and reviewed the initial
port of the arcmsr driver (from FreeBSD).
Many of their adapters include an ethernet port which can be used for
supervision, removing the need to install a special low-level utility in the
host OS.

I only had access to one 3Ware adapter, but it also worked out of the box and
I have reports of different recent models also working flawlessly.

There are some LSI models known to work flawlessly. Unfortunately, they seem
to be old products which are not sold anymore.

With all the troubles I had getting Adaptec cards to work reliably before
DragonFly was even booted, I cannot seriously consider to purchase products
of this brand.


If you want a RAID adapter to use with DragonFly, Areca and 3Ware are the two
best choices of the moment, my first choice beeing Areca.

-- 
Francois Tigeot