How to scan the SCSI bus with a 2.6 kernel

If you are playing with SCSI devices (like Fibre Channel, SAS, ..) you sometimes need to rescan the SCSI bus to add devices or to tell the kernel a device is gone. Well, this is the way to do it in CentOS with versions that have a 2.6 kernel. This means CentOS 5 and CentOS 4 (starting from update 3).

1. Find what's the host number for the HBA:
   

   ls /sys/class/fc_host/

   (You'll have something like host1 or host2, I'll refer to them as host$NUMBER from now on)
   
   
2. Ask the HBA to issue a LIP signal to rescan the FC bus:
   

   echo 1 >/sys/class/fc_host/host$NUMBER/issue_lip

3. Wait around 15 seconds for the LIP command to have effect
   
   
4. Ask Linux to rescan the SCSI devices on that HBA:
   

   echo - - - >/sys/class/scsi_host/host$NUMBER/scan

   The wildcards "- - -" mean to look at every channel, every target, every lun.
   

That's it. You can look for log messages at "dmesg" to see if it's working, and you can look at /proc/scsi/scsi to see if the devices are there. In CentOS 5 there is also the "lsscsi" command that will show you the know devices plus the device entries that point to those devices (very usefull).

For more information about how this works see the the upstream release notes for 4.3.

Anaconda, Kickstart & iSCSI

The previous days I've been trying to get Anaconda to install a Xen domU with it's harddisk being a iSCSI target. And this all automated using kickstart. It is working now, but Anaconda needed a bit of help.

First a quick overview of the seutp. I have set of servers that will host all the virtual machines. The storage comes from 2 servers that have 12 disks in a RAID array (see some of the other articles on this blog). I use LVM to carve up this storage and export each logical volume as a iSCSI target for the different virtual machines.

Since I would like to be able to use the Xen migration feature to move virtual machines between the servers, I basically have 3 options to have a shared storage so the migration works. The first one is to use a cluster/distributed filesystem that is present on all the servers that contain image files of the different virtual machines. The second one is to use CLVM that makes logical volumes available to a set of machines. And the third one is to let the virtual machine import the disk directly. It's a bit to much to discuss the merit of the 3 choices here. Maybe in another post.
Anyway, the choice was made to use the last option. Our virtual machine would act itself as a iSCSI initiator and get a disk to install on that way.

If you read the CentOS 5 docs about kickstart (http://www.centos.org/docs/5/html/5.1/Installation_Guide/s1-kickstart2-options.html) you can see that there are options for iSCSI. So this was supposed to be easy. A first test indicates that it all worked, so that looked good. Now we had some requirements. To prevent the different virtual machines from touching the wrong iSCSI targets we using a username and password on each target.

The documentation indicates that Anaconda should support username and password during the import of the iSCSI targets. But testing indicates that this does not work. Then I looked at the code and see nothing that indicates that it should work.

A second issue that we are seeing is that we have quickly have a lot of targets on our storage server and Anaconda tries to do a login to all targets that it discovers. This slows down the installation process a lot.

So it was time to refresh my Python skills and do some Anaconda hacking. BTW, a good reference for this is the Anaconda wiki page. Using the updates support you can easily use modified version of the anaconda code during a installation. And after some testing but problems are now solved. Anaconda will now use the --user, --password and --target options if they are present.

I've created 2 entries in the upstream bugzilla about this with the patches containing my 402431 and 418781. I have also attached a updates.img to bug 418781. This can be used to start a install with the patches I've made but without the need to recreate all the installation images. See the Anaconda wiki or the docs that come with Anaconda on how to use them (it depends on how you actual do the install).

CentOS 5.1 Released

For those of you that have not been behind there computer this weekend. CentOS 5.1 for the i386 and x86_64 platforms has been released into the wild. As part of the QA team I can say that it looks like a solid release (except the autofs issue, but that is already fixed with a new kernel in the updates repo). Especially Xen looks like it is a lot more stable then the 5.0 version was.

Anyway you can find the official announcement here : http://lists.centos.org/pipermail/centos-announce/2007-December/014476.html.
And the release notes (considered essential reading) are here : http://wiki.centos.org/Manuals/ReleaseNotes/CentOS5.1/.

Enjoy 5.1 and thanks again to all the people involved to make it happen.