Mercury gets a HBA upgrade

Dale Ghent | August 3, 2008

mercury.elemental.org is the server which hosts my $HOME and this website. It’s my Solaris 10 play-box, and I guess you can say that maintaining it is something of a hobby.

Its hardware is a quad core Xeon-equipped Dell PowerEdge 860, a small 1u server. Its pair of internal drives are Seagate SATA2, and were connected to the on-board Intel ICH7-based SATA controller. But there was something fishy about this in that the Solaris ahci SATA driver never attached to it and instead the drives ran in IDE mode. Despite my best efforts, I couldn’t change this. I eventually found out the reason – Dell crippled the SATA controller in the system BIOS to allow only IDE mode!

So this server was sold with “SATA drives”, which would imply a fully functioning SATA controller to drive them… but not quite. IDE mode means there were no benefits of SATA NCA and other niceties.

To fix this, I got a LSI SAS3041E-R controller – a 4x PCIe card that uses the LSISAS1064E chipset and offers 4 SATA ports. In Solaris land, this card would be driven by the mpt driver, a proven driver as the LSI SAS 1064 and 1068 chipsets are used to drive the on-board hard drives in pretty much every current Sun x86 and Niagara-based SPARC systems.

I installed this card in the single 8x PCIe slot in the PE860, and ran a 24″ SATA cable from it to HDD1, and used the existing Dell cable that connected the on-board controller to HDD1 to connect HDD0 to the card. After some fiddling in /boot/solaris/bootenv.rc to tell the kernel the new device path to its boot drive, the mpt driver attached and I was good to go.

I kicked off a SVM mirror resync as a basic test of sequential IO, and I hit 75MB/s reading from one drive and writing to the other. Not bad. A zpool scrub of my mirrored ZFS pool of 66.5GB of data (pool is 444GB in size) took just over an hour.

So if you’re thinking about a 4 or 8 port SAS/SATA card, consider the LSI SAS3041 or SAS3080/3081 cards, respectively. Both come in PCI-X and PCIe flavors and are supported by Solaris (and OpenSolaris) just fine.

/usr/X11/bin/scanpci output:
pci bus 0x0001 cardnum 0x00 function 0x00: vendor 0x1000 device 0x0056
LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS

Kernel boot messages:
scsi: /pci@0,0/pci8086,2779@1/pci1000,3090@0 (mpt0): Rev. 8 LSI, Inc. 1064E found.
scsi: /pci@0,0/pci8086,2779@1/pci1000,3090@0 (mpt0): mpt0 supports power management.
pcplusmp: pciex1000,56 (mpt) instance 0 vector 0x38 ioapic 0xff intin 0xff is bound to cpu2
scsi: /pci@0,0/pci8086,2779@1/pci1000,3090@0 (mpt0): mpt0 Firmware version v1.17.2.0 (IR)
scsi: /pci@0,0/pci8086,2779@1/pci1000,3090@0 (mpt0): mpt0: IOC Operational.
scsi: /pci@0,0/pci8086,2779@1/pci1000,3090@0 (mpt0): mpt0: Initiator WWNs: 0x500605b0000fa840-0x500605b0000fa843
pcie_pci: PCIE-device: pci1000,3090@0, mpt0
genunix: mpt0 is /pci@0,0/pci8086,2779@1/pci1000,3090@0
scsi: sd4 at mpt0: target 4 lun 0
genunix: sd4 is /pci@0,0/pci8086,2779@1/pci1000,3090@0/sd@4,0
genunix: /pci@0,0/pci8086,2779@1/pci1000,3090@0/sd@4,0 (sd4) online
scsi: sd3 at mpt0: target 5 lun 0
genunix: sd3 is /pci@0,0/pci8086,2779@1/pci1000,3090@0/sd@5,0
genunix: /pci@0,0/pci8086,2779@1/pci1000,3090@0/sd@5,0 (sd3) online

OpenSolaris 2008.11 – A Preview For The Storage Admin

Dale Ghent | July 16, 2008

Many reviews have been written about OpenSolaris since its release, but all of them barely tread beyond the desktop aspect, with the obligatory screenshots of the GNOME environment and a high-level description of only the major features most are already familiar with, or at least have heard of.

I’d like to take a different approach with this review, one that descends below the GUI to highlight aspects that server administrators in particular would be more interested in.
Read the rest of this entry »

Making Solaris HFS-aware

Dale Ghent | March 3, 2008

I’ve started a project of my own to port the HFS/HFS+ filesystem driver from Apple’s XNU kernel to OpenSolaris/Nevada.

Hopefully this will work well enough to allow Solaris users to read and write to HFS or HFS+ formatted disks and disk images. This includes iPods that were initialized on a Mac. Please check out the page I made for it and lend a hand if you’re interested!

Technorati Tags: , , , , ,

Comments
Comments Off
Categories
Geekery, Solaris
Tags
, , , , ,
Comments rss Comments rss
Trackback Trackback

Server upgrade time – elemental.org gets modern

Dale Ghent | November 17, 2007

After almost 8 years of running elemental.org mail, mailing lists, shell accounts, many websites (such as this one), database servers and essentially being a one-server ISP, the Sun Ultra 2 which ran all those things as lithium.elemental.org was retired and replaced this past weekend with a new server. Say hello to mercury.elemental.org.

Mercury is a Dell PowerEdge 860 with a Intel Xeon X3220 (quad core, 2.4Ghz) and 4GB 8GB of 667Mhz DDR2 RAM. Unlike lithium, mercury’s storage is entirely internal in the form of two mirrored 500GB SATA drives. This is to keep the entire package in 1 rack unit of space to keep colocation costs down.

What really excites me about this new server is that it is running Solaris 10 8/07 (lithium was running a very patched Solaris 8 FCS!). Solaris installed without a hitch and the 860’s onboard BCM5721 NICs are recognized by the bge driver, as are its IPMI baseboard controller by the bmc driver. The chipset on this system is the Intel ICH7 and unfortunately the Solaris ahci driver supports only the ICH6 at the moment, so the drives are running just fine in IDE compatibility mode.

This upgrade wasn’t just a mere update of hardware and OS. I also completely changed how the mail storage works and also make use of ZFS file systems for each user home directory and virtual web site:

  1. Out with uw-imap, in with Cyrus. All mail is delivered to Cyrus, so there are no more maildir-style spools sitting in each person’s home directory.
  2. To take advantage of Cyrus’s features, elemental.org is now operating its own Kerberos realm, ELEMENTAL.ORG. This is my first time running my own Keberos KDC, and I love it. Cyrus and Sendmail, via SASL, now offer GSSAPI authentication. Using Solaris’s pam_krb5_migrate.so.1 PAM module, as people log in with their UNIX passwords, a Kerberos principle is made for them and they are granted tickets. Pine is configured to connect to Cyrus and authenticate with GSSAPI, so shell users don’t have to type in or save their password when accessing their email!
  3. As I mentioned, all user data is now stored on a mirrored ZFS pool. Each user and virtual website gets their own ZFS file system and this will allow me to keep tabs on disk usage (and easily delete a user or site if the need should arise.) The zpool’s net size is 442GB.
  4. All incoming email is goes through greylist, ClamAV, and finally SpamAssassin milters.
  5. I’m more at ease and familiar with Solaris’s SMF facility now, having made a point to write SMF manifests for the services I’m running rather than plain old init scripts.

In addition, I’m now monitoring several aspects and services on the new system using Cacti.

Here’s to another 8 years of hopefully trouble-free operation!

Technorati Tags: , , ,

Comments
Comments Off
Categories
Geekery, Solaris
Tags
, ,
Comments rss Comments rss
Trackback Trackback

Solaris 10 8/07 “What’s New” docs are up

Dale Ghent | August 29, 2007

The first official list of new stuff added in Solaris 10 via its 4th update is available on docs.sun.com now:

http://docs.sun.com/app/docs/doc/817-0547/getjd?l=en&a=view

Technorati Tags: ,

Comments
Comments Off
Categories
Solaris
Comments rss Comments rss
Trackback Trackback

My presentation at the 2007 AFS & Kerberos Workshop

Dale Ghent | May 13, 2007

This past week, the 2007 AFS & Kerberos Workshop went on at the Stanford Linear Accelerator Center (SLAC) near Palo Alto, CA. Many people from an array of places eductational, government, and commercial came and presented papers and discourse on a wide range of topics involving AFS and Kerberos.

I was lucky enough to present a slide show on how we at UMBC have been combining OpenAFS with new ZFS and Zones features of Solaris 10 to obtain a more resilient AFS server infrastructure. You can view a PDF of my presentation here.

Technorati Tags: , , , ,

Comments
Comments Off
Categories
Geekery, Solaris
Comments rss Comments rss
Trackback Trackback

Funny Sun bug fix of the day

Dale Ghent | May 1, 2007

So today I was reading over the release notes for patch 126400-1, which is the latest OpenBoot PROM and SC update for the T1000/T2000, and came across an interesting bug ID listed under the Problem Description section:


6510364 “War Mode” in ALOM-CMT is required by the US NAVY which is currently missing

Awesome. I love knowing that my T1000s have a “war mode” now. Perhaps Sun should call it the SkyNet T1000 ;) Hopefully it’s the feature and not the US Navy that was “currently missing.”

Technorati Tags: , , , , , ,

More Linux/Solaris FUD wars

Dale Ghent | April 13, 2007

It’s all too often that I read posts such as this one and can’t help but to think that the writer is a tad on the myopic side of things… so much so that after a paragraph or two it becomes apparent that the writer hasn’t actually used Solaris in its current incarnation. And I don’t mean “used it” as in “I installed it and played around with it for a few hours/days, didn’t like the default GNOME theme, and promptly replaced it with Debian Etch” or some such. I mean “used it” as in implementing it in a real world production environment with an attempt to treat its features as the tools they are instead of toys.

In particular, I take issue with this comment from Mr. Zaitcev:

“This is the problem OpenSolaris is facing today in the nutshell: it has no breadth. It has a very limited number of excellent technologies, such as ZFS.”

No breadth? Care to, well, add some breadth to that statement, Mr. Zaitcev? Making a comment like that doesn’t mean you can toss out just one perceived example and end your argument at that.

It would appear that all of Mr. Zaitcev’s experience with Solaris/OpenSolaris comes from reading 3rd party accounts of the big new features in Solaris. This is exactly what I referred to in my opening paragraph… all these anti-Solaris pundits more than likely have zero hands-on expeience with the stuff they’re harshing on. People like Mr. Zaitcev read anecdotes and stories, come up with their own idea as to how things are based on those stories, and produce comically uninformed jabs posts such as the one linked above.

No breadth? Just what is the breadth that Mr. Zaitcev thinks is missing? Is breadth in this case even quantifiable? Is his supposition based solely on the age old (and aged) driver count argument? Does Mr. Zaitcev think that all Solaris is, is an ancient kernel which happened to have a few new concepts tacked on top of it?

I would bet that if Mr. Zaitcev sat down and tried to use Solaris in a real-world environment, he’d soon learn that Solaris has everything one needs in a data center environment… he just hasn’t discovered them (or read about them, natch) yet for himself. Who knows, perhaps he’d even appreciate them.

Technorati Tags: , , , ,

Comments
Comments Off
Categories
Geekery, Solaris
Comments rss Comments rss
Trackback Trackback

Crying “FUD” doesn’t always mean you’re right

Dale Ghent |

…and it sure doesn’t grant you instant vindication.

It appears that DaveM (Linux networking and SPARC port guru) has gotten seriously wound up in response to a blog post by Jeff Bonwick (Sun’s storage and kernel guru.)

As one can see in Jeff’s post, the suject he wrote about was within the greater context of using Solaris as a storage appliance OS (something I have an interest in) and why Solaris/OpenSolaris can and would excel when it comes to being the kernel of a storage OS.

I’m a storage guy. In the course of my work I have to not only work with Solaris hosts on my SAN, but also Windows and Linux (and soon, AIX). So I have a front-row seat when it comes to witnessing and dealing with how these various OSes deal with storage, from the filesystem to multipathing, to the HBA… and let me tell you, Linux is quite not the joy in this specific area as most people think it is on a general, all-encompassing level.

And that’s what Jeff’s angle was.

Now on to Dave’s rebuttle.

“The implication is that Linux is not rock-solid and that it does break and corrupt people’s data. Whereas on the other hand Solaris, unlike the rest of the software in this world, is without any bugs and therefore won’t ever break or corrupt your data.”

No OS comes without fault, but some OSes have faults that are more glaring than others in their analogous areas. Staying within the storage context of this discussion, I have to say, again, Linux is no shining star here.

ReiserFS is arguably the most advanced fs in terms of features when it comes to the portfolio of Linux file systems, but its issues with stability are such that you’re really walking on eggshells whenever you employ it. I have been personally told too many first-hand accounts and read plenty more on the Internets regarding its tendency to be fine and then fail spectacularly. It has been likened to a time-delayed /dev/null of sorts, and the future of it is in doubt with the legal troubles of its designer and Namesys limbo. Is any version of ReiserFS a viable Linux storage technology for a production environment? I say No. That’s sad because I dare say at one point ReiserFS had some promise.

EXT2 and 3… tried and true. Very stable and moderately fast for most tasks. But it’s an “old guard” file system. As such, it’s not very flexible, and any flexibility it gets comes from using a volume manager underneath of it. In the days where the notion of handing a server a 1TB LUN is nothing to blink at, this inflexibility can be suffocating in a dynamic environment. These “old guard” file systems (yes, Solaris’s UFS is one of them, too) are more like mere utility file systems than practical ones for today’s mass storage needs. It’s good for holding a machine’s OS and that’s about it.

XFS… Of all the file systems in the Linux file system portfolio, this one gets the gold star. Stable, fast, and decently scalable with the large amount of data you can stuff in it… but it still suffers the same problems EXT[23] and other “old guard” file systems do in terms of flexibility. In other words, it’s just a file system. Keep in mind that this critique is coming from a guy who worked with XFS on IRIX often and absolutely loved XLV… back in its the day.

As it stands now, the mainline Linux kernel doesn’t offer anything which embodies the file system triple play: being stable && fast && flexible. Solaris’s ZFS has this. I’ve so far entrusted 30TB of spinning rust to it, and it has yet to let me down. Sure, there are projects here and there that have the eventual goal endowing Linux with a ZFS analog, but as of right now they’re nothing production quality and are definitely not something a admin can call RedHat to get support for.

There are plenty of other aspects to the storage context… the fibre channel stack, for one, and other things such as multipath IO implementations and volume manager and management layers (which Linux has a host of… not necessarily a good thing… LVM, LVM2, MPIO, RDAC… it makes your head spin.)

But as far as this storage-oriented discussion goes, file systems are indeed the make or break aspect. This is why Jeff said what he said. Linux has no ZFS. Windows has no ZFS. It is not that Linux or Windows need ZFS itself in order to compete, it’s that they need to develop and employ the concepts that ZFS implements and do so as clearly and concisely as ZFS has.

Anyway, enough about storage. Now, why is it that the Linux community (let alone a prominent member of it) has to react so violently to any questioning of its perceived superiority? Is it misplaced or excess pride? Have they not tried things other than Linux recently and they’re just flying with blinders on? Is it just the social culture which prevails within it? What ever it is, seeing posts like Dave’s makes my toes curl with embarrassed amazement.

A friendly message to Dave: Chill the ad hominems, mkay? Crying “FUD!” at the mere sight of someone who you perceive as poo-poo’ing an aspect of your interest doen’t typically translate into a well thought-out rebuttle. You took the low road and tried to convey Jeff as being some instrument of some nefarious, Mr. Burns-like person at Sun. Is vilifying instead of cool-headed technical discourse really your desired style? Has anyone come biting at you saying “oh, he’s a Linux kernel developer, so he has an agenda”?

Technorati Tags: , , , , , ,

Sun and Qlogic to open up the source of storage software

Dale Ghent | April 11, 2007

Sun has announced that they will be releasing several of their storage products to the OpenSolaris community!

From their announcement:

  • Sun StorageTek 5800 storage system (Honeycomb) client interfaces along with the Honeycomb software developer kit (SDK) and Honeycomb emulator/server. Honeycomb is a third-generation digital repository solution for data capture and management.
  • SAM-FS (Storage Archive Manager) provides data classification, policy based data placement, protection, migration, long-term retention, and recovery capabilities for organizations to effectively manage and utilize data according to their business requirements. SAM-FS is used exensively in security/surveillance, digital video archiving, and medical imaging data environments.
  • QFS Sun’s shared file system software delivers significant scalability, data management, and throughput for the most data-intensive applications. Well known today in the traditional high performance computing (HPC) arena, QFS is increasingly being used in commercial environments that require multiple host, high speed access to large data repositories.

Also, and pretty awesome yet:

In addition, today QLogic is contributing their Fibre Channel HBA driver code to the OpenSolaris storage community. For the first time, developers have access to an I/O stack from the application through to the operating system.

Now how cool is that? That last sentence says it all… openness from from the app to the HBA. Most excellent. Thank you to Sun and a really surprised thank you to Qlogic!

Technorati Tags: , , , ,

Comments
Comments Off
Categories
Solaris
Comments rss Comments rss
Trackback Trackback
This page took 0.284 seconds to generate.