FreeBSD 5.2.1 on SPARC64

FreeBSD has a solid reputation in terms of features and performance on x86, powering sites from Hotmail to Yahoo, yet it doesn’t tend to be the first (or even second) OS that comes to mind with many people when thinking of Solaris alternatives for the SPARC platform.

While I’ve run plenty of other non-commercial operating systems on various non-x86 platforms, I’ve only ever used FreeBSD on x86 hardware. So in my research on non-Solaris SPARC operating systems, I was actually a bit surprised to find that not only does FreeBSD operate on UltraSPARC-based SPARC systems, but it’s considered a Tier 1 port as well, along with x86, x86-64, and Alpha.

Thus in my continuing series evaluating the operating systems for my Sun Ultra 5, I took at a look at FreeBSD 5.2.1, SPARC64 platform.

Hardware

My test system is a Sun Ultra 5, and the specs can be found here. The complete list of supported systems for FreeBSD SPARC64 can be found on FreeBSD’s 5.2.1 Hardware Notes.

FreeBSD only supports UltraSPARC/sun4u systems. There is no current FreeBSD port for sun4/sun4d/sun4c/sun4m systems, so if you’ve got an older system such as a SPARCStation 5, then you’ll have to look at other operating systems.

FreeBSD currently doesn’t officially support frame buffers/graphics for the SPARC platform, so this will be a server-only review (the format of this document changed with 5.1, and it doesn’t mention graphics at all). There has been discussion of getting it to work, but I haven’t seen anything official.

Installation

Here’s important issue to be aware of before installing FreeBSD on a SPARC system, and I shall italicize and bold-face it to highlight its importance:

FreeBSD for SPARC (as of 5.2.1) will not
install from the screen/keyboard, you must use the serial port.

The install taunts you from the keyboard/screen, as the installation menu will come up and teases you with its attractive menus. However, the keyboard drivers don’t work for a Sun keyboard, and will mock you as you try to navigate a menu.

There’s no mention of this anywhere during the install, and it is not documented in the most recent SPARC64 installation instructions (it even refers to the keyboard), or the 5.2.1 SPARC64 release notes. There is indirect mention of it in the 5.0 hardware notes, but not in the subsequent hardware notes. As a result, the FreeBSD SPARC64 mailing list has a couple of “why doesn’t this work” messages scattered about.

This can be annoying if you don’t keep handy a serial cable with null modem cable, which can be the case if you’re accustomed to dealing with SPARC hardware and its serial ways. I had already built myself a quick adapter (from my Serial Madness article), and proceeded with the installation.

To start the install, I downloaded the installation ISOs from FreeBSD’s FTP site and burned them to a CD. A simple boot cdrom from OpenBoot brings up the installation.

When the system boots from the CDROM drive, it
gives you 5 options for terminal types:

These are the predefined terminal types available to
sysinstall when running stand-alone. Please choose the
closest match for your particular terminal.

1 ...................... Standard ANSI terminal.
2 ...................... VT100 or compatible terminal.
3 ...................... FreeBSD system console (color).
4 ...................... FreeBSD system console (monochrome).

5 ...................... xterm terminal emulator.

Your choice: (1-5)

Selecting ANSI will render nice and clean looking menus during the install, but lacks arrow key functionality. I used VT100, and while the borders of the textual menus showed up as garbage characters, the arrow keys did indeed work. The other options didn’t work for me at all.

        +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A Network Configuration "A"A"A"A"A"A"A"A"A"A"A"A+
        |  Host:                            Domain:                    |
        | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+        +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+        |
        | |zorak                  |        |vegan.net         |        |
        | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+        +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+        |
        |  IPv4 Gateway:                    Name server:               |
        | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+               +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+          |
        | |192.168.0.1     |               |192.168.0.1     |          |
        | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+               +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+          |
        |       +"A"A"A"A"A Configuration for Interface hme0 "A"A"A"A"A"A+        |
        |       |  IPv4 Address:            Netmask:          |        |
        |       | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+       +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+ |        |
        |       | |192.168.0.15    |       |255.255.255.0   | |        |
        |       | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+       +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+ |        |
        |       |  Extra options to ifconfig:                 |        |
        |       | +"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+       |        |
        |       | |                                   |       |        |
        |       +"A+"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+"A"A"A"A"A"A"A+        |
        |                                                              |
        |              +"A"A"A"A"A"A+            +"A"A"A"A"A"A"A"A"A"A+                |
        |              | [OK] |            |  CANCEL  |                |
        +"A"A"A"A"A"A"A"A"A"A"A"A"A"A+"A"A"A"A"A"A+"A"A"A"A"A"A"A"A"A"A"A"A+"A"A"A"A"A"A"A"A"A"A+"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A"A+

As you can see, the menus with the VT100 option are a bit on the ugly side, but plenty readable.

Installation went pretty quick, and took about 20 minutes to complete a full install of developer tools, including a kernel source.

PCI
Devices

My Sun Ultra 5 has an Intel 10/100 card installed and FreeBSD recognized it immediately as fxp0, in addition to the built-in hme0 interface. Both interfaces worked flawlessly.

Compiling

As with the SPARC64 NetBSD port, FreeBSD SPARC64 exists entirely as a 64-bit operating system, with no 32-bit userland or kernel support. Unlike NetBSD, however, this didn’t become an issue. I had no trouble compiling any software, including OpenSSL, from either the ports or from source tarball.

FreeBSD 5.2.1 includes GCC 3.3.3 as the default compiler, where most of the other open source OS distributions for SPARC come with GCC 2.95. This sacrifices compile-time in some respects (in an article on O’Reilly’s devnet, Marc Espie comments on GCC 3.3’s longer compile times), but it gains on the fact that GCC 3.3.3 is current and generally more 64-bit friendly to the 64-bit-only FreeBSD than GCC 2.95 is (I had a few problems myself with GCC 2.95 on 64-bit NetBSD).

Since it’s a 64-bit operating system, the included GCC has been adjusted to compile 64-bit binaries (-m64) by default. It also appears to use -mcpu=ultrasparc by default, as an OpenSSL run compiled with no -mcpu declaration and one with -mcpu=ultrasparc were identical. This makes sense, since FreeBSD SPARC only runs on UltraSPARC processors.

It’s important to remember that the make utility included with FreeBSD is BSD version of make. Some applications require GNU’s make, so you’ll have to install that yourself. A common and useful convention is to install GNU’s make as gmake, in order to differentiate between the two.

Performance
Oddity

MySQL 4.0.17 compiled cleanly from a source tarball (yes, I know 4.0.18 is out, but to keep results consistent with the other reviews, I’m sticking with 4.0.17). While looking through MySQL’s site I saw that they have a binary distribution for FreeBSD SPARC64, so I downloaded that and ran sql-bench benchmarks against that as well.

The results showed some interesting results. When compared to the results from Solaris and Linux running on the same Ultra 5, a few operations were slightly faster on FreeBSD, but many were significantly slower.

MySQL 4.0.17 results, various OSs

MySQL 4.0.17 results, various OSs (cont)

For the Linux run, I compiled a 32-bit MySQL binary and ran it on Linux kernel 2.4.24 (64-bit). Under Solaris 9, the run was a 64-bit MySQL binary that I compiled with GCC that I had recorded from earlier articles.

As you can see from the graphs, both the MySQL I compiled and the pre-compiled binary I obtained from MySQL.com exhibited similarly slow behavior.

I thought perhaps it had something to do with soft-updates, so I disabled soft-updates for the /usr partition (where the binaries and data files reside for my tests), and ran the test again with my compiled binary. The results of that run were actually slightly worse across the board.

Given that both Solaris and Linux on the same hardware where generally comparable wit each other, and much quicker in certain operations than FreeBSD, I have to conclude that there is something wrong with the SPARC port of FreeBSD to account for the slowness. Without further testing and more variables, I couldn’t say what the issue is for certain, but I would hypothesis that it was something to do with the IDE driver. One way to find out would be to run these test on a system with a SCSI bus, and see if the same anomaly shows up. However, I don’t have access to a SCSI-based UltraSPARC system.

FreeBSD 5.1 Performance Worse

Interestingly enough, these results are an improvement over FreeBSD 5.1. Before I started these evaluations, I had installed FreeBSD 5.1 a few months ago when 5.1 was the most recent release.

I did a quick write test using dd, and wrote out a 100 MB file. For FreeBSD 5.2.1, the results were pretty much the same with the same test done in Solaris at roughly 10 MB/s.

104857600 bytes transferred in 10.957025 secs (9569897 bytes/sec)

In FreeBSD 5.1, the results were much worse, at 2 MB/s.

104857600 bytes transferred in 50.845939 secs (2062261 bytes/sec)

FreeBSD 5.1 versus 5.2.1

FreeBSD 5.1 versus 5.2.1, cont

I did see a note about major ata work in the release notes for 5.2 and 5.2.1, but I’m not sure what the exact issue was for 5.1, or what the current issue is for 5.2.1. So while there is still an issue with 5.2.1, it looks like they’ve worked out something even worse in 5.1.

Software

The default system comes with a wide variety of binaries pre-installed including Perl and tcsh, and anything not installed can easily added via the ports system. There wasn’t an app that I came up against that I didn’t compile and install cleanly, including apache, htdig, and OpenSSL.

Conclusion

I was very pleasantly surprised by FreeBSD on SPARC, and I’m surprised more people don’t use it on their UltraSPARC systems. When people think non-Solaris SPARC operating systems, many tend to think of Linux, but as long as you’re not using it for a desktop (and your system is an UltraSPARC), FreeBSD 5.2.1 makes a
very complete, useful, and flexible server.

Given my experience with the last 64-bit-only operating system NetBSD, the 64-bit only nature of FreeBSD gave me pause. However, it wasn’t an issue for any of the software I tried. A combination of good application configure support, good ports maintenance, plentiful pre-compiled binaries, and GCC 3.3.3 contribued to this. FreeBSD didn’t exhibit any of the problems

My only reservation was the extraordinarily long result for MySQL’s insert, which even with MySQL.com’s binary suffered from performance degradation compared to other operating systems on the same hardware. Still, if you’re just using it as a development system, or non-disk intensive system, then that may not be a major factor to you.

It is FreeBSD, and it is true to it’s Tier 1 claim, in that it works and feels talmost the same on SPARC as it does on x86 (with the exception of the performance problem). If you’re looking for an alternative operating system to run on an Ultra 5, then FreeBSD is definitely worth a look, especially if you’re comfortable and familiar with FreeBSD.

Appendix: dmesg

Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 5.2.1-RELEASE #0: Tue Feb 24 07:47:33 GMT 2004
    [email protected]:/usr/obj/usr/src/sys/GENERIC
Preloaded elf kernel "/boot/kernel/kernel" at 0xc0452000.
Timecounter "tick" frequency 333000000 Hz quality 0
real memory  = 268435456 (256 MB)
avail memory = 245907456 (234 MB)
cpu0: Sun Microsystems UltraSparc-IIi Processor (333.00 MHz CPU)
nexus0: 
pcib0:  on nexus0
pcib0: Sabre, impl 0, version 0, ign 0x7c0, bus A
pcib0: [FAST]
pcib0: [FAST]
DVMA map: 0xc0000000 to 0xc3ffffff
pci0:  on pcib0
pcib1:  at device 1.1 on pci0
pci1:  on pcib1
ebus0: revision 0x01
ebus0:  mem 0xf1000000-0xf17fffff,0xf0000000-0xf0ffffff at device 1.0 on pci1
ebus0:  addr 0x140072f000-0x140072f003,0x140072c000-0x140072c003,0x140072a000-0x140072a003,
0x1400728000-0x1400728003,0x1400726000-0x1400726003 (no driver attached)
ebus0:  addr 0x1400724000-0x1400724003 irq 37 (no driver attached)
ebus0:  addr 0x1400504000-0x1400504002 (no driver attached)
sab0:  addr 0x1400400000-0x140040007f irq 43 on ebus0
sab0: [FAST]
sabtty0:  on sab0
sabtty0: console 9600,8,n,1,-
sabtty1:  on sab0
ebus0:  addr 0x14003083f8-0x14003083ff irq 41 (no driver attached)
ebus0:  addr 0x14003062f8-0x14003062ff irq 42 (no driver attached)
ebus0:  addr 0x1400700000-0x140070000f,
0x140030015c-0x140030015d,0x14003043bc-0x14003043cb irq 34 (no driver attached)
ebus0:  addr 0x1400720000-0x1400720003,0x1400706000-0x140070600f,
0x14003023f0-0x14003023f7 irq 39 (no driver attached)
eeprom0:  addr 0x1400000000-0x1400001fff on ebus0
eeprom0: model mk48t59
eeprom0: hostid 80a829a6
ebus0:  addr 0x1000000000-0x10000fffff (no driver attached)
ebus0:  addr 0x1400722000-0x1400722003,0x1400704000-0x140070400f,0x1400702000-0x140070200f,
0x1400200000-0x14002000ff irq 36,35 (no driver attached)
hme0:  mem 0xe0000000-0xe0007fff at device 1.1 on pci1
hme0: Ethernet address: 08:00:20:a8:29:a6
miibus0:  on hme0
nsphy0:  on miibus0
nsphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
pci1:  at device 2.0 (no driver attached)
atapci0:  port 0xc00020-0xc0002f,0xc00018-0xc0001b,
0xc00010-0xc00017,0xc00008-0xc0000b,0xc00000-0xc00007 at device 3.0 on pci1
atapci0: [MPSAFE]
ata2: at 0xc00000 on atapci0
ata2: [MPSAFE]
ata3: at 0xc00010 on atapci0
ata3: [MPSAFE]
pcib2:  at device 1.0 on pci0
pci2:  on pcib2
fxp0:  port 0x400-0x43f mem 0x100000-0x1fffff,0x2000-0x2fff at device 3.0 on pci2
fxp0: Ethernet address 00:90:27:c2:88:5e
miibus1:  on fxp0
inphy0:  on miibus1
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Timecounters tick every 10.000 msec
GEOM: create disk ad0 dp=0xfffff800008726a0
ad0: 8693MB  [17662/16/63] at ata2-master WDMA2
acd0: CDRW  at ata3-master PIO4
Mounting root from ufs:/dev/ad0a

If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSAlert.