OSAlert has been reporting on the Debian/Ubuntu/GNU/Opensolaris hybrid for several years. But for those of you who’ve never looked more closely at this interesting OS, a Nexenta developer has laid out some of its more noteworthy features and advantages.
The Nexenta project, started in 2005, has had 6 releases (NexentaOS and NexentaCore), and is preparing for the upcoming NexentaCore Platform 2 release. If you are a Debian/Ubuntu developer, consider taking a little time to take a look at an emerging platform that provides a feature rich developer environment.
Nexenta has built a small but vibrant community. It has been downloaded over half a million times, and is being used in around 10,000 systems, mostly data-centers.
Nexenta uses Debian’s dpkg packaging system, and provides everything provided by Debian/Ubuntu environment. However, these are only a subset of the features in Nexenta’s toolchain. We’ll take a look at the additional features and tools provided by Nexenta.
The Opensolaris technologies: ZFS
The open source world has gone gaga over the next generation file system from the Opensolaris community. It has been ported to Apple’s OSX and FreeBSD. Licensing incompatibilities have held up a Linux port. It is a file system and volume manager built into one. And with lightning fast snapshots and automatic integrity and error checking, this is the filesystem of the future. Lets take a look at how you as a developer can benefit by the use of ZFS:
Never have to worry about data loss, or the system and packaging system going into a mangled state
ZFS’s quick and painless snapshot feature allows you to take incremental backups of your entire filesystem or the packaging related data. So if the last kernel upgrade botches up the system or renders it unbootable, easily revert back to the earlier working state
Nexenta provide a tool called apt-clone which is a wrapper over apt-get. Roughly translated
apt-clone = system clone + apt-get
so a command apt-get install apache2 is equivalent to
zfs clone beforeapache2 rootfs
#add grub lines to boot to ‘beforeapache2’
apt-get install apache2
If things go wrong and apache2 isn’t installed right, you can revert back to the snapshot “beforeapache2” and the system reverts to the original configuration. Further information in the apt-clone manpage [0].
Use snapshot capabilities to create restore points
..all along your development/build cycle allowing for easy reversion to an earlier state
Another use of ZFS for package maintainers is the capability to easily take a snapshot of the environment (like the current working directory) and make those deadly changes. There no longer a need to worry or limit your changes for the fear of “ruining this working state” or “losing all these changes I’ve made”. Use ZFS like a quick and dirty versioning system for those short development sessions.
Increase build environment storage as and when required
If you’ve run into situations where you had dedicated space for a certain need, but later found yourself out of space. This could be for your package development needs or for the music you play when you develop. With ZFS, you no longer have to decide between removing package A or sources B. You can have both the Rock and the Blues collection. A new hard disk or a partition plus a simple command will take care of all your space needs.
ZFS pools and RAID-Z
..for superior volume and data redundancy management
ZFS filesystems are laid out on ‘pools’ which are storage spaces. This is a different from the traditional ‘disks’ and ‘partitions’ way of thinking about laying out filesystems. You can now add any type of storage (disk, network attached storage, a file) to the pool, and everything in the pool is made available to the filesystems that are created from the pool. This allows us to very easily increase the size of a pool by simply giving it more storage.
To create a pool from a disk
zpool create
zpool create diskpool c1d0p0
To increase the size of the pool, simply add another disk to it
zpool add
zpool add diskpool c2d0p0
ZFS supports various raid configurations out of the box, including RAID-Z, which is a better implementation of RAID5 [1]. It implements features into the filesystem that previously required specialized hardware. A comparison of ZFS with Linux LVMs can be seen at [2].
This allows you to setup your storage in mirrored disks by simply (eg)
zpool create mypool mirror c1d0p2 c2d0p2
Creating a zfs filesystem on our newly created pool is done via
zfs create mypool/myroot
zfs create mypool/john
Both myroot and john will use the mypool storage space created by zpool. Set the mount points for these via
zfs set mountpoint=/space/john mypool/john
zfs set mountpoint=
A complete documentation of the zpool and zfs commands can be found at docs.sun.com [3][4].
ZFS supports quotas and reservations for each dataset. You can thus allocate varying amounts of space for various purposes. The built in checksumming of everything does away within the filesystem allows you to breathe easy while ZFS does everything that can be done to keep your data error free. ZFS also supports compression and encryption (currently under review by the ZFS community and to be added to the Opensolaris kernel shortly)
The Opensolaris technologies: Zones
(..and ZFS zones with Debian environment)
Zones are lightweight virtual instances of the operating system. The main (or parent) instance is known as the global zone. A global zone can have as many local zones as resources like memory and disk space allow for.
Zones are a boon to package maintainers. they allow for quick and easy setup and generation of development environment. Once the tasks are done, the zone can be done away with, until required later. Thus you do not have to allocate a specific amount of your resources towards a build setup.
Nexenta provides a set of tools in a package called devzone, which allow for easy creation of development zones for package maintainers. These can be easily tuned to other specific needs. A typical zone takes less than a minute to setup and use.
As each zone is self-contained, you can provide root access to them to those you may not fully trust. This setup can be used in build machines where “potential” developers to a project can be given access to their own development zone to play around with. This setup is used by the Nexenta Project during its hackathon events. further information at [links].
A zone can be created using
devzone_create
Now, enter the zone using
devzone_enter
and poof!.. you are now root in the new virtual instance of the machine. you would now typically setup your /etc/apt/sources.lst and start installing and building packages.
You can reconnect and disconnect the zone as many times as you like. Once you’re done with using a zone, simply run the following command in the global zone
devzone_free
to kill the zone, and free up it’s resources.
Devzones provide the following advantages to developers
- Better share resources: As development zones can be created on the fly, you can easily share resources. Developers in different parts of the world can access the machine at different times.
- Peace of mind: If you have a dedicated build machine, easily allow multiple developers access to it, without worrying about interfering with each other, or one’s mistake bringing down the system.
To see how you can build Debian/Ubuntu packages in a devzone, take a look at the screencast at [5].
Simplified service management using SMF
SMF, or Service Management Framework, is the Opensolaris replacement for init.d scripts. Services need to be run by a system, and SMF does this intelligently.
Traditionally, we’ve used /etc/init.d method of service management, where the services are started upon init depending on the init state. The scripts in the rc.d directory are run one after the other, and care must be taken to place the important core services are listed before the others that depend on it.
Like apt, SMF services are maintained as a tree of dependencies. The ssh service, for example, has network/physical as its dependency. There would be no use of running the the ssh service without the network services. Thus, any SMF service when started, first starts all its dependencies, and then itself. Starting/stopping a service is as simple as
svcadm enable ssh
svcadm disable ssh
This allows for very easy administration of services. In the above example, you dont have to worry about starting its dependencies first: SMF will take care of it. If one of the dependencies failed (eg: due to a disk failure), this is reported to the system via the Fault Management Architecture (FMA).
Another advantage of SMF is the speed of service startups upon bootup. Traditional /etc/init.d starts services one after the other depending on their numbering in the rc.d directory. But this is not efficient as a slow service wastes CPU time. SMF, with its knowledge of dependencies, will start 2 independent parts of the tree in parallel. This the CPU is optimally used, and leads to faster system bootup and services startup
More information on the Opensolaris SMF and FMA technologies, with comparison of equivalent /etc/init.f commands can be gleaned at the Sun documentation [6]
Dtrace: Probe the nervous system
Dtrace is another unique and powerful technology present in Nexenta. It provides a developer with a “read-only” interface into each and every function of the operating system. Set breakpoints (or probes) and run your script as and when they’re fired. And it is not just limited to the kernel. More and more packages are adding Dtrace probes, allowing you to easily process the function call paths in the application, and collecting various statistics about the running system/application.
For example, the AMP stack have probes enabled in them[7]. This allows you easily monitor your web services and applications, and tune them as necessary. Using tcpdump and snoop to look at the network traffic? Why not a line in Dtrace?[8]
dtrace -qn ‘ip:ip:*:receive{ printf(“Packet recieved from %s: %d byte packetn”, args[2]->ip_saddr, args[4]->ipv4_length ); }
Take a look at hundreds of other one-liners. Dtrace has a full fledged scripting language, which is similar to C. Many Dtrace resources are linked from here. Take a look at screencasts introducing Dtrace.
Example: Setting up a Webserver
If you have prior experience with setting up a webserver with Debian or Ubuntu, then you’ve already learnt setting one up on Nexenta.
First, download apache2, mysql and php
apt-get install apache2 mysql php5
Now setup the configuration files as required. they’ll be available at /etc/apache2/httpd.conf
Once done, restart apache to update the configuration changes using
apache2ctl restart
and your webserver is up and running. Content goes under /var/www
Example: Setting up a CIFS server
CIFS is a protocol that allows for request of files and services over the network. It is the default file sharing service in Windows, and has been implemented by projects like Samba.
Nexenta makes it very easy via the ZFS filesystem to setup a file server.
First let us enable the SMF service, and setup the domain in which the server belongs (let us assume the domain is win.nexenta.org)
#svcadm enable -r smb/server
#smbadm join -u Administrator win.nexenta.org
Now, lets setup the ZFS to export a particular dataset
#zfs set sharesmb=on data/myshare
#zfs set sharesmb=name=myshare data/myshare
You might also want to set the case sensitivity of files if accessing from other windows boxes.
#zfs set casesensitivity=mixed data/myshare
Restart the smb/server and we’re done.
#svcadm restart smb/server
For a complete step by step procedure take a look at the article here[9].
Give it a try!
Nexenta is available at www.nexenta.org. The developer community meets via IRC on #nexenta@freenode or the web interface at http://www.nexenta.org/chat. We’re currently porting packages from Ubuntu’s hardy repository to NexentaCore.
Join in here and request for a login to our build machines to test Nexenta live, and try out the devzone environments.
References
[0] http://www.nexenta.org/os/apt-clone[1] http://blogs.sun.com/bonwick/entry/raid_z
[2] http://www.unixconsult.org/zfs_vs_lvm.html
[3] http://docs.sun.com/app/docs/doc/819-2240/zpool-1m
[4] http://docs.sun.com/app/docs/doc/819-2240/zfs-1m
[5] http://www.gulecha.org/2008/08/22/getting-started-with-nexenta-development-part-2/
[6] http://www.sun.com/bigadmin/content/selfheal/SMF-quickstart.jsp
[7] http://blogs.sun.com/angelo/entry/dtrace_meet_the_amp_apache
[8] http://www.cuddletech.com/blog/pivot/entry.php?id=952
[9] http://jmlittle.blogspot.com/2008/03/step-by-step-cifs-server-setup-with.html
About the author:
Anil Gulecha is Nexenta developer. He maintains a blog at gulecha.org. He tries to play a little guitar in his free time.
But this is touting the Solaris advantages, not necessarily the Nexenta advantages. Why pick this over another Solaris distro?
Driver support, licensing, and program support. I run OpenSolaris snv95 on my “soon-to-be” fileserver, but I know people who’ve tried installing it without luck on their hardware. It’s getting much better (I’m running the new 780G AMD Chipset), but still not all the way there. My Realtek 8111c nic is still buggy and auto-dhcp doesn’t seem to work. Other things like USB+ZFS craps out after so much use because of “memory” issues. 4GB of ram doesn’t seem to work either, and I’m not the only one with this issue.
I’m not a big fan of linux (I’m more of a BSD guy myself), but I do believe it does have its rightful place in this world to keep other communities in check.
What are you talking about?
Nexenta and OpenSolaris share the same kernel.
There’s no better driver support and no Linux here.
If ZFS “craps out” on your OpenSolaris box, chances are Nexenta won’t make it any better.
OpenSolaris drivers quality (especially storage drivers) generally better then in Linux. The problem is that not all white-box HW is supported – but this is matter of time.
I have no idea where you pulled that load of baloney from. A lot of Solaris’s drivers have been neglected for years, and throwing in the word ‘quality’ won’t make them better. A big part of Solaris’s problem, still, is that it doesn’t have the hardware support of Linux, especially on commodity x86 stuff. This comes from Linux’s history (it’s the reason it exists) of being a kernel and OS intended to run on that kind of hardware (and years of work), and the Unix vendors’ (like Sun) insistence on not porting their operating systems to commodity hardware when they could get people to buy their really expensive stuff instead. That worked for many years and Sun got rich off it, but it has caught up now.
Much of the point of OpenSolaris is that Sun can’t fill that gap themselves, and they would love Linux’s drivers and kernel developers as a result.
In all probability, it’s going to be a long time. Linux has run on lots of off-the-shelf hardware for a long time, but even a few years ago, filling in the gaps of this network card, that network card, this onboard chipset, these TV cards etc. took years, a lot of driver developers and creating the environment in which to do it. There’s still a lot to do as well.
I just don’t see a kernel community developing for Solaris that is going to be able to achieve that.
Because it gives people normal Linux/Debian environment
apt-get is a Debian feature, not a solaris feature. The 18000 apps that come with it are not a Solaris feature either.
It mentions the GNU userspace, that makes it easy for Linux users to jump over to the Solaris kernel.
If it would work – last time I tried things blew up pretty soon.
You’ll still need to learn the Solaris tools to take advantage of the solaris specific advantages.
Another interesting project in that area is portage-prefix trying to implement Gentoo’s portage with Solaris.
After using, or rather trying to use, Nexenta I sort of doubt the 18000 apps figure, are you sure that is not for Ubuntu? Anyway, it doesn’t matter if they have 18000 apps, you only need a handful or so of important ones to bi missing to make life miserable.
For me the deal breaker was Java. Sun java won’t install. I suppose you could try to port openjdk but a lot of the dependencies are just not there yet, so that would be a major project.
Most of the tools for administrationg it is Solaris oriented. E.g. SMF for starting and stopping services, ldapclient to set up LDAP,… I would think the only thing an Ubuntu user would find familliar is apt-get, exept that there are a lot fewer apps to get. Apart from apg-get Nexenta feels like Solaris and have very little in common with Ubunto from the adminster point of view.
The main advantage over standard Solaris of Nexenta is that it boots from ZFS, but so does OpenSolaris from Sun. OpenSolaris have a lot more complete set of libs (and important to me, Java). My guess is that it is much easier even for a Linux person to start out with that instead of Nexenta and then port whatever GNU he nees you need than going with Nexenta.
Give this a try:
http://wimpi.coalevo.net/2008/08/how-to-jdk6-on-nexenta-core-platfo…
That’s the point, you don’t have to port GNU stuff to Nexenta, because it is already using the GNU userland.
Actually, there are quite a lot of GNU packages that comes with OpenSolaris, or even standard Solaris as well. E.g. gcc, gmake, gtar,.. The difference is that it is often easier compile the GNU things that are not there, on real Solaris than it is on Nexenta. In fact you may not even have to compile them in real Solaris as most things can be pkg-get:ed from blastwave.org.
In theory Nexenta should be easier, as you should more or less just be able to port/recompile an Ubuntu package if its not already compiled for Nexenta, but in reality, you end up with a Gentoo like feeling where you spend a lot of time compiling your software.
You also have to remember, that there are some things from Linux that is missing in the Solaris kernel and vice versa. This means that there will be a few Ubuntu packages that not port easily without doing some real programming.Then take dependencies on these packages into account and you will realize that easy life on Nexenta is not apt-get:able.
So, if you wan’t the GNU stuff on a Solaris kernel OpenSolaris or even standard Solaris is a better starting point at the moment. This may of course change in the future when more precompiled packages becomes available to Nexenta.
In spite of all my troubles with Nexenta, I kind of liked it. I think that this was mostly because it came without a GUI, that I had to turn off (not much use of a GUI in my server room), and that it booted directly from ZFS. However, persons coming to Nexenta with the hope and expectation that it will feel like Ubuntu will get very disappointed.
uh, the article states, and I quote:
“Nexenta uses Debian’s dpkg packaging system, and provides everything provided by Debian/Ubuntu environment”
According to Nexenta’s wiki, it can use ubuntu’s repos directly:
http://www.nexenta.org/os/BuildingPackages
But I guess there is a four step process to port them, so I guess you do have to port them, but it looks extremely easy.
One supposes the version that eradicates death and taxes is still in cvs…
I think the most coolest thing they produced is NexentaStor – NetApp killer software which delivers features of NetApp but without HW vendor lock-in:
http://www.nexenta.com/nexentastor-overview
the truth is that people don’t use more than 15 apps, and out of these, it is like 5-6 apps for 97% of the time.
Actually they use more: stuff like gdm, kerberos or the syslog also count as applications.
And it’s not like those 5-6 apps you mention are the same apps for everyone anyway.
Sun just needs to gpl solaris ,zfs, zones, dtrace and what ever else it has going for it and set it free. Solaris is dying a slow death and making it more Linux like is just postponing the inevitable. Donate it to the world.
She is very sexy. It is said that she is dating with a guy at hot interracial dating club for blacks&whites named ‘ kissinterracial.com ‘
Really?
I think that file the pre-allocation is going to be the hot ticket for latency intensive things such as databases. Even in solid-state drives, using less memory means better data reliability.