I was lucky enough to receive an invitation to the HP Technology Forum 2010 via OSAlert and just spent most of this last week in Las Vegas with five thousand other nerds of varying caliber. The tech forum is focused more on enterprise technology than that of the consumer, and– let’s face it– even if any of us could afford a $30,000 rack of servers, most of us have little idea of what we’d do with so many resources except brag. Despite the focus on an area not quite as natural to OSAlert and many of its readers, there was a plenitude of good and interesting information shared– aside from that, the forum was simply fun. There were a few subjects that were especially eye-catching, though many of them not necessarily comprehensive enough to base an entire article on; thus this overview.
Converged Infrastructure: What Is It, and Why Care?
“Converged Infrastructure” seemed to be the main obsession throughout the entire conference. I’ll admit it– I could only guess at what it meant, and I still don’t have my head fully wrapped around the concept, but, despite the thorough enterprise-ness of the idea, it’s definitely noteworthy.
Converged Infrastructure, in HP’s dictionary, “provides a blueprint for the data center of the future that eliminates costly and rigid IT silos so you can spend much more of your IT budget on business innovation. This is achieved by converging server, storage and networks with facilities ^aEUR“ all managed through a common management platform.”
Essentially, you have quite a few traditional servers and a whole slew of switches connecting these servers to all sorts of things via hundreds or even thousands of feet of cable. HP’s Converged Infrastructure aims to consolidate the servers, switches, storage, cables, and etcetera to the most compact, robust, and streamlined data center solution possible. Combine Converged Infrastructure with advancing virtualization and microchip technologies and you can reduce a traditional 20-server room to a 1-server rack (and that’s not exaggerating), saving thousands on what you’d spend maintaining the old technology to help pay for the new technology within a year or two. If the company applying said technology decides to use the saved money to expand the datacenter, computing power per watt and per square foot increases dramatically. This is pretty neat stuff, and any enterprise big enough that doesn’t at least take a look at this advancing technology is a bit soft in the collective head.
HP released quite a few new products at the Tech Forum that fit the Converged Infrastructure bill. Though they’re not outlined here, take a look at some of them; even if you’re not familiar with enterprise-grade servers, just reading the specs makes them rather drool-worthy.
Since this was the main focus at the forum, several of the HP partner booths that I visited also concentrated on the topic, including Samsung (who showed me some spiffy advancements in SSD and DDR3 technology that’ll be used in new HP servers), QLogic, ATTO (who talked to me about some of their neat fibre-channel networking technology also for use with the new HP servers), and many others, some of which I didn’t even get to visit with.
Silentium: Cool and Quiet Server Storage
Something that especially caught my eye was Silentium‘s booth in the expo. Silentium is a company that builds special server enclosures that both intelligently cool the server and intelligently reduce the noise pollution of both the server and the cooling apparatus. For example, if a classroom required a server of higher magnitude for whatever purpose, one of Silentium’s products would be perfect for that classroom: little noise and simple, compact cooling; the server can sit wherever in the room and not cause a distraction, impair communication, or be in the way. The same can apply to a small office that requires a server of that magnitude but doesn’t need or desire to have the server in a datacenter or a closet hidden somewhere in the back.
Blobo: Alcatel-Lucent’s Idea of Fun
One of the more popular attractions at the expo actually had very little to do with the technology being shown and more to do with the game that was being played. Alcatel-Lucent’s booth showcased various technologies, but the only one I (and most other attendees) took part in was that of the Blobo (especially because the few with the highest scores would win HP netbooks). I assume that this product has some connection with Alcatel-Lucent seeing as how that seemed to be all the employees who ran the booth seemed to concentrate on, but I’ve yet to see some connection between Blobo and Alcatel-Lucent while researching.
At any rate, Blobo is an interesting spin on video gaming with a similar aim as the Wii: be “fit” while playing video games. However, the Blobo’s console is actually the controller itself (AKA the Blobo– a small, somewhat squishy ball the size of a golf ball), which transmits via Bluetooth to any PC or even a mobile phone for the display; the computer that ran the game at the booth was a netbook connected to a 40″-or-so screen. Mac will also be supported soon. Lastly, Blobo is also available as a development platform for games and apps.
I personally don’t see this as any real contender to the Wii and other gaming systems, but then you never know. Very interesting.
And Then Some…
This is really only a drop in the bucket of what was going on with all of the excitement at the HP Technology Forum 2010– a few of the highlights from what I saw and heard. At least two follow-up articles are in line on the topics of Linux and open source at HP and a behind-the-scenes look at the technological side of how the tech forum was run, so stay tuned. Also, if you’re interested in this kind of thing, a more personal rendition of my adventures there is available via my blog, though I’ve currently yet to complete a few final posts on the subject.
I’ve never really understood the real benefits of consolidation. It seems like if you have a situation , where our servers are not under utilized consolidation becomes pointless.
It would seem to increase points of failure in the system.
Existing system of 20 servers:
1 physical server dies 1 server dies
New system of 20 servers virtualized onto on server.
1 physical server dies 20 servers die.
It also seems to be more difficult to incrementally grow.
Existing architecture needs one more server:
Buy one more server.
new architecture needs one more server:
Visualize one more, unless you run out of space, then buy a server big enough for 20 more.
What am I missing? Or is this kind of thing only for legacy mainframe or underutilised servers that do nothing all day in a static business with limited growth?
Enter live migration. When one physical server dies all the vm’s get migrated to a different physical machine on the pool. Supposedly without noticable downtime.
As for under-utilization, I’d wager most servers are under-utilized CPU-wise and only really utilizes memory. Few server applications are CPU-driven these days. You can save gobs of money by consolidating these on a standalone or pool virtualization solution.
Plus it has some nice management advantages like rebooting individual servers remotely with full access to the console and yu can add new servers without actually having to get a whole new physical machine.
There are many other advantages too (and disadvantages).
Edited 2010-07-01 08:52 UTC
But that is undermining the reason for 20 servers in the first place! You are arguing for a set of servers to do the job of another set of servers, and if that is the case, why not just have the first set of servers?
Do not forget that virtualisation is a strain on _absolutely all_ aspects of a system. The overhead simply does not make sense.
I mean, if 20 servers could be consolidated into one, then why not do that with the original configuration? Apparently, the original configuration must be absolutely lousy, or else what is the point?
Secondly, it is simply stupid to blindly lump servers together. If you want to lump intranet servers together, for example to provide centralised print and storage servers that possibly service outside requests, it is still alright. If you want to lump internet facing servers together, then I’d rather you just buy a gun and shoot yourself in the foot. Even if you really want to lump servers together, please ask yourself why you would rather run multiple instances of <insert your OS that is never going to be 100% efficient> instead of running different services within one big OS that will control all your resources with a lot more precision?
If you have too many CPU cycles to spare, go help folding@home. Don’t waste your time on virtualisation. We simply just need live migration of services, not live migration of OS instances, for one OS to completely dominate virtualisation for _everything_!
You must being to see what virtualisation is good for, and only what it is good for! That niche it is good for is to test, and to run wildly different OS systems on one machine (and for the latter, it should purely be because you have too much resources to spare on one system or you are too poor to buy another server and either conditions are woeful).
EDIT: lumping internet facing servers together is stupid because you cannot do load balancing on one server, nor can you protect against invasions if you only have one physical server because infiltration of one virtualised server means that access to the physical server is imminent (because the virtualised server can initiate cheap connections to the physical server that will bypass the networks and be simply connected via loopback. Not to mention the obvious: hypervisors are also vulnerable targets).
Edited 2010-07-01 09:17 UTC
You must not have much experience with enterprise level virtualization or IT…
I’d wager mine are not under utilized. They’re busy doing stuff. My individual 1 U servers also have nice management faculties which allow for remote rebooting, with full access to the console.
Like I tried to say I think they are for businesses with static computational needs and under utilized servers.
Is it too much to ask for companies to list the reasons not to use their particular solutions, as well as the reasons to use them? It doesn’t help when “Industry Analysts” just parrot press releases.
You just don’t get the enterprise. Virtualization is the new trend to get your customers to spend money and it doesn’t cost you much. You can build datacenters, buy one server and sell 20 virtual servers. Your customers will not notice they don’t have real servers. Most servers are idle more often than not anyway. And when customer are unlucky and get slow performance, it’s time to sell another buzzword: physicalization.
Bingo! <insert rant about commenting system>
Am I the only one who thinks that the current situation of stagnated pool of manufacturers buzzing with <insert flying pest> sucks?
We need _real_ progress! Someone!
But that’s the whole thing – physically, when 1 server dies, 1 server dies.
But when a virtual server dies, we replace that hardware with NO downtime. It’s about continuity and continued service.
Plus replicating the data elsewhere is cake when it’s SAN stored VMs.
I don’t think its possible to replace a server, virtual or not with “No Downtime”. Whatever the server was working on when it died, will be killed with it. Unless you are doing a map-reduce style system with Hadoop or similar system where you send requests to multiple machines.
SAN’s are nice, but they can’t match the cost/performance ratio of local disks. But, if your costs have no restraints you can get some sweet fiber connected sans. Those are pretty darn cool.
Check out VMWare’s “fault tolerance.”
http://www.vmware.com/products/fault-tolerance/
1 physical server dies 20 servers die.
True, but you can duplicate those twenty servers to another physical server, providing failover capability for all twenty at a much lower cost. Providing failover capability for 20 physical servers has a rather large cost.
Visualize one more, unless you run out of space, then buy a server big enough for 20 more.
You probably shouldn’t be in a situation where just one more virtual server breaks capacity.
The alternative is to buy an additional physical server, and have yet another under-utilized box drawing power and spewing heat.
It seems like if you have a situation , where our servers are not under utilized consolidation becomes pointless.
That is correct. Heavily utilized servers should not be consolidated.
The point of virtualization is so those 20 servers that sit mostly idle can be put on a much smaller number of machines, saving power and cooling costs.
Which is one of the few reasons for virtualisation. However, can’t you see that if you have 20 physical servers sitting almost idle, your IT management is the one that is already at fault?
If you, instead of 20 virtual servers, consolidate them into 2 physical servers running the same OS on both and load balancing them, and implement service-level live migration such that when one server of the 2 dies, the service on the dying server live migrates to the good server, you would have this main benefit of virtualisation without the ridiculous overhead.
No matter what, you should be tackling the main problem instead of the symptoms. Management deficiency creates the 20 underutilised servers problem, and you should be tackling that management deficiency instead of working around that problem by virtualising and incurring overhead.
Virtualisation as a buzzword is mainly to make money for the big name brands.
Having two physical servers running 20 different services with load balancing and service-level migration only works if all your software runs on the same OS.
This is often not the case, however.
If you are trying to consolidate a variety of platforms into a smaller number of systems, virtualization is an appropriate solution.
Virtualization obviously isn’t a one-size-fits-all solution, but there are plenty of situations where it is useful, even with well managed IT.
Most services are not built with clustering in mind. You can have legacy stuff; specialized stuff etc…
Piling up tons of services on a single OS instance is just plain dangerous for various reasons. Imagine you install an update for one of the services and it kills your box. Imagine you need to reboot your system because you’re doing some maintenance on a particular service and you immediately cut service for everyone else; or you have to work late. And good luck when the time comes to replace your server.
Virtualizing an entire OS might not be too efficient because of the duplication of resource both in memory and on disk; but it’s still the best option we have right now.
I maintain a 5 nodes VMWare cluster; with a total of 142GB of memory and 24 cores. We currently have 28 VMs running but we could probably go past 40 painlessly; most are Windows but there are also a few flavors of Linux. Our storage is on a SAN connected by Fiber Channel and it’s fast; really fast. I won’t try to calculate the cost of all of that (including VMWare) vs. the cost of 28 individual servers; either way would fall in what I consider expensive.
On the other hand; whenever a VM needs more resource than its host has available, VMWare migrates it to a different host.
Whenever I need to install updates to VMWare itself or do some hardware maintenance, I just do a live migration of the VMs to another host.
I can setup a new server on my box using VMWare workstation and easily migrate it to the cluster if I feel so inclined.
When a VM is no longer needed, we just delete it and the resource is immediately available to all other boxes.
When a host dies, the VM is restarted within seconds on another host and we’re not even using the latest VMWare that runs two copies of a VM for improved redundancy.
Also hardware updates happen without ever shutting down a single VM. Simply install VMWare on the new server and add it to your cluster. Then you can migrate your VMs and eventually retire an old server.
Oh and there’s even a software Cisco switch for the latest VMWare so your network engineers can do all of their fancy advanced network stuff all the way to the VMs.
Virtualization is not a perfect technology and it’s not cheap either if you go with VMWare; but it has so many advantages that it’s definitely worth looking at if you manage a decent number of servers.
+1 informative
(The other one about inexperience with enterprise IT is not funny. I always hated enterprise IT anyway.)
Not true. If your services are live migrate-able and load balanced, then the downtime of one physical server only means slower access to the users, exactly the same as for the virtualisation case.
Not to mention that no amount of live-migration can handle someone kicking the power line. On the other hand, as long as the whole installation retains partial power, services can still run when load balanced, unlike VMs.
The above would negate most of the advantages of virtualisation, except for the notable case of different OS/hardware. Please note that live migration is but _one_ advantage, so you will have to expound on “many advantages”.
Keyword there is “if”. In the real world, even if you’re lucky enough that you have a single OS, you’re not very likely to only be running services that can be load balanced etc…
You will always have the possibility of your power going out. If your services are load balanced on raw metal then they’re likely to be keeping their data on a SAN or something like that. What happens when that SAN loses power? Yes, if one of my VM hosts loses power then the VMs on it die and have to be restarted on another host so it’s still not 100% reliable; unless I use that new feature from VMWare which runs the same VM on two hosts for redundancy. And like someone else mentioned, you can still run your load balanced boxes as VMs and get the best of both worlds.
I wouldn’t give up on all the every day benefits of virtualization just because there is a chance that someone kicks a powercord; which would be unlikely anyways since it’s snug in a rack. Plus with redundant power supplies one would have to stick their foot in the rack and kick two plugs.
This is not nice, because your virtualisation stuff is also talking about a lot of “if”s. EDIT: and those “if”s are important.
Come on, if you are competent enough to make load balanced single OS servers, load balancing and redundant SANs will be too easy to talk about.
About the running load balanced boxes as VMs, that is probably me, given that I’m so active and interested in this thing. I’m saying that running VMs on multiple hosts will always be less efficient and clean than running multiple services load balanced across multiple hosts.
Oh, this again. As I was talking about, please enumerate the “every day benefits” outside of the one benefit of live migration. It does not make sense that live migration has to be applied to VMs only. It also does not make sense to double-count, no, this is multiple-count, benefits. Even economists understand that double-counting is fundamentally wrong.
Finally, I shall summarise the same points yet again:
1) You must have multiple hosts to offer real load balancing and redundancy, VMs or not.
2) VMs are never going to be more efficient than the OSes at doing anything.
3) VMs, theoretically, have only one real advantage above OSes, and that is the ability to run another OS, which may depend on a completely different architecture.
4) live migration is a huge plus, but can still be, and should be, done by OSes, even though VMs do it natively right now.
5) your many benefits are mainly just pluses of live migration.
Edited 2010-07-02 16:12 UTC
Well then let’s just not virtualize anything and instead wait for that theoretical OS that will be capable to do live migration of services and resource capping for each of them; and of course let’s wait for the services that go with that. Oh and let’s rewrite all legacy apps that companies still run around the world…
In principle, I agree that virtualizing the entire OS is suboptimal. But in the mean time, I’ll leave you to the world of theory and I’ll take advantages of the real world benefits of a virtualized environment. If you’re unable to see those benefits, it might be because as you say you don’t like enterprise IT so I suppose you’re just not familiar with it altogether.
Some services are already live migrate-able. It is all just a matter of sane design and effort. Sadly, the OSes do not have the kind of functionality exposed as a clean API.
Also, Plan 9 and Solaris have something similar or maybe extend-able to get essentially live migration, but they are not big time players.
I’m not talking about theoretical wonderland here. I’m talking about right here right now. We are just inches away from there.
Actually I would disagree that there are even a handful of live migratable services. I am racking my brain right now trying to think of anything stateful you can live migrate. Maybe RAC but that’s really kinda cheating and connections must be reestablished….
I don’t think your definition of live migrate is the same as mine.
For example I can RDP into a VM and run any app and move the VM to another machine and not only will my data in memory stay up but I won’t even drop my connection.
We don’t even lose a ping during a live migration 99 out of 100 times….
Finally something concrete we can talk about.
I suppose the ability to not lose pings, and especially not drop connections, is just wonderful.
However, VMs themselves have stateful, so your first point is half answered.
I agree that it would be a great feat for pings and connections to be alive during the live migrations. However, as ventajov admitted, live migrations are usually done for slow deaths — this means that live migrating services can just not accept connections for a while before completing. That way, services-level live migration will achieve the same level of continuity as compared to VM-level. i.e. abrupt deaths still cause loss of connection and some data loss.
Practically, services-level live migration should be inferior to VM-level ones because it is quite a bit harder to do without OS support. The inferiority, however, would be paid for by performance.
First Disclaimer I work for HP and regularly implement most of these technologies for customers, Virtualization, Virtual Connect etc….
I describe Virtualization and VMWare as good enough computing. Which is really just an extension of existing IT dept practices of buying a single service for every service. However the reality is while the hardware has grown exponentially more powerful the user demand has not. Most IT shops have a 5 year recap cycle but the customer demands haven’t increased with the new server capabilities. Hence you had servers that where 20% utilized now 1 to 2 % utilized. So virtulization allows us to consolidate those workloads while maintaining the logical separation that IT has adopted. Luckily open systems virtualization has great features to really provide some benefits over a single box, live migration allows me to move a VM with no downtime between physical servers is a key one. Now with virtualization I can proactively fix hardware problems without bringing down services and without clustering complexities. Virtualization allows me to reserve specifically CPU and Memory resources in a pool to gurantee VM performance. Not to mention some customers love the single hardware platform for development to deployment…. It does simplify many things.
However Virtualization everything is not a good motto. Some things don’t scale\work\make sense to be virtualized and in those cases Bare Metal make most sense.
Virtulization will always have a place unless there is a radical change in the way mainstream OSes operate. Someday maybe but my guess it will always be easier to virtualize.
In fact I would argue we will see OS’s change to virtualize better. Thinner OS’es with less complexity, just enough to host the application.
Nice.
Which is, again, management problems. Why do we already work around problems instead of tackling them head on as it should be?
What has more information with which to manage resources better, OS or hypervisor?
OS evolved so as to abstract away hardware in order to allow applications to work in as close to ideal conditions as possible and for applications to benefit from centralised abstraction of hardware. Now hypervisors are trying to abstract applications to allow live migration of software between physical hardware instances. And it does so by abstracting OSes? I really don’t understand why we don’t just have something like clusters that share workload where the OS itself helps the workload to slush around, instead of all these virtualisation rubbish.
The proliferation of frameworks mean that traditional OSes can only increase in bloat rather than decrease. It also means that the overhead of virtualisation is doomed to keep increasing and thinning is just the opposite. If you want to promote your virtualisation business, you really need to consolidate frameworks first.
As long as it less expensive to through everything on their own OS instance then to work out the kinks and resource contention at the OS level people will virtualize. Even the app vendors do it as the expect the machine to give them all it can. I can chop up at the hypervisor exactly how much memory and clock cycles a given vm gets. That is not a layer of granularity avaialble to me in most operating systems. For example take this example
One box running
MySQL
Mail
Apache
The MySQL database goes crazy for whatever reason and starts occupying a tremendous amount CPU time. Mail and Apache will be affected. Hence why people will put them on three different servers to protect the services.
I can instead VM those three server and say
VM1 – My SQL – 2 CPU’s at 1000MHZ and 4GB Ram
VM2 – Mail – 1 CPU at 500MHZ and 1GB Ram
VM3 – Apache – 1 CPU at 500mhz and 2GB Ram
Once virtualized I can move both the VM and Storage tied to it between hardware non disruptively. If I want a true HA solution I can even run them in lockstep on another box on VMWare. Heck even clustering works so I can cluster between different VM’s
It is indeed powerful features we get. Granted if Owned a big sunfire\superdome it may not make sense but that is a different market segment.
Remember competent IT engineers are not easy to come buy but I most people can set up vmware without too much effort and with hardware now a days it’s easy enough to buy your way out a problem versus engineer it corectly.
By the way I agree Virtualization is band-aid for a broken architecture. I think the reality is that the legacy cruft won’t let us do it the right way and right now I don’t see Windows and Linux giving us the same advantages at the host level to dial in the resource usage by process we need virtualization does. By the way this won’t change until the cost of Virtualization outweighs the “right way” Hardware is moving so quick most people have more Memory and CPU then they know what to do with. What they don’t have enough AC\Cooling\FloorSpace….
Edited 2010-07-01 18:23 UTC
I don^aEURTMt know anything about the enterprise side of things, but I always prefer an elegant solution. Our OSes have mostly failed us and we^aEURTMve come up with over-engineered solutions almost beautiful in their complexity.
I wonder what would have happened if Plan9 got where Linux is. All your resources across all servers and VMs could be freely accessed and moved around the pool just with symlinks.
Again, +1 informative.
I do not agree with the 3 server scenario, though. Theoretically even OSes allow you to assign resources to applications, so that example should be moot, unless, of course, the OS is itself incompetent. In fact, the resource should be parcelled out in quotas rather than just assignment because MySQL should get more than the assignment when the other 2 are just idle.
(bold for obvious mistakes, and I don’t have strikethrough for the extra “I”)
Competency is a problem throughout the millenia. No matter what time it is, leaders have found out that paying the extra to hire competent ones (paying in effort, face and much more than just money) pays off. This is because incompetent <insert occupation> wastes so much that the ecosystem itself becomes unsustainable.
Witness all those big name brands that almost completely stop innovating. Ford after Henry’s spectacular failure to move on from the model T, and IBM anyone? It is always the problem with management. Creative and competent types are the most difficult to retain because they know that they are scarce. The biggest problem with management is that they are trying too hard to tame volatility by policies that serve to both punish bad/dodgy behaviour and restrain the employees from creative production.
If you want to argue that competent workers are hard to come by, it smacks of incompetence (either as a manager or worker) yourself. Do not forget that you are in a recession, which is the best time to snatch talent. You get what you pay for, especially if what you pay for is clever work.