The SoftPear Project aims to create compatibility software between the IBM PC and the Apple Macintosh architecture. This is not an emulator for PowerPC Macintosh machines (like “PearPC”, “Basilisk” or “UAE”), the project has more similarities to Digital’s “FX!32”, FreeBSD’s “Linux Binary Compatbility” and “WINE” instead. Darwin/x86 or GNU/Linux will run on the PC, and the Mac OS X user interface, its libraries and all applications running on top of it will run on Darwin or GNU/Linux, using SoftPear’s compatibility layer. Since today, the PowerPC to i386 recompiler works with many small test programs. The code is in the CVS.
SoftPear’s PowerPC to i386 Recompiler Now Works
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSAlert.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
>This is not an emulator for PowerPC Macintosh machines (like “PearPC”, “Basilisk” or “UAE”)<
UAE is the *nix Amiga Emulator, not Macintosh
Now try going the other way
.
More cool Alpha technology that DEC took with them to their graves. FX!32 was awesome — it’d let a 500MHz Alpha emulate a ~300 MHz PII, back when a 266MHz PII was the fastest thing on the block. If SoftPear can get similar performance, it could be quite exciting indeed.
FX!32 looked for specific compiler structured code blocks and replaced them with native code blocks. It was only fast as long as a specific compiler was used to make the code. It had know to exactly how the compiler created code sequences.
Hmm, this is an interesting project, however, it seems more likely to me that a straight powerPC emulator will have more success in the race to get OSX on x86.
I’ve heard of dynamic recompilers used in many console emulators. Is this the same idea?
now everyone can run MorphOS on WinTel! :B
Just kidding, we know everybody wants to run OSX.
Good job.
I like to here about things like this. It is a good intellectual exercise.
IF the AMD Opteron is a more powerful processor than the G5, and a good recompiler was created based on the powerPC, wouldn’t it be possible to get a better OSX machine than a native G5?
this is sort of like fink?
“IF the AMD Opteron is a more powerful processor than the G5, and a good recompiler was created based on the powerPC, wouldn’t it be possible to get a better OSX machine than a native G5?”
Doubt it. There’s a lot more to it than the processor.
This is a nasty little project. Going from i386 NT to Alpha NT as DEC did is actually quite a bit easier. It’s pretty easy to emulate i386 on an Alpha, MIPS, or 32-bit PPC. Of course, NT also ran on all of those platforms, so the API was the same and predictible — w32 running little-endian.
All Mac code is big endian, so you’re having to change the bit order to make anything work. Icky. You also still only have four general registers to work with on an i386, while a powerpc has something like 32…have to juggle those around.
It’s still an interesting project; I wish them luck.
>UAE is the *nix Amiga Emulator, not Macintosh
UAE is also emulating the 68k CPU (among other Amiga-specific things), which is what the Mac used to run as well before the PowerPC times. Basillisk is the equivelant of UAE for the Mac platform, but they both have similarities in regards to the CPU they emulate.
Emulating a 33MHz 68040 based Mac is a far cry from emulating a 1GHz G4 based Mac. Also, no matter that the author doesn’t like calling it an emulator, if you have to translate the CPU, it’s an emulator.
I do like the idea of x86 native cocoa apps running on top of an emulated OSX layer. Then as more of the OSX layer becomes native, the speed increases without having to recompile the programs.
Whether dynamic or static binary translation, it’s no walk in the park. It’s an active area of research and no, I don’t think this project that is “just starting out” will have any chance of performing any better than PearPC which actually has dynamic binary translation (also known as just-in-time compilation) already. What PearPC doesn’t have is any form of optimisation of its translation cache. I expect this will happen sometime within the next year or so. As for a nice static binary translator, well, essentially you’re just doing decompilation followed by compilation on the new platform. If someone wants to code a PPC front end to Boomerang (boomerang.sourceforge.net) we could have PPC-to-x86 Darwin apps running in a few months with zero overhead.
Odd J.F. – from the documentation I read about FX!32 it did not require any compiler information to run properly. It simply profiled code, and then used the profile to optimize in the background. Do you have any references to this issue (drawback?) of FX!32?
Judging by how this is described they’re still pretty much an elaborate emulator. They describe themselves as a compatibility layer however if we read what they’re doing you bascically use linux or darwin/x86 as the base and on top of that run their software which will in turn let you load up MacOS X. It’ll be full screen and mostly seemless but this still seems like an emulator to me, just a very dedicated one. The only exceptional feature is the ability to run x86 apps that’ll directly access the real processor, (read will run MUCH faster.)
Still, WINE is a compatibility layer. Through it you run Windows apps inside X11. Through this thing you load OS X and then run OS apps in OS X. Sounds like an emulator to me.
Isn’t this the same guy behind XBOX-Linux?
if this thing is not an emulator, then it could only run individual applications, right? the comparisons with WINE suggest to me that the closest we could ever get with this software is maybe run Safari on windows, and not necessarily OS X in it’s entirety.
—
on a different note, does anyone here know if there is something similar to Transmeta’s code-morphing technology for the PPC architecture? maybe I’m trying the wrong keywords, but I couldn’t much info on this. the only thing I found were some references to IBM’s DAISY project.
to those people thinking this is an emulator, it’s not. it’s more like running wine on linux to load the stuff on top of dos that makes windows 3.1 (i dont know if this is possible..but it would be the same basic idea). saying that because osx is loaded then apps run on osx is false, because osx won’t be loaded, only the libraries and windowing systems, which will be run a top of linux or darwin/x86, not on top of darwin/ppc.
Years back when FX!32 was “hot”, me and my boss had some discussions with the folks working on it. It wasn’t anything too specific, so we didn’t need an NDA. It was a “normal” dynamic recompiler like everyone else does, but with a special feature – the initial recompilation did a hashing checksum which was used to recognize specific compiler constructions. If the hash flunked, normal translation occured. If it matched, a native version of the compiler code block was used instead. Since they had access to the compiler code, they could recognize and replace nearly every bit of compiled code with native subroutines. This made it blazingly fast. The thing worked best when you didn’t optimize the code. Just a straight compile.
Dude, yes it is. You are emulating the CPU, regardless of anything else. It’s a virtual environment with an emulated CPU. It’s not a FULL emulation – a virtual machine, but it IS an emulator.
Interesting… that sounds more like a hash binary translation lookup. From the docs, what would happen is the actual execution would simply profile the code and do a rough-cut dynamic recompile of the code.
After that it would use another program to optimize the code offline with the generated profile and then use code hashing to match code fragments (or basic blocks) and jump to the compiled code over running the dynamic recompiler again (which is slower). I’d imagine with common basic blocks (such as function entries/exits) you could have a preset compiled code to handle it as the hash lookup would enable you to do this (and could be per-compiler specific). I’d imagine a small speed hit for unknown compilers, though.
FYI : I checked out http://www.usenix.org/publications/library/proceedings/usenix-nt97/… to get a good idea about FX!32 (but it’s probably missing some stuff, if they had some extra shortcuts in there it’s probably not mentioned).
These days what with the amount of optimizations the compilers do the dynamic recompilers/binary translators certainly have a tough time since the assembly code’s pretty obfuscated now, half the job is working out what the programmer meant in the first place!
I see his name pop up all over the place. Didn’t he write that QubeOS 3p thing as well?
From what I gather, the code is translated to native i386 code, and will run natively on Darwin/x86. They are developing a loader for the code to work under Linux/x86.
WOW! This is the first really positive discussion I have read on OS News. It is also on a much higher intellectual plane. I am impressed. Thank you, all of you.
More cool Alpha technology that DEC took with them to their graves.
Not quite, I believe Intel are planning on using it for x86 emulation on the Itanium, the hardware emulator didn’t work very well so they’re trying this instead.
There’s an article which covers emulation and binary translation here (including FX32! and Transmeta’s code Morphing):
http://www.realworldtech.com/page.cfm?ArticleID=RWT122803224105
IF the AMD Opteron is a more powerful processor than the G5, and a good recompiler was created based on the powerPC, wouldn’t it be possible to get a better OSX machine than a native G5?
That depends on having a *very* good recompiler which I don’t think we’ll see any time soon. Also the G5 and Opterons strengths are different, the G5 seems stronger on FPU stuff and if it out-guns an Opteron on that no recompiler will fix that. A recompiler *may* give you the strengths of the target processor but it will definately give you it’s weaknesses.
Nice idea! But I could think of a better solution. this projekt will need new hardware drivers for OS X to be written, which is a lot of work.
I would love to see nativ x86 suport in PearPC, like it was introduced in WinUAE, so the emulator can switch to nativ mode if binaries are x86-copliled.
So you could use nativ darvin-kernel, some nativ drivers, and of course all OSS for OS X.
this would speed up the emulation dramatically!
No, you are thinking of Michal Stencl who wrote Uniforms, Seal (dos gui:s) and Qube (dos windows and linux gui).
I didn’t look at their code, so maybe there’s more substance here than i think there is. However, if these people really think what they are doing is comprable to WINE, then i’m not sure they really know what they are doing.
All WINE does is implement the win32 API calls. It doesn’t even attempt to emulate processor calls or act as a “virtual machine”.
It sound to me like SoftPear’s goal would be very benificial in getting PearPC to run more efficiently. They should be working together, not working seperate with only slightly seperate goals. Once we have an impressive PPC emulator/virtual machine, it’ll be a lot easier to make the “Compatibility Layer” the SoftPear people want to do.
Doesn’t Apple use gcc 3.x for the newer releases of OSX, which should help since we know intimately how gcc works?
Emulator: this is only vaguely an emulator. AS far as I know, what this software (and FX!32) does is look at a binary, and translate the assembly instructions from PPC to i386 (presumably a small subset of i386). You won’t find *as much* of a performance hit, since the CPU isn’t translating stuff at run-time.
Think of SoftPearPC as a program that translates a shell script into a perl script. You run sh2perl once, then run the program forever more as native perl. You run into some performance issues since there may be a better perl way (ie, ‘m//’ vs ‘if grep foo then’) to do things.
PearPC would be “SoftPearPC -o – script.sh | sh”, so you’re running the translation every time.
There was a project from AT&T way back in the day that took compiled motorola 68000-based code and translated it into PowerPC 601 based code. It was for Mac apps with no source code…you’d just plow it through flashport, and a PPC binary would pop out.
From what I heard, it worked really well. Go figure.
I can’t tell from the docs/whatever if the SoftPear a static recompiler or a dyamic recompiler. There’s actually no reason for it to be dynamic; a static recompiler can take its time, and it probably would make life much easier.
If you want to see a bunch of the issues involved, see:
http://softpear.sourceforge.net/doc.php
It’s amusing that endian issues are coming up. When I was porting stuff, that was always a major problem…and endian swapping is really hard to do automagically.
I wish ’em luck!
Think of it as a translator, like someone translating english to arabic. Or any translation between two non-directly related languages (ie not romance to romance).
[Quote]Think of it as a translator, like someone translating english to arabic. Or any translation between two non-directly related languages (ie not romance to romance).[/Quote]
…OR think of it as “translating” between two CPU instruction sets (ie PPC to ix86).
And guess what that is? an emulator.
The distiction the softPear people are trying to make is that it isn’t emulating the entire comptuter (ie. a “virtual machine” ie. bochs/vmware/virtualPC). Rather, it only runs a single program. HOWEVER. It will still need to emulate the instruction set simply because the software was compiled for PPC. It is not comprable to WINE because WINE only implements the win32 api so when a x86 program is trying to make calls to that api, WINE can fulfill those calls.
SoftPear will need to recompile the application at runtime, translate it to x86 and then implement the API calls. The nice part about this is you won’t need to install MacOS to still run Mac applications, and MAYBE, in the distant future, a ReactOS equivalent for OSX will exist.
>> Mike: …OR think of it as “translating” between two CPU instruction sets (ie PPC to ix86). And guess what that is? an emulator.
Nope. You are merely overgeneralizing.
>> Mike: The distiction the softPear people are trying to make is that it isn’t emulating the entire comptuter (ie. a “virtual machine” ie. bochs/vmware/virtualPC).
It doesn’t emulate anything because…
>> Mike: Rather, it only runs a single program.
…it doesn’t run anything. Emulators are designed to emulate computers; binary translators are designed to translate code from one type of computer to another. Some emulators do binary translation to increase the speed at which software runs emulate the machine, but this translation is not what makes it an emulator. A binary translator on the other hand merely scans a binary image, does NOT necessarily attempt to run it, and then writes a completed copy to disk. So while the techniques may appear to overlap at points, the purpose is entirely different, and the underlying implementation is quite different.
>> Mike: HOWEVER. It will still need to emulate the instruction set simply because the software was compiled for PPC.
Nope, sorry. It does not emulate the instruction set because it does not need to run the program. It just translates a file on a disk from one language into another language; you can claim that this involves emulating the instruction sets all day long, but that won’t make you correct. translator != emulator. Anybody with a basic education about programming languages should know this.
>> Mike: It is not comprable to WINE because WINE only implements the win32 api so when a x86 program is trying to make calls to that api, WINE can fulfill those calls.
Since we’re prejudging the project’s comments about WINE entirely on guesswork, I’ll just go ahead and assume the obvious: that they meant to prevent ignorant people from running around claiming that SoftPear was an emulator by pointing out that not everything which allows you to run software written for a different platform is somehow an emulator.
To answer the question if SoftPear is an emulator or not one has to know what an emulator actually is. Quoting Computer Organization & Design by John Hennessy and David Patterson, 2nd edition, page 424:
“Steward Tucker of IBM was saddled with the responsibility of porting software from the IBM 7090 to the new IBM 360. Thinking about the possibilities of microcode, he suggested expanding the control store to include simulators, or interpreters, for older machines. Tucker [1967] coined the term emulation for this, meaning full simulation at the microprogrammed level.”
So emulation, historically speaking, is full simulation (especially of the CPU) in microcode. Since it is quite hard to change the microcode of the CPU these days, and since microcode is legally seen a software, we can say that emulation is simulation of a system, or primarily it’s CPU, in software.
That covers interpretation of one system on another, but what about binary translation?
If it’s dynamic recompilation only code blocks that are “visited” during the execution of the program are translated and stored in the translation cache. The translation cache is thrown away when the program is terminated or when it is too full to hold further code blocks. So although code blocks do get translated you still need a special application that translates the original code every time you run it, ie. it’s still emulation.
You don’t think it’s right to say that dynamic binary translation is emulation? Then compare it to threaded interpretation. In this case code blocks get translated as well, but into jump tables instead of machine code. Each jump table represents the code block and transfers control to the routines in the interpreter that emulate the intructions.
The only difference between a threaded interpreter and a dynamic binary translator is that the latter generates a code block for each code block, while the former generates jump tables to interpreter routines (that can be written in high-level language, and thus be portable, unlike binary translation). Apart from that they share the same problem with self-modifying code, because both temporarily translate the original code into something else.
Since a threaded interpreter kind of is the evolutionary link between a normal interpreter and a dynamic binary translator, I think it’s safe to assume that all these methods are just different ways to implement an emulator.
What about static binary translation then?
If you manage to translate the whole binary of an application that it can run on a different platform without needing the original code anymore then that isn’t emulation anymore, but some way to port the software.
The problem is, it’s very hard to translate a whole application that it runs flawlessly. Just take indirect memory access via address registers, even after very precise flow control you cannot be sure that you know all possible values that the address register can contain, so you effectively don’t know which address will be accessed during runtime.
Due to this most static binary translators have a so-called “fallback interpreter”, which basically means that you a) still have to include the original code for the interpreter, and b) the application still won’t be able to run without an emulator.
Just for completeness:
What is a virtual machine?
The difference between a virtual machine and a real emulator is that the VM only simulates the previleged instructions and some additional hardware (like graphics and sound), but the majority of the code runs directly on the same CPU the host operating system is running on. Since there is no complete simulation of the CPU, this is no emulator. Due to this a VM only runs on the CPU it was designed for, unlike an emlator that can be ported (yes, with the right design you can even port a dynamic binary translator).
What is WINE?
WINE might appear a bit like a VM, but it doesn’t emulate *any* of the hardware, so it’s not even a VM. WINE just replicates the Win32 API under Linux (if the original sources were used it would be a straight port of the API) and features a PE loader to run the applications, since Linux uses ELF, not Microsoft’s Portable Executable format.
Now, let’s take a look at some of the stuff written by the developers of SoftPear:
We do not develop an emulator for PowerPC Macintosh machines (like “PearPC”, or like “Basilisk” or “UAE” for other platforms), though; the project has more similarities to Digital’s “FX!32”, FreeBSD’s “Linux Binary Compatbility” and “WINE”.
Several errors in just this one sentence:
1) UAE is not Macintosh but an Amiga emulator.
2) WINE runs user level applications on a different OS, but on the same processor.
3) FX!32 *is* an emulator.
A few more comments on FX!32 might be in order, since there seems a bit of false information around:
First of all, FX!32 interprets new applications, so it’s at least in part a typical emulator. The difference is that the interpreter collects profiling data for a binary translator. According to what I read the profiling data isn’t as advanced as it might sound, but is mainly locations of branch targets (to identify basic blocks) and unaligned memory accesses, because Alpha cannot handle these natively.
The binary translator doesn’t work during runtime, or at least it works independently, but it translates only the code blocks that were interpreted and stores them in a database. Thus FX!32 doesn’t have a dynamic recompiler, but rather a partial static recompiler. So if you’ve run every single part of the program, the translation database should contain translations of every single code block, and the interpreter theoretically isn’t needed anymore. Due to the problems with indirect memory access the interpreter is still needed as a fallback method, though.
OK, more from SoftPear:
Currently, we focus on two seperate sub-projects: Michael Steil is working on a PPC/x86 recompiler to make Darwin/PPC applications run on Darwin/x86.
This is a bit like FX!32, since that ran WinNT/x86 applications on WinNT/Alpha, with the difference that FX!32 has no dynamic recompiler. So this project of SoftPear does deal with an emulator, no matter how the authors like to call it.
One problem is that Darwin is just a part of MacOS X, so it’s a long way from being able to run MacOS X applications.
Tobias Bratfisch is working on on making Darwin/PPC’s Mach-O binaries run on Linux/PPC. Therefore we will need a Mach-O loader as well as an implementation of the dynamic linker for Linux.
Here is where the similarity to WINE comes in, because you have a special loader and probably some porting of the API, if it cannot be handled by mapping Darwin system functions to Linux system functions.
But again, this is like WINE for Darwin, but not for MacOS X yet.
Basically I think these are two totally different projects that will only share the MacOS X API if they should be finished one day. Apart from that they don’t have anything in common. To have the same name for two totally different approaches is kind of confusing.
SoftPear is not a traditional emulator. What SoftPear does is called binary translation. Emulators like PearPC simulate a complete hardware architecture including CPU and I/O to make an operation system think it is running on the real hardware it was designed for.
What the authors of SoftPear must have missed is that fact that PearPC has a binary translator for x86 (although not totally finished yet), so while it does have an interpreter, it also generates dynamic code for the same processor SoftPear does.
Thus the main difference between the two is that PearPC is a low-level emulator, which means that it is capable of running various operating systems for PPC, and SoftPear features high-level emulation that only enables it to run Darwin applications, and probably MacOS X apps in the future.
Both approaches serve different needs, but they both are emulators – if we leave out the WINE-like sub-project of SoftPear, which is something totally different.
One interesting notion is that the authors talk of dynamic and caching recompilation. While dynamic recompilation always temporarily caches translated code, otherwise it would be slower than an interpreter, I don’t think they mean the normal translation cache with that.
My guess is that they want to use a translation database similar to FX!32’s, which combined with the fact that they don’t have to emulate the operating system, because they are porting the API instead, shoul lead to a far better performance with MacOS X applications than PearPC when the project is done.
But it doesn’t change the fact that this part of the project deals with an emulator, no matter how sophisticated or high-level it might be.