| Engineers at the forefront of today's software-based
emulation technology shun the word that best describes
their efforts. Not that their goal has changed — they're
still working on software that transparently runs
applications and sometimes operating systems on nonnative
platforms. But over the years, emulation has acquired a
reputation for poor performance and precarious
compatibility. The next-best label is translation, which
happens to be technically as well as politically correct,
because it describes a new approach that is paying off
with significant improvements in performance. The allure of emulation is decades old, dating back to 1964, when IBM's then-new System/360 could run programs written for the IBM 1401, albeit slowly. Emulation was invented to bridge the gaps that inevitably appear when computer hardware evolves more rapidly than software. If you can run your old software on a new machine, or run nonnative software on the host machine you own, you can postpone obsolescence and win a degree of independence from cross-platform incompatibility. In short, emulation can free you to run the programs you want on the computer you want, with little concern for any technical differences in the underlying hardware and system software. Emulation may sound like an impossible fantasy, but the technical challenge is not that great — if performance is not a factor. All an emulator has to do is convert the original binary instruction codes into equivalent instructions for the host CPU and simulate any other platform-specific dependencies that distinguish the two machines. It's stretching the point only a little to say that any computer can emulate any other computer. The real challenges that have dogged emulation for 30 years are to do all this while achieving near-native performance and to emulate the alien platform so thoroughly that even ill-behaved applications will run without crashing. Emulation does not have to occur through software alone, of course. Add-in boards with their own processors have been available for many systems for years. The first IBM PCs, for instance, could run CP/M with one of several Z80-based boards. This article will focus on software solutions to emulation because although the hardware approach provides better performance, it is too costly. Like nearly all programming problems, emulation can be tackled with clever software design and plain old-fashioned brute force. The latest-generation microprocessors put at least 100 MIPS on the desktop, and that kind of power goes a long way toward solving the performance problem, especially if the nonnative software is written to run acceptably on machines with significantly less speed. For example, if a program runs smoothly on a 33-MHz 386 that delivers 12 MIPS, it's a good candidate for emulation on a microprocessor that is cranking out 112 MIPS. The speedier CPU absorbs much of the overhead of emulation. Still, even the brute force of today's fastest microprocessors isn't enough to make all emulation problems go away. Clever software design still has its place. Some designers are now adapting optimized compiler technology to their latest emulation engines, achieving impressive gains in speed. New to this field, but a key player, is IBM, whose Instruction-Set Translator goes far beyond the traditional model of emulating binary code in a serial fashion. The IST applies a level of analysis and optimization that is unheard of in real-time emulation. Designers are getting enormous help from recent trends in the evolution of system software. The API layers originally designed to shield applications programmers from the complexity of the underlying operating system are a boon to emulators, which can take advantage of the API to further their agenda of hardware/software abstraction. API calls made by the nonnative program can be translated to similar API routines on the native platform, thus bypassing the tedious steps of instruction emulation. Taken together, these three developments — faster CPUs, refined emulation engines, and API translators — amount to a breakthrough in emulation technology. The first clue that a watershed was reached came last year, when Microsoft included an 80x86 emulator in the RISC versions of Windows NT. This lets computers with Mips R4000 and DEC Alpha AXP microprocessors run Windows 3.1 binaries compiled for Intel microprocessors. Also last year, SunSelect (Chelmsford, MA) announced Wabi, an API translator that runs several of the most popular Windows 3.1 binaries on Unix workstations. Microsoft countered by licensing one of its greatest corporate treasures — the Windows API source code — to Insignia Solutions (Mountain View, CA), a leading company in emulation technology. The result is Insignia's SoftWindows, which runs Windows 3.1 on major non-Intel platforms. This March, Apple (Cupertino, CA) took emulation technology still further — taking perhaps the greatest risk in its history. The new Power Macs (see "Apple, IBM Bring PowerPC to the Desktop" on page 44) run a PowerPC version of System 7 that includes a hybrid 68LC040/68020 emulator. The emulator provides backward compatibility with 680x0-based Mac software on the new RISC platform, much as the 80x86 emulator in the RISC versions of Windows NT provides backward compatibility with Windows 3.1 software. But Apple is also using the emulator to run portions of System 7. Thus, System 7 becomes the world's first recursively emulated operating system, relying on its built-in emulator to run parts of itself as well as existing 680x0 programs and native PowerPC applications. In a real sense, Apple is staking its future on the viability of emulation (see "The Power Mac's Run-Time Architecture" on page 131). API Translation Operating systems have long provided services to programmers for handling some low-level tasks that are specific to the underlying hardware. For instance, an operating system might provide routines for printing characters on the screen, reading incoming bytes through a communications port, or writing data on a disk. All the programmer has to do is pass the appropriate arguments to the routine, and the operating system does the rest. But the shift to graphics-oriented computing that occurred in the 1980s caused an explosion in both the number and sophistication of services provided by the operating system. When Apple introduced the Mac in 1984, programmers complained it was difficult to program. Suddenly, programmers were responsible for constructing a graphics front end replete with pull-down menus, windows, dialog boxes, and icons, all managed by a complex event loop. Everything had to conform to a series of rigid "guidelines" laid down by Apple. To make the job a little easier, Apple equipped the Mac with the Toolbox, a rich library of operating-system routines that saved programmers from hundreds of the most onerous chores. Common Toolbox routines like NewWindow (draw a new window on the screen), FillRect (fill a rectangular area with a pattern), and CopyBits (copy a block of pixels from one screen location to another) isolate Mac programmers from many of the lowest-level tasks and are often highly optimized. In effect, the Toolbox imposes a thick abstraction layer between the programmer and the hardware. Apple strongly discourages programmers from bypassing the Toolbox because the program might break on future releases of the operating system. This model has been adopted by all modern GUI-based operating systems. The various Windows APIs, X Window System, Motif, Open Look, NextStep, GEOS, and PenPoint all have API layers that simplify the task of programming for a GUI while standardizing the look and feel of the application riding above it. If the API is ported to another CPU, programmers writing in high-level languages may not notice the difference. The API calls look exactly the same, accepting the same arguments and returning the same results. This trend plays right into the hands of emulators, which seek to impose another abstraction layer that isolates nonnative binaries from the underlying differences in hardware and system software. All APIs share a number of common functions, and an emulator can take advantage of that by mapping — or "translating" — API calls from one library to another. Alternatively, an API can be natively ported to the target platform or completely rewritten. Whether the API is translated, ported, or cloned, as long as it accepts the same parameters and provides the same services, the differences are virtually invisible to programmers writing in high-level languages. These approaches do not entirely eliminate the need to emulate binary instructions. There is always some code that doesn't call the API. Fortunately, the most time-consuming code (e.g., screen drawing) tends to rely heavily on API routines. Modern GUI applications spend enormous amounts of time redrawing the screen and performing other graphics chores, and almost all of that time is spent in the API. Performance profiles reveal that a typical Windows program spends about 60 percent of its time executing API calls, while the average Mac program spends about 80 percent of its time in the Toolbox. Mac software tends to be more API-intensive since the Toolbox is significantly larger than the Win16 API: more than 4000 calls versus 800 to 1500, depending on which extensions are counted. Look and Feel When the routines of one API are substituted for the routines of another, a side effect is that the program may acquire the look and feel of the host API. For instance, if Apple's Toolbox call for displaying a pull-down menu is mapped to the corresponding routine in Motif, the result is a Motif-style menu, not a Mac-style menu. The Mac program adopts the appearance and behavior of a program that's written natively for Motif. An example of this approach to emulation is the Equal Application Adapter from Quorum Software Systems (Menlo Park, CA). Equal allows you to run the Macintosh versions of Microsoft Word and Excel on a Unix workstation under Motif or Open Look. Quorum spent three years laboriously translating more than 4000 Toolbox routines into their Motif, Open Look, and X equivalents. A 680x0 emulation engine handles non-Toolbox code. Equal makes no attempt to emulate the entire Mac OS. Instead, you launch the Mac applications directly from the Unix desktop. "People don't want to buy system software, middleware, because it doesn't do anything," says Nick Sturiale, Quorum's marketing director. "What they really want is an application that works." For this reason, Quorum has taken an application-based approach. Depending on your point of view, this approach is either an advantage or a disadvantage. The emulated program looks and feels like a native port, integrating more smoothly with the host environment. Sometimes the emulated program runs even faster on the host than it does on its own platform, if the routines in the host API are more efficient than those in the original API. But some users don't want to lose the original look and feel of their applications, and they may also prefer to run the complete nonnative environment — operating system, GUI, and all. For those users, there's Liken, a Mac-on-Unix emulator from Andataco (San Diego, CA). Liken strives to preserve the Mac's distinctive look and feel. It runs the Finder within an X window on the Unix desktop, and the Mac session works like a Mac in almost every respect. You can even launch multiple Mac sessions in their own independent X windows. Liken takes a broad approach to API translation. First, it takes advantage of the fact that before System 7, Apple freely distributed copies of its system software through users groups, BBSes, Internet FTP sites, and other sources. Thus, Liken requires a copy of System 6. However, large chunks of the Mac OS are also located in ROM, and you can't just plug Mac ROMs into a Unix workstation. So about 1500 ROM-based Toolbox routines were rewritten with more than 90,000 lines of code, mostly in C. Those routines include all or parts of the Mac's HFS, boot-up procedures, Resource Manager, and Font Manager. Toolbox calls that aren't critical to look and feel are mapped to underlying routines in Unix's CLib and X's XLib. There is also a 68000 emulation engine that handles applications and parts of the Mac OS that Andataco didn't rewrite. As a result, Andataco says Liken runs as fast as a low-end Mac Quadra on a Sun SparcStation 10 and as fast as a high-end Mac Quadra on Hewlett-Packard's best PA-RISC box. Preserving original look and feel was also a key goal of SunSelect's Wabi, a Windows-on-Unix emulator. Wabi has a 386/486 emulation engine to execute non-API code, plus a library of work-alike Win16 routines that bypass Motif. Thus, a Windows application retains its normal attributes under Wabi. Hundreds of those work-alike calls that don't affect look and feel are mapped to corresponding routines in CLib and XLib, letting some Windows operations run faster under Wabi than they do on native PCs. To demonstrate this, SunSelect ran the Texas Instruments WinTach 1.2 benchmark program on two identical 50-MHz 486 PCs, one with Wabi and the other with Windows 3.1. Wabi scored an RPM (relative performance measurement) of 19 versus 12 for Windows, a difference of more than 63 percent. (One RPM equals the performance of a 20-MHz 386.) WinTach is highly API-intensive, so it's a good measure of relative API performance, although not necessarily a good benchmark of overall performance. One drawback of Wabi is that it does not duplicate the entire Win16 API. Instead, it implements a core API — including some calls undocumented by Microsoft — required by thirteen "certified" Windows applications. Those programs include popular packages from Lotus, Microsoft, Borland, WordPerfect, Corel, Aldus, Software Publishing, and Datastorm. Wabi will run additional programs that aren't certified, but the results are unpredictable. Wabi also doesn't include the Windows desktop (i.e., Program Manager and File Manager), although it can run PROGMAN.EXE and FILEMAN.EXE if you have a copy of Windows. SunSelect expects most Wabi users to manage files from their Unix desktops. Also missing from Wabi 1.0 are multimedia extensions, sound (except for simple beeps), and PC networking. "Official" Emulators Third-party products like Wabi, Equal, and Liken are prodding the operating-system vendors into action. Microsoft and Apple want to ensure that their competing operating systems will spread to nonnative platforms on officially sanctioned emulators. Microsoft is largely delegating the job to Insignia Solutions, while Apple is readying its own emulation technology. All this neatly dovetails with the cross-platform strategies of Microsoft and Apple. Windows NT is already spreading the Windows GUI to different CPU platforms — including DEC Alpha, Mips R4000, and PowerPC — with integral 80x86 emulation. Apple is working on Mac emulation for Unix and has decided, for the first time, to openly license the Mac operating system to PowerPC clone makers (see "Apple Opens the Mac OS" on page 24). Microsoft wasted no time making its intentions clear. No sooner had SunSelect announced Wabi than Microsoft stole some of the thunder by revealing it had licensed the Windows API source code to Insignia. Insignia was already a major force in the emulation market, thanks to its SoftPC series of DOS and Windows emulators for Unix and the Mac. But SoftPC relies heavily on instruction-level emulation, and its performance is snail-like on all but the fastest machines. With access to Windows source code, Insignia can port or translate the API to run at native speeds on any RISC platform. Another bonus is that Insignia can ship its product with the latest versions of MS-DOS and Windows. The result is Insignia's SoftWindows, the successor to SoftPC. SoftWindows began shipping late last year for some Unix workstations and debuted in March on the new Power Macs. Its performance approximates the feel of a fast 386 or a low-end 486, depending on the speed of the host CPU (see "Windows on RISC" on page 109). Naturally, SoftWindows does much better on API-intensive tasks, because those calls can either run natively or be mapped to host API routines that don't alter the look and feel of Windows. Non-API code is pumped through the same 80x86 emulation engine that's found in SoftPC. Access to the Windows API source code isn't a magic bullet, however. Insignia faces a challenge similar to Apple's port of the Toolbox to the PowerPC — and for many of the same reasons. Both the Win16 libraries and the Toolbox trace their lineage back a decade or more, which means there's lots of old code written in mixed languages by long-gone programmers. The most recent parts of both are written in C, but older chunks were written in Pascal, and the most critical routines were hand-coded in assembly language. The assembly routines tend to be highly optimized, resorting to every trick in the book, including self-modifying code. And, of course, the documentation for this old code ranges from adequate to hopeless. As with Apple's port of the Toolbox, therefore, Insignia's port of the Win16 API is not entirely native in the first release. In fact, much of Win16 in SoftWindows is emulated in the usual way. Porting this code (or remapping it to the host API) is an ongoing effort that over time promises to reap even greater gains in speed. Insignia is also revamping its core emulation engine, which currently simulates an Intel 286/287. SoftWindows can't run some programs that require 386 instructions (e.g., Borland's Paradox for Windows) or Windows 3.1 in enhanced mode. Without enhanced mode, you can't have a virtual memory swap file, so you must allocate more RAM to run major Windows applications. For Power Macs, Insignia recommends allocating a total of 12 MB to SoftWindows: 5 MB for the emulator itself, 4 MB of simulated extended or expanded PC memory, and 3 MB for a special cache. Insignia hopes to have a 486 emulation engine ready this summer. Meanwhile, Apple is pursuing Mac emulation on Unix. Because the Toolbox is about three times larger than Win16, it's much more difficult to port. The longtime absence of Mac clones has as much to do with the scale of this project as with the aggressiveness of Apple's lawyers. The closest attempt to date is a hybrid operating system from NuTek U.S.A. (Cupertino, CA) that duplicates most of the function, but not the look and feel, of the Mac OS. NuTek remaps the Toolbox to a 680x0 version of Motif. Apple wants to preserve the Mac's look and feel while running a Mac session in an X window — much as Andataco's Liken does, but with full support for System 7 and its latest extensions. Now that System 7 is up and running on the PowerPC chip, Apple is retooling it for Unix under the name Macintosh Application Services. MAS is an optional layer that will ride atop future Unix operating systems certified by the PowerOpen Association (Burlington, MA). It supports a fully functional System 7 session in an X window and runs both 680x0-based and PowerPC-based Mac software. In addition, on March 14, Apple introduced a new Mac-on-Unix emulator called the Macintosh Application Environment. MAE brings a full implementation of System 7 to Sun SparcStations running Solaris 2.3 and HP Series 700 workstations running HP/UX 9.0. As with MAS, MAE floats in an X window on the Motif or Open Look desktop. Some Toolbox calls are mapped to XLib and CLib for near-native performance, without sacrificing the Mac's look and feel. Unlike MAS, however, MAE 1.0 runs only 680x0-based Mac software, not the latest PowerPC-based applications. Emulation Engines API translation yields a quantum leap in performance but still isn't enough. Even the most API-intensive program spends time executing binary code that can't be mapped to high-level libraries. Handling that code requires an emulation engine to translate the program's native binary instructions into corresponding instructions on the host CPU. The emulation engine also has to simulate all the other functions of the original processor — registers, interrupts, status flags, privileged modes, and so on — plus other differences between the two platforms (e.g., absolute memory addresses referenced by programs). This is where emulators have historically bogged down in performance. One problem is that different microprocessors have different instruction sets, so translation isn't straightforward. Sometimes it takes dozens of instructions to duplicate the function of a single original instruction. Microprocessors also have widely varying arrangements of registers, status flags, and interrupt mechanisms. All these differences can be resolved by writing more emulation code, but the overhead soon becomes prohibitive. Another problem is that straight emulation works much like an interpreted programming language. Incoming binary instructions are translated serially, and repeating sequences of code have to be translated again and again. For example, consider how the QuickBasic interpreter that comes with MS-DOS would handle the following simple loop: For Loop = 1 To 100This code fragment prints Hello on the screen 100 times, and for each pass through the loop, QuickBasic must interpret the Print instruction all over again. A traditional emulator has the same limitation when it's translating repeated sequences of binary code. To attack these problems, emulation architects are increasingly turning to compiler technology. An emulation engine, after all, is essentially a code generator, like the back end of a high-level language compiler. The main differences are that a compiler has the advantage of working from source code instead of binary code, and it doesn't have to build its binary while simultaneously executing the code. Compilers are free to make multiple passes through the source code, carefully resolving branches and address offsets. An emulator has to translate and execute a stream of binary code on the fly. Again, the unprecedented speed of today's advanced microprocessors is coming to the rescue. On-the-fly compilation techniques that were unthinkable a few years ago are now being adapted to emulation engines. Insignia Solutions and IBM are at the forefront of this technology. Both companies are taking similar approaches. Insignia's SoftPC emulation engine and IBM's IST depart from the traditional model of serial emulation by storing blocks of translated code in a special cache for later execution. Both caches have mechanisms for discarding blocks of code that aren't needed anymore, making room for new blocks. Consider the QuickBasic example above. Instead of laboriously translating Print "Hello" 100 times, Insignia's emulator would notice that the code fragment is within a loop. After translating the fragment the first time, it would store the recompiled binary in the cache. On subsequent passes through the loop, the emulator would hit the cache, thus eliminating the overhead of retranslating that piece of code. This technique yields big gains in performance. Insignia refers to its cache as the delta cache. When running SoftWindows with the recommended 12 MB of RAM, the delta cache is 3 MB. Allocating more than 3 MB yields diminishing returns, although it's difficult to measure, because it partly depends on the instruction mix of the emulated program. IBM appears to be pushing the caching technique even further. The IST caches blocks of code whether they are frequently repeated or not and then "wires" the blocks together for execution. It's remarkably like a multiple-pass, optimizing compiler. In a process that's called discovery, the IST analyzes the original binary to construct a directed graph representing the program's flow of execution. In concept, the directed graph is like a flowchart, with each series of related instructions divided into basic blocks. Typically, the basic blocks are delineated by branches, jumps, calls, returns, and other instructions that redirect program flow. However, if a jump or call references a target address within 256 bytes, it falls within a basic block and discovery continues. In any case, the build of the directed graph stops when the number of instructions hits an arbitrary limit that balances the benefit of deeper analysis against additional overhead. Larger blocks might be desirable for optimization, but they would take too long to create. In the next step, analysis, the IST optimizes the code while translating it. In some cases, it might fold two or more 80x86 instructions into a single instruction on the target CPU. In other cases, the IST is intelligent enough to recognize frequently repeated pairs or triplets of 80x86 instructions, such as CMP (compare) followed by JNE (jump if not equal). These common sequences — or idioms — are quickly translated into their RISC equivalents. Another optimization eliminates redundant CPU condition states. For example, if a series of ADD instructions that repeatedly set the 80x86 carry flag are followed by a CMP instruction that generates all new condition codes, there is no need to simulate and track the condition-code states for the ADDs. IBM says this optimization really pays off in reduced overhead, because almost every 80x86 instruction affects some kind of flag. Finally, the IST stores the translated and optimized blocks of code in its special memory cache for execution. Soon the cache holds the referenced sections of the original program, reducing the need for further discovery and optimization. Amazingly, all this analysis, optimization, and code generation happens on the fly, while the program is running. IBM says the IST currently delivers the SPECint performance of a 486DX/33 CPU when running on a PowerPC 601 processor, approximately one-ninth the performance of native code on the PowerPC. With further refinements, the goal is one-third the performance. Future versions of the IST might spawn a separate thread that continues to optimize the cached code, even while the discovery and translation steps are executing on another thread. One-third to one-ninth of the original CPU performance may not sound impressive, but remember, it's worst-case emulation: pure 80x86 code that doesn't call high-level API routines. Ironically, Windows programs may run faster than DOS programs under this kind of emulation. The IST is new technology that will probably show up later this year in IBM's version of Wabi. The IST may also appear in future RISC versions of IBM's Workplace operating system. Although Insignia and IBM are making great strides with code caching, the idea may not work as well when it's applied to other platforms. Dave McMillen, who is manager of software technology for Andataco, says code caching isn't nearly as effective when emulating the Mac. As a legacy from the days when the Mac had only 128 KB of RAM, Mac programs use a great deal of indirection to shuffle blocks of code and other resources around in memory. If an emulator translates, caches, and attempts to use this code later, it may find that references to memory addresses are no longer valid. The cache would have to be flushed and the code retranslated. These cache misses happen so often, says McMillen, that little or nothing is gained. Nevertheless, Liken does take advantage of other on-the-fly techniques to boost performance. The approach is becoming more and more commonplace, and programmers who are experienced in writing compiler back ends are becoming a frequent sight in the labs of emulation developers. Optimized compiler technology holds even more promise for the future. As microprocessors keep getting faster, emulators will benefit not only from linear gains in brute force, but also from the additional on-the-fly optimizations that become possible. And as APIs mature and encompass more of the same high-level functions, translations from one library to another will become more straightforward. Someday soon, you may be able to walk into a computer store blindfolded, pick a software package off the shelf, and be sure it will run on your computer — no matter what kind of computer you prefer to use. Comparison Of Software-Based Emulators
†Future versions of SoftWindows for Unix platforms will allow users to select between the Motif and Windows GUIs.  Illustration: Emulation in Apple's PowerPC System 7. System 7 for PowerPC relies on built-in emulation not only for backward compatibility with 680x0-based Mac software, but also for its own Toolbox routines that aren't yet ported. Even native PowerPC programs call some Toolbox routines that must be handled by the 68LC040 emulation engine. Also, due to the effect of call chains, some native Toolbox routines call 680x0-based routines. If Insignia's SoftWindows is installed, some Windows code passes through two emulation layers: the SoftPC 80x86 engine and the 68LC040 engine. Illustration: Three Approaches to Emulation Illustration: Andataco's Liken runs the Mac Finder within an X window under Unix. Illustration: The Equal Application Adapter from Quorum runs Mac applications under Motif or Open Look, losing the Mac look and feel. Illustration: IBM Instruction-Set Translation. IBM's instruction-set translation technology works like on-the-fly compilation. The stream of Intel 80x86 instructions from the application is first analyzed and divided into logical blocks of code. Those blocks are then translated into RISC code for the host platform, "wired" together in proper sequence, and stored in a special memory cache for subsequent execution. IBM says a fast RISC-based computer can carry out all these operations without perceptible degradation of performance. Tom R. Halfhill is a BYTE senior news editor based in San Mateo, California. You can reach him on the Internet or BIX at thalfhill@bix.com. Letters / August 1994Emulation AlternativesTom R. Halfhill's article "Emulation: RISC's Secret Weapon" (April) fascinated me. But I have a question: With all the effort going into writing good run-time translators and emulators, why not put that effort into writing a good batch-mode binary-code translator? Instead of translating at run time every time an 80x86 program needs to be run, such a translator would just translate the entire program once and store the translated binary on disk. This way, the run-time speed would be faster because no time would be spent translating on-the-fly, and more effort could be devoted to optimizing the output code. With this technique, virtually any program written for the 80x86 could easily be ported to the PowerPC (barring legal issues). I hope that someone reading this letter will develop a program to do this. I think it would solve the compatibility problem — and make my life using the PowerPC easier as well. Peter Shell Pittsburgh, PA Several other people wrote with the same idea, and it's already been done. Echo Logic (Holmdel, NJ) has a tool called FlashPort that translates 680x0 binaries into PowerPC binaries. Some Macintosh developers are using it to port all or parts of their 680x0-based software to the new Power Macs. DEC has a similar technology that translates legacy software written for its minicomputers to the Alpha-series RISC processors. It's not trivial, though, and there are legal issues that emulation neatly sidesteps. — Tom R. Halfhill Copyright 1994-1997 BYTE |