Selecting a Real-Time Operating System

by Greg Hawley

The decision to buy a real-time operating system rather than create your own is hard enough. But the decision on what to buy and from whom presents even more variables.

Embedded software engineers insist on developing applications almost completely from scratch. Why is that? We might do well to take a cue from our friends the hardware engineers. When they start a new design, they select off-the-shelf integrated circuits and only design custom logic as a last resort. One of the first steps toward a goal of reusing other people¡¯s work is selecting a real-time operating system (RTOS). While there are quite a few issues to consider as you choose an RTOS, an organized approach makes success reachable.

Do I really need a real-time operating system?
Before jumping headlong into a discussion about selecting a real-time operating system, many of you may be asking yourselves why you even need one. Are all embedded software systems best served by a real-time operating system? Of course not. Many products are simply not large enough or complex enough to demand the extra overhead.

The debate of whether to use an RTOS is a lot like the debate whether to use a high-level language. Just like high-level languages, RTOSes allow you to develop applications faster. They can require a little more overhead, but as the technology improves, the overhead seems to diminish. When the marketplace initially opened, good tools were not available, but we are long past that point.

Just as some applications still warrant assembly language, there are applications simple enough that require only limited operating system services. In these cases, more simple structures¡ªlike round-robin scheduling of state-machine based functions¡ªmight be sufficient. After all, would you really expect a real-time operating system in your bread machine? Beyond that, RTOSes should be considered.

Build vs. buy
Within the embedded world, a lively discussion brews over whether a team should buy or build its real-time operating system. The unfortunate fact of the matter is that, like any religious war, there is a frightening lack of good statistics. That aside, buying your RTOS, in most cases, is the better choice. As I say this, please realize I have no personal or professional stake in any company in the RTOS industry.

The argument for buying your RTOS can be made with a simple anecdote. In the not too recent past, I was on a team developing the embedded software for a medical device. We used CMX Company¡¯s CMX-RTX. In the continuum of choices available to embedded developers, this OS is typical in the feature set it provides users. The source code, provided with the OS, contains about 11,000 raw lines of code. Keeping that in mind, can you imagine specifying, designing, implementing, and testing such a module for the couple thousand dollars CMX Company charges? Unlikely.

Nevertheless, advocates make compelling arguments for building an RTOS from scratch and for using an existing, proprietary RTOS. Where performance is absolutely paramount, writing your own real-time operating system might allow you at great expense to squeeze out a few percent more speed improvement.

Further, certain industries have specific regulations or standards they demand from software. In some cases, off-the-shelf operating systems may not meet those needs (such as avionics, medical devices, and safety systems).

KADAK Products Ltd.'s RTOS
KADAK Products Ltd.'s real-time operating system, AMX, includes a Windows-based configuration manager.


Finally, the installed base of existing embedded systems code using proprietary code is certainly large. Ripping up old code to use a new operating system would rarely be a wise move. One case where this often makes sense is when you are moving a product to a new microprocessor. If engineers haven¡¯t ported the proprietary RTOS to the new microprocessor, this might be a good time to consider using an off-the-shelf operating system.

Tool interrelationships
Engineers choosing their real-time operating system without considering the rest of their tool set are tempting fate. Microprocessors, in-circuit emulators, compilers, assemblers, linkers, debuggers, and simulators all¡ªin one way or another¡ªinteract with the operating system.

Some in-circuit emulator (ICE) vendors provide software that allows their in-circuit emulators to interface with real-time operating systems. Check to see if your ICE works with your RTOS. This can be useful when debugging the trickiest of bugs. It is, however, important to understand the performance implications of the in-circuit emulator¡¯s operation. Sometimes extra overhead is incurred when your ICE performs operations, like breaking on a given line of source code when that line is executed in a certain task.

For any given operating system in a given microprocessor family, it¡¯s likely the OS vendor supports only a subset of the available compiler chains (compiler, assembler, linker). Make sure the vendor supports yours. One of the first developments our group did with an off-the-shelf real-time operating system suffered one of the disasters you¡¯ll want to avoid. The selected RTOS vendor provided the OS to us in source code form. What we failed to consider was that this RTOS had never been bolted to the compiler we were using. After six weeks of hard work, the engineer responsible for modifying the source for the RTOS finally brought it up.

Selection criteria
Beyond the interaction with other tools in your development toolbox, if you organize the information you gather during your RTOS investigation, the selection process is a little easier. Start by creating a short list of possible RTOSes. When it comes time to select your RTOS, you¡¯ve probably already selected your microprocessor. Given this, the quickest way to create the short list is by immediately ruling out OSes that don¡¯t support your MCU. If you have selected the ubiquitous 68000 or x86 families, you¡¯re going to need to find some additional screening criteria.

After you have a short list, the hard work really starts. First, you need to figure out which criteria are really important for your application. While this article discusses several important features that have repeated on our selection matrices, each development is a little different, requiring a careful examination of what is truly important. From your list of chosen criteria, make a table. Evaluate each RTOS based on the listed items. Even after you¡¯ve filled out the table, the decision will probably still be hazy, as these things are rarely cut and dried. Everyone involved in the decision-making process for your development will need to meet to discuss the findings in your table. Out of that discussion should, however, come a decision¡ªor a plan for a decision.

There are two basic contexts in which you need to select your RTOS. The first basic group of criteria revolves around the details of a specific product. What are you going to use for the product you¡¯re working on right now? Making all of your decisions based on this simple, short-sighted vision probably isn¡¯t best. Expand your vision to consider your company¡¯s whole product line. If you do this, you need to look at the compatibility of your RTOS, perhaps with an entire product line. Will the RTOS be around for the years to come? Will the RTOS be compatible with the various microprocessors you expect to use?

With a very limited feature set, you can generally build the constructs you need, but that partially defeats the purpose of buying an RTOS off-the-shelf. When it comes to an RTOS¡¯s architecture, a couple of issues consistently seem to foil developers. Among them are the following: can the RTOS create and delete tasks dynamically? Can a task wait on multiple events simultaneously? How many priorities can tasks have?

Short of designing your entire application, it can be difficult to predict which services you will need. Generally, most basic features allow you to do what you want. If you are having difficulty, use the vendor¡¯s technical support and applications engineers as a sounding board. Also, if you already have experience with RTOSes and are bringing in another new OS, look for the features with which you are familiar. Vendors often have different styles for solving the same problem. Choose a style that¡¯s similar to what you¡¯ve done in the past.

Footprint
Real-time operating systems can be fit in surprisingly small memory footprints. Nevertheless, when OS vendors give a minimum footprint size, it is important to understand what is included in that minimalist build. Often the smallest build only includes an extremely limited set of features. Typical builds can be significantly larger. If your design is tight for ROM or RAM, ask for clarification. Sometimes vendors can give you a table that details how much RAM and ROM various services require.

Performance
In as many projects as not, performance is a big issue, but understanding an RTOS¡¯s impact on your system isn¡¯t easy. When you compare vendors¡¯ benchmarks make sure you understand what they are measuring. What evaluation boards did each vendor use? What was the microprocessor¡¯s clock speed? What memory system was used? How many wait states were used for memory accesses? Only then can you make a fair comparison.

Several performance modeling tools, from vendors like Tri-Pacific Software and CARDtools Systems, can help you model your system¡¯s performance. If you use these tools, understand that the tool¡¯s output is only as good as the estimates you give it. As your design tightens up, continue to refine your performance model.

Software components and device drivers
At the Embedded Systems Conference in November 1998, Jerry Fiddler, co-founder of Wind River Systems, painted the picture of the next decade of embedded systems as networked, ubiquitous, and common devices. Computers will be everywhere, but their presence will be inconspicuous. For this vision to come true, embedded systems will increasingly develop a need to interoperate using a variety of standards. To make all of this happen, developers will need to rely on components developed by others. If your application requires communication protocols, services, libraries or other components (such as TCP/IP, HTTP, ftp, telnet, SNMP, CORBA, and graphics), consider their availability. Similarly, in designs where off-the-shelf boards or ICs are used, determine if device drivers are available.

Some operating systems vendors provide these features or drivers either as a part of the operating system or as optional packages. In other cases, these services are available from third party vendors. As you talk to vendors, try to learn what is required to integrate the component into your system.

Debugging tools
RTOS vendors may have debugging tools that help find defects that can be harder to find with source-level debuggers (such as deadlocks, forgotten semaphore puts, and so on). Many tools allow the developer to watch as tasks pass semaphores to each other, as tasks switch, and as interrupts run (at the price of some CPU overhead).

A few vendors present users with an integrated development environment. For host-target debuggers, it works best (if it is not a requirement) that the application runs in RAM. If you expect to be running your code out of ROM, find out how much of the debugging services will still be useful.

Standards compatibility
Does the RTOS you¡¯re looking at support common standards? For example, there is a POSIX standard for RTOS services. While most developers don¡¯t need POSIX, it may be a consideration for you. If you¡¯re developing safety-critical systems, consider the safety standards your industry demands. Some RTOS vendors are beginning to certify their products.

Technical support
After you¡¯ve made your RTOS purchase, you¡¯re going to need support. RTOS vendors provide a variety of support sources. Some vendors have regional field applications engineers. All will have some sort of telephone and/or electronic support; however, make sure you understand how long this support lasts after your purchase. Try to get a feel, especially through references, of the quality of the vendor¡¯s technical support. If you are new to RTOSes in general, vendors¡¯ training services may be useful. Often the training can be brought to your site. If the vendor provides quality documentation with a few good examples, the need for training might actually be quite low.

Source vs. object code
Some vendors provide you with full source code when you purchase a development license. Others only provide object code. The first time you work with an RTOS that doesn¡¯t include source code, it can be unsettling. While both methods can result in a fine product, if you¡¯re making any but the simplest changes to the RTOS¡¯s source code, this should throw up red flags in your mind. If you find yourself making big changes, stop, pick up the phone, and call for support. If you are debugging or significantly modifying the operating system¡¯s code, you are probably defeating the whole purpose of buying your real-time operating systems from someone else.

For those who are concerned that not having source code will prevent reasonable kernel configuration¡ªdon¡¯t worry! Vendors will bring out the necessary constants into header files, so developers can tweak the kernel as needed.

Licensing
Purchasing some of the more sophisticated RTOSes is a significant business transaction. There can be a lot of costs to consider. Real-time operating systems typically require vendors to pay for development tools and to license the OS for products. While some vendors charge a one-time lump fee, other vendors¡¯ fees can be as extensive as charging per user per platform per product per site. I¡¯ve worked on products that have been at both extremes. Neither method is inherently better. Just make sure you understand what you¡¯re paying for.

Reputation
One of the intangibles of selecting a real-time operating system is getting a grasp on the vendor¡¯s reputation. While this can be difficult, there are a few hints you might find helpful.

First, get references and call them. While a vendor would be crazy to give you a bad reference, talking to real customers should give you a better view of the operating system¡¯s quality. Below are a list of things you might ask prospective references:

? How was technical support?

? How long did it take to get questions answered?

? How long did it take to get the OS running?

? Do you feel like you got value from your OS investment?

Second, do some research on the company. Below is a list of questions that might help you evaluate the company.

? How long has the firm been in business?

? Is it a stable business?

? How many employees does the company have?

? Does the vendor¡¯s Web site contain valuable information?

? How many design wins does the OS have?

? In what industries does this RTOS excel?

? Are there particular application domains to which the operating system has been tweaked (such as small microprocessors, safety systems, VME cards, and embedded PCs)?

? What is the state of the company¡¯s quality systems?

? At what level is the company on the SEI CMM?

? Is the company ISO 9001-certified?

In the final analysis
With a little legwork at the beginning of a project, you can create a very respectable analysis of how RTOSes can meet your needs. Choosing one can save you a lot of effort. What¡¯s even better, is that someone else will have debugged some of the most challenging software in your application. Your team spends less effort writing your code. You get your product to market faster. You spend less time in the lab late at night fighting nasty bugs. Everybody wins.

Greg Hawley is project manager with seven years of product development experience at Plexus Technology Group in Neenah, WI. A graduate of the University of Wisconsin, Madison, his software experience ranges from simple household appliances to complex, high-availability industrial control systems. He can be reached via phone at (920) 722-8748 or through e-mail at greg.hawley@plexus.com.

Return to Embedded.com

Send comments to: Webmaster
Copyright © 1998 Miller Freeman, Inc.,
a United News & Media company.