Designing an Extensible API in C Charles Mirho Charles Mirho is a consultant in New Jersey, specializing in Multimedia. He holds a Masters Degree in Computer Engineering from Rutgers University. He can be reached on CompuServe at: 70563,2671. Definition of an Extensible API An application program interface is the set of function calls that a software library exports to applications. These functions, along with their parameters, are usually prototyped in a header, or include file. In addition to prototypes, the header file also contains definitions for structures used by the functions, and #defines for flags and return values. All these components of the header file make up the complete definition of the API. An extensible API can accomodate growth in the library without requiring changes to existing applications (beyond a possible recompile). Listing 1 contains a simple, extensible API. The function in Listing 1 is useful in GUI APIs. It defines a region of the display where a mouse click can be trapped. A typical call to the function might look something like #define ID_HELP_BUTTON 10 int vertices[] = {10,20, 50,20, 50,40, 10,40}; REGION HelpButton = {sizeof(vertices)/sizeof(int)/2, &vertices[0]}; . . . DefineHotSpot (ID_HELP_BUTTON, &HelpButton); The value of ID_HELP_BUTTON varies depending on the application. ID_HELP_BUTTON also provides a unique ID for the region. The second parameter, &HelpButton, defines the boundary of the region as a set of vertices. Notice that the REGION structure is defined in API.H, the header file for the API. REGION contains a field for the number of vertices in the region, and a pointer to a list of vertices (coordinate pairs). Early versions of the library might only support rectangular hot spots (four vertices), but the API is extensible because more complex shapes can be used in the future without altering the prototype for DefineHotSpot. Compare Listing 1 to a non-extensible version of the same API in Listing 2. Since the region is always rectangular, only two vertices are required, specifying the upper-left and lower-right corners of the rectangle. While this function may seem cleaner and more intuitive at first glance, it is extremely confining. If future versions of the library must support regions with more than four vertices, then you must choose one of two undesirable alternatives: You can create an extensible version of DefineHotSpot. Now developers must learn two functions, since the old version of DefineHotSpot must be retained for compatibility. The result is a cluttered API. You must modify existing applications that use DefineHotSpot to support the new function API. Structured Parameters The extensible version of DefineHotSpot (Listing 1) uses a structured parameter, while the non-extensible version (Listing 2) passes all the parameters through the function prototype. Using structured parameters is one of the best ways to design an API that is more extensible. As the capabilities of the library expand, fields can be added to the structures without changing the function prototype. A good application of this technique is in functions that return information, such as int GetSystemInfo (SYSTEM_INFO * sinfo); This function is used to get information about devices in the computer system. The SYSTEM_INFO structure: typedef struct tagSYSTEM_INFO { int num_displays; /* number of attached displays */ int num_printers; /* number of attached printers */ int num_drives; /* number of attached disk drives */ } SYSTEM_INFO; has fields to hold the important properties of the system. (I kept it short for clarity.) Later versions of the API can expand the structure to accomodate new additions to the system, such as tape drives, as in: typedef struct tagSYSTEM_INFO { int_num displays; /* number of attached displays */ int_num_printers; /* number of attached printers */ int_num_drives; /* number of attached disk drives */ int_num_tapes; /* number of attached tape drives */ } SYSTEM_INFO; Because the features of the system are passed through the API in the form of a structure, rather than as separate parameters, it is easy to add a field for tape drives. The Size Field You can add even more flexibility to structured parameters with the size field. The size field holds the size, in bytes, of the structure containing it. When using a size field, you must make it the first field in the structure, as in typedef struct tagSYSTEM_INFO { int size; /* size of this structure */ int num_displays; /* number of attached displays */ int num_printers; /* number of attached printers */ int num_drives; /* number of attached disk drives */ int num_tapes; /* number of attached tape drives */ } SYSTEM_INFO; The size field makes it possible for existing applications to use newer versions of the library without performing a recompile. This is especially useful on platforms that use dynamic linking, because dynamic link libraries are often packaged separately and sold directly to customers. Application developers often have no control over which version of the library customers are using. To see how the size field can save a recompile, look at the SYSTEM_INFO structure again. When the num_tapes field is added, the size of the structure changes. It would normally be necessary to recompile applications that use the structure so that static and dynamic allocations reserve the correct amount of storage. Otherwise, the newer library would write too much data into the structure parameter, corrupting memory. However, if the first field of the structure contains the structure's size, and you are careful to add fields only to the end of the structure, then the structure can be extended without the need to recompile existing applications. The library simply examines the size field to determine which version of the structure the application is passing. If the application is passing the older structure, the size will be smaller, and the library knows not to fill the extended fields. Listing 3 contains an example. In Listing 3, the library keeps the declaration of the old SYSTEM_INFO structure as oSYSTEM_INFO. The oSYSTEM_INFO structure does not appear in the header file that applications use. Interpretation Flag Suppose the GetSystemInfo function is extended in the future to report details about particular devices in the system. You can use the same function to get the number of displays in the system, and details about the displays the system is using, as in: typedef struct tagDISPLAY_INFO { int size; /* size of this structure */ int displayno; /* display to get info on */ int xpixels; /* display width in pixels */ int ypixels; /* display height in pixels */ int bits_per_pixel; /* bits per pixel */ int planes; /* video planes */ } DISPLAY_INFO; You can insure that the GetDisplayInfo function will support this and any other device-specific structures that come along by changing the original prototype to int GetSystemInfo (int flag, unsigned char *info); GetDisplayInfo now accepts a byte-aligned pointer instead of a pointer to a specific structure. The function interpretes the pointer differently, depending of the value of the flag parameter. You call the function for general system information with /* API.H */ #define GET_SYSTEM_INFO 1 #define GET_DISPLAY_INFO 2 . . /* application */ SYSTEM_INFO sinfo = { sizeof(SYSTEM_INFO), 0, 0, 0 }; . . GetSystemInfo (GET_SYSTEM_INFO, (unsigned char *) sinfo); For details on display devices, you call the function with /* application */ DISPLAY_INFO dinfo = { sizeof (DISPLAY_INFO), 1, 0, 0, 0, 0}; . . GetSystemInfo (GET_DISPLAY_INFO, (unsigned char *) dinfo); Inside, the GetSystemInfo function would look something like Listing 4. Different structures describing entirely different things evolve differently, and so it is entirely possible for them to be the same size by coincidence. When different structures (as opposed to different versions of the same structure) are passed through the API as in Listing 4, the size field alone is not sufficient. The interpretation flag resolves any ambiguity. Variable-Sized Structures Variable-sized structures typically have a fixed-sized header portion and a variable-sized data portion. The header usually defines or limits the data in some way. The header and data are stored contiguously in memory, so that the data can be referenced as elements of the structure. This often makes the C code that manipulates the data easier to write and read. A variable-sized structure is also extensible because the data portion can be any size. The REGION structure in Listing 1 can be made variable-length by changing its definition to /* api.h */ typedef struct tagREGION { int vertex_count; int vertices[1]; } REGION; int DefineHotSpot (int id, REGION *pRegion); Suppose you want the user to decide the shape of the region in which to trap mouse events. The number and values of the vertices in the region are not known at compile time. You first prompt the user for the number of vertices, then allocate a region of the proper size, as in /* application */ REGION *pRegion; printf ("Enter the number of vertices:\n"); scanf {"%d", &cnt); pRegion = (REGION *) malloc (sizeof (REGION) + (2*cnt-1)*sizeof(int)); The pointer pRegion points to a region large enough to hold cnt vertices. In the malloc statement, sizeof(REGION) allocates enough space for the header (the field vertex_count) and one vertex, since the typedef for REGION contains one vertex. Since each vertex is a pair of integers, cnt vertices requires 2*cnt integers. You thus allocate space for 2*cnt-1 integers in addition to the space already allocated for the base structure. You then cast the return value of malloc to a pointer to a REGION structure. From then on, you can refer to the vertices as members of the structure. A loop is used to read the vertex pairs, as in pRegion->vertex_count = cnt; for (i=0;ivertices [2*i], &pRegion->vertices [2*i+1]); } The region with all its vertices is now passed cleanly to the DefineHotSpot function DefineHotSpot (ID_HELP_BUTTON, pRegion); Templates Sometimes the arguments to a function are so unpredictable that even structured parameters are limiting. The classic example of this is the Standard C library function printf. The prototype for printf declares a single parameter, a string that acts as a template for optional arguments. The printf function scans the template for clues to the number and size of the optional arguments. This is an extremly powerful technique, since it allows the function to accept any number of arguments of any size, in any order. Consider, for example, a function that draws an arbitrary set of line segments. Each segment has its own attribute for width and color. Segments may or may not be connected. We create a simple language to tell the function how to draw one or more segments. The language describes the motion and attributes of an imaginary pen which moves across the drawing surface. Table 1 describes the Simple Drawing Language. Figure 1 shows sample output from the Simple Drawing Language. Figure 2 shows pen styles. Listing 5 contains an example of the Draw function using the Simple Drawing Language. Extending the API is as simple as extending the drawing language. For example, to support different line styles, the language is extended to include an s (for style) followed by a number 1-5. Callback Functions Callback functions provide a useful way for developers to enter the API. They provide developers with a means of extending the API without altering it. Suppose, for example, that an API included a function for copying one file to another, with optional compression, such as int zcopy (char *szSourceFile, char *szDestFile, int (*fnCompress)()); This function takes two files as arguments. The first file is the source to copy from and the second is the destination to copy to. The third argument specifies an optional callback function to perform compression, so that the destination file takes up less space than the source. The callback function, if used, is provided by the developer who uses the library. The zcopy function calls the callback function repeatedly during the file copy. Developers are free to use any compression algorithm they desire. This makes the API extensible, since better compression algorithms can be developed and inserted without altering the API. Not only that, developers who use the library have a means of differentiating their products by offering better compression. The callback function resembles int fnCompress (unsigned char *pData, int *iSize) { //code to perform compression here } The zcopy function passes to the function a buffer, pData, which contains the raw data from the source file and the size of the buffer. Function fnCompress is expected to compress the data in pData (possibly using intermediate buffers) and return the buffer and new size to the zcopy function. This example is slightly oversimplified. A commercial version of zcopy would require additional (possibly structured) parameters to specify things like the compression block size. This example is meant only to illustrate the utility of callback functions in extending the API. Conclusion Following simple guidelines when designing an API can save headaches down the road as the API expands to accomodate new features. Structured parameters, variable-sized structures, size fields, interpretation flags, templates, and callback functions are some of the ways to prepare an API for future growth. Figure 1 Sample output from the Simple Drawing Language Figure 2 Pen Styles Table 1 Simple drawing language p (lower p) - pick up the pen; subsequent moves do not mark the drawing surface P (upper p) - put down the pen; subsequent moves mark the drawing surface c set pen color to the value which follows: R=red, G=green, B=blue, b=black Example: cR - set pen to color red w set pen width to the value which follows: T=thin, M=medium, F=fat Example: wF - set pen width to fat %x move to X coordinate specified by the next parameter %y move to Y coordinate specified by the next parameter Listing 1 An extensible API /* API.H */ typedef struct tagREGION { int vertex_count; int *vertices; } REGION; int DefineHotSpot (int id, REGION *pRegion); /* End of File */ Listing 2 A non-extensible version of the header file API.H in Listing 1 /* API.H */ int DefineHotSpot (int id, int x1, int y1, int x2, int y2); . . . /* Application */ #define ID_HELP_BUTTON 10 DefineHotSpot (ID_HELP_BUTTON, 10,20,50,40); /* End of File */ Listing 3 An example of how the size field can save a recompile int GetSystemInfo (SYSTEM_INFO *sinfo) { sinfo->num_displays = _getNumDisplays (); sinfo->num_printers = _getNumPrinters(); sinfo->num_drives = _getNumDrives(); if (sinfo->size == sizeof (oSYSTEM_INFO)) { /* don't touch extended fields */ } if (sinfo->size == sizeof (SYSTEM_INFO)) { /* fill extended fields */ sinfo->num_tapes = _getNumTapes(); } return 0; } /* End of File */ Listing 4 The function GetSystemInfo int GetSystemInfo (int flag, unsigned char *ptr) int GetSystemInfo (int flag, unsigned char *ptr) { if (flag == GET_SYSTEM_INFO) { if ( (int)*ptr == sizeof (oSYSTEM_INFO)) { oSYSTEM_INFO *sinfo= (oSYSTEM_INFO *)ptr; /* don't touch extended fields */ sinfo->num_displays = _getNumDisplays(); sinfo->num_printers = _getNumPrinters(); sinfo->num_drives = _getNumDrives(); } if ( (int)*ptr == sizeof (SYSTEM_INFO)) { SYSTEM_INFO *sinfo = (SYSTEM_INFO *)ptr; /* fill extended fields */ sinfo->num_displays = _getNumDisplays(); sinfo->num_printers = _getNumPrinters(); sinfo->num_drives = _getNumDrives(); sinfo->num_tapes = _getNumTapes(); } } if (flag == GET_DISPLAY_INFO) { DISPLAY_INFO *dptr = (DISPLAY_INFO *)ptr; dptr->xpixels = _getDisplayWidth(dptr->displayno); dptr->ypixels = _getDisplayHeight(dptr->displayno); dptr->bits_per_pixel = _getDisplayBPPix(dptr->displayno); dptr->planes = _getDisplayPlanes(dptr->displayno); } return 0; } /* End of File */ Listing 5 The Draw function using the Simple Drawing Language in Table 1 #include #define PENUP() pen_up=1 #define PENDOWN() pen_up=0 #define SETCOLOR(x) color=x #define SETWIDTH(x) width=x #define BLACK 0 #define RED 1 #define GREEN 2 #define BLUE 3 #define FAT 0 #define MEDIUM 1 #define THIN 2 #define TRUE 1 #define FALSE 0 int pen_up; int color; int width; int x1, x2, y1, y2; int gotx1, gotx2, goty1, goty2; /* Function Draw - demonstrates use of the Simple Drawing Language Does not actually draw lines, simple parses the first parameter and prints commands on screen */ Draw (char *str, int val) { int *pval=&val; //points to optional parameters //restore defaults gotx1=goty1=gotx2=goty2=FALSE; x1=x2=y1=y2=0; pen_up=1; color = BLACK; width=MEDIUM; //loop to parse the command string while (TRUE) { switch (*str++) { case 'p': PENUP(); break; //pick pen up case 'P': PENDOWN(); break; //put pen down case 'c': //set color switch (*str++) { case 'R': SETCOLOR(RED); printf ("Color set to red\n");break; case 'G': SETCOLOR(GREEN); printf ("Color set to green\n");break; case 'B': SETCOLOR(BLUE); printf ("Color set to blue\n");break; case 'b': SETCOLOR(BLACK); printf ("Color set to black\n");break; default: return -1; } //End switch (on character) break; case 'w': //set width switch (*str++) { case 'T': SETWIDTH(THIN); printf ("Width set to thin\n");break; case 'M': SETWIDTH(MEDIUM); printf ("Width set to medium\n");break; case 'F': SETWIDTH(FAT); printf ("Width set to fat\n");break; default: return -1; } //End switch (on character) break; case '%': //get next optional parameter switch (*str++) { case 'x': //set x coordinate if (gotx2) {x1=x2; x2=*pval++; break;} if (!gotx1) {x1=*pval++; gotx1=TRUE; break;} if (!gotx2) {x2=*pval++; gotx2=TRUE; break;} break; case 'y': //set y coordinate if (goty2) {y1=y2; y2=*pval++; break;} if (!goty1) {y1=*pval++; goty1=TRUE; break;} if (!goty2) {y2=*pval++; goty2=TRUE; break;} break; default: return -1; } //End switch (token) //do we have enough info to draw the line? if (gotx2 && goty2 && !pen_up) { printf ("Drawing line <%d,%d>-<%d,%d>\n", x1,y1,x2,y2); x1 = x2; y1 = y2; goty2 = FALSE; gotx2 = FALSE; } //end if (got both coordinates - draw line) break; case '\0': //end of command string return 0; default: return -1; } //End switch (on character) } //End while (TRUE) return 0; } //End function (Draw) /* End of File */ Using Wrappers to Improve Portability of Commercial Libraries Kenneth E. Van Camp Kenneth Van Camp is a Senior Programmer/Analyst for IMI Systems. He has been developing professional software applications for over ten years, and holds a B.S. in Mechanical Engineering from the State University of New York at Stony Brook. He can be reached at: R.D. #1 Box 1255, East Stroudsburg, PA 18301. You've seen the ads for them: the Ultimate Portability Libraries. One supports Windows, PenPoint, and Presentation Manager. Another supports UNIX, MS-DOS, and VMS. Each one is the "only" tool you'll ever need. Promise. Well, maybe not. But you can improve the portability of software that uses third-party libraries by isolating all library-specific code and writing wrappers. Most programmers are familiar with wrappers as applied to functions, but wrappers can also be written for all data structures, constants, and global variables. In C++, this concept is known as encapsulation, but it works equally well in C. Why Use Wrappers There are several compelling reasons to use wrappers: Future portability New version protection Better parameter and return value checking Programming for the lowest common denominator Simplification Enforced corporate standardization Uniformity of naming standards See the sidebar "Why Use Wrappers" for more information. Wrapper Functions In their simplest form, wrapper functions are a one-to-one match of library functions. Wrappers simplify the interface to a library by combining multiple function calls or eliminating unused parameters. If you later determine you need a parameter you thought was unnecessary, it is a simple matter to write a second wrapper with a more complete parameter list. The simpler wrapper can then call the more complex one. In C++, unused parameters can be given default values so a future need can be easily accommodated. Alternatively, over-loading the wrapper can provide a simple mechanism for dealing with multiple complexity levels. One of the most important rules of a wrapper function is that none of its parameters can be of a type declared by a third-party library. This means that all data structures must be wrapped, too. C++ handles this by encapsulation. Simply write a class that includes the library data structure as a private member, and write access functions for any structure members that are required in application code. While you are at it, consider including related wrapper functions as members of the class, so they can pass the original data structure to library functions without having to be a friend to the class. Listing 1 shows an example of a C++ class that encapsulates the Windows 3.1 API WNDCLASS structure and a related API function. In C, this behavior can be easily mimicked. For example, Listing 2 shows an equivalent set of wrappers in C for the WNDCLASS structure. typedefs rename the Windows structures, and several access functions (all beginning with the prefix WCL_) allow the application programmer to define or retrieve values of the structure members. Notice the initialization function (WCL_init) used in place of a C++ constructor to initialize values to reasonable defaults. In this case, I included several parameters to set structure members that are usually initialized in any application. Additional access functions allow me to set or retrieve the values of any other members. These can be added as the need arises. Once you have become used to C++ constructors, you quickly become dependent on the idea of an initialization function for your data structures. The only problem with using one in C is that the programmer must remember to call it before using the data structure. The best approach to this is to write an initialization function for every structure type, even if you don't need it, and require that an initializer be called before using any data structure. That way it is less likely to be forgotten. A case can be made for an exit function for every data structure, analogous to the C++ destructor. In practice, I have found this is rarely necessary and merely places an undue burden on the C programmer. Callback Functions Sometimes, you must write a function with an interface determined by the library in use. For instance in Windows, callback functions are commonly written to handle messages for dialog boxes, for enumerations, and timer processing. In each case, your application's function must process parameters that are beyond its control. It is possible to write a macro to wrap callback functions and give them an interface of your choosing. Personally, I do not use this approach because I find this type of macro leads to code that is confusing to read and debug. Instead, I place all callback functions in a module of their own. The only thing these functions do is load the parameter values into a data structure and pass the structure to the real callback function, which is kept in the normal application code. The problem with this approach is that each callback function has a separate wrapper, so each one must be rewritten if the library's callback interface changes. For instance, one of the differences between the WIN16 and WIN32 APIs is that some data was moved from the lParam to the wParam for several messages. A translation function could handle this if you properly encapsulated the parameters in a data structure, but every callback wrapper must be modified to call the translation function. In fact, it is not necessary to have a separate callback function for every Windows dialog box. Since a window handle is provided to the callback function, a single callback function can handle all dialog messages and dispatch them to the appropriate function in your application code for the window in use. The situation is the same for enumerations and timer procedures. Conclusions Wrappers and encapsulating classes are a simple way to increase your software's longevity. If you have experience working with several different libraries, you may be able to put an even higher-level interface on your wrappers -- one which removes the application programmer even farther from the specifics of one library. This will give your application even greater longevity as you declare your independence of any one library. There is an initial investment in all of these techniques, but this investment usually pays for itself long before the initial project ends. In later projects, the payback will be nearly instantaneous. In addition, increased reliability and standardization provide benefits that are difficult to quantify but obvious to any experienced developer. Why Use Wrappers Future Portability A simple but solid goal when developing software should be to remain as independent as possible from third-party software, so that your project does not depend on any one vendor or environment. However, you should not reinvent the wheel and write your own libraries. Instead you should isolate all of your library-specific code in a few modules, including all references to a library's functions, data structures, classes, header files, constants, and global variables. When you isolate all system-dependent code in a few modules, porting to new environments in the future might be as easy as rewriting the wrapper modules to support a new library. There is no guarantee that your new library can be mapped onto the function calls of the old library, but chances are most of it will. Don't be tempted to ignore this argument, even if you can write compatible wrappers on future systems to make them emulate your current environment. This can be a trap. No matter how much time you allocate for a future port, you will never implement all of the options of your current library. By taking the time to isolate your system-dependent code now, parameter checking can be tightened in the future to prevent use of features that are not implemented in all of your environments. While you are wrapping third-party library functions, you will probably also want to wrap the file- and memory-access functions that you use. Even though the ANSI file-access functions may be all you need right now, there is a good chance that a future environment (a network, Windows, OS/2, etc.) will require you to use some special access functions. For memory access, you may want to use a handle for major data structures to allow future porting to virtual-memory environments. A simple, easy-to-maintain handle table allows for future migration. New-Version Protection No one can predict how a future release of a commercial library will behave. There will always be bug fixes and enhancements that change the behavior of the library, for better or for worse. If your application depends on the behavior of the old version, you may be able to correct for the changes in the wrappers. For example, Microsoft changed the return value of the fputs function in Microsoft C between versions 4.0 and 5.0. Although the new return value is now an ANSI-standard function, no one expected fputs to change. An application developed with a wrapper for fputs could probably have been revised to accommodate this change with minimal effort. Better Parameter and Return Value Checking In a perfect world, programmers check the values of all parameters before passing them to functions. They also check all function return values to catch any errors. In the real world, many junior-level programmers don't know all of the conditions they should check for. Even senior-level programmers may overlook unlikely error conditions, especially under the stress of a deadline. Wrapper functions can check for any unlikely conditions and call a global error-reporting function. The application can determine where such error messages should be directed, depending upon the current state. For instance, when retrieving a database record you normally write application code that handles only two conditions. If the call fails, it was most likely caused by a missing record. In the wrapper, you can take care of checking for unusual conditions like a corrupted database and branch to an appropriate routine to shut down and perform an integrity check on the database. This approach allows you to write your code checking software in increments. As you gain experience with a library and discover new conditions that merit checking, you can add these checks without modifying all of your application code. Programming for the Lowest Common Denominator While checking parameters, you can also check for options that are not available on all the platforms your software supports. If a future port determines that an option should not be used, the wrapper can provide a warning for programmers in the old environment as well as the new one. This will stop future programmers from using options that can't be supported on all systems. Simplification If you have six programmers on a team developing a Windows application, should you train all six in Windows development? Not necessarily. By assigning two or three members to write the wrappers and document the available options, you can decrease the learning curve for the remaining team. The remaining team can then focus on problems they are better qualified to solve. The team's database experts, for instance, can concentrate on database issues. Documentation for the wrappers can be as simple as, "This duplicates the functionality of the xyz function in library ABC, with the following changes..." However, the documentation for library ABC will change from one release to another, but your wrapper's interface should not. If time permits, you should write complete documentation for the wrapper. Otherwise, the library's release number should be noted and the old manuals retained for reference. If you have the time to properly document your work, you might as well try to simplify the application developer's job. If you find that calls to a library function are always preceded by one or more "setup" functions, it may make sense to combine the functions into a single wrapper call. Also, if you find yourself always initializing data structure members to a certain value before calling a function, this can easily be placed in the wrapper. Enforced Corporate Standardization By limiting parameter choices and function calls in the wrappers, you can make a more consistent application. The look and feel of the application should not be a decision left to junior-level programmers, but rather should be a matter of corporate policy. Wrappers are the perfect place to enforce this policy. Uniformity of Naming Standards Instead of using whatever naming standards are used in a third-party library, the wrappers can be written to conform to your company's naming standards. This shortens the learning curve for new programmers, and makes code easier to read. For instance, if your company does not normally use Hungarian notation, calls to Windows API functions probably look out of place. Listing 1 A C++ class that encapsulates the Windows 3.1 API WNDCLASS structure and a related API function #include "windows.h" typedef int BOOL; typedef WNDPROC MYWNDPROC; typedef HINSTANCE MYHINSTANCE; typedef HCURSOR MYHCURSOR; typedef HBRUSH MYHBRUSH; class MyWndClass { public: MyWndClass (char *, MYWNDPROC, MYHINSTANCE, char *); void SetCursor (MYHCURSOR hCursor) { wndclass.hCursor = hCursor; } void SetBackground (MYHBRUSH hbrBackground) { wndclass.hbrBackground = hbrBackground; } void SetMenu (char *szMenuName) { wndclass.lpszMenuName = szMenuName; } BOOL RegisterClass (void) { return (::RegisterClass (&wndclass)); } private: // Default constructor is inaccessible. MyWndClass (void); WNDCLASS wndclass; }; MyWndClass::MyWndClass (char *szAppName, MYWNDPROC WndProc, MYHINSTANCE hInstance, char *szMenuName) { wndclass.style = CS_HREDRAW CS_VREDRAW; wndclass.lpfnWndProc = WndProc; wndclass.cbClsExtra = 0; wndclass.cbWndExtra = 0; wndclass.hInstance = hInstance; wndclass.hIcon = LoadIcon (NULL, IDI_APPLICATION); wndclass.hCursor = LoadCursor (NULL, IDC_ARROW); wndclass.hbrBackground = GetStockObject (WHITE_BRUSH); wndclass.lpszMenuName = szMenuName; wndclass.lpszClassName = szAppName; } /* End of File */ Listing 2 The equivalent of Listing 1 using wrapper functions in C #include "windows.h" typedef int BOOL; typedef WNDCLASS MYWCLASS; typedef WNDPROC MYWNDPROC; typedef HINSTANCE MYHINSTANCE; typedef HCURSOR MYHCURSOR; typedef HBRUSH MYHBRUSH; void WCL_Init (MYWCLASS *wndclass, char *szAppName, MYWNDPROC WndProc, MYHINSTANCE hInstance, char *szMenuName) { wndclass->style = CS_HREDRAW CS_VREDRAW; wndclass->lpfnWndProc = WndProc; wndclass->cbClsExtra = 0; wndclass->cbWndExtra = 0; wndclass->hInstance = hInstance; wndclass->hIcon = LoadIcon (NULL, IDI_APPLICATION); wndclass->hCursor = LoadCursor (NULL, IDC_ARROW); wndclass->hbrBackground = GetStockObject (WHITE_BRUSH); wndclass->lpszMenuName = szMenuName; wndclass->lpszClassName = szAppName; } void WCL_SetCursor (MYWCLASS *wndclass, MYHCURSOR hCursor) { wndclass->hCursor = hCursor; } void WCL_SetBackground (MYWCLASS *wndclass, MYHBRUSH hbrBackground) { wndclass->hbrBackground = hbrBackground; } void WCL_SetMenu (MYWCLASS *wndclass, char *szMenuName) { wndclass->lpszMenuName = szMenuName; } BOOL WCL_RegisterClass (MYWCLASS *wndclass) { return (RegisterClass (wndclass)); } /* End of File */ Template Classes for the iostreams Library Randal Kamradt Randy has been programming since 1984, and is working for Bordart Automation where he develops CD-ROM based applications for libraries. His special interests include object-oriented programming/design and C/C++ programming in general. One of the nicer additions that C++ made to C is the iostreams facility. Even if you only use C++ as "a better C," iostream provides an elegant method of I/O and allows easy extensions for user-defined types. This ease of extension only works one way, though, as creating new stream types that work with existing code can be fairly complicated, and requires a complete understanding of the streams hierarchy. When I first got my hands on a C++ compiler I wanted to create a streams class that would work with a compact disk. My compact disk driver provided basic read routines, and I already had a class that I used as an interface to the driver. To provide seamless integration with existing code I needed to create new classes that fit in with the existing iostreams hierarchy. (See the sidebar "iostreams Hierarchy" for a discussion of this class and virtual inheritance.) The first step was to copy the definitions of the ifstream, fstreambase, and filebuf classes and rename them to cdstream, cdstreambase, and cdbuf classes. (I was not concerned with output classes since I was working with CDs.) Then I changed the open parameters and constructor parameters to suit the class I used to interface with the CD driver. Within the cdbuf class I replaced the file-handle integer with a pointer to the CD interface class, and added appropriate new and delete statements in the open and close routines of cdbuf. Finally, I replaced the calls to the global open, close, read, and seek with ones that used the CD class pointer. After doing all that and then fixing the few bugs I let creep in, I was convinced that there had to be a better way. A few months after creating these classes I was faced with doing it again. By this time we had a new version of Borland C++ with templates. After thinking about it for a while, I decided that templates could be the better way I wanted. By creating generic versions of the classes I had copied before, I could create an endless number of different stream types for anything that resembled a stream. In this article, I will present these template classes, and a serial port stream class as an example. These classes were compiled and tested with Borland C++ v3.1. The portability of the templates depends on consistency in the iostreams facility across compilers. Template Classes To avoide duplicating classes unnecessarily, I created templates for the classes. (See the sidebar "Templates" for more information.) The classes I made templates of included filebuf, fstreambase, fstream, ifstream, and ofstream, I call the template classes tbuf, tstreambase, tstream, itstream, and otstream. The suffix indicates a templeate class. The replaceable type is a basic class that has open, close, read, write, and seek member functions. (I will call this class T when referrring to this replaceable type.) When creating a class for an entity that does not have one or more of these functions, you can either create a dummy function that does nothing, or one that calls an error routine. See Listing 1 for all template class definitions. The first template class, tbuf is the most complex. It contains one object of the variable type, and controls the operation of that object. tbuf's open, close, and seek member functions directly call T's open, close, and seek member functions. The overflow and underflow member functions call read and write, along with setting the various buffer pointers. It also has the ability to attach an already open T to itself. tbuf is derived from streambuf, which provides it's interface. Some of the member functions of tbuf override the virtual functions of streambuf. Since ios contains a pointer to streambuf, ios has access to these virtual functions, and is able to read and write tbuf. The second template class, tstreambase contains a tbuf. It is derived (virtually) from ios. In its constructor tstreambase initializes ios with a pointer to its tbuf object. It can also open its tbuf object if called with the necessary parameters. Otherwise, it has an open, close, and attach call that map directly to the tbuf open, close, and attach member function. The last set of template classes are tstream, itstream, and otstream. These are multiply derived from istream/ostream and tstreambase. They are shell classes that simply combine the capabilities of the two inherited classes. The only thing necessary in the definition is the duplication of the constructors, and an open and rdbuf member function, that calls the tstreambase open and rdbuf member functions. The open function is redefined to give default mode values to itstream and otstream. The rdbuf function is redefined to avoid ambiguities with ios which contains its own rdbuf function. Note that when creating a deep hierarchy, constructors need to be defined for all classes, even if they don't change from the base class's constructor. Duplicating numerous constructors, or constructors with long parameter lists, can be a nuisance. There are four constructors for itstream/otstream that are duplicates of the tstreambase constructors: tstream() tstream(const char *name, int mode, int prot) tstream(T &f) tstream(T &f, char *buffer, int length) The default constructor, tstream() initializes the buffer using default parameters for buffer size. The stream is considered closed. tstream(const char *name, int mode, int prot) initializes the buffer, and opens the named stream. tstream(T &f) initializes the buffer with the T parameter. tstream(T &f, char *buffer, int length) initializes the buffer with T, and sets the buffer to the char * with the length specified. These four constructors are duplicated in the three classes itstream, otstream, and tstream. In all of these cases, the parameters are passed on directly to tstreambase's constructors. The constructors for tstreambase call the constructors either for the default tbuf constructor, or in the case of constructors tstream(T &f) and tstream(T &f, char *buffer, int length) it passes the parameters to tbuf's constructors. It then calls the ios init function to set ios pointer to its T data member. For constructor tstream (const char *name, int mode, int prot), it calls T's open function. The tbuf class has three constructors: tbuf() tbuf(T &f) tbuf(T &f, char *buffer, int length) tbuf(), the default constructor, builds a buffer of default size. The T stream is considered closed. tbuf(T &f) builds a default buffer, but attaches T as its stream. The T stream is considered open for read or write. tbuf(T &f, char *buffer, int length) builds a buffer using the buffer and length parameters, and attaches T as its stream. SerialStream and ComBuffer classes As an example of how to use these templates, I decided to use a serial port stream. In MS-DOS you can access the serial port as a stream, but it is not interrupt driven, so it is nearly useless without special drivers. The serial port stream is divided into two classes, SerialStream, and ComBuffer. The ComBuffer provides an interrupt-driven circular buffer for input, and passes output directly to the port. The Serial-Stream class uses the ComBuffer class to do its I/O, and has the correct interface to plug into the template. I split the classes in two in case there was more then one stream per port. However, ComBuffer needed to be mapped directly to a port. The only instance of ComBuffer is a static global (see Listing 2 and Listing 3) CommPorts which is an array of two, one for each port. Since it is a static global, it is initialized before main. It uses the static data member initPort to ensure the correct port number is initialized. In the constructor and destructor for ComBuffer I included a print statement to visually assert the order of the construction. ComBuffer is not a safe or complete class that could be used anywhere in a program, so I made the definition private to the module. I could alternatively have made the definition private to the SerialStream class definition. The SerialStream class uses ComBuffer to communicate with the physical ports, via a pointer. When a SerialStream is opened, the name parameter is decoded to give the port number. The port number is used as an index into the CommPorts array, and the pointer to that index is saved. During reading and writing the request is passed on to ComBuffer via that pointer. Only one character is read or written at a time. This inefficiency does not concern me, since the iostream should be unbuffered, and should only request one character at a time anyway. Any class that is to be used in the streams templates must meet certain requirements. First it needs a default constructor. This constructor should leave the stream in a closed state. It needs a open function that takes a const char *, and two ints for parameters. This is perhaps the most extreme restriction, as not all streams will be able to map these parameters. For streams that take fewer parameters, as my SerialStream class does, it is easy enough to ignore the rest. For a class that needs more information to get started, this can be a problem. One alternative could be to use the name parameter to pass in additional information: x.open("COM1, INT=7, SPEED=2400",ios::in); Although this appears sloppy, and requires character translation in the open function, it is not without precedent. Another alternative is to access the T class directly: x.open("COM1",ios::in); x.rdbuf()->fd()->setspeed(1200); x.rdbuf()->fd()->setint(7); The stream class must also define a close function that takes no parameters. A read and write function that takes a char * and an int, as well as a seek function that takes a long and a ios::seek_dir type must be present. Finally, a const void * conversion operator needs to return the open/close state, as the this pointer or a null pointer. The open, close, read, write, and seek function can all be dummied up if not needed. If seek is dummied, you need to make sure the stream is unbuffered. I mentioned previously that the portability of the templates depends on how similarly different libraries implement the internals of the iostreams classes. This code was made with the Borland C++ v3.1 libraries in mind, and might need to be changed for other implementations. In the header files, Borland mentions minor differences with the AT&T library, so I assume that the templates will work as well under AT&T and any other vendor that follows them. If the internals of iostreams are not under discussion in the ANSI X6J16 commitee, then perhaps vendors should include a set of templates similar to these in the C++ library to allow different streams types to be portable from one implementation to another. iostreams Hierarchy To provide seamless integration with existing code you need to create new classes that fit in with the existing iostreams hierarchy. To illustrate some of the relationships, Figure 1 contains a diagram of the ifstream path of the iostreams class hierarchy. At the bottom of the hierarchy is the ifstream, ofstream, and fstream classes. You would normally use these classes when dealing with files as streams. These classes provide no new functions or data members, and each is completely dependent on its base classes for its abilities. They are derived multiply from fstreambase and one of the istream, ostream, or iostream classes. The istream class gives ifstream all of its formatted input functionality. The >> operator is defined for all basic types, and get, getline, and read are also defined for reading. In addition, the control functions gcount, ignore, peek, putback, seekg, and tellg are defined. You should use the istream class whenever you create input functions for your user-defined types. The ostream class is similar but for output streams. The ostream class defines the functions flush, put, seekp, tellp, and write along with the < < operator. The iostream class is a "shell" class that simply inherits istream and ostream. All of these classes are virtually derived from the ios class. I will explain more on this later. fstreambase gives fstream the ability to open and close a file. fstreambase defines the functions attach, close, open, rdbuf, and setbuf. It also contains a single data member, a filebuf type. This filebuf type gives fstream the ability to read and write, though not directly. The read/write capability is given to istream/ostream through the virtual base class ios. Both fstreambase and istream/ostream are derived from ios. Since it is a virtual derivation, both of the classes share the same copy of ios's data members. Among these members are a pointer to a streambuf type. fstreambase gives ios a pointer to its filebuf object during initialization. During use, istream/ostream uses this pointer to access the filebuf for reading and writing. The ios class also has various format and state flags and numerous functions for setting and reading these flags. The filebuf class is the workhorse of the hierarchy. It is the one that actually calls for reads and writes from the file. It also handles opening, closing, seeking, and buffering. When an fstream is opened, the fstream-base open function calls filebuf's open which calls the global open. The filebuf class is derived from streambuf. The streambuf class is an abstract type that allows ios to communicate with different types of buffer classes. Since ios contains a pointer to a streambuf type it can communicate with filebuf through virtual functions. The virtual functions overflow, underflow, seekoff, setbuf, and sync provide ios and istream/ostream the ability to read, write, and seek. Templates and Virtual Functions Template classes allow a generic class to be defined. Individual objects can be declared from it by filling in type information in the individual declarations. The classic template example is the array-class. Once the array class template is defined somewhere, various types of arrays can be declared: // an array of ints: Array ArrayOfInts; // an array of doubles: Array ArrayOfDoubles; The compiler will generate the code needed for access and initialization of each of the above arrays. If the compiler/linker can optimize, it will automatically remove similar code from different modules. Be aware that each of these Array declarations will produce its own code. This may be more code then you might want. There are ways around this, but that is the general limitation of templates. Virtual functions can be seen as alternatives to templates. While templates provide generic source code, virtual functions provide generic object code. Where templates are generally faster, virtual functions generally produce less code. This trade off is not always true, and there are other disadvantages on both sides. I have found however that virtual functions and templates often complement one another, and that very good solutions can be found in the mixture of the two. In the above example one might define: // an array of anything derived from // Numerics: Array ArrayOfNumbers; Then if Numerics is used as a base class, anything derived from Numerics can be used in this array, and Numerics provides the least-common-denominator for operations available for ArrayOfNumbers. Figure 1 The ifstream path of the iostreams class hierarchy Listing 1 tstream.h -- template class definitions #if !defined TSTREAM_H #define TSTREAM_H #include // for base class definitions // class tbuf<>. Derived publicly from streambuf to // allow class ios which contains pointer to streambuf // access to virtual functions. tbuf<> implements // basic buffering, reading, writing and seeking on // a stream. It also provides open, close, attach, // and utility functions. template class tbuf : public streambuf { public: // openProtection provides a default parameter to the // open functions to specify what protection a file // will be created with. You can ignore this if it is // not necessary static const int openProtect; // tbufSize specifies the default buffer size. It is // set to 516 bytes. static const int tbufSize; // Default contructor. Make a buffer without a // stream attached. mode has a dual meaning, if it // is zero it means that any operation is allowable, // and the stream should not be deleted when closing. tbuf() : stream(0), mode(0), opened(0) { makbuf(); } // create buffer and attach to t. t is assumed to be // already opened in read/write mode. t will not be // deleted or closed when closing this buffer tbuf(T &t) : stream(&t), mode(0), opened(1) { makbuf(); } // create buffer from parameters, and attach to t. tbuf(T &t, char* b, int l) : stream(&t), mode(0), opened(1) { setbuf(b, l); } // destroy buffer. If mode is not zero, t will be // closed and deleted. Otherwise just flush the // output buffer. ~tbuf() { if(mode) close(); else overflow(EOF); } // return open status int is_open() const { return opened; } // return reference to stream T &fd() const { return *stream; } // open stream. mode must not be zero. stream will // be closed and deleted when closing buffer. tbuf *open(const char *name, int mode, int prot = tbuf::openProtect); // close buffer and optionally delete stream. tbuf *close(); // attach stream to buffer. Stream is assumed to // be opened in read/write mode. tbuf *attach(T &); // write buffer to stream and reset pointers. virtual int overflow(int = EOF); // read data into buffer and reset pointers. virtual int underflow(); // sync input and output. virtual int sync(); // seek to offset and flush output buffers. virtual long seekoff(long, ios::seek_dir, int); // set buffer. For unbuffered i/o set char * to 0. virtual streambuf *setbuf(char *, int); protected: int putBackSize() { return (blen() > 8) ? 4 : 1; } void resetpg(int end = 0); void makbuf() { setbuf(new char[tbufSize], tbufSize); } int unbuffered() { return streambuf::unbuffered() !base(); } T *stream; int mode; short opened; char lookAhead[2]; }; template const int tbuf::tbufSize = 516; template const int tbuf::openProtect = 0; // Attach an open stream to this buffer. When this // buffer is closed don't try to close stream. If // not yet buffered, try to create a buffer. Reset // the put and get pointers. Return 0 on error, or // this if OK. template tbuf* tbuf::attach(T &t) { if(opened) // if already opened, return 0 return 0; stream = &t; // attach stream. opened = 1; // set to opened. mode = 0; // buffer doesn't own stream. if(!base()) // if no buffer yet... makbuf(); // try to make one. else resetpg(); // reset put and get pointers. return this; } // Close buffer, and optionally stream attached to it. // Flush buffer if data inside. Return 0 on error or // this if OK. template tbuf* tbuf::close() { if(!*stream) // if stream closed, opened = 0; // set opened off. if(!opened) // if not on return 0. return 0; int ret = 0; if(out_waiting()) // if output in buffer. // flush output buffer. ret = (overflow(EOF) == EOF) ? 1 : 0; if(mode) // if mode is 0, don't delete stream; // close or delete stream. opened = 0; // set opened off. return ret ? 0 : this; } // Write data from buffer to stream. This function // is called when the buffer is full and we need to // output more characters. The parameter c is the // character that caused the overflow. If the // stream is buffered, it will be placed in the // buffer, otherwise it is written to the stream. // If input char is EOF, don't write, just flush. // Returns EOF on error, or 1 on success. template int tbuf::overflow(int c) { if(!opened // check to see if stream mode == 0 // is on and mode is out. !(mode&ios::out)) return EOF; if(unbuffered()) { if(c ! = EOF) { // if unbuffered, char b = c; // write single char if(stream->write(&b, 1) != 1) return EOF; } } else // else if buffered. { if(sync() != 0) // sync input and output return EOF; // when writing resetpg(1); // reset the put/get pointers if(c != EOF) { sputc(c); // add c to the buffer gbump(1); // move the get pointer } } return 1; // return OK } // Open stream. If mode ios::ate (at end) is on, // seek to end template tbuf* tbuf::open(const char* n, int m, int p) { if(opened !m) //if already on, or no mode, return 0; // return error. stream = new T; // make new stream pointer. stream->open(n, m, p);// open stream. if(!*stream) // if stream not open, return 0; // return error. opened = 1; // set to on. mode = m; // remeber mode. if((mode & ios::ate) && stream->seek(OL, ios::end) == EOF) return 0; // seek to end if ios::ate. resetpg(); // reset put/get pointers. return this; // return OK. } // Set the buffer, reset the put/get pointers. // Return 0 on error, this if OK. template streambuf* tbuf::setbuf(char* b, int l) { if(!b) // turn off buffering. { streambuf::unbuffered(1); return this; } if(opened && base())// check if stream is opened, return 0; // , and no buffer yet. setb(b, b+l, 0); // set buffer pointers. resetpg(); // reset put/get pointers. return this; // return OK. } // Seek to offset. dir indicates the relativity of // the offset, either from beginning, current position, // or end (ios::beg, ios::cur, ios::end). // First make sure there's nothing in the buffer. template long tbuf::seekoff(long off, ios::seek_dir dir, int /* mode ignored */) { long loff = off; int count = out_waiting(); // flush output first. if(count) { if(stream->write(pbase(), count) != count) return EOF; } else if(dir == ios::cur && // if relative seek, (count = in_avail()) != 0) loff -= count; // discount input. resetpg(); // reset pointers. return stream->seek(loff, dir); } // sync input and output buffer pointers. If output // is waiting, write it, if input is waiting, // back up to read it again template int tbuf::sync() { if (!opened) // check if opened. return EOF; int count = out_waiting(); // check for output, if(count) // in buffer. { if(stream->write(pbase(), count) != count) return EOF; // write output. resetpg(1); } else if(in_avail()) // check for input { // in buffer long pos = stream->seek(long(-in_avail()), ios::cur); setg(eback(), gptr(), gptr()); setp(gptr(), gptr()); // set up to re-read. if(pos == EOF) return EOF; } return 0; } // Read from stream to fill buffer. Must be opened and // in input mode. If there is input in the buffer, // return it. Otherwise call sync, read from stream, // and set pointers. If the input is unbuffered, // use the lookahead buffer. template int tbuf::underflow() { if(!opened mode == 0 !(mode&ios::in)) return EOF; // check for correct mode. if(in_avail()) // if available from buffer, return *gptr()&0xFF; // don't bother. int c; int count; if(unbuffered()) // if unbuffered. { count = stream->read(lookAhead, 1); if(count == EOF) // read one char { // into look ahead c = EOF; // buffer. setg(0, 0, 0); } else { c = lookAhead[0]&0xFF; setg(lookAhead, lookAhead, lookAhead+1); } } else // else buffered. { if(sync() != 0) // sync pointers. return EOF; int pb = putBackSize(); char *b = base(); // read into buffer. count = stream->read(b+pb, blen()-pb); if(count == EOF) // check read return. return EOF; setg(b, b+pb, b+pb+count); setp(b+pb, b+pb); // set pointers. if(count) // return first char. c = *gptr()&0xFF; } if(!count) c = EOF; return c; } // reset the put and get pointers. If end is // true set the end pointers to the end of // the buffer. template void tbuf::resetpg(int end) { char *buf = base(); // get the start of buffer. char *sbuf = buf + (buf ? putBackSize() : 0); char *ebuf; // set start and end pointers. if(end) ebuf = buf + blen(); else ebuf = sbuf; setp(sbuf, ebuf); // set put pointer setg(buf, sbuf, sbuf);// set get pointer } // tstreambase is virtually derived from ios. ios // does not define any functionality for tstreambase // but allows communication between tstreambase and // istream, ostream and stream classes. The functions // defined in tstreambase typically call the same // named function for its tbuf<> member and set ios // status flags. When a tstreambase is constructed // ios is initialized with a pointer to the tbuf<> // member. ios see this pointer as a streambuf // pointer, and only has access to tbuf<> through the // virtual functions, overflow, underflow, sync, and // seekoff. template class tstreambase : virtual public ios { public: // Default constructor. Make an unattached // buffer, and initialize ios with it's pointer. tstreambase() : tbbuf() { ios::init(&tbbuf); } // Make a buffer attach to named stream, and open // stream tstreambase(const char* n, int m, int p = tbuf::openProtect) : tbbuf() { ios::init(&tbbuf); open(n, m, p); } // Make a buffer attached to already opened stream. tstreambase(T &f) : tbbuf(f) { ios::init(&tbbuf); } // Make a buffer using parameters, and attach // to already opened stream. tstreambase(T &f, char* b, int l) : tbbuf(f, b, l) { ios::init(&tbbuf); } // Destroy buffer ~tstreambase() { ; } // close attached stream and set ios flags. void close() { if(tbbuf.close()) clear(ios::goodbit); else setstate(ios::failbit); } // set buffer and set ios flags. void tstreambase::setbuf(char* b, int l) { if(tbbuf.setbuf(b, l)) clear(ios::goodbit); else setstate(ios::failbit); } // open stream and set ios flags. void open(const char *, int, int = tbuf::openProtect); // attach opened stream and set ios flags. void attach(T &); // return the buffer. tbuf *rdbuf() { return &tbbuf; } private: tbuf tbbuf; }; // Attempt to open tbuf<>, and set ios flags // accordingly. Opening an already open stream // results in an error. If ios::app (append to // file) is set, also set ios::out. If ios::out // is set, and ios::app, ios::in, and ios::ate // (start at end) are not set, set the ios::trunc bit. template void tstreambase::open(const char *n, int m, int p) { if(m & ios::app) m = ios::out; else if((m & (ios::outios::ateios::appios::in)) == ios::out) m = ios::trunc; if(tbbuf.is_open()) clear(ios::failbit); else if(tbbuf.open(n, m, p)) clear(ios::goodbit); else clear(ios::badbit); } // Attach an opened stream to buffer, and set the ios // bits accordingly. template void tstreambase::attach(T &f) { if(tbbuf.is_open()) setstate(ios::failbit); else if(tbbuf.attach(f)) clear(ios::goodbit); else clear(ios::badbit); } // The itstream, otstream and tstream class are merely // "shell" classes, and serve only to combine the // functionality of it's base class. The only functions // defined are the constructor and destructors, and // open and rdbuf. There are no addition data members // and all functions call directly the functions of the // base class. The default open mode is ios::in for // itstream, ios::out for otstream, and both for // tstream. template class itstream : public tstreambase, public istream { public: itstream() { ; } itstream(const char* n, int m = ios::in, int p = tbuf::openProtect) : tstreambase(n, m ios::in, p) { ; } itstream(T &f) : tstreambase(f) { ; } itstream(T &f, char* b, int l) : tstreambase(f, b, l) { ; } ~itstream() { ; } tbuf *rdbuf() { return tstreambase::rdbuf(); } void open(const char *n, int m = ios::in, int p = tbuf::openProtect) { tstreambase::open(n, m ios::in, p); } }; template class otstream : public tstreambase, public ostream { public: otstream() { ; } otstream(const char* n, int m = ios::out, int p = tbuf::openProtect) : tstreambase(n, m ios::out, p) { ; } otstream(T &f) : tstreambase(f) { ; } otstream(T &f, char* b, int l) : tstreambase(f, b, l) { ; } ~otstream() { ; } tbuf *rdbuf() { return tstreambase::rdbuf(); } void open(const char *n, int m = ios::out, int p = tbuf::openProtect) { tstreambase::open(n, m ios::out, p); } }; template class tstream : public tstreambase, public iostream { public: tstream() { ; } tstream(const char *n, int m, int p = tbuf::openProtect) : tstreambase(n, m, p) { ; } tstream(T &f) : tstreambase(f) { ; } tstream(T &f, char *b, int l) : tstreambase(f, b, l), iostream() { ; } ~tstream() { ; } tbuf *rdbuf() { return tstreambase::rdbuf(); } void open(const char *n, int m, int p = tbuf::openProtect) { tstreambase::open(n, m, p); } }; #endif /* End of File */ Listing 2 tserial.h -- serial stream class definitions #if !defined SERIAL_H #define SERIAL_H #include class ComBuffer; // forward declaration // This class represents everything needed for the // template parameter. It has a default constructor, // open, close, read, write, seek, and status. class SerialStream { public: // Default constructor. Initialize an unopened stream SerialStream() : port(-1) { } // Destructor. calls close. ~SerialStream() { close(); } // The open function sets up the default serial port // parameters, and hook up the interrupts. The name // is check for "COM1" and "COM2". mode and prot are // ignored. int open(const char *name, int mode, int prot); // Read fills buf with len characters from the // circular buffer. This class and any class that // cannot seek should be unbuffered, so read typically // asks for one character at a time. int read(char *buf, size_t len); // Write sends the characters in buf to the port. int write(char *buf, size_t len); // Seek is a dummy function and always retruns error. long seek(long, ios::seek_dir) { return -1; } // close set the port number to -1 for open testing. int close(); // This function is the status function. operator const void *() { return port == -1 ? 0 : this; } private: int port; ComBuffer *buffPtr; }; #endif /* End of File */ Listing 3 tserial.cpp -- serial port stream classes #include #include "tserial.h" #include #include #include #include // interrupt functions. typedef void interrupt far intFunc(...); // Programmed Interrupt Controller constants. const PICO : 0x20; const PIC1 : 0x21; const EOI : 0x20; // Offsets from port address. const InterruptEnable = 1, InterruptIdent = 2, ModemControl = 4, LineStatus = 5, ModemStatus = 6; // Value for the modem control register. const ModemControlValue = 11; // A serial port circular buffer class. There // are only two instances of this class, one // for each port COM1, and COM2. They are // static globals, and are initailized before // main(). They contain a circular buffer, // the port number and port address, and the // interrupt vector they replace, so that // they can be restored at destruction. This // is not a complete or robust class, and is // private to this module. It is intended // only for use by SerialStream. class ComBuffer { public: // Size of circular buffer enum{ BufSize = 0x100 }; // Default constructor. ComBuffer(); ~ComBuffer(); // Set up vector, and enables interrupts void hookInterrupt(); // Get a single character from the buffer int getc(); // Send a single character to the port void putc(int c); // Read the line status register. int status() const { return ::inp(portAddr+LineStatus); } // Check if characters are available int avail() const { return in != out; } // Receive a character from port. This function // is called by the interrupt routine. void receive(); private: // The mask needed for setting the PIC. int interruptMask(int port) const { return 1 << (4 - port); } // The interrupt number for each port. int interruptVec(int port) const { return 12 - port; } char *in, *out, *buff; // The circular buffer int port; // The port number 0-1 int portAddr; // The port address intFunc *oldVector; // The previos vector static int initPort; }; // initPort is used by the constructor to initialize // the ports in sequence. int ComBuffer::initPort = 0; // Only two ports are set up here. If you add more // you need to identify the port addresses and // interrupt vectors. static ComBuffer CommPorts[2]; // Initialize the buffers. Allocate the circular // buffer, set the in/out pointers, set the port // and port address values, and clear the old // vector value. ComBuffer::ComBuffer() : buff(new char[BufSize]), port(initPort++), oldVector(0) { // this line is for sceptics to see initializing. cout << "initializing port #" << port << endl; portAddr = port ? 0x2f8 : 0x3f8; in = out = buff; } // Delete the circular buffer. If the old vector // is a valid address, restore it. ComBuffer::~ComBuffer() { // another line for sceptics. cout << "de-initializing port #" << port << endl; if(oldVector) ::setvect(interruptVec(port),oldVector); delete[] buff; } // These two routines reset to PIC, and put // the new character in the buffer void interrupt far comInterrupt0(...) { ::outp(PICO,EOI); CommPorts[0].receive(); } void interrupt far comInterruptl(...) { ::outp(PICO,EOI); CommPorts[1].receive(); } // This function sets up the interrupts, and // enables the PIC void ComBuffer::hookInterrupt() { if(oldVector==0) oldVector = ::getvect(interruptVec(port)); switch(port) { case 0: ::setvect(interruptVec(port),comInterrupt0); break; case 1: ::setvect(interruptVec(port),comInterrupt1); break; default: return; } ::outp(portAddr+ModemControl, ::inp(portAddr+ModemControl)ModemControlValue); ::outp(PIC1,::inp(PIC1) & ~interruptMask(port)); ::outp(portAddr+InterruptEnable,1); ::outp(PICO, EOI); ::inp(portAddr); ::inp(portAddr+InterruptIdent); ::inp(portAddr+ModemStatus); status(); // clear status reg. } // Receive a byte from the port and place in the // circular buffer, called from the interrupt // handler. void ComBuffer::receive() { if(in == buff + BufSize) in = buff; // circular buffer *in++ = ::inp(portAddr); } // Get a character from the circular buffer. int ComBuffer::getc() { while(!avail()) if(::kbhit()) return -1; if(out == buff + BufSize) out = buff; // circular buffer return *out++; } // Send a character directly to the port void ComBuffer::putc(int c) { while((status() & 0x20) == 0) if(::kbhit()) return; ::outp(portAddr, c); } const default_init =_COM_1200_COM_CHR8 _COM_STOP1_COM_NOPARITY; // The open function needs to translate the name // to the port number, initialize the port, and // enable interrupts. int SerialStream::open(const char *name, int, int) { if(::stricmp(name,"COM1") == 0) port = 0; else if(::stricmp(name, "COM2") == 0) port = 1; else return -1; unsigned int stat = :: bios_serialcom( _COM_INIT,port,default_init); buffPtr = CommPorts+port; buffPtr->hookInterrupt(); if(stat & 0xFF00) return -1; return 0; } // The close routine sets the port number to -1 // to indicate it's closed int SerialStream::close() { port = -1; return 0; } // The read routine repeatedly calls ComBuffer::getc // to fill it's buffer. len should always be 1. int SerialStream::read(char *b, size_t len) { for(int i = 0; i < len; i++) b[i] = buffPtr->getc(); return i; } // The write routine repeatedly calls ComBuffer::putc // to empty it's buffer. len should alway be 1. int SerialStream::write(char *b, size_t len) { for(int i = 0; i < len; i++) buffPtr->putc(b[i]); return i; } /* End of File */ A Safer setjmp in C++ Philip J. Erdelsky Philip J. Erdelsky is an R&D engineer with over eight years experience in C programming. He can be reached at Data/Ware Development, Inc., 9449 Carroll Park Dr., San Diego, CA 92121, or on Compuserve at 75746,3411. Calling setjmp to mark a place in your C program and then calling longjmp to return to it, even from a deeply-nested function call, is considered useful but slightly hazardous. If control has left the function that called setjmp, longjmp will usually crash. Yet the careful use of this perverse pair is an efficient way to transfer control when an error detected in a low-level function must be handled at a much higher level. When using the alternative, passing the error condition laboriously up the chain of functions, testing a flag at each step, you must be careful not to go flying past an intermediate function's garbage collection, leaving memory blocks unreleased, files open, and motors running. Surprisingly, setjmp and longjmp are still available in C++, and they still work the same way. However, C++ provides a way around the principal danger of setjmp and longjmp. Technique The most important part of this technique is to encapsulate the jmp_buf buffer in a class object that has a constructor and a destructor: #include class error { error *previous; public: static error *current; jmp_buf jmpbuf; error() { previous = current; current = this; } ~error() { current = previous; } }; error *error::current; The class has one static member called error::current, that points to the current buffer, the one an error exit will use to make its escape. If a function needs to handle an error discovered at a lower level, just begin it this way: function f(...) { error error_x; // calls constructor for // error_x if (setjmp(error_x.jmpbuf)) { return; // calls destructor for // error_x } // destructor for error_x also // called here } When the buffer error_x is created, its constructor is called. The constructor code links the buffer to a chain of previous error buffers, if any, and makes it the one pointed to by error::current. The if statement puts the context into its jmpbuf member and returns a zero, so processing continues with the function code. When a lower level function detects an error, it calls longjmp (error::current->jmpbuf,1); This brings control back to the garbage-collection routines after the if statement. When the return statement is executed, the destructor for error_x is called, the buffer is removed from the chain, and any subsequent longjmp (error::current->jmpbuf, 1) will use the previous buffer, not this one. Moreover, if control returns from the function f without detecting an error, the destructor for error_x is called automatically. These are the principal safety features of this technique. Actually, an important part of the garbage collection is performed automatically by C++. The destructors for other variables, if any, are called automatically. If garbage collection is needed in some intermediate function, you have to be a little more careful. You might put something like this at the head of the intermediate function: error error_x; if (setjmp (error_x.jmpbuf)) { error *p = error:: current->previous; error::~error(error_x); longjmp (p->jmpbuf,1); } You can safely reuse the name error_x because each is local to its own function. Here is where you run into a little difficulty. C++ is very good about calling destructors for local objects when control leaves their scope, but not when exit is made via longjmp. Therefore, the destructors for error_x and other local variables, if any, must be called explicitly before bailing out. You can avoid this drudgery if you are willing to sacrifice portability and are using a compiler like Borland C++ 2.0 that uses negative frame addressing. Just use the function declare_error, followed by an immediate return, as shown in Listing 1. Negative frame addressing makes it possible for a carefully coded (but admittedly nonportable) function like declare_error to find the return address and patch it so that when the following return statement is executed, the destructors for error_x, and other local variables, if any, are called, and then return is made, not to calling function, but to the special function error_exit, which then calls longjmp. Of course, this involves some additional runtime overhead, but error exits are not common and need not be fast. The function declare_error, followed by an immediate return statement, can also be called when the error is first discovered, if the function from which it is called needs to call destructors for some of its local variables. Even if you have to do this for every intermediate function, you can still save a lot of code if each intermediate function is called from many places. Occasionally, an error occurs in a non-function block and needs to be handled in a larger enclosing block. This technique doesn't work so well because there is no practical way for declare_error to find the block exit. However, the old break and goto statements will usually suffice in such cases. Limitations This technique isn't foolproof. It can still fail if an error is declared in a function called by a constructor or initializer for a variable declared before error_x: f(...) { class something old; int x = g(); error error_x; if (setjmp (error_x.jmpbuf)) { declare_error(); return; } If an error is declared in g, control will pass to a higher level, and the destructor for old won't get called. Don't even think of trying to get around this by declaring error_x first. If an error is declared in g, an attempt will be made to pass control with an uninitialized error_x.jmpbuf. If memory deallocation is the only garbage collection involved, a more sophisticated memory allocator should be able to straighten things out, but such a technique is beyond the scope of this article. This is a safer setjmp, about the safest available in C++. For a much safer setjmp, you have to use other languages that may keep you from doing what you really want to do. Listing 1 An example of using negative frame addressing via the declare_error function in Borland C++ to find a return address static void near error_return(void) { longjmp(error::current->jmpbuf, 1); } void declare_error(void) { _AX = (int) error_return; asm MOV BP,[BP]; // move frame pointer up one level asm MOV [BP+2],AX; // patch return address #if defined(__MEDIUM__) defined(__LARGE__) defined(__HUGE__) asm MOV [BP+4],CS; // if return address is far, patch segment, too #endif } error error_x; if (setjmp(error_x.jmpbuf)) { declare_error(); return; } /* End of File */ The UUCP g Protocol Ian Lance Taylor Ian Lance Taylor took a course in COBOL at the Cambridge Rindge and Latin High School, from which he graduated in 1982. Since then he has written an ANSI C library for a comuter system which nobody has ever heard of, has ported the DECUS version of Zork from Fortran to C, has written a complete (UNIX only) UUCP package, and has read Knuth from cover to cover. He works for Cygnus Support, a company dedicated to supporting free software. He can be reached at ian@airs.com. Introduction The UUCP suite of programs is widely used in the UNIX world to transfer mail and news between computers. UUCP implementations are also available for many types of personal computers. This article is a detailed description of the original, and most frequently used, UUCP transport-layer protocol. The basic UUCP service is unattended file transfer over telephone lines or a network connection. Other programs build on that base to provide remote program execution. The first UUCP implementation was written by Mike Lesk at AT&T in 1976. UUCP has been rewritten and enhanced several times since then, most notably by Peter Honeyman, David A. Nowitz, and Brian E. Redman in 1983, who wrote what is known as HoneyDanBer UUCP. I recently rewrote the program suite from scratch, in order to distribute it in sourcecode form under the GNU Public License. My implementation is modestly entitled Taylor UUCP. UUCP is a point-to-point protocol, which means that it is used between two computers at a time. The protocol has two layers, a session layer and a transport layer. The two computers use the session layer to identify each other and to agree on which files should be transferred. The session layer relies on the transport layer to provide error-free data transfer across the communication link. UUCP uses several different transport-layer protocols, which are optimized for different types of communication links. Not all UUCP implementations support every protocol. When two computers begin a UUCP session, they negotiate which transport layer protocol to use. The transport protocols are known by single character names. The most common ones are g, f, t, and e. In the article I am going to discuss the g protocol. It is the original UUCP transport-layer protocol, and it is the only one supported by all UUCP implementations. It is based on a packet-driver protocol originally written by G.L. Chesson at Bell Labs. It is intended to work over unreliable connections, such as phone lines, but it requires an eight-bit transparent connection. If any special characters cannot be sent over the line, such as XON or XOFF, it will not work. There is not enough room here to present the complete g protocol implementation used by Taylor UUCP, so this article merely shows the high points. Listing 1 shows a number of typedefs, macro definitions, variables, and prototypes that are used in the later code. The complete Taylor UUCP code is available for anonymous FTP from various sites, including ftp.uu.net. Since the g protocol works over noisy phone lines, it has to be prepared for data to be modified, or even lost, as it is sent from one computer to the other. (Some types of network connections can also duplicate data. The g protocol does not always handle this correctly). This protocol provides flow control. The computer sending data is permitted to send only a certain number of bytes to the receiving computer before it must wait for an acknowledgement. That prevents a fast computer from overwhelming a slower computer with more data than the latter can handle. The g protocol communicates in packets. There are two types of packets, control packets and data packets. The header of a data packet tells the receiver how much data is present. It also has a checksum which is used to detect transmission errors. Packet Header All packets begin with a six-byte header. Control packets have no attached data, and are therefore exactly six bytes long. Data packets contain data following the header bytes. The header bytes are shown in Figure 1. The first header byte is always the ASCII character DLE, which is octal 020 or control-P. If there is a transmission error, this byte can be used to locate the start of the next header. The second header byte is known as K. For a control packet, the value is always 9. For a data packet, K indicates how many bytes of data follow the six-byte header. The amount of data is 2K+4. The K value is always between 1 and 8, which means that the amount of data in a single packet can be any power of 2 between 32 (21+4) and 4,096 (28+4). The third and fourth header bytes form a two-byte (16-bit) checksum. The third byte is CHECKLOW and the fourth byte is CHECKHIGH. The full checksum is (CHECKHIGH << 8) CHECKLOW The checksum differs for control and data packets, and is described further below. The fifth header byte is the control byte. It is composed of three bit fields, known as TT (2 bits), XXX (3 bits), and YYY (3 bits). The value of the control byte is (TT << 6) (XXX << 3) YYY TT indicates the type of the packet: 0 -- control packet 1 -- alternate data channel 2 -- data packet 3 -- short data packet The meanings of XXX and YYY depend upon the type of packet, and are described further below. The alternate-data-channel packet type (with a TT value of 1) is not used by UUCP. The sixth and last header byte is a simple check on the validity of the header. It is the exclusive-OR of the second through fifth header bytes (the first byte is always DLE, and so does not have to be checked further). If the exclusive-OR check fails, the header data is invalid and is ignored. Windows and Flow Control Each data packet has a packet sequence number, which is the XXX value. The packet sequence number is always between 0 and 7. At the beginning of the session it starts at 1, goes up to 7, then wraps around to 0. The sequence numbers for incoming and outgoing packets are independent. After one computer sends out packet 7, the next packet it sends out is packet 0, regardless of how many intervening packets it has received. The packet sequence number is used to detect data loss. If the packet following packet 5 is not packet 6, then information has been lost. Some protocols, such as the Internet TCP protocol, permit packets to arrive out of order, but the g protocol considers this an error. Control packets have no sequence number, so there is no way to detect a lost control packet. The packet sequence number is also used for flow control. During protocol initialization each computer announces a window size. The window announced by one computer is the number of packets the other computer may send before it sees an acknowledgement. There are two ways to send an acknowledgement. One is the YYY field of a data packet. The other is the YYY field of an RR control packet. (Control packets are described further below.) The two methods allow the acknowledgement to be either combined with data or not combined, depending on whether the acknowledging computer has any data to send. In either case, the acknowledgement gives the last packet number which was correctly received. Every packet must be acknowledged, and every packet must be acknowledged in order. For example, suppose that computer A has announced a window size of 3, and that at some point during the conversation A has sent an acknowledgement for packet 6. This means that computer B may send 3 more packets--namely packets 7, 0, and 1--but it may not send packet 2 until it has received an acknowledgement for packet 7. Note that the window size announced by A is a restriction on which packets B is permitted to send. The window size may range from 1 to 7. A window size of 1 means that each packet must be acknowledged before the next one is sent. To see why the window size may not be 8, suppose that computer A has just sent packet 0, and received an acknowledgement for it. If the window size were 8, A could then send packets 1, 2, 3, 4, 5, 6, 7, and 0. If all eight packets were lost, B might time out, assume that its acknowledgement was lost, and acknowledge packet 0 again. When A saw the acknowledgement, it would not know whether the eight packets it sent were lost or whether B saw them all but all the acknowledgements but the last one were lost. With a maximum window size of 7, each packet acknowledgement is unambiguous. The original UUCP implementation always used a window size of 3, and some implementations have followed its lead. Many newer implementations default to the maximum window size of 7. The window size prevents a fast computer from overwhelming a slow computer. If there were no way to slow down the fast computer by forcing it to wait for an acknowledgement, it could eventually fill the input buffers of the slow computer and cause data to be lost. The problem would be detected, since there would be checksum failures, but the time required for the computers to get back in sync would slow the overall data transfer rate drastically. Many systems rely on XON/XOFF handshaking for flow control. Data transmission stops when an XOFF character (control-S) is received, and starts again when an XON character (control-Q) is received. This does not work with the g protocol, which requires an eight-bit transparent communication line. (For example, one of the checksum bytes in the header might be XON or XOFF.) Some modern error-correcting modems and serial ports can use various forms of hardware handshaking for flow control. This does not make the g protocol useless, since error detection is still necessary for the connection between the modem and the computer. In any case, a large window is relatively efficient even when it is not needed. Data Packets A data packet is indicated by a TT value of 2 or 3. The K value must be between 1 and 8, for a packet size between 32 and 4,096 bytes. This is the number of bytes which follow the header. For example, a header with a K value of 2 indicates a total packet size of six header bytes plus 64 data bytes for a total of 70 bytes. During protocol initialization, each side announces the largest data packet it is prepared to receive. The original UUCP implementation always set the maximum data-packet size to just 64 bytes. Many later implementations have followed suit. While this small packet size was once reasonable, modern high-speed error-correcting modems are much more efficient when larger packet sizes are used. Not every file contains a multiple of 32 bytes, which is the minimum packet size. Therefore, there has to be a way to transfer fewer bytes in a single packet. This is done with a short data packet, with a TT value of 3. The K value always indicates the physical number of bytes that are sent over the communication link. However, a short data packet contains a smaller number of logical bytes. If the first data byte of a short data packet is less than 128, then that byte is subtracted from the physical packet length to get the logical packet length. The real data starts with the second byte. Otherwise a 15-bit value is formed with the low-order seven bits of the first byte and the complete second byte. The logical packet length is this value subtracted from the physical packet length, and the real data starts with the third byte. The logical length is given using subtraction because, if the logical length is only one byte less than the physical length, there is only a single byte available to specify the logical length. The data-packet layout is designed so that the data can be manipulated as a block, without having to consider each individual byte (other than when computing the checksum). Some other communications protocols, such as Zmodem, use escape characters embedded within the data. This forces the code to examine each data byte and make a decision about it, which is less efficient. Listing 2 shows the fgsenddata function, which sends out a data packet. The zdata argument points to the data, and the six bytes before zdata are available to hold the header (ensuring that these six bytes are available saves a call to memcpy in the common case). The cdata argument is the length of the data in zdata. If the packet size is larger than 64, then this code assumes that it is dealing with a newer UUCP implementation which will be able to handle different packet sizes within the same session. Listing 2 shows a call to the igchecksum function, which is shown in Listing 3. Note that every byte sent with the data packet participates in the checksum, even though the values of some of the bytes may be unimportant in a short data packet. The g protocol checksum is an ad hoc hashing scheme. It is the most time-consuming part of the entire protocol, and it is not even particularly good at detecting errors. A cyclic redundancy check would be more efficient to compute and would also catch more errors. For a brief but interesting discussion of this, see the book Design and Validation of Computer Protocols, by Gerard J. Holzmann. Note that after the checksum is computed it is manipulated still further before being placed in the packet header, as shown in Listing 2. Control Packets A control packet is indicated by a TT value of 0 and a K value of 9. The XXX value indicates the type of control packet: 0 -- not used 1 -- CLOSE 2 -- RJ 3 -- SRJ 4 -- RR 5 -- INITC 6 -- INITB 7 -- INITA A CLOSE packet is sent when the conversation is finished, and the g protocol is shutting down. It is also sent if some fatal error is forcing one computer to drop the connection. The YYY value is ignored. The RJ control is also known as NACK. It means that there was some problem with a received packet, such as a checksum error, and that data must be resent. The YYY value is the last successfully received packet. Every subsequent packet must be resent. The SRJ control is actually not used by UUCP. Most UUCP programs will not recognize it. It is supposed to request retransmission of just packet YYY. The RR control is also known as ACK. It is used to acknowledge a data packet without sending a data packet in return. (Acknowledgements were discussed above in the section on windows and flow control). The YYY value is the packet being acknowledged. The INIT controls are used during protocol initialization. When the protocol starts up, each computer exchanges pairs of each type of INIT packet. The computer which placed the call sends an INITA, then the other computer responds with an INITA, and similarly for INITB and INITC. The YYY value for an INITA or INITC packet is the window size that the other computer should use. The YYY value for an INITB packet indicates the packet size that the other computer should use. This is encoded as one less than the K value for the desired packet size, so that it is a value from 0 to 7. The initialization exchange is not a negotiation. Each computer simply makes a request. Both computers may wind up using different window and packet sizes. Listing 4 shows the code to send a control packet, and shows how the checksum value is computed. Error Handling Error handling is an important aspect of any communications protocol. The bulk of the Taylor UUCP implementation is devoted to it, although there is unfortunately not enough space to present the actual code. A protocol must be able to detect communication errors to ensure that it receives correct data. After an error occurs, the protocol must be able to quickly recover and continue transmitting information. There is no way to ensure completely reliable data transfer across telephone lines. They can, in theory, arbitrarily corrupt data. In practice, however, telephone lines are fairly reliable. A fairly simple checksum is thus adequate to detect data corruption. The g protocol relies on checksums for error detection, and negative acknowledgements and timeouts for error recovery. An error in a packet header can be detected by an invalid exclusive-OR byte or an invalid K value. After an error, the receiving computer must start looking for a new DLE byte. It must start looking at the second byte of the header. It may not skip the rest of the header, and it may certainly not skip the entire packet. If data were lost, then the start of the next packet might appear in the skipped data. Another DLE byte might be found that is actually part of the data in the corrupted packet, but the exclusive-OR byte should prevent it from being treated as a legitimate packet header. An error in packet data is detected by an invalid checksum. If an invalid packet is seen, the receiving computer will send an RJ packet to tell the sending computer that the packet must be resent. The receiver must then start looking for a new DLE byte. Once again, it must not skip the packet data, since the checksum error might have been caused by lost data. The next packet header might be somewhere within the bytes which were assumed to be the data for the erroneous packet. An RJ packet includes the sequence number of the last correctly-received packet. All subsequent packets must be resent. This is referred to as a go-back-N protocol. After an error, the sending computer must resend all the packets from a particular point, rather than just resend the erroneous packet. An SRJ packet could be used to request just the erroneous packet, but most UUCP implementations do not implement it. A go-back-N protocol can be less efficient than just resending a single packet, but it is easier to implement. There are some concerns with a go-back-N protocol. If computer A is temporarily overloaded with work, it might not find time to process its input buffers, and data might be lost. When it notices the loss, it will send an RJ packet. Computer B is permitted to respond with an entire window full of data, but if it does computer A might simply get overloaded again. Taylor UUCP uses a simple slow-start algorithm, which temporarily shrinks B's sending window to a single packet and opens it up slowly as each acknowledgement is received. Part of this code can be seen at the end of Listing 2. A lost packet will cause the next packet to be seen as an out-of-order packet, a case which requires a bit of thought. If A detects an invalid packet, it will send an RJ packet. It will then expect to see another copy of the invalid packet. However, B will probably have already sent the next few packets. It may even have sent an entire window full of packets. If A sends an RJ packet for each out-of-order packet, B will see a sequence of RJ packets all requesting the same invalid packet, which it will have to send multiple times. Taylor UUCP simply ignores out-of-order packets, and relies on a timeout to detect completely lost packets. More sophisticated approaches are possible. Although phone lines cannot duplicate data, certain types of network connections can. This can confuse the g protocol. Suppose that the window size is 7, and that A sends out packets 4, 5, 6, 7, 0, 1, 2. After A received an acknowledgement for packet 4, it will send out packet 3. If packet 4 is duplicated, and the duplicate arrives at B just after packet 3 arrives, B will see the wrong data. This is a potential problem for any windowing protocol. It is generally avoided by making the window large enough to ensure that any duplicate packets will arrive before a complete window is sent. For example, the sequence number used by the Internet TCP protocol is a full 32 bits. The UUCP Session Layer The g protocol is used by the UUCP session layer to transmit UUCP commands and data files. UUCP commands are simple strings. When using the g protocol, they are sent as a sequence of data packets (not short data packets). The last data byte of the last packet in the sequence is a null byte, and the string is itself terminated by a null byte. Normally only one or two packets are required. UUCP sends a file by simply sending the data in a sequence of data packets or short data packets. At the end of the file a short data packet is sent with a logical packet length of zero bytes. References Holzmann, Gerard J. 1991. Design and Validation of Computer Protocols. Englewood Cliffs, NJ: Prentice-Hall. Figure 1 UUCP g data-packet header bytes DLE K CHECK CHECK CONTROL XOR LOW HIGH Listing 1 Various typedefs, macro definitions, variables, and prototypes /* The boolean type. */ typedef int boolean; #define TRUE (1) #define FALSE (0) /* Names for the bytes in the frame header. */ #define IFRAME_DLE (0) #define IFRAME_K (1) #define IFRAME_CHECKLOW (2) #define IFRAME_CHECKHIGH (3) #define IFRAME_CONTROL (4) #define IFRAME_XOR (5) /* Length of the frame header. */ #define CFRAMELEN (6) /* Macros to break apart the control bytes. */ #define CONTROL_TT(b) ((int)(((b) >> 6) & 03)) #define CONTROL_XXX(b) ((int)(((b) >> 3) & 07)) #define CONTROL_YYY(b) ((int)((b) & 07)) /* DLE value. */ #define DLE ('\020') /* Get the length of a packet given a pointer to the header. */ #define CPACKLEN(z) (1 << ((z)[IFRAME_K] + 4)) /* field value for a control message. */ #define KCONTROL (9) /* Get the next sequence number given a sequence number. */ #define INEXTSEQ(i) ((i + 1) & 07) /* Compute i1 - i2 modulo 8. */ #define CSEQDIFF(i1, i2) (((i1) + 8 - (i2)) & 07) /* Packet types. These are from the TT field. */ #define CONTROL (0) #define ALTCHAN (1) #define DATA (2) #define SHORTDATA (3) /* Control types. These are from the XXX field if the type (tt field) is CONTROL. */ #define CLOSE (1) #define RJ (2) #define SRJ (3) #define RR (4) #define INITC (5) #define INITB (6) #define INITA (7) /* Maximum amount of data in a single packet. */ #define CMAXDATAINDEX (8) #define CMAXDATA (1 << (CMAXDATAINDEX + 4)) /* Maximum window size. */ #define CMAXWINDOW (7) /* The timeout to use when waiting for a packet. Protocol parameter ''timeout''. */ #define CTIMEOUT (10) /* The number of times to retry waiting for a packet. Each time the timeout fails we send a copy of our last data packet or a reject message for the packet we expect from the other side, depending on whether we have any unacknowledged data. This is the number of times we try doing that and then waiting again. Protocol parameter ''retries''. */ #define CRETRIES (6) /* Next sequence number to send. */ int iGsendseq = 1; /* Last sequence number that has been acked. */ int iGremote_ack = 0; /* Last sequence number to be retransmitted. */ int iGretransmit_seq = -1; /* Last sequence number we have received. */ static int iGrecseq; /* Remote segment size (set during protocol initialization). This is one less than the value in a packet header. */ int iGremote_segsize; /* Remote packet size (set based on iGremote_segsize). */ int iGremote_packsize; /* Remote window size (set during handshake). */ static int iGremote_winsize; /* Timeout (seconds) for receiving a data packet. Protocol parameter ''timeout''. */ static int cGtimeout = CTIMEOUT; /* Maximum number of timeouts when receiving a data packet or acknowledgement. Protocol parameter ''retries''. */ static int cGretries = CRETRIES; /* Get a packet. This is called to wait for a packet to come in when there is nothing to send. If freturncontrol is TRUE, this will return after getting any control packet. Otherwise, it will continue to receive packets until a complete file or a complete command has been received. The timeout and the number of retriesare arguments. The function returns FALSE if an error occurs or if cretries timeouts of ctimeout seconds were exceeded. */ boolean fgwait_for_packet (boolean freturncontrol, int ctimeout, int cretries); /* Send data to the other system. If the fdoread argument is true, this will also read data into abPrecbuf; fdoread is passes as TRUE if the protocol expects data to be coming back, to make sure the input buffer does not fill up. Returns FALSE on error. */ boolean fsend_data (const char *zsend, int csend, boolean fdoread); /* End of File */ Listing 2 The fgsenddata function sends out a data packet. boolean fgsenddata (zdata, cdata) char *zdata; int cdata; { char *z; int itt, iseg, csize; unsigned short icheck; itt = DATA; csize = iGremote_packsize; iseg = iGremote_segsize + 1; if (cdata < iGremote_packsize) { /* If the remote packet size is larger than 64, the default, we can assume they can handle a smaller packet as well, which will be more efficient to send. */ if (iGremote_packsize > 64) { /* The packet size is 1 << (iseg + 4). */ iseg = 1; csize = 32; while (csize < cdata) { csize <<= 1; ++iseg; } } if (csize != cdata) { int cshort; /* We have to move the data within the packet, unfortunately. It only happens once per file transfer. It would also be nice if we computed the checksum as we move. We zero out the unused bytes. */ itt = SHORTDATA; cshort = csize - cdata; if (cshort <= 127) { memmove (zdata + 1, zdata, cdata); zdata[0] = (char) cshort; memset (zdata + cdata + 1, 0, cshort - 1); } else { memmove (zdata + 2, zdata, cdata); zdata[0] = (char) (0x80 (cshort & 0x7f)); zdata[1] = (char) (cshort >> 7); memset (zdata + cdata + 2, 0, cshort - 2); } } } z = zdata - CFRAMELEN; z[IFRAME_DLE] = DLE; z[IFRAME_K] = (char) iseg; icheck = (unsigned short) igchecksum (zdata, csize); /* Wait until there is room in the receiver's window for us to send the packet. We do this now so that we send the correct value for the last packet received. Note that if iGsendseq == iGremote_ack, this means that the sequence numbers are actually 8 apart, since the packet could not have been acknowledged before it was sent; this can happen when the window size is 7. */ while (iGsendseq == iGremote_ack CSEQDIFF (iGsendseq, iGremote_ack) > iGremote_winsize) { if (! fgwait_for_packet (TRUE, cGtimeout, cGretries)) return FALSE; } /* Ack all packets up to the next one, since the UUCP protocol requires that all packets be acked in order. */ while (CSEQDIFF (iGrecseq, iGlocal_ack) > 1) { iGlocal_ack = INEXTSEQ (iGlocal_ack); if (! fgsend_control (RR, iGlocal_ack)) return FALSE; } iGlocal_ack = iGrecseq; z[IFRAME__CONTROL] = (char) ((itt << 6) (iGsendseq << 3) iGrecseq); iGsendseq = INEXTSEQ (iGsendseq); icheck = ((unsigned short) ((0xaaaa - (icheck ^ (z[IFRAME_CONTROL] & 0xff))) & 0xffff)); z[IFRAME_CHECKLOW] = (char) (icheck & 0xff); z[IFRAME_CHECKHIGH] = (char) (icheck >> 8); z[IFRAME_XOR] = (char) (z[IFRAME_K] ^ z[IFRAME_CHECKLOW] ^ z[IFRAME_CHECKHIGH] ^ z[IFRAME_CONTROL]); /* If we're waiting for acks of retransmitted packets, then don't send this packet yet. The other side may not be ready for it yet. Instead, code infggot_ack will send the outstanding packets when an ack is received. */ if (iGretransmit_seq != -1) return TRUE; return fsend_data (z, CFRAMELEN + csize, TRUE); } /* End of File */ Listing 3 The igchecksum function int igchecksum (z, c) register const char *z; register int c; { register unsigned int ichk1, ichk2; ichk1 = 0xffff; ichk2 = 0; do { register unsigned int b; /* Rotate ichk1 left. */ if ((ichk1 & 0x8000) == 0) ichk1 <<= 1; else { ichk1 <<= 1; ++ichk1; } /* Add the next character to ichk1. */ b = *z++ & 0xff; ichk1 += b; /* Add ichk1 xor the character position in the buffer counting from the back to ichk2. */ ichk2 += ichk1 ^ c; /* If the character was zero, or adding it to ichk1 caused an overflow, xor ichk2 to ichk1. */ if (b == 0 (ichk1 & 0xffff) < b) ichk1 ^= ichk2; } while (--c > 0); return ichk1 & 0xffff; } /* End of File */ Listing 4 This listing shows the code or sending a static boolean fgsend control (ixxx, iyyy) int ixxx; int iyyy; { char ab[CFRAMELEN]; int ict1; unsigned short icheck; ab[IFRAME_DLE] = DLE; ab[IFRAME_K] = KCONTROL; ict1 = (CONTROL << 6) (ixxx << 3) iyyy; icheck = (unsigned short) (0xaaaa - ict1); ab[IFRAME_CHECKLOW] = (char) (icheck & 0xff); ab[IFRAME_CHECKHIGH] = (char) (icheck >> 8); ab[IFRAME_CONTROL] = (char) ict1; ab[IFRAME_XOR] = (char) (ab[IFRAME_K] ^ ab[IFRAME_CHECKLOW] ^ ab[IFRAME_CHECKHIGH] ^ ab[IFRAME_CONTROL]); return fsend_data (ab, CFRAMELEN, TRUE); } /* End of File */ An Essential String Function Library William Smith William Smith is the engineering manager at Montana Software, a software development company specializing in custom applications for MS-DOS and Windows. You may contact him by mail at P.O. Box 663, Bozeman, MT 59771-0663. The include file in the Standard C library defines 22 functions for manipulating character strings. Seventeen of these functions begin with the prefix str and another five begin with the prefix mem. The functions that begin with the prefix str work on null-terminated strings. They accomplish such critical tasks as finding the length of a string, concatenating strings, or comparing strings, to name just a few of the tasks. The functions that begin with mem work on any buffer of memory. These functions do not interpret a null character as a terminator. At first, the functions in appear to offer a broad smorgasbord of functionality. I originally expected them to satisfy most string-processing requirements I would encounter. In actuality, I repeatedly encountered situations where what I needed to do I could not accomplish with a single call to one of the Standard C functions. But they are good building blocks. I found myself using and grouping them to accomplish what I really needed. Most of the string processing tasks I am faced with center around manipulating text data for input and output. I nearly always have to parse and convert some text script file or user input into a data structure and vice versa. Over time, in just about every program I wrote, the specific needs for text processing started to repeat themselves. I frequently needed to delete and insert strings, or trim leading and trailing tabs and spaces from text. These and many other requirements were common to nearly every project I would work on. After almost ten years of programing in C, a group of about an additional 20 functions has precipitated and become a crucial part of my C function library. I am going to share with you the most recent incarnation of my bare bones but essential string function library. These functions complement the Standard C functions defined in Dynamic Memory Issues When writing string functions, you can go in a couple of different directions with regards to dynamic memory. You can dynamically allocate memory to store the string that results from a function's execution of a task. This approach allows you to avoid modification of the original string passed to the function. For example, when performing search and replace, you can use dynamic memory to store the string that contains the modifications. The function can then leave the original string unchanged. However, when using this approach, the programmer must keep track of memory allocation and make sure to release the allocated memory eventually. This can be a challenge in certain situations. Use of dynamic memory may be more suitable in C++. C++ is better organized to provide object creation and deletion. This helps with dynamic memory management and relieves some of the burden on the programmer. Since the functions presented here are pure Standard C, I choose to avoid dynamic memory. In fact, I also choose to avoid creating buffers on the stack as a scratch or work space. Some of the editing functions require a temporary work space, but I get around this by using the memmove function defined in . memmove provides safe memory copying of overlapping buffers. You would need a temporary copy of the source buffer to do it yourself. Although convenient, using memmove has the disadvantage of being more costly with respect to processor time. This varies from system to system, but generally there are usually faster ways to accomplish the same task as memmove. Modifying the functions to avoid the use of memmove can wring a bit more performance and efficiency out of them. In the future, it might be worthwhile to create object wrappers in C++ for these string functions. For now, I will leave them in standard C. This means that all the functions assume that the strings that you pass to them are NULL-terminated. The functions also assume that the strings are pointers to memory areas that are large enough to accommodate the resulting string generated by the function. The burden of avoiding buffer overflows rests on the programer. Implementation I break the string functions up into two categories. I group functions for extracting or finding a substring in a string into the file named STR_NGET.C (Listing 1 and Listing 2). The second group, in STR_EDIT.C (Listing 3 and Listing 4), contains functions that I use for editing strings. Functions for Getting Substrings STR_NGET. C contains the functions str_nleft, str_nmid, str_nright and str_rstr. The first three functions extract a specified number of characters from a string. These functions modify the string itself by moving the desired characters into it. str_nleft extracts the n left- most characters. str_nright extracts the n right-most characters. str_nmid extracts n characters from a specified position. str_rstr resembles the function strstr defined in . But instead of finding the first occurrence of a substring, str_rstr finds the last occurrence of the substring. The relationship between strstr and str_rstr is analogous to the relationship between strchr and strrchr. I have seen a function called strrstr in some libraries that come with commercial compilers. It is equivalent to my function str_rstr. It is not a part of the standard. Functions for Editing Strings All the 13 functions in the file STR_EDIT.C do some type of modification or editing to a string. The functionality ranges from the simple padding of strings to a fixed length for justification to complete search and replace. str_center, str_ljust, and str_rjust justify strings. These functions first trim leading and trailing spaces and tabs from a string. They then move the string so it is either centered, left-justified, or right-justified within a specified length. The trimming functions, str_ltrim, str_rtrim and str_trim execute the trimming tasks required by the justification functions mentioned above. These functions trim all characters from the end or ends of a string that match a list of characters to trim. The function str_delete removes a specified number of characters from a string starting at a designated location within the string. The function str_insert inserts a string into a string at a designated location in the string. The function str_rplc uses both str_delete and str_insert implement a search and replace capability. str_mrplc does search and replace for all matches. str_rplc just replaces the first match. The function str_repeat builds a string of desired length by repeating a string. The function str_vcat is a variable-argument version of the Standard C function strcat. This function concatenates a list of strings. The last string or parameter passed to str_vcat must be a null-pointer. str_ocat is a version of strcat that can handle overlapping strings. An example of overlapping strings would be a single string with multiple pointers to different locations in the string. Depending on the compiler vendor, sometimes strcat will work with overlapping strings, sometimes it will not. For safety and constancy I created the function str_ocat. str_ocat is just wrapper for memmove. Conclusions Nearly every major program I have written has involved text processing in some form. The Standard C library provides a useful, but shallow group of string-manipulation functions. Over time and out of need, I have come up with the group of string functions presented here. These functions build upon the standard library functions and provide the functionality that I have found important in practice. There are an endless number of more functions you can invent. And you can probably find more efficient ways to implement the functions demonstrated here. Nevertheless, these are the functions I have found useful and essential in my work with C. Listing 1 STR_NGET.C - functions for extracting or finding a substring /***************************************************** File Name: STR_NGET.C Description: Library of functions for geting substrings in a string Global Function List: str_nleft str_nmid str_nright str_rstr Portability: Standard C *****************************************************/ #include #include #include /***************************************************** Name: str_nleft Expanded Name: Get Left N Characters Parameters: Str - string to get left characters Num - number of characters to get Return: Str Description: Get Num leftmost charcters in Str. Modifies Str. *****************************************************/ char *str_nleft( char *Str, size_t Num ) { if ( Num < strlen( Str )) { Str[Num] = '\0'; } return ( Str ); } /* function str_nleft */ /***************************************************** Name: str_nmid Expanded Name: Get Middle N Characters Parameters: Str - string to get substring in Pos - index into Str of start of midstr Num - count of charcters to get Return: Str Description: Get Num chars from middle of string. *****************************************************/ char *str_nmid( char *Str, size_t Pos, size_t Num ) { char *Mid; size_t Len = strlen( Str ); if ( Pos >= Len ) { /* Outside of string */ *Str = '\0'; return ( Str ); } /* Adjust count if it extends outside of string */ if ( Pos + Num > Len ) { Num = Len - Pos; } Mid = &Str[Pos]; memmove( (void *)Str, (void *)Mid, Num ); Str[Num] = '\0'; return ( Str ); } /* function str_nmid */ /***************************************************** Name: str_nright Expanded Name: Get Right N Characters Parameters: Str - string to get right characters Num - number of characters to get Return: Str Description: Get Num righmost characters in Str. Modifies Str. *****************************************************/ char *str_nright( char *Str, size_t Num ) { size_t Len = strlen( Str ); return ( str_nmid( Str, ( Num > Len ? 0 : Len - Num ), min( Num, Len ) ) ); } /* function str_nright */ /***************************************************** Name: str_rstr Expanded Name: String Right (Reverse) Search Parameters: Str - string to search Find - string to search for Return: Pointer to last occurrence of substring Find in Str or NULL if not found Description: Searches for last occurrence of sub string Find within Str. *****************************************************/ char *str_rstr( char *Str, char *Find ) { char *StrResult = NULL, *StrWork = Str; while ( ( StrWork = strstr( StrWork, Find ) ) != NULL ) { StrResult = StrWork; StrWork++; } return ( StrResult ); } /* function str_rstr */ /* End of File */ Listing 2 STR_NGET.H - contains function prototypes for Listing 1 /***************************************************** File Name: STR_NGET.H Description: Include file for STR_NGET.C *****************************************************/ #if !defined ( STR_NGET_DEFINED ) #define STR_NGET_DEFINED char *str_nleft( char *Str, size_t Num ); char *str_nmid( char *Str, size_t Pos, size_t Num ); char *str_nright( char *Str, size_t Num ); char *str_rstr( char *Str, char *Find ); #endif /* End of File */ Listing 3 STR_EDIT.C - functions for editing strings /**************************************************** File Name: STR_EDIT.C Description: Library of functions for editing strings Global Function List: str_center str_delete str_insert str_ljust str_ltrim str_mrplc str_ocat str_repeat str_rjust str_rplc str_rtrim str_trim str_vcat Portability: Standard C ****************************************************/ /* Standard C */ #include #include #include /* Own */ #include /***************************************************** Name: str_center Parameters: Str - string to center Len - num of chars of centered string Return: Str Description: Centers a string in a desired length by removing tabs and adding spaces to both sides of string. *****************************************************/ char *str_center( char *Str, size_t Len ) { size_t LenOrg; /* Trim spaces and tabs off the string */ str_trim( Str, "\t" ); LenOrg = strlen( Str ); if ( Len <= LenOrg ) { /* The desired string length is shorter than ** the original so return */ return ( Str ); } /* Add the spaces to each side */ str_rjust( Str,( LenOrg + Len ) / 2 ); str_ljust( Str, Len ); return ( Str ); } /* function str_center */ /***************************************************** Name: str_delete Parameters: Str - string to edit Pos - index to start deleting chars at Num - number of charcters to delete Return: Str Description: Modifies Str, by deleting Num chars beginning at Pos. *****************************************************/ char *str_delete( char *Str, char *Pos, size_t Num ) { size_t Len = strlen( Str ); if ( ( Pos >= &Str[Len] ) ( Num == 0 ) ) { /* Outside string or no chars to delete */ return ( Str ); } Num = min( Num, strlen( Pos ) ); if ( Num ) { /* Delete characters by contactenating */ memmove( Pos, &Pos[Num], strlen( &Pos[Num] ) + 1 ); } return ( Str ); } /* function str_delete */ /***************************************************** Name: str_insert Parameters: Str - string to edit Pos - pointer to location withing Str Insrt - string to insert into Str Return: Str Description: Inserts a string Insrt into Str at Pos *****************************************************/ char *str_insert( char *Str, char *Pos, char *Insrt ) { size_t Len = strlen( Insrt ); char *Tmp = &Pos[Len]; memmove( Tmp, Pos, strlen( Pos ) + 1 ); memmove( Pos, Insrt, Len ); return ( Str ); } /* function str_insert */ /***************************************************** Name: str_ljust Parameters: Str - string to left justify Len - length of string Return: Str Description: Pads right end of Str with spaces to left justify Str to a new length Len. *****************************************************/ char *str_ljust( char *Str, size_t Len ) { size_t LenOrg = strlen( Str ); char *StrEnd = &Str[LenOrg]; Len = max( Len, LenOrg ) - LenOrg; StrEnd[Len] = '\0'; while ( Len ) { Len--; StrEnd[Len] = ' '; } return ( Str ); } /* function str_ljust */ /***************************************************** Name: str_ltrim Parameters: Str - string to trim Trim - string containing chars to trim Return: Str Description: Delete characters from the left end of Str that are contained in Trim *****************************************************/ char *str_ltrim( char *Str, char *Trim ) { size_t Num = strspn( Str, Trim ); str_delete( Str, Str, Num ); return ( Str ); } /* function str_ltrim */ /***************************************************** Name: str_mrplc Expanded Name: String Multiple Search and Replace Parameters: Str - string to edit Find - search string Rplc - replacement string Return: Str Description: Multiple search and replace. All occurrences of Find within Str are replaced with Rplc. *****************************************************/ char *str_mrplc( char *Str, char *Find, char *Rplc ) { char *StrWork = Str; size_t LenRplc = strlen( Rplc ); while ( ( StrWork = strstr( StrWork, Find ) ) ! = NULL ) { str_delete( Str, StrWork, strlen( Find ) ); str_insert( Str, StrWork, Rplc ); StrWork += LenRplc; } return ( Str ); } /* function str_mrplc */ /***************************************************** Name: str_ocat Expanded Name: Concatenate overlapped strings Parameters: Dest - destination string Str - string to concat Return: Dest Description: Behaves the same as strcat in string.h. This version will work for strings that overlap in memory. *****************************************************/ char *str_ocat( char *Dest, char *Str ) { return ( (char *)memmove( (void *)&Dest[strlen( Dest )], (void *)Str, strlen( Str) + 1 ) ); } /* function str_ocat */ /***************************************************** Name: str_repeat Parameters: Str - string buffer to load Rpt - repetition string Num - number of repetitions Return: Str Description: Builds a string of length Num, by repeating a substring Rpt. *****************************************************/ char *str_repeat( char *Str, char *Rpt, size_t Num ) { size_t Len = strlen( Rpt ); if ( Len == 1 ) { /* The string is only one character */ memset( Str, *Rpt, Num ); } else { size_t i, j; /* Build Str with repetitions of Rpt */ for ( i = 0, j = 0; i < Num; i++ ) { Str[i] = Rpt[j++]; j %= Len; } /* for i */ } /* else */ Str[Num] = '\0'; return ( Str ); } /* function str_repeat */ /************************************************** Name: str_rplc Expanded Name: String Search and Replace Parameters: Str - string to edit Find - search string Rplc - replacement string Return: Str Description: Search and replace. First occurrences of Find within Str is replaced with Rplc. ***************************************************/ char *str_rplc( char *Str, char *Find, char *Rplc ) { char *StrWork = strstr( Str, Find ); if ( StrWork ) { str_delete( Str, StrWork, strlen( Find ) ); str_insert( Str, StrWork, Rplc ); } return ( Str ); } /* function str_rplc */ /***************************************************** Name: str_rjust Expanded Name: String Right Justify Parameters: Str - string to edit Len - new length of string Return: Str Description: Pads the left end of string so that it is right justified to a total length of Len. *****************************************************/ char *str_rjust( char *Str, size_t Len ) { size_t LenOrg = strlen( Str ); Len = max( LenOrg, Len ) - LenOrg; if ( Len ) { memmove( &Str[Len], Str, LenOrg + 1 ); while ( Len ) { Len--; Str[Len] = ' '; } } return ( Str ); } /* function str_rjust */ /***************************************************** Name: str_rtrim Parameters: Str - string to trim Trim - string of characters to trim Return: Str Description: Delete characters from the right end of Str that are contained in Trim *****************************************************/ char *str_rtrim( char *Str, char *Trim ) { char *StrWork = &Str[strlen( Str) - 1]; /* Look for last character in string not being ** in trim string */ while ( (Str != StrWork ) && ( strspn( StrWork, Trim ) != 0 ) ) { *StrWork-- = '\0'; } return ( Str ); } /* function str_rtrim */ /***************************************************** Name: str_trim Parameters: Str - string to trim Trim - string of characters to trim Return: Str Description: Delete characters from both ends of Str *****************************************************/ char *str_trim( char *Str, char *Trim ) { str_ltrim( Str, Trim ); str_rtrim( Str, Trim ); return ( Str ); } /* function str_trim */ /***************************************************** Name: str_vcat Parameters: Dest - destination string Strl - required first string Return: Dest Description: Variable argument version of strcat. Concatenates a list of strings into a destination string. The last argument must be a NULL pointer. *****************************************************/ char *str_vcat( char *Dest, char *Strl, .... ) { va_list VarArgList; char *Str; /* Initialize variable arguments */ va_start( VarArgList, Dest ); /* Get first var arg string */ Str = va_arg( VarArgList, char * ); strcat( Dest, Strl ); while ( Str != NULL ) { /* Loop though all the arguments */ strcat( Dest, Str ); /* Get the next string */ Str = va_arg( VarArgList, char * ); } /* Clean up */ va_end( VarArgList ); return ( Dest ); } /* function str_vcat */ /* End of File */ Listing 4 STR_EDIT.H - contains function prototypes for Listing 3 /****************************************************** File Name: STR_EDIT.H (Listing 4) Description: Include file for STR_EDIT.C ******************************************************/ #if !defined( STR_EDIT_DEFINED ) #define STR_EDIT_DEFINED char *str_center( char *Str, size t_Len ); char *str_delete( char *Str, char *Pos, size_t Num ); char *str_insert( char *Str, char *Pos, char *Insrt ); char *str_ljust( char *Str, size_t Len ); char *str_ltrim( char *Str, char *Trim ); char *str_mrplc( char *Str, char *Find, char *Rplc ); char *str_ocat( char *Dest, char *Str ); char *str_repeat( char *Str, char *Rpt, size_t Num ); char *str_rjust( char *Str, size_t Len ); char *str_rplc( char *Str, char *Find, char *Rplc ); char *str_rtrim( char *Str, char *Trim ); char *str_trim( char *Str, char *Trim ); char *str_vcat( char *Dest, char *Strl, ... ); #endif /* End of File */ Hiding ISAM Function Libraries with OOP Thomas Murphy Thomas J. Murphy has a bachelor's degree in Physics from St. Ambrose College, Davenport, IA, and a Ph.D. in Biophysics from the University of Illinois. He has worked in the software-development field for 13 years and has owned his own computer-consulting and custom software business for over seven years. His major area of expertise is database systems. He can be reached at Computer Management Consultants, Ltd, Box 132, RR #10, Oswego, NY 13126. Function libraries each have a unique set of functions, unique lists of arguments, and a unique user manual. Quality of documentation ranges from excellent to so indecipherable that you must analyze the source code to successfully use the product. On a project of any size, project programmers have to surmount the learning curve, not only for the application and hardware/operating system environment but for the function libraries in use. In addition, you may be stuck with your first choice of libraries or face some rather nasty problems if you want to switch to another library. Should you convert the code for all your past clients so your current staff of programmers (even if that is only you) can continue to maintain it? Should you keep knowledgeable about every library you've ever used? Or should you just stick with your original choice? This article presents an example programmer interface to a special-purpose function library designed to ease these problems for the case of a B+Tree, Indexed Sequential Access Method (ISAM) data-handling function library. The interface uses the standard Object-Oriented Programming (OOP) approach to protecting the application programmer and the function library from each other. It handles all the gory details of dealing with the function library by inserting a layer of function calls between the application programmer and the library itself. The functions called by the application programmer do not depend on the particular library product in use. They depend only on what the application programmer wants to do. OOP is not a language; it is an approach to programming. The class presented in this article is written in C++, but it does not use any of the flashier bells and whistles most often talked about when one is discussing C++. There is no operator overloading, no inheritance, no polymorphism. All this class has is a little bit of data and function hiding and an OOP approach to doing business. The code could be rewritten in Standard C by throwing all the class-member data into a structure, declaring instances of the structure, and passing as an argument the address of the declared structure to what are presented here as member functions. The examples in this article are for a specific product (CBTREE v3.0 by Peacock Systems, Inc.). Without changing the calling parameters or the names of the functions called by the application programmer, however, you could rewrite the body of these functions for any of the several ISAM function libraries I have used, and for many (no doubt) that I have not. The application code that uses the class is no longer dependent on what data handler is in use. The details of the data handler are hidden from the application programmer. What the Class Does Before looking at the code, look at what it was designed to do. Class Isam is a programmer interface to a data handler that uses ISAM files. After the application programmer/analyst has set up the ISAM files in the application (designed the record layouts, decided on index keys, created the files, etc.), I want him/her to be as free as possible from the nitty-gritty details of working with the data files themselves. The application programmer: needs to be able to gain access to the data and index files with a single, simple statement. If the files are already opened in another part of the program, they should not be opened again; the programmer should just have access to them. after gaining access to a file, needs to have space automatically allocated for the records he/she will be reading from the file. If the programmer is going to be dealing with 20 records at once, space must be allocated for 20 records. if modifying, deleting, and adding records, needs to be able to put the new set of records in an array and write them. The programmer should not have to keep track of which records have been worked on or what has been done to them. The programmer should not have to keep track of which keys have to be removed and added to the btree indices. In other words, the programmer needs to be able to use indices without having to think about them. should not have to write programs to re-index files. The class should do most of the work. If the programmer had to re-index the file because he/she added another index, all the remaining code should not need to be modified. should have the freedom to write a function to create a key that manipulates the data as desired. be able to address the indices by name, instead of an index number. How the Class Does It Listing 1 contains the header file defining class Isam. Notice that one of the member data items and one of the member functions is marked as specific to CBTREE. The rest of the member data and functions are generic and would be present regardless of the library being used. The code for the member functions is shown in Listing 2. The class contains ten public functions, eight of which are routinely called by the application programmer. The destructor, ~Isam, is typically called only automatically when the instance of the class goes out of scope. Isam::reindex is called to recover from hopefully infrequent disasters. public data consists of one string array, rec, which will contain records read from the ISAM file. The Constructor The programmer gains access to an ISAM file by declaring an instance of the class, such as Isam employeefile ("employee"); If the programmer will be dealing with more than one employee record at a time, such as all the employees in a department (for a small company), the maximum number can be specified in the declaration. The constructor, Isam, will allocate space for that number, as in Isam employeefile ("employee", 20); The first time the Isam constructor is called in a program, it calls a static function, isam_init, which loads a static array of structures, btparms[], with file and index parameters for all files in the application. This information is read from a setup file for the application. As it happens, CBTREE already uses such a file in its data-file and b-tree creation utilities and, optionally, in its runtime library. I found that CBTREE's parameter file, btparms.btr, contained almost all the information needed for the Isam class. Moreover, CBTREE provided a utility to display and modify these parameters. The two parameters missing from the file were kstart, the starting character position in the data record for a key, and keygen, the name of an application function that will generate a key if specified. Rather than re-invent the wheel, I modified the data entry and display utility for the file provided by CBTREE to include these two parameters. Because the data entry and display utility is proprietary, the modified code is not presented here; but modification was completely straightforward. Adding the parameters on the end of each record in the file had no ill effects on CBTREE's use of the file. If you are not using CBTREE, you may have to create your own parameter file. In addition to file and index parameters, the btparms array also contains a count (int count) of how many instances of class Isam are currently active that access each data file. If the count is greater than zero for a particular data file, btparms[] contains the file handles for the data and index files (datafd and indxfd). Thus, multiple instances of Isam for the same data file (in the same program) can share the file handles. The class constructor and destructor (Isam and ~Isam) manage all bookkeeping involved. The constructor, based on the parameters in btparms[] and on the callers specification for the number of records to be handled at a time, allocates all required space, thus satisfying the requirements for easy access and automatically-allocated space. Read and Write Functions Isam::read and Isam::write work as a pair for reading records to be modified and writing them back to the file. You can retrieve a record, modify it, and write it back to the file using the get... functions (see the first if statement in Isam::write), but the result would be the addition of the new version of the record, not the replacement of the old version. Isam::read asks the caller for a key (the only required parameter). Optionally, the caller can specify how many records are to be read (default is zero), what index to use (default is the first one), and where in the rec array to start storing records (default is the beginning). The function returns the number of records found matching the specified (perhaps partial) key, regardless of how many records were specified to be read. Many would have set the default number of records to be read at one; but I tend to do a lot of "authority checking," reading a cross-reference file to determine whether or not there is a record with the key entered by the user in a data entry program. In this instance, I don't even care about the record; I just want to know whether or not it is there. If you'd like the default to be one, simply change the declaration in Listing 1. The maximum number of records to be read plus the starting location in the rec array must not exceed Isam::elements. (This check is made in the code.) If you don't know the number of records to expect and you want all matches, you must create an instance of Isam with room for one record, make a test read, and create another instance of Isam with room for the number of records found (integer returned by Isam::read). Isam::read places each record read in public array, rec. It makes a second copy in private array, oldrec. Isam::write uses the copy in oldrec to compare the original version of the record as read with the new version to be written, and to generate values for old keys to be removed from the b-tree(s). A record can be added by calling Isam::clear, then placing the new record in the array. Isam::rec and calling Isam::write. A record can also be added after a failed attempt to read a record using a user-entered value for a key. Records can be deleted by reading them using lsam::read, setting the appropriate array element of rec to a null string, and calling lsam::write. Modifying a record involves reading it using Isam::read, changing the data in rec, and calling Isam::write. A typical calling sequence (using non-Isam function names now) might look like Isam invoice ("invoice"); Isam lineitm ("lineitm", 20); while (1) { paint_invoice_screen(); if (!get_invnum (inv_num)) break; invoice.clear(); lineitm.clear(); if (invoice.read (inv_num,1)){ get_inv_flds(invoice.rec[0]); lineitm.read(inv_num, 20); get_lin_flds(lineitm.rec); fill_inv_screen(); } if (inv_edit()){ package_inv(invoice.rec[0]); package_lin(lineitm.rec); invoice.write(); lineitm.write(); } else break; } In this code segment, get_inv_flds extracts the fields from the record, get_lin_flds extracts fields from the whole array of line item records, and package_inv and package_lin put the (perhaps modified) fields back into the records. These functions are still the responsibility of the application programmer. Application functions get_inv_flds, get_lin_flds, and inv_edit may have their own instances of the Isam class (e.g., for data generation from customer and product files based on customer and product numbers in the invoice and lineitem records). Once Isam::read has been called, the class should be cleared (Isam::clear) in order to forget what it read if the next write is not related to the records read. Notice in Listing 2 that Isam::write calls Isam::clear when its work is done. All key generation is done in Isam::write. The function generates both keys to add to the b-tree(s) and keys to be removed from the b-tree(s). If a function is specified (btparms[].keygen), that function is called to generate the key. If not, a strnncpy from rec[] or oldrec[] is executed using btparms[].kstart and btparms[].keylen (Function strnncpy came in the CBTREE library. It works exactly like strncpy except that it null-terminates the copied string). Before any keys are added to the database, all characters in the keys are converted to upper case. Before any keys are used to retrieve records (in Isam::read or Isam::getge), the same case conversion is executed automatically. Should an error occur during execution of Isam::write (like disk full), Isam::backout is called to reverse any changes to the index or data file made for the record before the error occurred. Only the record being operated on at the time of the error is backed out. Listing 3 shows the source file for the catalog and cataloged programs used to generate keys. The application programmer is assumed to have access to this file (though not necessarily to the source file shown in Listing 2). In order to set up a function to generate keys, the function name must be entered in the parameter file (btparms.btr), the function prototype and code must be added to Listing 3, and finally the name and address must be added to the Catolog[] array in Listing 3. This process is referred to as cataloging the function. Listing 3 shows a sample function, descwds, that returns a list of words used in a description type field, ready for insertion into the b-tree. The application programmer must insure that the strings returned by cataloged functions are the correct length. The correct length can be any multiple of btparms[].keylen. Multiples larger than one indicate multiple values in the b-tree for the same record. Note how these are processed in the key sections of Isam::write (Listing 2). Re-Indexing Function Once we've put key generation totally in the hands of Isam::write, it is a relatively easy job to create a generic re-indexing function. (See the last function in Listing 2.) Listing 4 shows a sample program that uses this function. Note in Listing 4 that, not only are the parent ("invoice") and child ("line item") files re-indexed, but all orphan line items (lineitem records with an inv_num that corresponds to no record in the parent file) are eliminated. Index Keys For performance considerations, the read function and the getfirst and getnext functions described above require an integer argument specifying what index to use. Some addition and deletion of indices can take place for a file without the requirement for change to all code that uses the file. Thus, the programmer may not know what the current or future value for the integer corresponding to a particular index is. The Isam::keynum function returns the integer corresponding to the argument btname. Note in Listing 2 that I've chosen to make the name comparison case insensitive (stricmp). It takes time to compare a specified name with the list of indices (stored in Isam::btnames). Isam::keynum improves performance because it is typically called once and the returned integer used several times in an application function. If the function fails to find the specified btname, there is obviously a problem. The function executes a fatal-error exit (eprintf). Conceivably, however, the function could be called in response to entry of an index name specified by an end-user. If this type of interaction with the end-user is included in the code, the function should be written to return a negative integer to indicate failure rather than fatally exiting. Report-Creation Functions The functions, Isam::getfirst, lsam::getge, and Isam::getnext are designed for application procedures that go through the file in the order of an index (reports and data display programs). Isam::getfirst puts into Isam::rec[0] the first record in the file (as viewed through the specified index). Isam::getge puts into Isam:: rec[0] the first record with a key greater than or equal to the specified key. Isam::getnext gets the record following the one read by the last call to a function starting with get. These functions are the bare-bones minimum for putting together reasonable reports. All of the ISAM data-handling libraries have more functions of this type (getlast, getprevious, etc.). Add them to the class if you use them. A typical code segment using the get functions might look like Isam inv("invoice"); int idx=inv.keynum("DATE_INVNUM"); inv.getge(idx, begin_date); get_inv_flds(inv.rec[0]); print_inv_summary_header(); while (inv_date <= end_date) { print_inv_summary_line(); if (!inv.getnext (idx)) break; get_inv_flds(inv.rec [0]); } Utility Functions There are three non-class-member functions used and shown in Listing 2. Function nospace removes all spaces from a character string. Function ToUpper converts all characters in a string to upper case. (It's a big version of toupper.) Function eprintf, as it is shown in Listing 2, works like members of the printf family, except that, after printing the error message, it exits the program. My own version of eprintf uses the window class in my toolbox to print the error message in a bright red window, sound an error buzzer, and wait for the user to write down the error message and press a key before exiting. Conclusion The Isam class presented in this article is an example of how OOP can be used to set up a function-library interface. With the interface, application code can be independent of the function library. More importantly, with the interface, a function library written by somebody else can work the way you decide it should work. Using CBTREE with C++ The CBTREE function library was developed in standard C. In order to use it with C++, all CBTREE headers must be included using the extern "C" modification (see Listing 1). The code presented in this article was compiled using Borland's C/C++ 3.1 compiler. In my hands, the CBTREE library was originally compiled using Borland's Turbo C 2.0. While preparing this article, I retrieved the original distribution disks and recompiled the CBTREE library programs using Borland's C/C++ 3.1 compiler rebuilt the library. I recompiled and linked an application using the code presented here using the newly rebuilt library. Aside from a small reduction in the size of the library, I saw no change. To successfully compile and link my programs, I had to make one change to the original CBTREE code. In the main header file, CBTREE.H, there is a superfluous declaration for function lockunlk (line 142) that does not include the argument list prototype. In C mode, the Borland compiler ignored this. In C++ mode, the compiler interpreted this as an overloaded function declaration and refused to continue because of the extern C modification. Removing the declaration (I commented mine out) made everybody happy. Listing 1 isam.h //////////// // isam.h // //////////// #ifndef isam_h #define isam_h extern "C" { #include // CBTREE header #include // CBTREE header } typedef char *(*t_func)(char *); typedef int (*rel_func)(char *); class Isam { private: int elements, fd[2], btr[10], indices, backingout; long * loc; char ** oldrec, * okey, * nkey, ** inames; BTC * btc; // CBTREE specific int getxxx (int index, int opt); // CBTREE specific void backout (int ele, char op, int index = -1, int result = 0); public: char ** rec; Isam (const char *datafilename, int e = 1); ~Isam ( ); int read (const char *key, int ele_limit = 0, int idx = 0, int ele = 0); int write (); void clear (); int getfirst (int index = 0); int getnext (int index = 0); int getge (char *key, int index = 0); int keynum (const char *btname); void reindex (rel_func func); }; char * nospace(const char *arg); int eprintf(const char *format, ...); char * ToUpper(const char *c); /////////////////////////////// // catalog utility functions // /////////////////////////////// int catalog_number (char *name); t_func cataloged_func (int f ); #endif /* End of File */ Listing 2 isam. cpp ////////////// // Isam.cpp // ////////////// #include "isam.h" #include #include #include #include #include #include static struct BTParms { char name[9], filename[9]; int keylen, dreclen, maxrecs, indxfd, datafd, kstart, count; t_func keygen; } btparms[40]; static int items = 0; extern "C" { int (*Write)(int,const void *,unsigned) = write; int (*Read)(int,void *,unsigned) = read; } static void isam_init(void); static void isam_init(void) { ifstream btr("btparms.btr"); char nwln, item[81], buf[20], fname[13], fcname[9]; int i; if (!btr) eprintf("\n\nUnable to open btparms.btr.\n"); for (i = 0; i < 40; i++) { btr.get (item, 80); if (!strlen(item)) break; btr.get(nwln); strcpy(btparms[i].name, strtok(item, "^")); strtok(NULL, "^"); strcpy(buf, strtok(NULL, "^")); btparms[i].keylen = atoi(buf); strtok(NULL, "^"); strcpy(buf, strtok(NULL, "^")); btparms[i] .dreclen = atoi (buf); strtok(NULL, "^"); strcpy(fname, strtok(NULL, "^")); strtok(NULL, "^"); strcpy(buf, strtok(NULL, "^")); btparms[i].kstart = atoi(buf); strcpy(fcname, nospace(strtok(NULL, "^"))); strcpy(btparms[i].filename, strtok(fname,".")); btparms[i].count = O; btparms[i].indxfd = btparms[i].datafd = -1; if (strlen(fcname)) btparms[i].keygen = cataloged_func ( catalog_number(fcname)); else btparms[i].keygen = (t_func) NULL; } items = i; btr.close(); if (!items) eprintf("\n\nUnable to initialize isam system.\n"); for (i = items; i < 40; i++) { strcpy(btparms[i].name,""); strcpy(btparms[i].filename,""); btparms[i].keygen = NULL; btparms[i].keylen = btparms[i].dreclen = btparms[i].count = btparms[i].kstart = 0; btparms[i].indxfd = btparms[i].datafd = -1; } } Isam::Isam(const char *datafilename, int e) { int i, j; char buf[20]; backingout = 0; elements = e; if (!items) // this is the first Isam isam_init(); // initialize btparms strcpy(buf, nospace(datafilename)); indices = 0; // find btree params for (i = 0; i < items; i++) if (!stricmp(buf, btparms[i].filename)) btr[indices++] = i; if (!indices) // couldn't find any, fatal eprintf("\n\nCan't find parameters for %s!\n", datafilename); // set up btree interfaces if( !(btc = new BTC [indices]) !(inames = new char* [indices]) !(loc = new long [elements]) !(oldrec = new char* [elements]) !(rec = new char* [elements]) !(okey = new char [btparms[*btr].dreclen]) !(nkey = new char [btparms[*btr].dreclen]) ) eprintf("\n\nOut of memory.\n"); for (i = 0; i < indices; i++) { inames[i] = btparms[btr[i]].name; btc[i].btvcount = 1; if (btrinit(inames[i], btc+i) == ERR) eprintf("\n\nCouldn't initialize %s.\n", inames[i]); btc[i].btmulti = 0; // record locking off } for (i = 0; i < elements; i++) { loc[i] = OL; if (!(oldrec[i] = new char [btparms[*btr].dreclen + 1]) !(rec[i] = new char [btparms[*btr].dreclen + 1]) ) eprintf("\n\nOut of memory.\n"); } if (btparms[*btr].count) { // open files if neccesary fd[0] = btparms[*btr].datafd; fd[1] = btparms[*btr].indxfd; } else { // yup, it's neccesary strcpy(buf, btparms[*btr].filename); strcat(buf, ".dat"); if ((fd[0]:bt_open(buf, 0_RDWR,S_IRDWR)) == ERR) eprintf("\n\nCan't open %s.\n", buf); strcpy(buf, btparms[*btr].filename); strcat(buf, ".idx"); if ((fd[1]=bt_open(buf, 0_RDWR,S_IRDWR)) == ERR) eprintf("\n\nCan't open %s.\n", buf); btparms[*btr].datafd = fd[0]; btparms[*btr].indxfd = fd[1]; } (btparms[*btr].count)++; // record that we are here clear(); } Isam::~Isam() { int i, j; // clean up out mess for (i = 0; i < indices; i++) btrterm(btc+i); for (i = 0; i < elements; i++) { delete [] oldrec[i]; delete [] rec[i]; } delete [] btc; delete [] inames; delete [] loc; delete [] oldrec; delete [] rec; delete [] nkey; delete [] okey; // if we're last ones out, turn off the lights if (!(--(btparms[*btr].count))) { close (fd[0]); close (fd[1]); btparms[*btr].indxfd = -1; btparms[*btr].datafd = -1; } } int Isam::write () { int deltakey, oldl, new1, ele, index, result, len, start, notfirst; t_func fcn; if (*loc < 0) // programmer hasn't followed rules eprintf ("No writes after gets!"); for (ele = 0; ele < elements; ele++) { old1 : strlen(oldrec[ele]); new1 = strlen( rec[ele]); if (!(old1 new1) !strcmp(oldrec[ele], rec[ele])) continue; for (index = 0; index < indices; index++) { // generate the old and/or new key len = btparms[btr[index]].keylen; start = btparms[btr[index]].kstart; if ((fcn = btparms[btr[index]].keygen) != (t_func) NULL) { if (old1) strcpy(okey, fcn(oldrec[ele])); if (new1) strcpy(nkey, fcn( rec[ele])); } else { if (old1) strnncpy(okey, oldrec[ele] + start, len); if (new1) strnncpy(nkey, rec[ele] + start, len); } deltakey = (!old1!new1stricmp(okey,nkey)); if (old1 && deltakey) { notfirst = 0; while (strlen(okey) >= len) { strnncpy(btc[index].btkey, ToUpper(okey), len); btc[index].btoptype = (new1 index notfirst++ backingout) ? DELTKY : DELETE; btc[index].btloc = loc[ele]; result=cbtree(fd[O], fd[1], btc+index); if (result != BTCALLOK) backout (ele, 'D', index, result); strcpy(okey, okey + len); } } if (new1 && deltakey) { notfirst = 0; while (strlen(nkey) >= len) { strnncpy(btc[index].btkey, ToUpper(nkey), len); btc[index].btoptype = (loc[ele] == OL) ? INSERT : ISRTKY; btc[index].btloc = loc[ele]; result = cbtree(fd[0], fd[1], btc+index); if (result == BTCALLOK) loc[ele] = btc[index].btloc; else backout(ele, 'I', index, result); strcpy(nkey, nkey+ len); } } } if (!new1) continue; if (btseek (fd[0], loc[ele], btc->btdtalen) == -1L) backout(ele, 'S'); if(Write(fd[0],rec[ele],(unsigned)btc->btdtalen) !=btc->btdtalen) backout(ele,'W'); } clear(); return 0; } void Isam::backout(int ele,char op,int index,int result) { //backs out record add/change/delete on error char *trec; if(!(trec = new char [btparms[*btr].dreclen]) backingout)// Iquit! Error in error backout! eprintf("\n\nBackout error, error type was %c\n", backingout); if (index>=0) indices = index + 1; // those beyond index are ok backingout = op; elements = 1; strcpy(trec,rec[ele]); strcpy(rec[0],oldrec[ele]); strcpy(oldrec[0],trec); loc[0] = loc[ele]; write(); switch(backingout) { case 'I': eprintf( "\n\nINSERT error, element %d, index %s, result %d\n", ele, inames[index],result); case 'D': eprintf( "\n\nDELETE error, element %d, index %s, result %d\n", ele, inames[index],result); case 'S': eprintf ("\n\nSeek error, element %d\n", ele); case 'W': eprintf ("\n\nWrite error, element %d\n", ele); } } int Isam::read(const char *key, int ele_limit, int idx, int ele) { int i = 0, j = 0; if ((ele_limit + ele) > elements) eprintf("\n\nNot enough elements!\n"); free_svkey(btc+idx); btc[idx].btoptype = GETALL; strcpy(btc[idx].btkey, ToUpper(key)); btc[idx].btloc = OL; while (cbtree(fd[0], fd[1], btc + idx) == BTCALLOK) { while (btc[idx].btrecnum[i-j] != OL) { if (i < ele_limit) { loc[i+ele] = bte[idx].btrecnum[i-j]; btseek(fd[0], loc[i+ele], btc[idx].btdtalen); Read (fd[0], rec[i+ele], btc[idx].btdtalen); } i++; if (!((i-j) < btc[idx].btmax)) break; } if ((i-j) < btc[idx].btmax) break; j += btc[idx].btmax; } for (j = 0; j < ele_limit;j++) strcpy(oldrec[j], rec[j]); return i; } int Isam::getfirst(int index) { return getxxx (index, GETFRST); } int Isam::getnext(int index) { return getxxx(index, GETNXT); } int Isam::getge (char *key, int index) { *loc = -1; btc[index].btoptype = GETGE; strcpy(btc[index].btkey,ToUpper(key)); if (!(cbtree(fd[0], fd[1], btc + index) == BTCALLOK)) return 0; btseek(fd[0], btc[index].btloc, btc[index].btdtalen); Read (fd[0], rec[0], btc[index].btdtalen); return 1; } int Isam::getxxx(int index, int opt) { *loc = -1; btc[index].btoptype = opt; if (!(cbtree(fd[0], fd[1], btc + index) == BTCALLOK)) return 0; btseek(fd[0], btc[index].btloc, btc[index].btdtalen); Read (fd[0], rec[0], btc[index].btdtalen); return 1; } void Isam::clear() { int i; for (i = 0; i < elements; i++) { loc[i] = OL; memset (oldrec[i], '\0', btparms[*btr].dreclen + 1); memset ( rec[i], '\0', btparms[*btr].dreclen + 1); } } int Isam::keynum (const char *btname) { int i; if (!items) eprintf("\n\nBtparms not initialized!\n"); if (indices < 1) eprintf("\n\nNo indices.\n"); for (i = 0; i static struct cat { char *name; t_func func; } Catalog[] = { { "descwrds", descwrds }, { NULL, NULL } }; /////////////////////// // Catalog utilities // /////////////////////// int catalog_number (char *name) { int i = 0; while (Catalog[i].name) { if (!strcmp(nospace(name), Catalog[i].name)) return i; i++; } return -1; } t_func cataloged_func(int f) { return Catalog[f].func; } ///////////////////////// // Cataloged functions // ///////////////////////// char * descwrds (char *r); char *descwrds (char *r) { // Extracts up to 20 seven-letter words from // desc field. The description field starts at // posn 37 and goes on for 120 characters. static char keys[141]; int i = 37, j = 0, k = 0; memset (keys, ' ' , 140); while ((i < 157) && (k < 20)){ while ((ispunct(r[i]) isspace(r[i])) && i < 157) i++; while (!ispunct(r[i]) && !isspace(r[i]) && i < 157 && j < 7) keys[(7 * k) + j++] = r[i++]; while (!ispunct(r[i]) && !isspace(r[i]) && i < 157) i++; k++; j = 0; } keys[7 * k] = '\0'; return keys; } /* End of File */ Listing 4 A sample program using the re-index function Isam::re-index #include "isam.h" int orphan (char *rec); static Isam i ("invoice"); void main(void) { Isam l ("lineitem"); i.reindex (rel_func NULL); l.reindex (orphan); } int orphan (char *rec) { char inv_num[10]; strnncpy(inv_num, rec + 7, 9); return (i.read(inv_num)) ? 0 : 1; } /* End of File */ Standard C The Header P.J. Plauger P.J. Plauger is senior editor of The C Users Journal. He is secretary of the ANSI C standards committee, X3J11, and convenor of the ISO C standards committee, WG14. His latest books are The Standard C Library, published by Prentice-Hall, and ANSI and ISO Standard C (with Jim Brodie), published by Microsoft Press. You can reach him at pjp@plauger.com. Background Time and date calculations achieved a new level of sophistication under the UNIX operating system. Several of the developers of that system were amateur astronomers. They were sensitive to the need for representing times over a span of decades, not just years. They automatically reckoned time as Greenwich Mean Time (once GMT, now UTC), not just by the clock on the wall. They were, in short, more finicky than most about measuring and representing time on a computer. That same attention to detail has spilled over into the Standard C library. Its scope is basically whatever was available in C under UNIX that didn't depend on the peculiarities of UNIX. As a consequence, you can do a lot with times and dates in Standard C. The functions declared in provide the relevant services. It stretches the truth a bit to say that these functions don't depend on the peculiarities of UNIX. Not all operating systems distinguish between local time and UTC. Even fewer allow different users to display times relative to different time zones. Some of the smallest systems can't even give you the time of day. Yet all implementations of C must take a stab at telling time wisely if they want to claim conformance to the C Standard. The C Standard contains enough weasel words to let nearly everybody off the hook. A system need only provide its "best approximation" to the current time and date, or to processor time consumed, to conform to the C Standard. A vendor could argue that 1 January 1980 is always the best available approximation to any time and date. A customer can rightly quarrel about the low quality of such an approximation, but not whether it satisfies the C Standard. What this means in practice is that a program should never take times too seriously. It can enquire about the current time (by calling time) and display what it gets in a variety of attractive formats. But it can't know for sure that the time and date are meaningful. If you have an application that depends critically upon accurate time stamps, check each implementation of Standard C closely. There are too many functions declared in to cover in one installment. So I've divided the presentation into three parts: the basic time functions--those that yield results of type clock_t, time_t, or double the conversion functions--those that convert times between scalar and structured forms the formatting functions--those that yield text representations of encoded times The topic for this month is the basic functions. What the C Standard Says 7.12 Date and time 7.12.1 Components of time The header defines two macros, and declares four types and severalufunctions for manipulating time. Many functions deal with a calendar time that represents the current date (according to the Gregorian calendar) and time. Some functions deal with local time, which is the calendar time expressed for some specific time zone, and with Daylight Saving Time, which is a temporary change in the algorithm for determining local time. The local time zone and Daylight Saving Time are implementation-defined. The macros defined are NULL (described in 7.1.6); and CLOCKS_PER_SEC which is the number per second of the value returned by the clock function. The types declared are size_t (described in 7.1.6); clock t and time_t which are arithmetic types capable of representing times; and struct tm which holds the components of a calendar time, called the broken-down time. The structure shall contain at least the following members, in any order. The semantics of the members and their normal ranges are expressed in the comments.137 int tm_sec; /* seconds after the minute- [0, 61] */ int tm_min; /* minutes after the hour- [0, 59] */ int tm_hour; /* hours since midnight- [0, 23] */ int tm_mday; /* day of the month- [1, 31] */ int tm_mon; /* months since January- [0, 11] */ int tm_year; /* years since 1900 */ int tm_wday; /* days since Sunday- [0, 6] */ int tm_yday; /* days since January 1- [0, 365] */ int tm_isdst; /* Daylight Saving Time flag */ The value of tm_isdst is positive if Daylight Saving Time is in effect, zero if Daylight Saving Time is not in effect, and negative if the information is not available. 7.12.2 Time manipulation functions 7.12.2.1 The clock function Synopsis #include clock_t clock(void); Description The clock function determines the processor time used. Returns The clock function returns the implementation's best approximation to the processor time used by the program since the beginning of an implementation-defined era related only to the program invocation. To determine the time in seconds, the value returned by the clock function should be divided by the value of the macro CLOCKS_PER_SEC. If the processor time used is not available or its value cannot be represented, the function returns the value (clock_t) -1.138 7.12.2.2 The difftime function Synopsis #include double difftime(time_t time1, time_t time0); Description The difftime function computes the difference between two calendar times: time1 - time0. Returns The difftime function returns the difference expressed in seconds as a double. ..... 7.12.2.4 The time function Synopsis #include time_t time(time_t *timer); Description The time function determines the current calendar time. The encoding of the value is unspecified. Returns The time function returns the implementation's best approximation to the current calendar time. The value (time_t)-1 is returned if the calendar time is not available. If timer is not a null pointer, the return value is also assigned to the object it points to. Footnotes 137. The range [0, 61] for tm_see allows for as many as two leap seconds. 138. In order to measure the time spent in a program, the clock function should be called at the start of the program and its return value subtracted from the value returned by subsequent calls. Using the Basic Time Functions The functions declared in determine elapsed processor time and calendar time. They also convert among different data representations. You can represent a time as: type clock_t for elapsed processor time, as returned by the primitive function clock type time_t for calendar time, as returned by the primitive function time or the function mktime type double for calendar time in seconds, as returned by the function difftime type struct tm for calendar time broken down into separate components, as returned by the functions gmtime and localtime a text string for calendar time, as returned by the functions asctime, ctime, and strftime You have a rich assortment of choices. The hard part is often identifying just which data represention, and which functions, you want to use for a particular application. For this installment, I ignore functions that produce a struct tm or a text string. Here is a brief description of the individual types and macros defined in . It is followed by brief notes on how to use the basic time functions declared in . NULL--See "The Header ," CUJ December 1991. CLOCKS_PER_SEC--The expression clock() / CLOCKS_PER_SEC measures elapsed processor time in seconds. The macro can have any arithmetic type, either integer or floating point. Type cast it to double to ensure that you can represent fractions of a second as well as a wide range of values. clock_t--This is the arithmetic type returned by clock, described below. It represents elapsed processor time. It can have any integer or floating-point type, which need not be the same type as the macro CLOCKS_PER_SECOND, above. size_t--See "The Header ," CUJ December 1991. time_t--This is the arithmetic type returned by time, described below. Several other functions declared in also manipulate values of this type. It represents calendar times that span years, presumably to the nearest second (although not necessarily). Don't attempt to perform arithmetic on a value of this type. tm--A structure of type struct tm represents a "broken-down time." Several functions declared in manipulate values of this type. You can access certain members of struct tm. Its definition looks something like: struct tm { int tm_sec; /* seconds after the minute (from 0) */ int tm_min; /* minutes after the hour (from 0) */ int tm_hour; /* hour of the day (from 0) */ int tm_mday; /* day of the month (from 1) */ int tm_mon; /* month of the year (from 0) */ int tm_year; /* years since 1900 (from 0) */ int tm_wday; /* days since Sunday (from 0) */ int tm_yday; /* day of the year (from 0) */ int tm_isdst; /* DST flag */ The members may occur in a different order, and other members may also be present. The DST flag is greater than zero if Daylight Savings Time (DST) is in effect, zero if it is not in effect, and less than zero if its state is unknown. The unknown state encourages the functions that read this structure to determine for themselves whether DST is in effect. clock--This function measures elapsed processor time instead of calendar time. It returns -1 if that is not possible. Otherwise, each call should return a value equal to or greater than an earlier call during the same program execution. It is the best measure you can get of the time your program actually consumes. See the macro CLOCKS_PER_SEC, above. difftime--The only safe way compute the difference between two times t1 and t0 is by calling difftime(t1, t0). The result, measured in seconds, is positive if t1 is a later time than t0. time--This function determines the current calendar time. It returns -1 if that is not possible. Otherwise, each call should return a value at the same time or later than an earlier call during the same program execution. It is the best estimate you can get of the current time and date. Implementing the Basic Time Functions The functions declared in are quite diverse. Many wrestle with the bizarre irregularities involved in measuring and expressing times and dates. Be prepared for an assortment of coding techniques (at least in future installments). Listing 1 shows the file time.h. As usual, it inherits from the internal header definitions that are repeated in several standard headers. I discuss the implementation of both the macro NULL and the type definition size_t in "The Header ," CUJ December 1991. also defines two macros that describe properties of the primitive functions clock and time: The macro _CPS specifies the value of the macro CLOCKS_PER_SECOND. The macro_TBIAS gives the difference, in seconds, between values returned by time and the time measured from 1 January 1900. (This macro name does not appear in .) The values of these macros depend strongly on how you implement clock and time. This implementation represents elapsed processor time as an unsigned int (type clock_t). It represents calendar time as an unsigned long (type time_t) that counts UTC seconds since the start of 1 January 1900. That represents dates from 1900 until at least 2036. You have to adjust whatever the system supplies to match these conventions. The macro _TBIAS is a kludge. Normally, you want to set it to zero. The version of time you supply should deliver calendar times with the appropriate starting point. UNIX, however, measures time in seconds since 1 January 1970. Many implementations of C offer a function time that matches this convention. If you find it convenient to use such a time function directly, then should contain the definition: #define _TBIAS ((70 * 365LU + 17) * 86400 That counts the 70 years, including 17 leap days, that elapsed between the two starting points. In several places, the functions declared in adjust a value of type time_t by adding or subtracting _TBIAS. Listing 2 shows the file time.c. It defines the function time for a UNIX system. As usual, I assume the existence of a C-callable function with a reserved name that peforms the UNIX system service. For this version of time, the header can define the macro _TBIAS to be zero. UNIX also provides an exact replacement for the function clock. So do many implementations of C modeled after UNIX. Thus, you may not have to do any additional work. Just define the macro _CPS appropriately. For a PC-compatible computer, for example, the value is approximately 18.2. Listing 3 shows the file clock.c. It defines a version of clock you can use if the operating system doesn't provide a separate measure of elapsed processor time. The function simply returns a truncated version of the calendar time. In this case, the header defines the macro _CPS to be 1. Listing 4 shows the file difftime.c. It is careful to correct the biases of both times before comparing them. It is also careful to develop a signed difference between two unsigned integer quantities. Note how the function negates the difference t1 - t0 only after converting it to double. The remaining functions all include the internal header "xtime.h". Listing 5 shows the file xtime.h. It includes the standard header and the internal header "xtinfo.h". That internal header defines the type _Tinfo. It also declares the data object _Times, defined in the file asctime.c. _Times specifies locale-specific information on the category LC_TIME. Listing 6 shows the file xtinfo.h. The header "xtime.h" defines the macro WDAY that specifies the weekday for 1 January 1900 (Monday). It defines the type Dstrule that specifies the components of an encoded rule for determining Daylight Savings Time. And it declares the various internal functions that implement this version of . Conclusion The basic time functions I have shown so far meet a number of needs. You can measure execution times, obtain time and date stamps, and compare those stamps. That's more time support than many programming languages offer. For Standard C, however, it's only the beginning. References W.M. O'Neil. 1975. Time and the Calendars. Sydney, N.S.W.: Sydney University Press. Calendars are notoriously idiosyncratic. This book tells you more than you probably want to know about the history of measuring calendar time. It also explains why days and dates are named and determined the way they are today. This article is excerpted in part from P.J. Plauger, The Standard C Library, (Englewood Cliffs, N.J.: Prentice-Hall, 1992). Listing 1 time. h /* time.h standard header */ #ifndef _TIME #define _TIME #ifndef _YVALS #include #endif /* macros */ #define NULL _NULL #define CLOCKS_PER_SEC _CPS /* type definitions */ #ifndef _SIZET #define _SIZET typedef _Sizet size_t; #endif typedef unsigned int clock_t; typedef unsigned long time_t; struct tm { int tm_sec; int tm_min; int tm_hour; int tm_mday; int tm_mon; int tm_year; int tm_wday; int tm_yday; int tm_isdst; }; /* declarations */ char *asctime(const struct tm *); clock_t clock(void); char *ctime(const time_t *); double difftime(time_t, time_t); struct tm *gmtime(const time_t *); struct tm *localtime(const time_t *); time_t mktime(struct tm *); size_t strftime(char *, size_t, const char *, const struct tm *); time_t time(time_t *); #endif /* End of File */ Listing 2 time.c /* time function -- UNIX version */ #include /* UNIX system call */ time_t_Time(time_t *); time_t (time)(time_t *rod) { /* return calendar time */ time_t t = _Time(NULL) + (70*365LU+17)*86400; if (tod) *tod = t; return (t); } /* End of File */ Listing 3 clock.c /* clock function -- simple version */ #include clock_t (clock)(void) { /* return CPU time */ return ( (clock_t) time(NULL) ); } /* End of File */ Listing 4 difftime.c /* difftime function */ #include double (difftime)(time_t t1, time_t t0) { /* compute difference in times */ t0 -= _TBIAS, t1 -= _TBIAS; return (t0 <= t1 ? (double)(t1 - t0) : -(double) (t0 - t1)); } /* End of File */ Listing 5 xtime.h /* xtime.h internal header */ #include #include "xtinfo.h" /* macros */ #define WDAY 1 /* to get day of week right */ /* type definitions */ typedef struct { unsigned char wday, hour, day, mon, year; } Dstrule; /* internal declarations */ int _Daysto(int, int); const char *_Gentime(const struct tm *, _Tinfo *, const char *, int *, char *); Dstrule *_Getdst(const char *); const char *_Gettime(const char *, int, int *); int _Isdst(const struct tm *); const char *_Getzone(void); size_t _Strftime(char *, size_t, const char *, const struct tm *, _Tinfo *); struct tm *_Ttotm(struct tm *, time_t, int); time_t _Tzoff(void); /* End of File */ Listing 6 xtinfo.h /* xtinfo.h internal header*/ /* type definitions */ typedef struct { const char *_Ampm; const char *_Days; const char *_Formats; const char *_Isdst; const char *_Months; const char *_Tzone; } _Tinfo; /* declarations */ extern _Tinfo _Times; /* End of File */ Stepping Up To C++ The Function operator[] Dan Saks Dan Saks is the owner of Saks & Associates, which offers consulting and training in C and C++. He is secretary of the ANSI and ISO C++ standards committees, and also contributing editor for Windows/DOS Developer's Journal. Dan recently finished his first book, C++ Programming Guidelines, written with Thomas Plum. You can write to him at 393 Leander Dr., Springfield, OH 45504, or dsaks@wittenberg.edu (Internet), or call (513)324-3601. Dynamic array classes let you create arrays for which you can delay choosing the number of elements until runtime. In my last column, I described reasons why you might want to use such things in your programs (see "Dynamic Arrays", CUJ, November, 1992). I also presented a fairly typical implementation for class float_array, a simple dynamic array of float. Each float_array has two data members: array, a pointer to the first element of an actual array of float allocated from the free store, and len, the number of elements in that array. Listing 1 shows the class definition. It has two constructors, a destructor, an assignment operator, a subscript operator, and a function that returns the number of elements. const Member Functions Revisited Last time I also introduced const member functions. For the most part, you can think of a const member function as one that promises not to change the object addressed by its this pointer. Although you can call a const member function for both const and non-const objects, you can call a non-const member function only for non-const objects. You declare a member function as const by placing the keyword const after the parameter list in the member function declaration. Class float_array in Listing 1 has two const member functions: operator[] and length. You must also use the keyword const when you write the function definitions, as shown in Listing 2. If you accept my belief that you should declare objects const whenever feasible, then clearly the length function should be const. length simply returns the number of elements in the array without changing anything. It also appears that operator[] should be const. If you don't declare it const, you can't subscript a const float_array. The operation appears to be perfectly safe. After all, it doesn't change anything. It simply returns a reference to the selected element. But, when I concluded last time, I said that the operator[] in Listing 2 is an accident waiting to happen. It lets you write an expression, without a cast, that inadvertently modifies a const float_array. For example, given const float_array fb(fa); you can write fb[0] = 0; which overwrites the value of the fb's first element. This is valid C++. The assignment statement compiles and executes without error. But that assignment is probably an accident, and you might find the results surprising. constness is Shallow To see how the accident can happen, let's take a closer look at what it means for a member function to be const. In a non-const, non-static member function for class X, the type of this is X *const. That is, the function cannot change the value of its this pointer, but it can modify each non-const data member of the object addressed by this. In a const member function of class X, this has type const X *const. Not only is the this pointer const, but so is the object it points to. Each data member of a const object is in turn const. This applies recursively to any data members that are objects of a class type. Thus, a const member function cannot change the value of any data members in the object addressed by this. (A const member function must be non-static because the const qualifier modifies the object addressed by the this pointer. A static member function has no this pointer, so it cannot be const.) Aye, there's the rub: constness is shallow. A pointer member in a const object is a const pointer, but it is not pointer to const unless explicitly declared so. In other words, inside operator[], the object's array data member (that is, this>array) has type float_array *const, not the desired const float_array *. You cannot change the pointer, but you can change the values of the array elements. This violates the intuitive notion that the elements of a const array are themselves const. operator[] in Listing 2 is a const member function. Inside the function, the array member is const, but the element array[i] is not. The function returns a float &, not a const float &. Thus, even though operator[] doesn't change any values, it returns a modifiable reference to an array element that the caller might accidentally use to change a value in a const float_array. From the programming language's standpoint, this is not an error. The behavior of the float_array class, even with this questionable operator[], is well-defined. For a non-class type T, writing over a const T object (or a part thereof) yields undefined behavior because you might be writing into ROM (read-only memory). But C++ only places a const class object in ROM if that class has neither constructors nor a destructor. Since class float_array has constructors and a destructor, the translator will never put const float_array objects in ROM. In this case, const float_array objects behave at runtime just like non-const objects. However, if class float_array had neither constructors nor a destructor, the translator might place const float_array objects in ROM, and calling operator[] for a const float_array would have undefined behavior. C++ has no notation for specifying "deep" constness. That is, in a class X with pointer member T *p, you can't make the p member of a const X object have type const T *. However, you can design X to simulate "deep" constness. Overloading with const Let's consider correcting the problem in operator[] (Listing 2) by changing the function's return type from float & to const float & in both Listing 1 and Listing 2. Then when you declare const float_array fb(fa); and write fb[0] = 0; you will get a compilation error because the expression fb[0] refers to a const float object, which you cannot modify. Unfortunately, now every float_array subscripting expression is const, so you cannot do that assignment even if fb were non-const. What you need is two different subscripting operators: one for const float_arrays, and another for non-const float_arrays. C++ lets you overload a member function as const and non-const. That is, you can declare two member functions in a given class with identical names and parameter lists, provided one of the functions is const and the other is not. In this case, you overload operator[] as a const function returning a const float &, and as a non-const function returning a (plain) float &. Listing 3 shows the float_array class definition with both declarations for operator[]. Listing 4 shows the corresponding function definitions. When the compiler encounters a float_array subscripting expression like fb[i], it chooses an operator[] by the constness of fb. It does not matter whether fb[i] appears as an lvalue or rvalue. For example, given float_array fa(fx); const float_array fb(fy); then fa[i] = fb[i]; calls the non-const operator[] on the left of the assignment, and calls the const operator[] on the right. Everything works fine. However, the assignment fb[i] = fa[i]; calls the const operator[] on the left side and calls the non-const operator[] on the right. But the const operator[] returns a non-modifiable lvalue (a const float &), so it cannot appear as the destination of the assignment, and you get a compile-time error. Furthermore, the compiler selects the operator[] based on the constness of the float_array expression that refers to the object, and not the constness of the object itself. For example, const float_array *p = &fa; binds p, a pointer to const, to a non-const object, fa. Then (*p) [i] calls the const operator[] because *p has type const float_array, even though p points to a float_array that's declared non-const. Arrays That Grow When I first introduced operator[] in my previous column, I suggested that the function could handle a subscript out-of-bounds in a number of ways. Thus far, my float_array class has treated subscripts out of bounds as an error caught by assert. Now let's consider rewriting operator[] to automatically extend the array so that the subscript is in bounds. For example, if you declare float_array fa(4); then fa has four elements whose indices are 0 through 3. Now if you write fa[6] = 1.0; then operator[] adds three more elements to fa at indices 4, 5, and 6, so the assignment operator has some place to store the 1.0. Rewrite only the non-const operator[]. Extending an array changes the array (by making it bigger), and a const member function should not change the object addressed by its this pointer. Subscripting out of range in a const float_array should remain a runtime error. Listing 5 shows my revised implementation for the non-const operator[]. If the function determines that the subscript is out of bounds, it allocates a new, larger array. It copies the values of the elements of the old array into their corresponding positions in the new array, and fills the remaining elements in the new array with zeros. Then it deletes the old array, thus returning that storage to the free store. Listing 6 shows a trivial test program that uses operator[] to extend an array. Figure 1 shows output from that program. The second for loop in main (Listing 6) overwrites the values in fb and then adds new elements. The output expression just before the return statement forces a subscripting error on the const float_array fc. I added that expression just to reinforce that operator[] is different for const and non-const float_arrays. The subscripting error makes the assertion in operator[] fail. assert displays an error message and aborts the program. Unlike the exit function, abort does not flush and close the standard output stream, cout. Thus I changed the body of the display function from cout << s << " = " << fa << '\n'; (as it was in my previous column), to cout << s << " = " << fa << endl; endl is a special kind of function called a manipulator. It's defined in iostream.h. The expression cout << endl; writes a newline to cout and flushes cout's output buffer to standard output. Typically, you won't see the difference between \n and endl unless you redirect standard output to a file. You might recognize an opportunity to use realloc in the non-const operator[]. realloc is a Standard C library function that changes the size of a dynamically-allocated block of memory. However, realloc only works with storage previously allocated by calloc, malloc, or realloc. Many C++ environments implement the new operator in terms of malloc, so realloc might work with new. But there is no guarantee that it always will. For float_arrays, you could simply abandon the new operator and use malloc and realloc. The code for class float_array will still be portable. However, using malloc and realloc only works for arrays of objects without constructors and destructors. The new operator applies a default constructor to each element of the array it allocates. malloc just grabs storage. On occasion, members of the C++ standards committee have suggested adding a renew operator that works with new as realloc works with malloc. So far, nothing has come of it. Dangling Pointers and References A dangling pointer or reference is one that remains bound to an address after the program deallocates the object at that address. The potential occurrence of a dangling pointer or reference is an ever present fact of life for C++ programmers, as it is for C programmers. operator[] offers yet another opportunity for the unwary to get bitten. Consider, for example, the following code fragment: float_array fa, fb; ... float *p = &fa[i]; ... fa = fb; ... *p = 1.0; p's declaration binds it to the address of an element of fa. (Remember, fa[i] is a reference to an element, and so &fa[i] is the address of the referenced element.) If fb and fa have a different number of elements, the assignment fa = fb deletes the old elements of fa with new ones at a different address. This leaves p dangling. An even subtler problem occurs in the test program in Listing 7 when using the (non-const) operator[] that may extend a float_array (Listing 5). If you run the program with 4 as the input value for size, it initializes fa with four elements { 0 1 2 3 }. The subsequent statements size_t n = fa.length(); fa[n] = fa[n - 1] = 123; look like they extend fa by one element and copy 123 into the last two elements, so that fa = { 0 1 2 123 123} I tested the assignment against three MS-DOS C++ compilers (Borland, Comeau, and Zortech), and they all produced that result. The next two statements n = fa.length(); fa[n - 1] = fa[n] = 456; look like they should extend fa by one more element, and store 456 in the last two elements, so that fa = { 0 1 2 123 456 456 } Interestingly, only Borland and Zortech produced that result; Comeau produced fa = { 0 1 2 123 123 456 } Apparently, Comeau evaluates fa[n - 1] = fa[n] = 456; more or less as follows: float &t1 = fa[n - 1]; float &t2 = fa[n]; t2 = 456; t1 = t2; But the call to fa[n] in the second step extends fa and moves its elements to a different place in the free store. Unfortunately, t1 has already been bound to the original location of fa[n - 1], and the call to fa[n] leaves t1 dangling. Thus, the final step (t1 = t2), copies 456 from fa[n] into the original location of fa[n - 1], not the current one. The Comeau compiler is generating correct code. C++ compilers, like C compilers, have considerable freedom about the order in which they evaluate operands between sequence points, and the Comeau compiler is merely exercising its freedom. The problem is in the test program, because the expression depends on a particular evaluation order for success. Subscripting Objects operator[] opens the door for dangling pointers and references because it returns a reference to part of a float_array's internal representation. That representation tends to move around, but the references don't rebind to the new locations. You can eliminate many opportunities for dangling references by rewriting operator[] to return a subscripting object instead of a reference. A reference to a float can only keep track of a single float element in a float_array. The reference has no way of knowing if the value it's referring to has moved. A subscripting object keeps track of both the float_array and the value of the subscript, so it always subscripts using the current location of the array. Listing 8 is the header for class float_array using a subscripting class, fa_index. fa_index has two data member: fa, a pointer to a float_array, and ix, a subscript for an element of that float_array. fa_index has a two-argument constructor that fills in values for members fa and ix. Notice that the constructor is private. Thus, clients (users) of fa_index can't create fa_index objects. How, then, does the program ever create an fa_index? fa_index declares float_array as a friend class. Hence, every member function of class float_array is a friend of class fa_index. float_array member functions has access rights so they can construct fa_index objects. Listing 9 presents the implementations for both the fa_index and float_array member functions. (The listing includes all the float_array member functions for completeness.) The only float_array member function that changes to use fa_index is the non-const operator[]. Rather than return a float & as before, it returns an fa_index object. The fa_index class has a conversion operator that converts an fa_index to a float. When you use an expression like fa[i] as an rvalue in a context expecting an arithmetic expression, the program implicitly calls operator float to convert the fa_index object into the value of element that the object designates. That is, the compiler translates the assignment in float x; ... x = fa[i]; into something like fa_index t1(fa[i]); x = float(t1); The program constructs a temporary fa_index t1 to hold the result of fa[i]. Then it applies fa_index::operator float to t1 to get the value of the array element. When fa[i] appears to the left of an assignment, the program calls fa_index::operator=(float) to write through the fa_index object and into the float_array element it designates. For example, an assignment like fa[i] = 1; translates into fa_index t1(fa[i]); t1.operator=(float(1)); Using this new implementation for float_array with the test program in Listing 7 eliminates the dangling reference. It enables all three compilers to produce the same (intended!) result. But you pay for subscripting objects. They slow down every subscripting operation on a non-const float_array. Furthermore, you lose some functionality that you have to reconstruct by writing more member functions for fa_index. For example, fa_index::operator= only lets you use an expression like fa[i] as the left side of an = operator. It doesn't cover any other uses as an lvalue, such as fa[i] *= 10 or ++fa[i]. If you want these operators, you must define them for class fa_index. Other Uses for operator[] Some C++ applications use operator[] as a file positioning operator, replacing explicit calls to a function like lseek. For example, File f("myfile"); ... f[10] = 'X'; writes an X into the tenth character position of File f. Coplien (1992, 49-52) outlines a simple implementation for this scheme using subscripting objects. Cargill (1992, 91-112) provides an excellent description of what goes wrong with poorly-designed subscripting objects. His example uses a file object similar to Coplien's. Cargill also provides some thoughtful discussion on whether using operator[] this way is just too clever. The pitfalls may out-weigh the notational advantages. References Cargill, Tom. 1992. C++ Programming Style. Reading, MA: Addison-Wesley. Coplien, James O. 1992. Advanced C++ Programming Styles and Idioms. Reading, MA: Addison-Wesley. Figure 1 Output from the test program in Listing 6 size? 3 fa = { 0 1 2 } fb = { 0 1 2 } fb = { 0 1 2 } fb = { 0 1 2 } fb = { 0 1 4 } fb = { 0 1 4 9 } fb = { 0 1 4 9 16 } fb = { 0 1 4 9 16 25 } fc = { 0 1 2 } Assertion failed: i < len, file fa4.cpp, line 59 Abnormal program termination Listing 1 Class definition of float_array // fa2.h - a dynamic array of float #include class float_array { public: float_array(size_t n = 0); float_array(const float_array &fa); ~float_array(); float_array &operator=(const float_array &fa); float &operator[](size_t i) const; inline size_t length() const; private: float *array; size_t len; }; ostream &operator<<(ostream &os, float_array &fa); inline size_t float_array::length() const { return len; } /* End of File */ Listing 2 float_array::operator[] as a const member function // fa2.cpp - a dynamic array of float #include "fa2.h" #include // other float_array member function definitions ... float &float_array::operator[](size_t i) const { assert(i < len); return array[i]; } /* End of File */ Listing 3 Class definition of float_array with operator[] overloaded as const and non-const // fa3.h - a dynamic array of float with operator[] // as both const and non-const member functions #include class float_array { public: float_array(size_t n = 0); float_array(const float_array &fa); ~float_array(); float_array &operator=(const float_array &fa); const float &operator[](size_t i) const; float &operator[](size_t i); inline size_t length() const; private: float *array; size_t len; }; ostream &operator<< (ostream &os, const float_array &fa); inline size_t float_array::length() const { return len; } /* End of File */ Listing 4 float_array function definitions // fa3.cpp - a dynamic array of float with operator[] // as both const and non-const member functions #include "fa3.h" #include // other float_array member function definitions ... const float &float_array::operator[](size_t i) const { assert(i < len); return array[i]; } float &float_array::operator[](size_t i ) { assert(i < len); return array[i]; } /* End of File */ Listing 5 An implementation of float_array that extends the size of the array when it detects subscript out-of-bounds // float_array::operator[] that extends the array on // subscript out of bounds float &float_array::operator[](size_t i) { if (i >= len) { float *new_array = new float[i + 1]; assert(new_array != 0); size_t j; for (j = 0; j < len; ++j) new_array [j] = array[j]; for (; j < i + 1; ++j) new_array[i] = 0; delete [] array; array = new_array; len = i + 1; } return array[i]; } /* End of File */ Listing 6 A trivial float_array test program #include #include "fa4.h" void display(const char *s, const float_array &fa) { cout << s <<" = "<< fa << endl; } int main() { size_t i, size; cout << "size? "; cin >> size; float_array fa(size); for (i = 0; i < fa.length(); ++i) fa[i] = i; display("fa", fa); float_array fb = fa; display("fb", fb); for (i = 0; i < 2 * size; ++i) { fb[i] = i * i; display("fb", fb); } const float_array fc = fa; display("fc", fc); cout << "fc[size] = " << fc[size] << '\n'; return 0; } /* End of File */ Listing 7 Another float_array test program #include #include "fa4.h" void display(const char *s, const float_array &fa) { cout << s << " = " << fa << endl; } int main() { size_t i, size; cout << "size? "; cin >> size; float_array fa(size); for (i = 0; i < fa.length(); ++i) fa[i] = i; display("fa", fa); i = fa.length(); fa[i] = fa[i - 1] = 123; display("fa", fa); i = fa.length(); fa[i - 1] = fa[i] = 456; display("fa", fa); return 0; } /* End of File */ Listing 8 Class definition for float_array using a subscripting class // fa5.h - a dynamic array of float using a // subscripting object #include class fa_index { friend class float_array; public: fa_index &operator=(float f); operator float(); private: fa_index(float_array *f, size_t i); float_array *fa; size_t ix; }; class float_array { friend class fa_index; public: float_array(size_t n = 0); float_array(const float_array &fa); ~float_array(); float_array &operator=(const float_array &fa); float operator[](size_t i) const; fa_index operator[](size_t i); inline size_t length() const; private: void extend(size_t i); float *array; size_t len; }; ostream &operator<<(ostream &os, const float_array &fa); inline size_t float_array::length() const { return len; } /* End of File */ Listing 9 Implementation of float_array using a subscripting object // fa5.cpp - a dynamic array of float using a // subscripting object #include "fa5.h" #include fa_index::fa_index(float_array *f, size_t i) : fa(f), ix(i) { } fa_index &fa_index:: operator=(float f) { if (ix >= fa->len) fa->extend(ix); fa->array[ix] = f; return *this; } fa_index::operator float() { if (ix >= fa->len) fa->extend(ix); return fa->array[ix]; } fa_index float_array::operator[](size_t i) { return fa_index(this, i); } float float_array::operator[](size_t i) const { assert(i < len); return array[i]; } ffloat_array::float_array(size_t n) { if ((len = n) == 0) array = 0; else { array = new float[n]; assert(array != 0); for (int i = 0; i < n; ++i) array[i] = 0; } } float_array::float_array (const float_array &fa) { if ((len = fa.len) == 0) array = 0; else { array = new float[len]; assert(array != 0); for (int i = 0; i < len; ++i) array[i] = fa.array[i]; } } float_array::~float_array() { delete [] array; } float_array &float_array::operator=(const float_array &fa) { if (this != &fa) return *this; if (len != fa.len) { delete [] array; if ((len = fa.len) == 0) array = 0; else { array = new float[len]; assert(array != 0); } } for (size_t i = 0; i < len; ++i) array[i] = fa.array[i]; return *this; } ostream &operator<<(ostream &os, const float_array &fa) { os << '{'; size_t i; for (i = 0; i < fa.length(); ++i) os << ' ' << fa[i]; return os << " }"; } void float_array::extend(size_t i) { float *new_array = new float[i + 1]; assert(new_array ! = 0); size_t j; for (j = 0; j < len; ++j) new_array [j] = array [j]; for (; j < i + 1; ++j) new_array[i] = 0; delete [] array; array = new_array; len = i + 1; } /* End of File */ Questions & Answers Linked Lists, Strings, and Internationalization Ken Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member on the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet). Q Just got the August CUJ, so please excuse the response delay. To sort a linked list quickly, count the elements, allocate an array to hold them all, then sort the array. Pick off the elements to rebuild the list. Or else use a radix ("Postman's") sort, if possible. Or use a merge sort. Any sort not relying on element exchange will serve to sort a linked list, though it will of course need minor modifications. Quicksort becomes a merge sort, except that partitioning is based on a pivot rather than being blind (half the elements go in each sublist). Next, in your "array of strings" answer you have the statement: printf(get_string(1)); SHAME! If there are any % characters in the string, nasty things may happen. Use: puts(get_string(i)); /*adds \n */ or printf ("%s", get_string(1)); Keep up the (otherwise) good work. David Chapman San Jose, CA A Thanks for your reply. As I suggested in my original article, if one needs to sort a linked list, it might be just as simple to keep an array of pointers to the data items and sort that instead. Thanks also for pointing out the potential problem with the format specifier. Most likely, the percent character (%) will not appear in text followed by any character other than a space. This will be an invalid conversion specifier whose behavior is undefined by the ANSI standard. This is another "little gotcha" of the type described in the answer to the next question. On some machine/compiler combinations, the percent sign will show up on the output. On others, it will disappear. Of course if % is followed by a legitimate conversion character, garbage will appear on the output or the program may bomb. If the specifier was for a float number (e.g. %f), then it is possible that the value picked up will be an invalid floating-point number and a floating-point exception will occur. In practice, the return value of get_string will not usually be passed to printf. Instead, it will be a parameter to some display function that will display it with an attribute and/or in some window. For those further interested in making programs portable, you might want to investigate the internationalization functions that are specified under the Xopen Portability Guidelines (XPG). These functions are available under OSF UNIX, and probably are available on several other systems. Xopen is a consortium of European computer vendors. A brief description of the design of the functions may help you develop your own if you are on a system that does not support them. The functions use a message catalog file that is specific to a single language. The catalog file contains all strings that need to be translated to another language. The files for each language are kept in a separate directory. The strings in the catalog are arranged in sets. Each set has an identifying number and each string within the set has a unique number within that set. The initial function, catopen, opens the appropriate file corresponding to the filename passed to it. The directory for the file (i.e. the language of the file) is derived using the environment variables NLSPATH and LANG. catopen returns a catalog identifier used by the remainder of the functions. The function corresponding to the get_string example is catgets. Its four parameters are the catalog identifier, a set number within the catalog, a string number with the set, and a default string. It returns a pointer to the corresponding string. If the routine is unable to access the particular string, it returns a pointer to the default string. Listing 1 and Listing 2 give an example of the use of the file. The example code produces: String to output on standard output if the file is accessible and I can't find it if the file is not accessible. The message catalog is actually composed of two files. The first file is an ASCII file. A compilation program uses the ASCII file as input and creates the actual file read by the program. This makes the lookup quicker as either a single or double seek is all that is required. The compiled file can be read in as a whole if it is not too large. The catalog can use symbolic names, rather than absolute numbers. The compilation program generates a header file for use by the calling functions. Obviously this is a far superior method, as it is easier to match the strings with their intended usage. Listing 3 and Listing 4 give an example of the use of symbols. Variable Argument Lists, Portability, and Gotchas Q Regarding the answer you gave to reader Willi Fleischer of Moerfelden, Germany (CUJ, September 1992, page 114) on the use of variable parameter list, I have tried to solve a problem of the same kind: to pass the variable arguments from a function to a function (which can be nested). The solution I found, though it needs to be verified on different platforms (I tested on SGI and IBM UNIX workstations), is to call the function that needs the variable argument list and to prevent it from adding any element to the calling function's stack. For example, in Listing 5, Msg will call PrintString in order to convert the printf argument list into a static string. As you can see, PrintString is the first function/statement to be executed in Msg after declaring line. If not, you will fail (try if(1) line = PrintString;). I understand this is somewhat restrictive, but it can be handled easily in many cases. Its advantages are: making PrintString syntax simple (no need for _direct and _va_list functions of the same kind), and making the functions that call it not have to deal with va_start and so on. Joseph Z. Wang White Plains, NY A I consider this in the land of serendipitous trouble. A particular set of operations works on a couple of computers, but is not really part of ANSI standard. The dead giveaway is the fact that it does not work if another statement is inserted into the code. PrintString is only being passed one parameter (fmt). The way the parameters are passed on this particular set of machines, the va_start macro coincidentally sets up args to point to the first parameter after fmt. If the stack pointer is altered by the calling function (such as you suggested), the coincidence disappears. Unfortunately (or fortunately, depending on how you look at it), the only portable ways to pass the variable parameter list is by the method suggested or one that follows along a similar line. I teach a class in how to make C portable. It turns out there are a few "gotchas" that can appear while using the language. Code appears portable, as it works on a few machines, but in reality, it is not. Your code is one example. The previous question regarding the % in a string passed to printf is another example of non-portable behavior. The C standard lists nine pages of unspecified, undefined, or implementation-defined behavior. Many of the items concern situations that a good C programmer would avoid anyway. Some others are of the "gotcha" variety. For example, the order of two equal members in an array sorted by qsort is unspecified. Some implementations may leave equal members in the original order. Going to an implementation that does not may produce some unexpected surprises if your programming was counting on the original behavior. Many of the portability problems can be pointed out by a good lint. For those new to C, this program is a source-code analysis program that comes with many UNIX systems and is available separately on PC systems. I have used PC-Lint by Gimpel and found it very satisfactory, in many cases superior to UNIX lints I have tried. For example, the following loop may set each element of the array to the value of its index: int array[10]; int i; for (i = 0; i < 10;) { array[i] = i++; } Then again, it may not. The value of i may be incremented before or after it is used as the index value for array. The order of evaluation of the assignment operator is not specified by the language. Nor is the time specified that the increment will take place except that it is after the value of i on the right hand side is obtained and before the semicolon is reached. Any of the lint programs should point out this error of dependence on order of evaluation. My suggestion for writing portable code is to keep it simple. Another author suggests that "well written code is portable." In a paper by A. Dolenc, A. Lemmke, D. Keepl and G.V. Reilly, "Notes on Writing Portable Programs in C," the authors state that the expression f(p=&a, p=&b, p=&c) is ambiguous. The standard permits it to be interpreted as f(&a, &b, &c) or f(&c, &c, &c) [or many others--pjp]. My immediate response is that one should have coded it in whichever of the forms was really required, followed by the appropriate assignment for p. By the way, the paper includes some excellent suggestions for making your program portable. They would permit your program to be backfitted to old K&R compilers, if you need to do that. I will admit that I do not code in an absolute portable fashion. It takes a lot of hard work. The one facet which I ignore entirely is the limitation of six characters of significance in an external name. Although the ANSI C library must adhere to this limitation, I find it difficult to obey it while writing a maintainable program. If I come across that odd system that requires it, I will simply make up some header files that look like: #define my_reasonably_named_function(a,b,c)\ IHE345(a,b,c) #define another_useful_function(d,e) IHE346(d,e) Truncating a File in Place and Portability Is it possible to truncate a file in place using ANSI C? Currently, I am copying the truncated data to a temporary file, removing the original file, and renaming the temporary file to the original file's name. Is there a more sophisticated method? BSD4.3 UNIX provides the (non-portable) system function calls truncate and ftruncate. Do these implement the method described above? Jeff Dragovich Urbana, Illinois A The answer is as you have just described it. The BSD functions can play games with the inode (which contains the length of the file and data block-allocation information). Obviously you can run out of space when you truncate the file, as you will be making a copy of it. Since our theme is portability this month, let me suggest how to make functions like these portable. You can create a compatibility library of functions that you commonly use. For example, instead of using ftruncate, you might make up functions as: int file_truncate_by_fd(int file_descriptor,\ unsigned int size); int file_truncate_by_name(char *file_name,\ unsigned int size); These functions would be kept in a package of file functions whose implementation might vary from machine to machine. A header file, say file.h would be used for the function prototypes and any other #defines or typedefs required by your functions. In your BSD version of the library, these functions could call truncate and ftruncate. Alternatively, you could #define them in the header file to be the appropriate call, so the functions will be called inline, without an extra function call layer. For the MS-DOS version, the functions would perform the operations you listed. For all versions, the functions should return an error indication if the truncation was unable to be performed. Note that it is logically impossible for the BSD version to fail to truncate, while the MS-DOS version may fail if there is insufficient disk space. scanf and ANSI I have some comments about the sscanf question in the September 1992 CUJ. I realize this is probably the thousandth letter you've received about this, but I've run into similar sscanf problems in the past and have a vested interest in these questions. According to my reading of Harbison and Steele, (I don't have a copy of the ANSI Standard document), sscanf should return 4 on string[0], and 5 on string[1]. I suspect the return of 4 may be the result of an error in the code as published (should string[0] b e garbageR A 3 0 4 ?), but, as you pointed out, string[1] scanning using the [^PR] conversion operator will stop immediately upon seeing the R; no conversion will take place. Since the assignment-suppression operator is involved, this won't affect the rest of the conversion. No pointers would be consumed in either case. Conversion will then pick up at the first %c, and continue with the other char and the three ints, all of which are present in string[1]. Am I correct in this, or am I missing yet one more subtlety in sscanf formatting? By the way, sscanf behaves this way in Borland C++ version 3.1; previous versions of Turbo C/C++ would, believe it or not, cause a null-pointer error with this usage of sscanf. Thanks for your attention in this. Dan Rempel Victoria, B.C. Canada A Thank you for your sharp eye in catching the code error. I'll repeat the relevant code here, so our readers won't have to go back. The strings (with your correction) are: char *string[] = { "some garbageR A 3 0 4", "R A 3 0 4" }; The sscanf with its format looked like: const char *format = "%*[^PR]%c %c %d %d %d"; ret = sscanf (string[i], format, &l, &q, &k, &m, &n); When scanning string[0], the * suppresses the assignment of any set of characters which match the [^PR] specifier. That specifier matches any nonempty sequence of characters that are not P or R. This directive eats up the characters "some garbage" in string[0]. The value R is then assigned to 1, and the remaining assignments take place normally. For string[1], there is no sequence of characters that match the initial [^PR] specifier. The directive fails with a matching failure (inappropriate input, according to the standard). When that directive fails, sscanf returns with a zero value as no assignments have taken place. As I read the C Standard, if a match to a directive fails, even if there is assignment suppression, the function terminates at that point. The order of the operations as specified by the C Standard is to match the specifier and convert it to the corresponding type. It then checks for assignment suppression. Listing 1 Message catalog file example .../english/example.cat Set ID = 10 ... Set ID = 24 1 "String to output" 2 "Another string to output" /* End of File */ Listing 2 Accessing the message catalog message_file = catopen("example.cat", 0); printf("%s", catgets(message_file, 24, 1, "I can't find it"); catclose(message_file); /* End of File */ Listing 3 Message catalog file example with symbols .../english/example.cat Set ID = ERROR_MESSAGES ... Set ID = SOME_STRINGS FIRST_STRING "String to output" SECOND_STRING "Another string to output" /* End of File */ Listing 4 Accessing the message catalog symbolically #include "example.h" ... message_file = catopen("example.cat", 0); printf("%s", catgets(message_file, SOME_STRINGS, FIRST_STRING, "I can't find it"); catclose(message_file); /* End of File */ Listing 5 Msg will call PrintString in order to convert the printf argument list into a static string. #include char *PrintString(const char *fmt, ...) /* Convert printf() arguments into a static string. */ { va_list args; static char line[999]; va_start(args, fmt); vsprintf(line, fmt, args); va_end(args); return line; } void Msg(const char *fmt, ...) /* Send message to stdout user. */ { char *line; line = PrintString(fmt); print("%s\n", line); ... } /* End of File */ On The Networks: Special Issue: USENET Network News Update Sydney S. Weinstein Sydney S. Weinstein, CDP, CCP is a consultant, columnist, author, and president of Datacomp Systems, Inc., a consulting and contract programming firm specializing in databases, data presentation and windowing, transaction processing, networking, testing and test suites, and device management for UNIX and MS-DOS. He can be contacted care of Datacomp Systems, Inc., 3837 Byron Road, Huntingdon Valley, PA 19006-2320 or via electronic mail on the Internet/Usenet mailbox syd@DSI.COM (dsinc!syd for those who cannot do Internet addressing). This is my third anniversary in writing this column, and its time to update the prior January columns. In January 1990 (CUJ Vol. 8, No. 1), I wrote about the internet, the Internet, USENET, and Network News. In January 1991 (CUJ Vol. 9, No. 2), I wrote about obtaining USENET Network News. In January 1992 (CUJ Vol. 10, No. 2), I wrote about obtaining the sources referred to in this column. This year, I am going to present a quick review and update to all three past columns. I do not intend to provide an in-depth review, so newcomers to USENET might want to try and seek out back issues of those articles. "On The Networks" covers articles posted to several of the source groups on USENET Network News: comp. sources.games, comp.sources.misc, comp.sources. reviewed, comp.sources.unix, comp.sources.x, and alt.sources. Each of these groups is like a section of a large electronic magazine called USENET Network News. I call it a magazine, and not a bulletin board partly because unlike a bulletin board, where each reader accesses a central machine to read the messages, USENET Network News is delivered on a subscription basis to each computer, and the articles are read locally. I try and limit my coverage to the highlights of the articles posted to each group. In addition, I also generally restrict my coverage to freely-distributable C and C++ sources. Freely-distributable means that you can freely (and for free) make copies of the software, use it as you see fit, and give it away as you desire. It does not mean the software is in the public domain. Most of the software is copyrighted. This means you cannot pretend you wrote it, or include code from it in a product you are selling. However, the authors have allowed you to use and distribute it for free. If you make changes, most authors do not let you call the changed version by the name of the original. This is to avoid confusion as to what is and is not part of the product and to reduce the authors' support headaches. Internet Update First an update. USENET, often times referred to as "the net" is a loose collection of cooperating computers. In the long-distant past, all USENET computers ran UNIX, but now they could be running anything from MS/DOS to VAX/VMS. It also used to be that to be considered a computer on USENET, you communicated via the UNIX to UNIX Communications Protocol (UUCP) to another computer. That also has changed. Now there are many protocols in use. So, to be considered a member of the USENET network, one must be able to exchange Electronic Mail with other computers on the USENET network. If your computer is part of a network, and that network has a gateway to any other network, you are considered on an internet, short for inter-network. Note this internet is spelled with a lower case i. The Internet (capital I) is the computer network whose addressing (name space) is loosely managed by the DDN Network Information Center in Chantilly, VA. It is a collection of networks that grew out of the Defense Department's ARPANET (Advanced Projects Research Agency Network). The Internet now includes not only the MILNET and NSFnet member networks (direct descendents of the ARPANET), but also several commercial TCP/IP networks which are members of the Commercial Internet Exchange (CIX). The Internet is mostly a set of networks with leased lines permanently connecting them to regional and backbone networks. Some outlying networks use gateways with dial-up links to reach their regional network, but most links are 56K, 1.544M, or 45M bits/second leased lines (that's megabits per second). Whereas only mail and news is usually available over the USENET via UUCP, the Internet runs the TCP/IP protocol and supports news (NNTP, Network News Transfer Protocol), mail (SMTP, Simple Mail Transfer Protocol), Wide Area Information Service (WAIS), remote access lookup (archie), remote logins to any computer on the network provided you have an account there (telnet), and remote file transfer (FTP, file transfer protocol) in addition to many other services. All of these services coexist and work in real time. Network News Update The sources mentioned in this column are released via several groups distributed as part of USENET Network News. USENET Network News is both a method of distributing information between a very large group of computers, and a somewhat organized collection of that information into news groups that are distributed via the news software. First, the software. There are now two current Network News transport software suites: the older, C News, and a newcomer, INN. The current version of the traditional Network News transport software is named C News, not because it is written in C but because it follows A News (previously called News) and B News as the third rewrite of the transport software. It supports transfer of the news articles (the individual messages) between every member of the USENET network. The C News software recently had a performance-enhancement release, and a "cleanup" release is expected shortly. C News is best suited for smaller sites that mostly have UUCP feeds, and feed a limited set of neighbors. New for the major backbone sites, especially those with Internet connections, is Internet Network News (INN) from Rich Salz. INN is largely responsible in cutting down the time it takes for an article to be propagated throughout the backbone networks. Where C news uses batching to save articles for distribution in bulk, INN uses an immediate transfer to its NNTP (TCP/IP-based network news protocol) neighbors. Thus an article now reaches most of the backbone and regional network sites in only 1-5 minutes. (Just three years ago it took close to a day). INN is relatively new, and takes advantage of the decline in the price of RAM. It boosts performance by keeping everything it can in memory. Second, there is the volume. At this time last year, the average was twenty-five megabytes per day. Now it is up to approximately forty megabytes per day. The number of newsgroups is also growing. Currently there are over 2,400 of them. Lastly, there is a new distribution method, CD-ROM. Sterling Software, 1404 Fort Crook Road S., Bellevue, NE 68005, publishes a set of ISO-9660 format CD-ROMs with almost all the groups of USENET Network News. The CD-ROMs come out when a new one fills up, which is about every 10-14 days. They even include a modified news reader that will access the information as stored on the CD-ROM. Sterling can be reached at (800) 643-NEWS, or (402) 291-2108 from outside of the US. A subscription runs about $350/year + shipping. Shipping varies from $60/year in the US up to $200/year for some overseas areas. How to Use This Column This column normally reports on articles in the comp.sources and alt.sources sub hierarchies. All of the groups I currently report on but alt.sources are moderated. For the moderated groups, the authors of the software submit their packages to the moderator for posting. Each group has its own rules for acceptance. alt.sources is unmoderated and a free-for-all. Sources in that group are posted directly by the author. For the moderated groups, when I list a package, I provide five pieces of information for each package. The Volume, Issue(s), archive name, the contributor's name, and electronic mail address. The Volume and Issue are specifically named in the listing. The archive name is in italics, and the contributor's name is followed by an electronic mail address, in < >'s. To locate a package via WAIS or archie, use the archive name. This is the short one-word name in italics given with each listing. To find the file at an archive site, use the group name (from the section of the column you are reading--I place all listings for each group together in the column), the volume and the archive name. Most archive sites store the postings as group/volume/archive-name. The issue numbers tells you in how many parts the package was split when posted. This way you can be sure to get all of the parts. In addition, I report on patches to prior postings. These patches also include the volume, issue(s), archive name, the contributors name, and electronic-mail address. Patches are stored differently by different archive sites. Some store them along with the original volume/archive name of the master posting. Some store them by the volume/archive name of the patch itself. The archive name listed is the same for both the patch and the original posting. alt.sources, being unmoderated, does not have volume and issue numbers. So I report on the date in the Date: header of the posting and the number of parts in which it appeared. If the posting was assigned an archive-name by the contributor, I also report on that archive name. Archive sites for alt.sources are harder to find, but they usually store things by the archive name. How to Find an Archive Site With that much information flowing into each site every day, most sites cannot keep the information on their local disks for very long, usually only a couple of days. So, by the time you read my articles, the sources have been deleted from the machines in the network to make room for newer articles. So, many sites have agreed to archive the source groups and keep these archives for several years. The problem then is finding out which sites archive which groups, and how to access these archives. I again refer to the articles by Jonathan I. Kames of the Massachusetts Institute of Technology posted to comp.sources.wanted and news.answers. These articles, appear weekly and explain how to find sources. As a quick review, here are the steps: 1. Figure out in what group, Volume, and Issue(s) the posting appeared. Also try and determine its archive name. If you know these items, it's usually easy to find an archive site that keeps that group. Most archive sites keep their information in a hierarchy ordered first on the group, then on the volume and last on the archive name. These together usually make up a directory path, as in comp.sources.unix/volume22/elm2 .3. In that directory you will find all of the articles that made up the 2.3 release of the Elm Mail User Agent that was posted in Volume 22 of the comp.sources.unix newsgroup. If you do not know the archive name, but do know the volume, each volume also has an Index file that can be retrieved and read to determining the archive name. One common publicly accessible archive site for each of the moderated groups in this article is UUNET. 2. If you do not know which sites archive the groups, or even if any site is archiving that item, but they are not archiving the entire group, consult Archie. (CUJ August 1991, Vol. 9, No. 8). Archie is a mail-response program that tries to keep track of sites reachable via FTP (file transfer protocol, a TCP/IP protocol used by internet sites) that have sources available for distribution. Even if you cannot access the archive site directly via FTP, it is worth knowing that the archive site exists because there are other ways of retrieving sources only available via FTP. 3. If you know the name of the program, but do not know to what group it was posted, try using Archie and search based on the name. Since most sites store the archives by group and volume, the information returned will tell you what newsgroup and volume it was posted in. Then you can retrieve the item from any archive site for that newsgroup. 4. If you do not even know the name, but know you are looking for source code that does xxx, retrieve the indexes for each of the newsgroups and see if any of the entries (usually listed as the archive name and a short description of the function) look reasonable. If so, try those. Or, make a query to archie based on some keywords from the function of the software and perhaps it can find items that match. 5. Next you have to determine what access methods the archive machine allows to retrieve software. Most archive sites are internet-based, and support the FTP service. If you have access to FTP on the internet, this is the easiest and fastest way of retrieving the sources. If you don't, perhaps a local college is a member of the internet and can assist you in using FTP to retrieve the sources you need. Other sites support anonymous UUCP access. This is access via the UUCP protocol where you don't need to register in advance to call in. UUNET Communications Services supplies this using the (900) GOT-SRCS number at a per-minute charge for nonsubscribers. Many other archive sites provide it for just the cost of your long-distance telephone call. If you cannot use FTP, this is the next best method to use. Many anonymous UUCP archive sites list what they carry in the Nixpub listing of public access UNIX sites maintained by Phil Eschallier. The Nixpub listing is posted in comp.misc and alt.bbs periodically. If you don't get News and need a copy, it can be retrieved via electronic mail using the periodic-posting mail-based archive "server." This is run by MIT on the system pit-manager.mit.edu. To use the server, send an electronic mail message to the address mail-server@pitmanager.mit.edu with the subject help and it will reply with instructions on how to use the server. If you are not on USENET, sending electronic mail to sites on the internet is also possible via the commercial mail services. CompuServe, MCI-Mail, ATT-Mail, and Sprint-Mail all support sending messages to Internet addresses. Contact the support personnel at the commercial mail service you use for details on how to send messages to Internet addresses. The last way to access the sources is via electronic mail. Several sites also make their archives available via automated mail-response servers. Note, this can be a very expensive way of accessing the information, and due to the load it places on the networks, most archive servers heavily restrict the amount of information they will send each day. The most commonly used mail-response FTP server is ftpmail@decwrl.dec.com. Digital Equipment Corporation runs a mail-based archive server that will retrieve sources via FTP and then mail them to you. To find out how to use this service, send it a message with the word help in the body. CD-ROM Archives Available Also available, are CD-ROMs with the archives of sources posted via USENET and from other sources as well. Two of the larger publishers are Walnut Creek CD-ROM and Prime Time Freeware. Walnut Creek CD-ROM, 1547 Palos Verdes Mall, Suite 260, Walnut Creek, CA (800) 786-9907 or (510) 947-5996 publishes several CD-ROMs each year. Some contain the Simtel20 MS-DOS Archive, others the X and GNU archives, and still others MS-Windows sources and other collections of sources and binaries. Disks run from $25 to $60 each (varies by title) plus shipping. In addition, the offer those hard to find CD-caddys at reasonable prices. Prime Time Freeware, 370 Altair Way, Suite 150, Sunnyvale, CA 94086, (408) 738-4832, FAX: (408) 738-2050, , publishes twice a year a collection of freely distributable source code, including the complete USENET archives. Their disks run about $60 each set plus shipping. The latest issue, August 1992, has over 3MB of source code spread over two disks. They also offer a standing subscription plan at a discount. Hopefully, this special edition of my column has given you a hint as to how to read my column and track down the sources. Note, I have been asked many times if I can make floppies or tapes containing the software mentioned in my column. I cannot spare the time to do this. I also have to work (and teach) for a living. If I started doing this, I could easily spend all my time trying to fulfill the requests and never get any of my work done. However, what I have offered to do in the past, and am still willing to do, is provide a list of USENET sites in your area code. Send me a self-addressed, stamped envelope (my address is in the bio attached to this column). Those living in major metropolitan areas, please include two stamps on your letter. Note: I can only offer this service for US area codes. If you have net access, but need a news neighbor, I will also reply to Electronic Mail asking for nearby news sites. Code Capsules Time and Date Processing in C Chuck Allison Chuck Allison is a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. Most operating systems have some way of keeping track of the current date and time. ANSI C makes this information available in various formats through the library functions defined in time.h. The time function returns a value of type time_t (usually a long), which is an implementation-dependent encoding of the current date and time. You in turn pass this value to other functions which decode and format it. The program in Listing 1 uses the functions time, localtime and strftime to print the current date and time in various formats. The localtime function breaks the encoded time down into struct tm { int tm_sec; /* (0 - 61) */ int tm_min; /* (0 - 59) */ int tm_hour; /* (0 - 23) */ int tm_mday; /* (1 - 31) */ int tm_mon; /* (0 - 11) */ int tm_year; /* past 1900 */ int tm_wday; /* (0 - 6) */ int tm_yday; /* (0 - 365) */ int tm_isdst; /* daylight savings flag */ }; local time overwrites a static structure each time you call it, and returns its address (therefore only one such structure is available at a time in a program without making an explicit copy). The ctime function returns a pointer to a static string which contains the full time and date in a standard format (including a terminating newline). strftime formats a string according to user specifications (e.g., %A represents the name of the day of the week). See Table 1 for the complete list of format descriptors. Time/Date Arithmetic You can do time/date arithmetic by changing the values in a tm structure. The program in Listing 2 shows how to compute a date a given number of days in the future, as well as the elapsed execution time in seconds. Note the optional alternate syntax for the time function (the time_t parameter is passed by reference instead of returned as a value). The mktime function alters a tm structure so that the date and time values are within the proper ranges, after which the day-of-week (tm_wday) and day-of-year (tm_yday) fields are updated accordingly. mktime brings the date and time values in the tm structure into their proper ranges, and updates the day of week (tm-wday) and day of year (tm-yday) values accordingly. This occurs when a date falls outside the range that your implementation supports. My MS-DOS-based compiler, for example, cannot encode dates before January 1, 1970, but VAXC can process dates as early as the mid-1800s. The asctime function returns the standard string for the time represented in tm parameter (so ctime (&tval) is equivalent to asctime (localtime(&tval)). The function difftime returns the difference in seconds between two time_t encodings as a double. If you need to process dates outside your system's range or calculate the interval between two dates in units other than seconds, you need to roll your own date encoding. The application in Listing 3 through Listing 5 shows a technique for determining the number of years, months and days between two dates, using a simple month-day-year structure. It subtracts one date from another, much as you might have done in elementary school (i.e., it subtracts the days first, borrowing from the month's place if necessary, and so on). Note that leap years are taken into account. For brevity, the date_interval function assumes that the dates are valid and that the first date entered precedes the second. Following the lead of the functions in time.h, It returns a pointer to a static Date structure which holds the answer. File Time/Date Stamps Most operating systems maintain a time/date stamp for files. At the very least, you can find out when a file was last modified. (The common make facility uses this information to determine if a file needs to be recompiled, or if an application needs to be relinked). Since file systems vary across platforms, there can be no universal function to retrieve a file's time/date stamp, so the ANSI standard doesn't define one. However, most popular operating systems (including MS-DOS and VAX/VMS) provide the UNIX function stat, which returns pertinent file information, including the time last modified expressed as a time_t. The program in Listing 6 uses stat and difftime to see if the file time1.c is newer than (i.e., was modified more recently than) time2.c. If you need to update the time/date stamp of a file to the current time, simply overwrite the first byte of a file with itself. Although the contents haven't changed, your file system will think it has, and will update the time/date stamp accordingly. (Know your file system! Under VAX/VMS, you get a newer version of the file, while the older version is retained). This is sometimes called "touching" a file. The implementation of touch in Listing 7 creates the file if it doesn't already exist. Note that the file is opened in "binary" mode (indicated by the character b in the open mode string--I'll discuss file processing in detail in a future capsule). Table 1 Format descriptors for strftime Code Sample Output --------------------------------------------- %a Wed %A Wednesday %b Oct %B October %c Wed Oct 07 13:24:27 1992 %d 07 (day of month [01-31]) %H 13 (hour in [00-23]) %I 01 (hour in [01-12]) %j 281 (day of year [001-366]) %m 10 (month [01-12]) %M 24 (minute [00-59]) %p PM %S 27 (second [00-59] ) %U 40 (Sunday week of year [00-52]) %w 3 (day of week [0-6]) %W 40 (Monday week of year [00-52]) %x Wed Oct 7, 1992 %X 13:24:27 %y 92 %Y 1992 %Z EDT (daylight savings indicator) Listing 1 time1.c -- prints the current date and time in various formats #include #include #define BUFSIZE 128 main() { time_t tval; struct tm *now; char buf[BUFSIZE]; char *fancy_format = "Or getting really fancy:\n" "%A, %B %d, day %j of %Y.\n" "The time is %I:%M %p."; /* Get current date and time */ tval = time(NULL); now = localtime(&tval); printf("The current date and time:\n" "%d/%02d/%02d %d:%02d:%02d\n\n", now->tm_mon+1, now->tm_mday, now->tm_year, now->tm_hour, now->tm_min, now->tm_sec); printf("Or in default system format:\n%s\n", ctime(&tval)); strftime(buf,sizeof buf,fancy_format,now); puts(buf); return 0; } /* Output The current date and time: 10/06/92 12:58:00 Or in default system format: Tue Oct 06 12:58:00 1992 Or getting really fancy: Tuesday, October 06, day 280 of 1992. The time is 12:58 PM. */ /* End of File */ Listing 2 time2.c -- shows how to compute a date a given number of days in the future, as well as the elapsed execution time in seconds #include #include #include main() { time_t start, stop; struct tm *now; int ndays; /* Get current date and time */ time(&start); now = localtime(&start); /* Enter an interval in days */ fputs("How many days from now? ",stderr); if (scanf("%d",&ndays) !=1) return EXIT_FAILURE; now->tm_mday += ndays; if (mktime(now) != -1) printf("New date: %s",asctime(now)); else puts("Sorry. Can't encode your date."); /* Calculate elapsed time */ time(&stop); printf("Elapsed program time in seconds: %f\n", difftime(stop,start)); return EXIT_SUCCESS; } /* Output How many days from now? 45 New date: Fri Nov 20 12:40:32 1992 Elapsed program time in seconds: 1.000000 */ /* End of File */ Listing 3 date.h -- a simple date structure struct Date { int day; int month; int year; }; typedef struct Date Date; Date* date_interval(const Date *, const Date *); /* End of File */ Listing 4 date_int.c -- computes time interval between two dates /* date_int.c: Compute duration between two dates */ #include "date.h" #define isleap(y) \ ((y)%4 == 0 && (y)%100 != 0 (y)%400 == 0) static int Dtab [2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date *date_interval(const Date *d1, const Date *d2) { static Date result; int months, days, years, prev_month; /* Compute the interval - assume d1 precedes d2 */ years = d2->year - d1->year; months = d2->month - d1->month; days = d2->day - d1->day; /* Do obvious corrections (days before months!) * * This is a loop in case the previous month is * February, and days < -28. */ prev_month = d2->month - 1; while (days < 0) { /* Borrow from the previous month */ if (prev_month == 0) prev_month = 12; --months; days += Dtab[isleap(d2->year)][prev_month--]; } if (months < 0) { /* Borrow from the previous year */ --years; months += 12; } /* Prepare output */ result.month = months; result.day = days; result.year = years; return &result; } /* End of File */ Listing 5 tdate.c -- illustrates the date_interval function /* tdate.c: Test date_interval() */ #include #include #include "date.h" main() { Date d1, d2, *result; int nargs; /* Read in two dates - assume 1st precedes 2nd */ fputs("Enter a date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &d1.month, &d1.day, &d1.year); if (nargs != 3) return EXIT_FAILURE; fputs("Enter a later date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &d2.month, &d2.day, &d2.year); if (nargs != 3) return EXIT_FAILURE; /* Compute interval in years, months, and days */ result = date_interval(&d1, &d2); printf("years: %d, months: %d, days: %d\n", result->year, result->month, result->day); return EXIT_SUCCESS; } /* Sample Execution: Enter a date, MM/DD/YY> 10/1/51 Enter a later date, MM/DD/YY> 10/6/92 years: 41, months: 0, days: 5 */ /* End of File */ Listing 6 ftime.c -- determines if time1.c is newer than time2.c /* ftime.c: Compare file time stamps */ #include #include #include #include main() { struct stat fs1, fs2; if (stat("time1.c",&fs1) == 0 && stat("time2.c",&fs2) == 0) { double interval = difftime(fs2.st_mtime,fs1.st_mtime); printf("time1.c %s newer than time2.c\n", (interval < 0.0) ? "is" : "is not"); return EXIT_SUCCESS; } else return EXIT_FAILURE; } /* Output time1.c is not newer than time2.c */ /* End of File */ Listing 7 touch.c -- updates time stamp by overwriting old file or creating the file if it doesn't exist /* touch.c: Update a file's time stamp */ #include void touch(char *fname) { FILE *f = fopen(fname,"r+b"); if (f != NULL) { char c = getc(f); rewind(f); putc(c,f); } else fopen(fname,"wb"); fclose(f); } /* End of File */ CUG New Releases Dynamic Link Library for Text Windows Steven Graham Steven K. Graham has worked for Hewlett-Packard, served on the Faculty at UMKC, and is currently a Senior Engineer with CSI. TextView, CUG375 (one disk) is a free Dynamic Link Library (DLL) for simplified manipulation of text windows under Microsoft Windows, written by Alan Phillips (Lancaster, United Kingdom). Alan Phillips is a systems programmer at the Lancaster University Computer Centre, where he writes UNIX communications software. Alan can be contacted at a.phillips@lancaster.ac.uk. Similar to WinDosIO, a previous CUG volume, TextView handles the details of window operations, permitting users to call functions for writing text (such as TVOutputText) in much the same way printf would be called in an MS-DOS application (with the exception of an extra parameter to identify the window where the text will be written). TextView can create multiple, independent windows that can be resized, minimized, maximized, and scrolled horizontally and vertically. A thoroughly-documented demonstration program illustrates the use of TextView windows to provide tracing and debugging information during application development. TextView requires the use of a compiler (such as Microsoft C) which can generate Windows code. The TextView volume includes a readable and carefully-organized 42-page manual. The TextView functions follow the same conventions as the Windows API, and the manual uses the same layout as the Microsoft Windows Programmer's Reference. TextView function names all begin with TV. The functions use Pascal calling conventions and must be declared FAR. Function prototypes are contained in the file textview.h. Adding this file to your source selects the right calling mode and performs necessary casts to far pointers. The TextView import library textview.lib must be included in the list of libraries to be linked. The stack size required for your application may need to be increased. Some functions in the TextView import library must be statically linked. Dual-Mode tools Volume 376 (four disks) adds OS/2 tools to the CUG library. Martii Ylikoski, of Helsinki, Finland, has provided a large number of free, dual-mode tools that support both OS/2 and MS-DOS. The tools are remarkably well packaged. Each tool includes accompanying source, makefile, documentation, and demo files, along with files (.bat or .cmd) to install and uninstall the tools. For OS/2 there is also a tools2.inf file, in the standard format for OS/2 help files. Full source code is included, generally with a single file per utility. The makefiles (.mak) indicate the required dependencies. A library was used in building the tools, and is included in two forms--mtoolsp.lib for protected mode and mtoolsr.lib for real mode. No documentation for the libraries exists, other than the examples of function use provided in the source code for the tools. The collection of 54 utilities provides a variety of functions such as: find file (ff), disk usage (du), head, tail, set priority (setprty), touch, cat, and scan (a find-like utility that searches for files and executes commands once the files are found). Hard-To-Find Information on Floppy Disks Diskette manipulations are the core of CUG 377 (one disk), provided by Ian Ashdown, P. Eng., of West Vancouver. This volume provides a wealth of information about diskette device-service routine (DSR) functions. The documentation addresses a variety of quirks in diskette access, and provides considerable hard-to-find information on floppy diskettes, diskette controllers, and the diskette DSR functions. The volume also provides extensive example and test routines, with source code (in both C and C++ versions), for reading, writing, formatting, and verifying almost any IBM System 34 format diskette on a PC compatible. The code includes support and interface functions that increase the diskette DSR's reliability and provide a consistent programming interface across PC platforms. The information was largely determined through extensive use of an in-circuit emulator and other debugging tools, along with careful study of various machines and various DOS and BIOS versions. Given the variety of ROM BIOSes available, and the necessity to derive the information by experimentation, the material in this volume cannot cover every case, but certainly provides a thorough and careful treatment. C++ Matrix Operations From Robert Davies, a consultant and researcher in mathematics and computing from New Zealand, formerly with the New Zealand Department of Scientific and Industrial Research (DSIR), we get NEWMAT (CUG 378, one disk), a C++ matrix package. This volume was written for scientists and engineers who need to manipulate a variety of matrices using standard matrix operations. It was developed by a scientist (Robert Davies has a Ph.D. from the University of California at Berkeley) to support real work. NEWMAT emphasizes operations supporting statistical calculations. Functions include least squares, linear-equation solve, and eigenvalues. Matrix types supported include: Matrix (rectangular matrix), UpperTriangularMatrix, LowerTriangularMatrix, DiagonalMatrix, SymmetricMatrix, BandMatrix, UpperBandMatrix, LowerBandMatrix, SymmetricBandMatrix, and RowVector and ColumnVector (derived from Matrix). Only one element type (float or double) is supported. Supported matrix operations include: *, +, --, inverse, transpose, conversion between types, submatrix, determinant, Cholesky decompositions, Householder triangularization, singular value decomposition, eigenvalues of a symmetric matrix, sorting, fast Fourier transform, printing, and an interface compatible with Numerical Recipes in C. NEWMAT supports matrices in the range of 4x4 to the machine-dependent, maximum array size 90x90 double elements or 125x125 float elements for machines whose limit for contiguous arrays is 64K. NEWMAT works for very small matrices, but is rather inefficient. NEWMAT works with Borland and Glockenspiel C++. The version current at this writing (NEWMAT03) doesn't work with GNU C++, but a new version (NEWMAT06) is expected (by November 1992) that will work with GNU C++. Robert Davies suggests the following as criteria for interest in NEWMAT: first, a desire for matrix operations expressed as operators; second, a need for various matrix types; third, a need for only a single element type; fourth, use of matrix sizes between 4x4 and 90x90; and fifth, tolerance for a large and complex package. There is a fairly large file documenting the package, which broadly addresses issues from particulars of functions and interactions with various compilers, through design issues in building a matrix package. If you fit the profile described, then NEWMAT may be the matrix tool you need. Archiving and Compression Program The final CUG volume for this month (CUG 379, one disk) is ZOO (version 2.1), a file archiving and compression program (standard extension .zoo), written by Rahul Dhesi, with assistance from J. Brian Walters, Paul Homchick, Bill Davidsen, Mark Alexander, Haruhiko Okumura, Randal L. Barnes, Raymond D. Gardner, Greg Yachuk, and Andre Van Dalen. This volume includes C source, executable, and documentation. Zoo is used to create and maintain collections of files in compressed form. It uses the Lempel-Ziv compression algorithm which yields space savings from 20% to 80% depending on the file data. Zoo can manage multiple generations of the same file and has numerous options accompanied by lengthy descriptions in the manuals. Zoo supports a range of hardware and operating systems, and includes makefiles with various options. Zoo is part of the GNUish MS-DOS project, an attempt to provide a GNU-like environment for MS-DOS, with GNU ports and MS-DOS replacements for non-ported GNU software. C Programming Guidelines and C ++ Programming Guidelines Dwayne Phillips The author works as a computer and electronics engineer with the U.S. Department of Defense. He has a PhD in Electrical and Computer Engineering from Louisiana State University. His interest include computer vision, artificial intelligence, software engineering, and programming languages. C Programming Guidelines and C++ Programming Guidelines are comprehensive style manuals that list specific standards for all aspects of C and C++ programming. They are good books for any group of programmers trying to work together to produce consistently high-quality, portable software. C Programming Guidelines covers standard ANSI C while C++ Programming Guidelines covers general C++. C++ Programming Guidelines repeats large parts of C Programming Guidelines, but it covers C++ specifics and has excellent material concerning migrating from C to C++. Purpose These books list coding standards, that is, rules governing the details of how to write source code. C and C++ are flexible languages and programmers can write correct, working programs using a wide variety of "styles" (including the obfuscated style). A coding standard offers rules and guidelines that help different developers to produce programs in the same style. The authors give three primary reasons for using a coding standard: reliability, readability/maintainability, and portability. If programmers follow a sound coding standard, the result will be more reliable, readable, and portable software. Some programmers argue that while noble, these attributes ruin program speed and efficiency. Although a legitimate argument in some special situations, long-term performance and monetary costs support the authors' reasons for advocating a coding standard. Audience Most C and C++ programmers could use these books. For example, if you are starting a software company, these books could aid your quality control program. In a university research lab, these books could help standardize the work of the beginners. A government employee wanting to specify software standards to a contractor could simply hand these over. A programmer working alone could use these to minimize the tendency to drift into a personal programming style. The publisher offers machine-readable versions of both books. This enables you to place them online as ready references or fold them into your group's own coding standard. Organization Because both books are references, the they are organized along similar lines. The chapters contain one- to ten-page sections with the majority being two pages long. These sections resemble man pages on UNIX systems. Each section has subsections titled STANDARD or GUIDELINE, JUSTIFICATION, EXAMPLE, ALTERNATIVES, and REFERENCES. The authors use STANDARDs as rules that you must follow while GUIDELINEs are suggestions. The subsections contain short, direct paragraphs about the subject at hand that resemble cohesive subroutines. This style of writing is appropriate for such reference books. These are not the types of books you cuddle up with in front of the fire and read from cover to cover. Nevertheless, the authors do an excellent job of tying all the sections together around the three primary reasons for using a coding standard. Sometimes the authors are too efficient. They often refer to other books published by Plum Hall for details. I prefer reading the information in the book at hand over holding my place with one finger while thumbing through a different book. The programming guidelines in the books fall into one of two categories. The first category comprises syntax rules (use of tabs, spaces, lines per function, lines per file). The second category comprises style rules (functional cohesiveness, functions per file, program and problem structure). This may seem to be a lengthy statement of the subject since most books on programming devote only one chapter to programming style. The subject, however, is more complicated than most people realize and does require one book per language. One criticism of the text is a shortage of code examples. The authors do give code examples, but not in every section. Programmers sometimes do not translate words into action well. They do, however, understand code and I wish the these books had more source code. Highlights There are several sections in each deserving special attention. C Programming Guidelines In C Programming Guidelines, the author discusses byte order and how this affects program portability. When you venture outside the PC world, you find the different machines store numbers with a different byte order. This is especially true with floating-point numbers. C Programming Guidelines suggests converting data to a canonical form before storing to disk and converting back immediately after reading from disk. One section discusses the different methods of placing braces ({}) in C programs. The author gives three different methods with many examples. This may sound trivial, but the author claims this is a highly-charged issue with most programmers. Another section states how to use a local standard header for a project to improve portability. I have been the victim of this mistake. Someone once gave me a program that had 40 C files and each started with #include /usr/users/dave/all.h. When we moved the program to a new environment we had to replace each of these with #include all.h. This may sound trivial, but it can be very time-consuming. Plum advocates the use of code reviews to ensure correctness and portability. Plum describes a "first-order correctness review" that produces a list of test cases for the software. You use these for first-order correctness testing and later for regression testing. A final section to note covers the environment of Standard C. In this section Plum defines a "strictly portable" C program as producing "identical behavior in any Standard C environment." Strictly portable programs must be careful of word sizes and number representations. Plum lists several dozen rules that make your program strictly portable. C Programming Guidelines concludes with a description of all the library functions in Standard C. This will not replace a vendor's reference manual. It is, however, a good, concise description of the standard library. C++ Programming Guidelines C++ Programming Guidelines repeats several dozen sections from C Programming Guidelines. Nevertheless, C++ Programming Guidelines contains significant additions about C++ items. C++ Programming Guidelines dedicates considerable attention to migrating from C to C++, a very important topic given that most of us C programmers will be using C++ in five years if not sooner. We would all like the luxury of not having to produce anything of value for six months while we learn C++, but that will not happen. We must remain productive while migrating. The authors describe three levels in the migration: Typesafe C, Object-Based C++, and Full C++. Typesafe C is the common subset of Standard C and C++. You should use it when the target platform currently supports C and will support C++ by the time you field your system. Object-based C++ is a form of beginner's C++ using basic classes and other C++ extensions. It is not full C++ in that you do not use class derivation and virtual functions. The authors describe these levels amply and as a programmer and manager of programmers I endorse their slow but sure approach to this problem. C++ Programming Guidelines does not dodge the issue of portable C++ programs. The ANSI C standard makes portable C possible. C++ does not yet share such a standard, so portable C++ programs are more difficult to realize. The authors discuss how to work around this throughout the text and recommend using Ellis and Stroustrup's text as a de facto standard (see reference below). C++ Programming Guidelines has several sections that discuss code reuse and management of reusable code. One of the selling points of full C++, object-oriented programming is reusable code. You do not receive the full advantages of reusable code without following certain rules in coding and managing code. The authors do a commendable job of explaining this subject and sprinkling tips through the entire book. C++ Programming Guidelines has one short but vital page on the importance of design in object-oriented, C++ programming. In C++ you create base classes from which you derive many other classes. If you change a base class, then this change can ripple through many derived classes and you have a quandary. The solution is design. You must spend a much larger proportion of time and effort when designing your base classes. You need more reviews and more outsiders asking questions about your design. The result is shorter overall development time and higher quality software. The authors conclude C++ Programming Guidelines with two benchmark programs. These programs produce a table of execution times for basic operations in C and C++. They also provide you with a sense of execution times for basic operations so you can estimate program performance. Conclusion These are working books that spell out coding standards. A coding standard will help improve the quality and portability of your group's software by helping you to have consistent source code in all your projects. If your group does not have a coding standard, it needs one. If your group needs a coding standard, start with one of these books. You may not agree with everything in the books, but that's not necessary. Use the books as a foundation and cut, paste, add, and delete until you have a standard that will work for you. This will improve the quality of your software, and we all need to do that. Reference Ellis, Margaret A. and Bjarne Stroustrup. 1990. The Annotated C++ Reference Manual. Reading, MA: Addison-Wesley. Book Information C Programming Guidelines, 2nd Edition by Thomas Plum Price: $30.00 ISBN: 0-911537-07-4 C++ Programming Guideliness by Thomas Plum and Dan Saks Price: $34.95 ISBN: 0-911537-10-4 Publisher: Plum Hall Inc. To order: (913) 841-1631 (books), (808) 885-6663 (machine-readable text) Editor's Forum Just two issues back, I was telling you about all the exciting things going on in the international standards arena, at least as related to the C programming language. (See the Editor's Forum, CUJ November 1992.) The past few weeks have seen a different kind of excitement welling up in the ANSI arena. I fear the result won't be nearly as positive, however. Standards, as you probably know, are produced largely by the effort of volunteers. Many of us who developed the C standard put in half a year of meeting time over a period of six years, not to mention a comparable investment in work between meetings. Throw in the cost of travel and lodging for two dozen meetings and you're looking at a serious investment in both time and money. (Multiply that by 50 or 60 active participants and you begin to appreciate the staggering cost of developing a programming language standard.) For the privilege of doing all this grunt work, we pay an annual fee to CBEMA, the secretariat to the ANSI-authorized committee X3 that oversees our efforts. That fee has climbed over the years to a hefty $300 per year. Not outrageous for those of us whose companies pick up the tab, but noticeable. As an independent, I shelled out $650 last year for membership in X3J11 (C) and X3J16 (C++). The extra $50 encourages CBEMA to distribute documents for X3J11. But now CBEMA has begun issuing a flurry of additional bills, retroactive to last year. The typical charge is an additional $300 per year for each X3 committee we participate in -- ostensibly to cover the costs of our international activities. I've been asked to pay an additional $1,200 for a variety of reasons. I can look forward to an additional $1,850 in bills for 1993 before you even read these words. Needless to say, a number of us standards drones are more than a little upset. I can't judge whether these additional fees are reasonable, by some metric. I do know that they were decided with no public debate and they were made retroactive for most of a year. That is not a good way to treat volunteers, techie or otherwise. The short term distraction is trifling compared to the loss of talent this can cause in the longer term. And it makes us all leery about what might happen next. For my part, I am resigning a number of posts that I have long valued. I shall continue as Convenor of WG14 -- I feel a responsibility for the stewardship of C for at least the near future. I plan to keep up with the X3 activities in C and C++. But I don't expect to travel quite as much or work quite as hard in the standards arena from now on. I hope that not too many others feel pinched enough to do likewise. P.J. Plauger pjp@plauger.com New Products Industry-Related News & Announcements TauMetric Upgrades C++ Compilers for DEC/MIPS TauMetric Corporation announced upgrades to its Oregon C++ Development Systems for SPARC and for DECstations and DECsystems workstations. The Oregon C++ system provides direct code generation, C++ level 2.1 compatibility, and a C++ debugger. The Oregon C++ compiler includes modes for both ANSI C and K&R C. Oregon C++ generates object code directly and the system provides a revised version of Oregon Debugger (ODB) for C++. Major new features include: nested class and type scopes, operator delete [] syntax, and overloading postfix ++ and --. Oregon C++ is available for VAX/VMS, SPARC, MIPS (DECstation), HP 9000/300, and Sun-3 Systems. Contact TauMetric Corporation, 8765 Fletcher Parkway, Suite 301, La Mesa, CA 91942, (619) 697-7607, or (800) 874-8501; FAX: (619) 697-1140. TFS Releases C++ Class Library and Hypertext Help System The Friendly Solutions (TFS) Company has released C*Drive++, a C++ class library, and FASTVIEW, a hypertext help system and help text compiler. C*Drive++ contains low-level routines for windowing, pop-up and pull-down menu functions, form entry, and printer control. The package also includes classes for screen I/O, mouse/ keyboard support, string handling, color control, and directories. Source code is provided. The class library is compatible with Borland and Microsoft compilers. FASTVIEW allows developers to create TSR help text. The help engine works with C*Drive++ or as a stand alone system. Source for the FASTVIEW engine is included. C*Drive++ is priced at $140, and FASTVIEW (with royalty free distribution) is priced at $199. Contact The Friendly Solutions Company, 6309 Chimney Wood Ct., Alexandria, VA 22306, (703) 765-0654. Quarterdeck Provides X11 and Motif Software Development Toolkits for DESQview/X Quarterdeck Office Systems, Inc. has announced its DESQview/X X11 and Motif Software Development Toolkits. Also available at no charge with the toolkits is the GNU C/C++ compiler. With the X11 Toolkit, developers will be able to: port X clients from other X Window System environments to DESQview/X, create new X clients, perform network-independent communication between machines, access Adobe Type Manager to use Type 1 scalable fonts, and customize DESQview/X through utility programs and configuration files. The DESQview/X X11 Starter Toolkit, for use with the GNU C/C++ compiler only, includes the X11 R4 programming libraries, the DESQview/X system library with Berkeley Socket Interface, sample programs, make files for GNU, the DESQview/X Road-map documentation, and the GNU C/C++ compiler. The starter kit sells for $50. The complete DESQview/X X11 Toolkit, priced at $750, includes (in addition): make files and library support for Microsoft C, Borland C++, Zortech C++, Rational Instant C, Watcom C/386, and MetaWare High C compilers, O'Reilly X Reference and Programming Guides (Volumes 1, 2, 4, and 5), Rational Systems DOS/4GX DOS extender and tools, Instant C development environment, Oxygen, and Quarterdeck's Developer Passport Support. The toolkit components are available separately. Quarterdeck has made the Quarterdeck X libraries for DESQview/X available to GNU for free distribution with its C/C++ compiler. For information, contact Quarterdeck Office Systems, Inc., 150 Pico Boulevard, Santa Monica, CA 90405, (800) 354-3222 or (310) 392-9851, FAX: (310) 399-3802. Innovative Data Solutions Upgrades PARAGen Code Generator for PARADOX Innovative Data Solutions, Inc., has upgraded its PARAGen code generator for the PARADOX engine. PARAGen creates C, C++, standard Pascal, object-oriented Pascal, and Pascal for Windows (TPW) code for accessing PARADOX database applications. PARAGen creates OOP code with classes and member functions that manipulate those classes. PARAGen creates an internal record structure that represents the actual structure of the PARADOX table, enabling record and search operations to populate the structure and access values stored in the transfer buffer immediately. Optimization, error checking, and generated code style are configurable. Users can create and examine PARADOX tables from within PARAGen, allowing productive work without the neeed to load PARADOX on the machine. PARAGen's look and feel has been changed to conform to the CUA/SAA compliant interface standard. PARAGen retails for $179 and comes in a DOS specific version and a native Windows 3.x version. Contact Innovative Data Solutions, Inc., 1757 Eastwood Court, Suite 9, Schaumburg, IL 60195, (708) 882-3713. Non Standard Logics Implements XFaceMaker X/Motif GUI Builder for SCO Open Desktop Non Standard Logics SA (NSL) has implemented its XFaceMaker (XFM) interactive graphical interface builder for The Santa Cruz Operation's (SCO) Open Desktop. For information, contact the US office of NSL at 99 Bedford Street, Boston, MA 02111, (617) 482-6393; FAX: (617) 482-0357. NSL European headquarters offices are at 57-59 Rue Lhomond, 75005 Paris France; telephone (33-1) 43 36 77 50, FAX: (33-1) 43 36 59 78. Quarterdeck Adds DOS Protected Mode Interface (DPMI) Host to QEMM-386 Quarterdeck Office Systems, Inc., has announced their DPMI Host, which supports virtual memory and provides a companion product to QEMM-386, their PC memory-management software. Quarterdeck's DPMI Host is compatible with Microsoft C/C++, Borland C++, and Intel's Code Builder Kit. The Quarterdeck DPMI Host requires QEMM-386, DESQview-386, or DESQview/X. The DPMI Host supports running multiple DPMI programs inside Quarterdeck's DESQview products. Quarterdeck DPMI Host is a full implementation of versoin 0.9 of the DPMI specification, including DOS extensions. Quarterdeck will make the DPMI Host available to registered Quarterdeck users at no cost, through the company's user BBS: (310) 314-3227. Online documentation is available. QEMM users can also download the DPMI Host from various online services. Users who want disks and hard copy documentation can order DPMI Host directly for $30 (or free to Quarterdeck Passport Support Subscribers). For information, contact Quarterdeck Office Systems, Inc., 150 Pico Boulevard, Santa Monica, CA 90405, (800) 354-3222 or (310) 392-9851; FAX: (310) 399-3802. Algorithmic Solutions Releases IFF Indexed File Management Library for UNIX System V Algorithmic Solutions has released IFF,the Indexed File Management Library for C programming environments. IFF is a C library that supports indexed files, with up to 16 keys, both unique and non-unique keys, seven key data types, user-defined key data type support, segmented keys with up to 10 components, file and record locking, and automatic, internal lock management that enables both single and multiuser access. IFF uses a single file that incorporates both data and indexing structure, avoiding the need for special utilities for copying or moving files. IFF prices begin at $595, and SCO UNIX, SPARC and HP-UX are supported. Contact Algorithmic Solutions, P.O. Box 382, Simpsonville, MD 21150-0382, (301) 421-0134; FAX: (301) 942-1452. Software Through Pictures Ships on SCO Open Desktop Release 2.0 Interactive Development Environments, Inc. (IDE) has announced the shipment of Software through Pictures Integrated Structured Environment (ISE) Release 4.2D for SCO Open Desktop Release 2.0. IDE's Software through Picture is a multiuser, integrated CASE environment that supports development for technical and commercial applications. A single license for Software through Pictures ISE for SCO Open Desktop ranges from $5,000 to $21,000, depending on the configuration. Suggested hardware configuration includes a 33MHz 486 PC, 300MB hard drive, and 16MB RAM. For information, contact Interactive Development Environments, 595 Market Street, 10th Floor, San Francisco, CA 94105, (800) 888-IDE1, (415) 543-0900; FAX: (415)543-0145. Sequiter Software Announces Clipper to C Translator Sequiter Software has released CodeTranslator 2.0, which translates Clipper '87 and dBASE III PLUS applications into C, and has announced plans for a November release of CodeBase 5.0. CodeTranslator generates ANSI compliant C code, designed to correspond to the original dBASE/Clipper application, preserving specific variable and function names, as well as familiar commands like goto and skip. CodeBase 5.0 is a C/C++ library which has file and multiuser compatibility with dBASE, FoxPro, and Clipper. New CodeBase features include bit optimization technology (for faster queries) and CodeReporter, a Microsoft Windows report design tool. CodeTranslator retails for $245 and CodeBase retails for $395. Contact Sequiter Software, Inc., Suite 209, 9644 - 54 Avenue, Edmonton, Alberta, Canada, T6E 5V1, (403) 437-2410, FAX: (403) 436-2999. RTXC and PC/104 Unite for Real-time Embedded Applications A. T. Barrett & Associates has announced real-time kernel support for the PC/104 Consortium with its real-time executive, RTXC. RTXC is a real-time multitasking executive for use in embedded systems requiring a deterministic design with pre-emptive scheduling for event-driven operation. RTXC is written primarily in ANSI C and supports all the standard PC-compatible resources. RTXC provides a library of services including task, memory, and resource management, intertask communication and synchronization, and timer support. RTXC is available in three configurations, Basic Library (26 kernel services), Advanced Library (38 kernel services), or Extended Library (55 kernel services). Source code is supplied and there are no royalties for continued use in multiple products. For information, contact A.T. Barett & Associates, 11501 Chimney Rock, Suite R, Houston, TX 77035, (713) 728-9688. AccSys for dBASE Version 2 Provides Faster and More Flexible API Copia International, Ltd., has released version 2.0 of AccSys for dBASE. Their announcement stated that the new version runs five times faster than previous versions, for multiuser applications with index files. Version 2 adds high level functions that can read, write, update, pack, index, and re-index the database. The AccSys for dBASE Expression Analyzer allows programmers to read, write, and update "foreign databases". Version 2 adds dEparse, dErun, and dEeval, enabling AccSys for dBASE to parse, evaluate, and interpret dBASE key and index expression. Over 70 dBASE look-alike functions have also been added. AccSys for dBASE is priced at $395 or $995 with source. Contact Copia International, Ltd., 1342 Avalon Court, Wheaton, IL 60187, (708) 682-8898; FAX: (708) 665-9841. Symantec Announces MultiScope Debuggers for Borland and Microsoft C++ Applications Symantec Corporation has announced version 2.0 of its MultiScope Debuggers, which supports Borland C++ and Microsoft C/C++ (6.0 and 7.0) for programming Windows and DOS applications. The MultiScope Debuggers offer a Windows-hosted (3.1) graphical user interface. The CommandBar provides quick access to frequently used debugging commands. The DOS debuggers can be Windows-hosted and allow debugging of DOS applications in a DOS window. A Symantec representative described the MultiScope Debuggers as designed specifically for debugging C++ code and noted the inclusion of features such as class browsing, automatic object mapping, object-oriented breakpoints, and name unmangling. MultiScope includes both Run-Time Debugging for controlling program execution and a Crash Analyzer System for examining the aftermath of program crashes. MultiScope Debuggers for Windows is priced at $379. The DOS version runs $179. For information, contact Symantec Corporation, 10201 Torre Avenue, Cupertino, CA 95014-2132, (800) 999-8846 or (408) 253-9600; FAX: (408) 253-4092; Telex: 9103808778. EnQue Upgrades UltraWin Text Windowing Library EnQue Software has upgraded UltraWin 2.10, its text windowing library, whose features include unlimited overlapping windows, background printing, PC timer control, mouse and graphic support, and enhanced data entry capabilities, along with a hypertext help engine and an EGA/VGA font editor. EnQue has also released InTUItion 1.10, a textual user-interface library which includes a tool-supporting interactive interface design. EnCom 1.00, a COMM library is a new introduction from EnQue, with interrupt driven transmit and receive, support for COMM ports one through four, CRC and message formatting facilities. UltraWin, EnCom, and InTUItion, with small model libraries are available free from various online services, or by sending $2 for shipping and handling to EnQue. Complete source with large model libraries and printed manuals are available for EnCom ($100), UltraWin ($100), or both UltraWin and InTUItion ($200). Contact EnQue Software, Route 1, Box 116C, Pleasant Hill, MO 64080, Voice/FAX: (816) 987-2515; EnQue BBS: (816) 353-0991. Library Technologies Releases Version 3.0 of C-Heap Memory Management Library Library Technologies has released version 3.0 of its C-Heap memory management library for Microsoft and Borland C/C++ compilers. C-Heap now supports the allocation of memory from upper memory blocks (UMBs) through malloc, either via MS-DOS calls or the XMS driver transparently. C-Heap can use 64K of expanded (EMS) memory as heap space. Version 3.0 of C-Heap adds local heaps to its list of memory tools. Local heaps are similar to Microsoft-based heaps, but are not restricted to 64K. Any block of heap memory of any size can be used as a local heap and allocated from as required. If the block is associated with a particular object, the block may be swapped out of memory to expanded, extended, or virtual memory as required. C-Heap is priced at $229, including source. Contact Library Technologies, P.O. Box 56031, Madison, WI 53705-9331, (800) 267-6171. SilverWare Inc. Adds COM Port Control to dBASE IV SilverWare, Inc., has released SilverComm 3.00 (SPCS), a communications library (written in C and assembly) that provides xBASE programmers with access to COM port control. xBASE programs can auto detect and invoke the 16550 FIFO UART, increasing data throughput and reliability at baud rates up to 115K. SilverComm 3.0 includes support for the IBM PS/2 Dual Async Multi COM Adapter and shared interrupt (IRQ) handling on Micro Channel adapters. Other new features include support for high number interrupts on multi COM boards, YMODEM Batch file transfer protocol. SilverComm also supports XMODEM, YMODEM and ASCII transfers, ANSI and TTY terminal emulations, and AST or STB multi COM boards. SilverComm 3.0 is priced at $249 (upgrages $89) and includes complete C and assembly source code. Contact SilverWare, Inc., 3010 LBJ Freeway, Suite 740, Dallas, TX 75234, (214) 247-0131, FAX: (214) 406-9999. Pryor and Pryor Releases Communications Libraries Pryor and Pryor, Inc., has released three communications products for C and assembly programmers. The SPI library is a communications library module for serial ports which supports sharing IRQs. SPI and a mouse can coexist on either IRQ3 or IRQ4 at rates up to 4,800 baud. The FCL library is a file-compression module that can be used in dynamically compressing file-transfer programs as well as simple disk file-compression programs. The SDA serial Data Analyzer program converts a PC into a tool for monitoring and displaying data flows over an RS232 link. The user may select display modes, capture and store data in real-time, playback captured data, and print information. Prices range from $49 (SPI, FCL) to $229 (SDA). For information, contact Pryor and Pryor, Inc., 602 -- 1230 Comox Street, Vancouver, B.C., Canada, Voice/FAX: (604) 669-2609. MetaCard Upgrades UNIX Development Environment MetaCard Corporation has upgraded their hypermedia and rapid application development environment, MetaCard 1.2. The release is Motif-compliant and will be supported on nine UNIX/X11 platforms, including new support for Silicon Graphics IRIS, Data General AViiON, and IBM RS/6000 workstations. A C-based extension allows MetaCard to be used for computationally-intensive or hardware-dependent applications. C functions can be called directly from MetaCard; no relinking or other processing is required. A save-disabled product is available free via Internet. MetaCard is self-contained, no C compiler is required. Single-user licenses for MetaCard cost $495, with site and volume licensing available. Contact MetaCard Corporation, 4710 Shoup Place, Boulder, CO 80303, (303) 447-3936. Adaptive Solutions Ships Parallel Processor CNAPS Computer Systems Adaptive Solutions, Inc., has begun shipping its parallel-processing CNAPS computer systems, designed for pattern recognition and signal-processing applications. CNAPS systems can be configured with 128, 256, or 512 processors. The CNAPS system connects to a UNIX host via Ethernet. The CodeNet development tools provide Motif interfaces and include the CNAPS-C compiler, with extensions to support massive parallelism and scaled fixedpoint arithmetic of the CNAPS architecture. Other tools include the CNAPS Programming Language (CPL) assembler and debugger, command-line and windowing interfaces, and data conversion utilities. Contact Adaptive Solutions, Inc., 1400 NW Compton Drive, Suite 340, Beaverton, Oregon 97006, (503) 690-1236; FAX: (503) 690-1249. SMDS Updates Aide-De-Camp and Allies with Centerline Software Maintenance and Development Systems, Inc. (SMDS), has released version 8.0 of the Aide-De-Camp (ADC) software-configuration management system. Version 8.0 includes the Lakota scripting language. ADC is object-based and can track software changes at a user-defined level, including file, subroutine, procedure, or components of individual changes. SMDS also announced a joint marketing and product integration alliance with Centerline Software, Inc. (formerly Saber Software). The integration effort will link ADC and Centerline's CodeCenter and ObjectCenter UNIX programming environments, to provide support for developing and managing large C/C++ software inventories. Contact Software Maintenance & Development Systems, Inc., P.O. Box 555, Concord, MA 01742, (508) 369-7398; FAX: (508) 369-8272. Promula Development Corporation Announced Fortran for COHERENT Promula Development Corporation and Mark Williams Company have announced Promula's PROMULA FORTRAN compiler for COHERENT. PROMULA FORTRAN supports FORTRAN 66, FORTRAN 77, VAX FORTRAN, PRIME FORTRAN, SUN FORTRAN, and some FORTRAN 90 extensions, and passes the Federal Software Testing Center's FORTRAN Compiler Validation suite, version 2. PROMULA FORTRAN provides natural integration with C-based libraries on COHERENT and is compatible with the PROMULA FORTRAN to C Translator, which generates C source. PROMULA FORTRAN for COHERENT is priced at $95. Contact Promula Development Corporation, 3620 North High Street, Suite 301, Columbus, OH 43214, (614) 263-5454; FAX: (614) 263-5573. Knowledge Dynamics Corp's INSTALL Improves Data Compression in Version 3.2 Knowledge Dynamics Corportion has updated INSTALL, with a new compression algorithm, Reduce, that compresses data to 60% for most products. Version 3.2 has an enhanced script language with fully general assignment statements, 12 new string-handling functions, on-the-fly modificaiton of environment variables, and the ability to cold or warm boot the end-user's computer. An option for INSTALL 3.2 is the HyperText Help System, a TSR which provides access to the INSTALL manual, requires 2K RAM with EMS/XMS, and is Norton Guides-compatible. Contact Knowledge Dynamics Corporation, P.O. Box 1558, Canyon Lake, TX 78130-1558, (512) 964-3994. Microtec Research Introduces Software Development Environment for Embedded Systems Microtec Research, Inc., has introduced XRAY MasterWorks, a C/C++ integrated software-development environment specifically designed for embedded-systems engineers. XRAY MasterWorks is a package of program building, code generation, debugging, and analysis tools that provides push-button control over a variety of development tasks. Tight integration provides features such as rebuilding a program and reloading it into the debugger with a single command. XRAY MasterWorks contains XRAY Master, XRAY Debugger, and the C/C++ cross-compiler package. XRAY Master includes the following components: an environment manager, XRAY Control Panel (configuration control), XRAY Source Explorer, XRAY Make, and Object Format Converter (converts code to formats usable with logic analyzers and PROM programmers). XRAY Debugger supports multiple windows and can debug code in various execution environments, including instruction-set simulation, in-circuit emulation, and in-circuit monitor. The ANSI C and C++ Cross Compiler Package generates ROMable, reentrant code, and includes compiler, macro assembler, liner, and librarian. XRAY MasterWorks is hosted on SPARCstations and initially supports embedded systems based on Motorola 68x0 and 683xx processors. For information, contact Microtec Research Inc., 2350 Mission College Boulevard, Santa Clara, CA 95054, (800) 950-5554; FAX: (408) 982-8266. Borland Ships BRIEF 3.1 for DOS and OS/2 Borland International, Inc., has begun shipping BRIEF 3.1 for DOS and OS/2. Formerly sold by Solution Systems as separate products Borland will bundle the two versions of the programmer's editor in a single package. New features in BRIEF 3.1 include mouse support, EMS, Redo, and Dialog Box support. Brief has a C-like macro language; statement completion; integration with compilers; language support including Borland C++, Turbo Pascal, dBASE and Assembly; and multiple, resizable windows. Contact Borland International, Inc., 1800 Green Hills Road, P.O. Box 660001, Scotts Valley, CA 95067-0001, (408) 439-4825. Softbridge Announces Automated Test Facility 2.0 Softbridge, Inc., has announced Automated Test Facility (ATF) 2.0. ATF is a software system designed for unattended testing of applications written under Windows, OS/2, and DOS, either standalone or in networked environments. ATF 2.0 includes interactive menu-based test script building, online hypertext help, revamped user interface, and enhanced test management. ATF 2.0 provides a more open architecture, with Dynamic Link Library (DLL) functions. The SQL Server database required in earlier versions is now an optional feature. Contact Softbridge, Inc., 125 Cambridge Park Drive, Cambridge, MA 02140, (617) 576-2257; FAX: (617) 864-7747. CHAMPS Software Announces CHAMPS/CASE CHAMPS Software, Inc, has announced CHAMPS/CASE, a tool providing a framework for Rapid Application Development. CHAMPS/CASE is designed as a VAX Rdb/VMS layered product and should be available first quarter 1993. CHAMPS/CASE was designed to expedite the transition from FMS, RMS, and Cobol-based development to IFDL, Rdb, and ANSI C. CHAMPS/CASE generates 100% of the C source code for applications and can create ANSI C, ANSI SQL, and IFDL programs. CHAMPS is NAS, ANSI/SQL, FIMS, and POSIX compliant and supports DEC's COHESION environment. Contact CHAMPS Software, Inc. 1255 North Vantage Point, Crystal River, FL 34429, (904) 795-2362; FAX: (904) 795-9100. We Have Mail Dear Mr. Plauger: I have been reading the C Users Journal since it was little more than a newsletter for the users group. The one thing I have noticed in C programmers is the use of the functions "FOO" and "FOOBAR" within a test module. Where are the origins of such naming conventions, and to what are they attributed? Although, not a technical question, it has aroused my curiosity over the years. Sincerely, Mr. Philip Felice U.S. Trust Company of New York 770 Broadway New York, NY 10003 The terms foo, bar, and foobar all derive from the old U.S. Navy slang FUBAR, their answer to the Army's SNAFU. FUBAR leaked into the programming community through MIT. The (somewhat laundered) Navy term is an acronym for "fouled up beyond all repair," while the Army term is short for "situation normal, all fouled up." -- pjp Mr Plauger, Thank you for doing such a fine job over the years. My first subscription issue came Saturday. Exciting. Exciting--an article on neural nets--then on page 85 the article falls off the face of the earth????? How beastly clever--they have implemented a '50s movie cliff hanger? (OK. OK. Just kidding.) I'm sure you have really heard about this problem by now. I hope that the end of the article will be in next month's issue. (Well OK, deadlines being what they are--the month after?) This is one issue that I am rushing a check for the source code disk. I really appreciate the cost of the disk to be affordable! Thanks & Cheers, deryl I can't claim credit for that particular gaffe. Some days, there's enough embarrassment to go around. -- pjp Diane Thomas, Managing Editor replies: Profuse apologies for our FUBAR. You can find the correction to that article on page 102 of the October 1992 issue. If you don't care to purchase the October issue, please call or write and I will fax or mail the correction to you. Editor: I am constantly impressed with the articles and the quality of The C Users Journal and I want to thank you for all the work that you do. It always disappoints me to read some flame from a reader who is mostly demonstrating his lack of tact or inability to express an opinion without insulting someone. I hope you take all flames with a grain of salt and recognize that most people that are satisfied or happy don't write. I, personally, rarely get the time. It's also great to see that the Gods and Gurus make mistakes, too. I noticed a goof that you'll probably get a hundred letters on and is one of the ones I've read a number of other gurus do commonly. In your "Bugs" article, your example of violation of coding standards showed the "improved version" as: char fn[L_tmpnam], *s; if ((str = fopen ((const char *) tmpnam(fn), "wb+")) == NULL) ; else if ((s = (char *) malloc(sizeof (fn) + 1)) == NULL) fclose (str); str = NULL; else str->_Tmpnam = strcpy(s, fn); The erroneous line is the one with the fclose on it. You will notice that there are two statements between the else if and the else with no curly braces. Tsk, tsk, tsk. :-) I've gotten into the habit of always using curly braces just because I made this mistake all the time. Also, I frequently want to put a print statement into the if statement somewhere and then have to put curly braces around it anyway. Thanks again for all the very interesting articles and wealth of information. Chris Carlson carlson@sgi.com Howdy; I just got the latest issue of The C Users Journal, and was zipping through your "Standard C" column when I noticed that one of your "fixes" contains yet another bug. The offending code appears on page 12. [same code as in previous letter follows -- pjp] I doubt this will even compile as written. I'm pretty sure you intended to write: fclose(str), str = NULL; Don't you just hate when that happens :-) Timothy G. Northrup ...!rutgers!brspyr1!jrsalb1!tim Yup. Luckily, the error occurred in transcription for publication in the magazine (my fault). The actual code does compile and run its test cases. -- pjp Dear Mr. Plauger, I enjoyed reading your article "Bugs" in the September '92 issue of The C Users Journal. Your book, The Standard C Library is quite informative. I did notice one example which, as written, has undefined behavior according to The Standard as I understand it. On page 209 your va_fputs function returns without calling va_end on a write error. In your implementation of the library it doesn't make any difference, but as the text points out that may not always be the case. While I'm on the subject, I don't recall ever reading a discussion of an implementation of the Standard C library suitable for use by multi- threaded programs. I program on the Amiga where spinning off multiple threads is quite easy to do, and while the system shared libraries are written to expect to be called from various tasks at any time, the C linker libraries for the compilers I've looked at expect to have only one caller at a time. Some functions work just fine anyway (strlen for example) while others like strtok (which make use of an internal static variable in the implementations I've seen) are clearly not going to like having multiple asynchronous callers. So if you're looking for an article idea you may want to consider "Library Use by Multiple Concurrent Threads." Otherwise, it's good to have The Editor reachable via e-mail. Now if we only had a tutorial on the correct spelling of "Plauger." (Wow! Your own domain! Is that anything like the fat jokes, "He's as big as New Jersey" or "He has his own zip code?") Todd M. Lewis utoddl@guitar.oit.unc.edu That's a bug all right. It's already fixed in the second printing, but thanks for reporting it. My brother, Dave Plauger, has just finished making a thread-safe C++ library for Hewlett Packard. I'm trying to coax him into writing an article on what he did. -- pjp Dear Mr. Plauger, This is an answer to Mr. Grabbe's letter in the July issue of CUJ. The point editor for the PC is available for free using anonymous ftp. Address: unmvax.cs.unm.edu. login: anonymous password: id It is located in /pub/Point/ptpc.tar.Z. This is a compressed archive file that will be extracted on (hopefully) any UNIX system using zcat ptpc.tar.Z tar xvf - Note: there is also an X-Windows version of point at the same location (best UNIX editor I have ever seen besides emacs/epoch). As far as I know point has been posted to comp.sources.*. Right, Mr. Weinstein? By the way, where is your column? Continue tracking comp.sources.x! PS: The last issues of CUJ have become somewhat "less academic" maybe due to the discussion some months ago. To me this is a serious drawback for the quality of your magazine. Yours truly, Gerhard Wilhelms Lehrstuhl fuer Informatik I Universitaet Augsburg Universitaetsstr. 2 W-8900 Augsburg GERMANY Which is a serious drawback, that the magazine is less or more academic? You'd think we could get 50,000 readers to agree on something. Seriously, all we can do is keep striving for a good balance in selecting from submissions. We appreciate your continuing input to help us tune that balance. -- pjp Editor: In the article titled "Bit Arrays with C++" in the July 1992 issue of CUJ, authors A. Giancola and L. Baker chose an approach that seemed very unorthodox to me. If possible I would appreciate a clarification of their design rationale. The code given in the article cannot, according to the authors themselves, be compiled with a licensed derivative of the AT&T C++ compiler. A second compiler, Borland's C++ v3.0, does compile the code but issues a warning message. The compilation problems stem directly from the use of a nested class inside a template class, a design that Giancola and Baker chose in order to "prevent inadvertent misuse of objects." Why do they consider this possibility to be of such importance? Of what value is "safe" code if it cannot be compiled, or compiles with warnings? The authors' stated goal was to create a bit array that could be addressed using arithmetic assignment statements of the form A(i,j) = x;. This was quite a difficult problem and required approximately a hundred lines of clever code to solve. An alternate approach would have been to access array elements through an assignment function. this would have required perhaps ten or twenty lines of straightforward code. There is no question that it also would have the advantage of faster execution. To call an access function, one must write a statement of the form A.set(i,j,x); instead of A(i,j) = x;. Is the second form so much cleaner that it is worth the coding effort and the sacrifice in execution speed? In the first sentence of the article, Giancola and Baker explain that they developed this code while working in a "Go" program. Their task was to represent the Go board, a 19x19 grid in which each location can be either white, black, or empty. This could have been done by using an array of char, e.g., char board[19][19], requiring 361 bytes of storage. No special code would be needed since access to 2-dimensional char arrays is built into the C/C++ language. The implementation chosen by the authors requires only 113 bytes of storage, but at the expense of added complexity. Unless the program must store a large number of Go positions simultaneously, the modest savings in storage space achieved by the authors hardly seems worth the effort. Furthermore, storing and retrieving values would be relatively efficient for a 19x19 array of characters; packing the information into 113 bytes makes accessing of data about an order of magnitude more complex (and thus slower). In these days where PCs are typically equipped with megabytes of memory, the authors' apparently extreme desire to conserve storage space is difficult to understand. If storage space was indeed at a premium for some reason, it is odd that Giancola and Baker chose an implementation in which all of the bit vector code is in the form of inline functions. The inline expansion of every bit vector function call will consume code space; why not use non-inline functions to reduce the program's code size? At a detailed level the code is not as efficent or direct as it could be. This fragment: int i; tmp_mask = 1; for (i=1;i #include #include /* Maximum argument significant digits. */ #define MAX_SIG_DIGITS 9 #define ARRAY_SIZE 2 #define I ( i ) % ARRAY_SIZE #define I MINUS_1 ( i - 1 ) % ARRAY_SIZE #define I_MINUS_2 ( i ) % ARRAY_SIZE typedef unsigned long Ulong; typedef long double Ldouble; int main( int argc, char **argv ) { int i, dec_digits, sign; Ulong n0, n, d0, d, temp; Ulong p[ARRAY_SIZE] = { 0, 1 }; Ulong q[ARRAY_SIZE] = { 1, 0 }; double x; Ldouble percent_err; /* Check for one command-line argument. */ if ( argc != 2 ) { fprintf(stderr, "Usage: %s number\n", argv[0); exit( EXIT_FAILURE ); } else /* argc == 2 */ x = strtod( argv[1], (char **) NULL ); /* Handle zero and negative arguments. */ if ( x < 0.0 ) { sign = -1; x = -x; } else if ( x == 0.0 ) { puts( "\n0:\n" ); puts( " 0 / 1 Exactly!" ); exit( EXIT_SUCCESS ); } else /* x > 0.0 */ sign = 1; /* Check for out-of-range arguments. */ if ( x >= pow( 10.0, (double) MAX_SIG_DIGITS ) ) { fprintf( stderr, "%s: Magnitude is", argv[1] ); fprintf( stderr, " too large.\n" ); exit( EXIT_FAILURE ); } else if ( x <= pow( 10.0, (double) -MAX_SIG_DIGITS) / 2.0 ) { fprintf( stderr, "%s: Magnitude is", argv[1] ); fprintf( stderr, "too small.\n" ); exit( EXIT_FAILURE ); } /* Determine the argument's radix-10 ratio. */ d0 = (Ulong) pow( 10.0, (double) MAX_SIG_DIGITS - ((x < 1.0) ? 0.0 : floor( 1.0 + log10( x )))); n0 = (Ulong) ( ( x * (double) d0 ) + 0.5 ); printf( "\n%.*g:\n\n", MAX_SIG_DIGITS, (double) n0 / (double) d0 ); /* Iteratively determine integer ratios. */ for ( i = 2, d = d0, n = n0 ; ; i++, temp = d, d = n % d, n = temp ) { p[I] = n / d * p[I_MINUS_1] + p[I_MINUS_2]; q[I] = n / d * q[I_MINUS_1] + q[I_MINUS_2]; /* Print ratios with non-zero numerators. */ if ( p[I] != 0 ) { printf("%11ld / %-10lu", sign * p[I], q[I]); if ( n % d == 0 ) { printf(" Exactly!\n" ); break; } /* * Compute the ratio's percent error * (taking care to avoid significance * loss when subtracting nearly equal * values): * * percent error = * * * 100 * ( p[i] * d0 - q[i] * n0 ) * ----------------------------------- * q[I] * n0 * * Display in %f format with at least two * significant digits. */ percent_err = (Ldouble)p[I] * (Ldouble)d0; percent_err -= (Ldouble)q[I] * (Ldouble)n0; percent_err *= 100.0L; percent_err /= (Ldouble)q[I] * (Ldouble)n0; dec_digits = ( fabs( (double) percent_err ) >= 10.0 ) ? 1 : 1 + ( int ) ( fabs( floor( log10( fabs((double) percent_err ))))); printf( "%+*.*Lf%%\n", dec_digits + 6, dec_digits, percent_err ); } } exit( EXIT_SUCCESS ); } /* End of File */ Sorting Networks Frederick Hegeman Frederick Hegeman is an amateur programmer and computer language hobyist. He can be reached at P.O. Box 2368, Rapid City, SD 57709, telephone (605) 343-7014. A sorting network sorts n items by performing a predetermined set of comparisons. A software implementation for a sequential computer might take the form swap(1,2); swap(4,5); swap(3,5); swap(3,4); swap(1,4); swap(1,3); swap(2,5); swap(2,4); swap(2,3); where swap(a,b) = if(b < a) exchange a and b This is a nine-comparison network for sorting five items. You might try it with pennies, nickels, dimes, quarters, and cheeseburgers. The best-, worst-, and typical-case number of comparisons are the same. Only the number of exchanges varies. Nine comparisons are, in fact, the fewest necessary to sort five items by exchanges when any combination of items is possible. When n is known to be small, these simple predetermined sequences out-perform the best algorithmic sorts. In theory, the minimum number of comparisons necessary to sort n items is the ceiling of log2(n!). As n grows larger, 1og2(n!) approaches nlog2(n). The reason why O(nlog2(n)) sorting algorithms, such as Quicksort, are so efficient as n grows larger should be obvious. Sorting networks generated by the Bose-Nelson algorithm (See Listing 1.) are O(n1.585), which diverges from nlog2(n) rapidly. However, the Bose-Nelson network for 16 elements is 65 comparisons, which is pretty nearly nlog2(n), and sorting 1000 random 16-element arrays using a Quicksort that pivoted on the last element in the array requires 85.43 comparisons on average, which is just over n1.585. This is probably the reverse of what you might have expected. Sorting is so often discussed in terms of increasing n that it is easy to fall into the trap of expecting "efficient" algorithms to behave nicely over the whole range of their inputs. The "gotcha" in Quicksort is the extra comparisons needed to properly partition the array. They are insignificant when sorting a large array but constitute a significant part of the total when the array is small. As n becomes very small, the behavior of sorting networks comes closer to the ideal. Bose-Nelson generates minimum comparison networks only for n œ 8. However, minimum comparison networks are available for 9œ nœ16 as well. Those generated by the code in Listing 2 are based on illustrations in The Art of Computer Programming, Vol. 3 (Knuth 1973). Please note that both listings generate sorts according to element numbers--one greater than the corresponding array index. Sorting small arrays is a problem all its own, difficult to understand in the usual terms. If you need to sort a small array in a time-critical section of code, you can count on only two things holding true: your algorithmic sort may perform much worse than expected, and no algorithmic sort can match the minimum comparison networks on all combinations of elements. When that code is part of something like an operating system, the networks have at least two other attractive features: many comparisons can be done in parallel, if appropriate; and they are inherently re-entrant, so that sorting may be interleaved with other tasks. Bibliography Knuth, Donald E. 1973. The Art of Computer Programming, Vol. 3. Reading, MA: Addison-Wesley. Pp. 220-229. Bose, R. C. and Nelson, R. J. 1962. "A Sorting Problem". JACM, Vol. 9. Pp. 282-296. Davis, Wilbon. September, 1992. "Time Complexity". The C Users Journal, Vol. 10, No. 9. Pp. 29-38. Listing 1 Bose-Nelson algorithm for generating sorting networks /* Calling bose(n) generates a network * to sort n items. See R. C. Bose & R. J. Nelson, * "A Sorting Problem", JACM Vol. 9, Pp. 282-296. */ bose(n) int n; { Pstar(1, n); /* sort the sequence {X1,...,Xn} */ } P(i, j) int i, j; { printf("swap(%d, %d);\n", i, j); } Pstar(i, m) int i; /* value of first element in sequence */ int m; /* length of sequence */ { int a; if(m > 1) { /* Partition into 2 shorter sequences, * generate a sorting method for each, * and merge the two sub-networks. */ a = m/2; Pstar(i, a); Pstar((i + a), (m - a)); Pbracket(i, a, (i + a), (m - a)); } } Pbracket(i, x, j, y) int i; /* value of first element in sequence 1 */ int x; /* length of sequence 1 */ int j; /* value of first element in sequence 2 */ int y; /* length of sequence 2 */ { int a, b; if(x == 1 && y == 1) P(i, j); /* 1 comparison sorts 2 items */ else if(x == 1 && y == 2) { /* 2 comparisons inserts an item into an * already sorted sequence of length 2. */ P(i, (j + 1)); P(i, j); } else if(x == 2 && y == 1) { /* As above, but inserting j */ P(i, j); P((i + 1), j); } else { /* Recurse on shorter sequences, attempting * to make the length of one subsequence odd * and the length of the other even. If we * can do this, we eventually merge the two. */ a = x/2; b = (x & 1) ? (y/2) : ((y + 1)/2); Pbracket(i, a, j, b); Pbracket((i + a), (x - a), (j + b), (y - b)); Pbracket((i + a), (x - a), j, b); } } /* End of File */ Listing 2 Minimum comparison network pairs /* 10 items - 29 comparisons; 9 - 25 */ int net10[29][2] = { {2,9},{1,5},{6,10},{3,7},{4,8},{1,4},{7,10},{3,6}, {1,2},{4,7},{9,10},{5,8},{1,3},{5,9),{2,6),{8,10}, {2,3},{4,5},{6,7},{8,9),{2,4},{7,9},{3,5},{6,8}, {3,4},{7,8},{4,6},{5,7},{5,6}}; /* 12 items - 39 comparisons; 11 - 35 */ int net12[39][2] = { {1,2},{3,4},{5,6},{7,8},{9,10},{11,12},{2,4},{6,8}, {10,12},{1,3},{5,7},{9,11},{2,3},{6,7},{10,11}, {2,6},{7,11},{6,10},{3,7},{2,6},{7,11},{1,5},{8,12}, {4,8},{5,9},{1,5},{8,12},{2,5},{8,11},{4,9},{3,4}, {9,10},{3,5},{8,10},{4,6},{7,9},{4,5},{6,7},{8,9}}; /* 16 items - 60 comparisons; 15 - 56; 14 - 51; 13 - 46 */ int net16[60][2] = { {1,2},{3,4},{5,6},{7,8},{9,10},{11,12},{13,14}, {15,16},{1,3},{5,7},{9,11},{13,15},{2,4},{6,8}, {10,12},{14,16},{1,5},{9,13},{2,6},{10,14},{3,7}, {11,15},{4,8},{12,16},{1,9},{2,10},{3,11},{4,12}, {5,13},{6,14},{7,15},{8,16},{6,11},{7,10},{4,13}, {8,12},{14,15},{2,3},{5,9},{2,5},{8,14},{3,9}, {12,15},{3,5},{6,7},{10,11},{12,14},{4,9},{8,13}, {7,9},{4,6},{8,10},{11,13},{4,5},{6,7},{8,9},{10,11}, {12,13},{7,8},{9,10}}; /* Extracts a network for n items from an array of * comparison pairs for m items when n <= m. Expects * the 2nd member of each pair to be the larger. For * example, to extract a minimum comparison network * for 9 items call * extract(9, sizeof(net10)/(2*sizeof(int)), net10); */ extract(n, m, network) int n, m; int network[][2]; { int i; for(i = 0; i < m; i += 1) { if(network[i][1] <= n) printf("swap(%d, %d);\n", network[i][0] , network[i][1]); } } /* End of File */ An Efficient Method for Optimizing Binary Trees David W. Schwartz David W. Schwartz is an undergraduate student in Computer Science and Economics at Oklahoma State University. He works in Systems Design for Dimensional Concepts Inc., a company specializing in hospital Medical Records software. He can be reached by mail at Oklahoma State University, c/o Dr. M. Samadzadeh, 219 Math Sciences, Stillwater, OK 74078. A binary tree is a data structure used to store dynamic, ordered data. The distribution of data within a binary tree greatly influences the efficiency of operations on the tree: the more unbalanced the tree, the less efficient the data access. Through normal use (i.e., additions and/or deletions), a binary tree will generally become unbalanced. Therefore, it is desirable to optimize the binary tree periodically to maximize execution time efficiency. Many techniques have been devised to prevent trees from becoming unbalanced, including AVL trees, red-black trees, and 2-3 trees. These methods generally complicate the simple insert, delete, and access routines available with standard binary trees. They also generally incorporate the need for additional memory in each node. It is possible, however, to optimize a binary tree after construction using an algorithm as short and simple as the ones used to create and access it. First, the tree is formed into a linked list through a standard in-order traversal. The tree can then be reformed into its optimally-balanced configuration with a simple recursive function. This solution allows all access routines to be concise and simple, uses no extra data space, and saves time by not balancing a tree when speed is needed most, i.e., when the tree is in use. Origin In his article "Optimizing Binary Trees," (Terry 1991) The author suggests rebalancing a tree only periodically, by first flattening the tree into a linked list (the least speed-efficient case), then "folding" the tree in half, forming two subtrees, which can then also be folded recursively. This process yields the optimum arrangement for the tree. In Terry's algorithm, the only exception to the folding process appears when a subtree contains exactly five nodes. Under the normal process of folding, the subtree in Figure 1 would result. Note that if the values in the five nodes are close together, it becomes impossible to add new nodes except at the extreme edges of the tree, thus extending the tree to three levels. If the tree is arranged as in Figure 2, any value less than 3 can be added without extending the tree. To avoid such sparsely filled trees, Terry always positions the short side of the subtree toward the outside of the parent tree. His routines are also generic, i.e., they work on any standard binary tree, without knowing what data it contains. His routines require only that each node have the pointers to its branches at a predictable location in the node. The easiest way to meet this requirement is to place the pointers at the beginning of the node; any tree whose node structure begins as follows can be optimized: struct treenode { struct treenode *left; struct treenode *right; ... }; A Better Way Although Terry's routine yields the optimal node arrangement for any given tree, it includes many counterproductive movements. If a tree contains N nodes, each time the routine folds the tree, it must alter 0(N) of the nodes. The tree must be folded approximately log2(N) times, yielding a total of 0(N * log2(N)) alterations to complete the process. It is possible to achieve the same result with code that is smaller, faster, and makes only 2N or 0(N) alterations in the tree. (For an empirical timing comparison of the two methods, see the sidebar entitled "Timing Comparison".) First, the tree must be formed into a linked list through a standard in-order traversal. Then the tree is reformed into its optimally-balanced configuration with a simple recursive routine (Listing 1). The tree passed to this routine is assumed to have two pointers as the first elements in the node. Data in the node is inconsequential to this discussion. The tree is also assumed to use NULL pointers as terminators on the leaf nodes. Thus, a program can use these routines by simply passing a void pointer to optree and replacing the old root with the return value. A program need only execute the following line of code to optimize any binary tree. root = (struct treenode *) optree((void *)root); The Code Of the code necessary for the optimization (see Listing 1), the only routine available to other source files is optree (from "optimize tree"). It serves as the entry point for the module and takes only a single parameter, a pointer to the root of the tree to be optimized. The optree function initializes the pointers for the linked list and then calls the routine to form the tree into a linked list, list_build. The list_build function is a modified in-order traversal of the tree. It traverses the tree attaching each node to the end of a list and counting how many nodes are placed in the list. This count later determines how many nodes are in the tree and how the tree should be subdivided optimally. When list_build completes, optree begins the optimization process by resetting the left variable, to indicate that the tree currently leans to the right, and then calling the recursive optimization routine, form_tree. The form_tree function works much like a recursive node-counting routine turned upside down. A node-counting routine would call itself to find the number of nodes in the left subtree, add one for the current node, and call itself to find the number of nodes in the right subtree. Conversely, form_tree receives as a parameter the number of nodes that should be included in the newly-constructed tree. To construct the appropriate tree, form_tree determines how many nodes belong in the left subtree, calls itself to remove that number of nodes from the linked list, removes a node from the list for itself, and then calls itself again to remove the rest of the nodes (the right subtree) from the list. The recursion terminates when form_tree is asked to construct a tree with fewer than one node (namely, a NULL tree). The only complexity in the algorithm involves calculating how many nodes belong in each subtree. If the number of nodes to be divided between the two subtrees is odd, the extra node should be put in the subtree that is closer to the center of the current node's parent. If the current node is a left child of node X, the extra node should go into the right subtree of the current node. This arrangement forces the shortest paths to the outside of the tree, where data is most likely to be added. The special five-node case (in which the subtree is unevenly divided, as in Figure 2) is also considered in calculating the number of nodes that belong in each subtree. Note that when folding a five-node tree, the desired effect can be achieved by simply moving the fold one node too far from the center of the parent tree. That is, instead of dividing the nodes so that there is one more node in one subtree than in the other, simply divide it so that there are two more nodes in one subtree than in the other. Thus handling the five-node case requires only the following line of code. if(num == 5) middle++; External Variables There are five global variables in Listing 1. The decision to make these variables global was not reached arbitrarily. The globals are all static and thus protected from other modules. The list_base variable is global so that one node with a right pointer can be placed on the list before recursion begins. This eliminates the need to check for a NULL header each time the function is entered, thus increasing efficiency and decreasing code size. The other four globals are used by multiple levels of recursion, although only one copy of each is necessary. If these variables had not been global, some routines would have required several extra parameters. Excess parameters consume scarce stack space and add expensive pointer references. Consequently, the drawbacks usually associated with the use of global variables are overshadowed by the compactness, efficiency, and clarity they provide here. Sorted Data One disadvantage to binary trees is that when adding sorted data they degenerate into linked lists and exhibit worst case behavior. The proposed optimization algorithm addresses this limitation in two ways. First, the optimization algorithm can conveniently "fix" any binary tree that has suffered the effects of sorted data previously added. Second, with only a small modification the algorithm can add sorted data to a tree as quickly and conveniently as the data's sorted nature warrants! Since a large volume of sorted data would normally only be added in batch processing, it could be done easily as part of the optimization process. During optimization, the tree is initially formed into a linked list. The sorted data is already in some form of a list (or it cannot be considered sorted). The two lists merge easily and quickly into a single list. This list becomes a balanced tree when given to the optimization function. Thus, adding a large volume of sorted data results in optimal performance, not worst-case performance. Possible Problems Perhaps the most obvious problem with the routines presented here lies in the tendency of list_build to cause a stack overflow when flattening a large, badly degenerated tree. A non-recursive in-order traversal algorithm, which can be found in many computer science textbooks, will solve this problem. Listing 1 uses a recursive approach in the interest of simplicity and brevity. The form_tree function should not need to be rewritten to avoid recursion. Nearly all implementations of C allow for enough levels of recursion to optimize millions of nodes. A well-optimized binary tree of height 32 can contain over four billion nodes, many times the number of nodes normally expected in a binary tree. Conclusion Any implementation of a binary tree can benefit from having the data in the tree redistributed evenly. The routines presented here provide a means to achieve the most speed-efficient distribution of the data, without the complexities involved in balancing the tree as it is growing and shrinking. The standard routines for manipulation of binary trees are renowned for being small, easy to understand, and fairly efficient. Every attempt was made to keep the optimization routines within that tradition. The optimization routines presented here are a significant improvement over the routines presented by Terry (1991) which inspired their creation. When compared to the original routines, these routines: are smaller--they compile to smaller executable code and take fewer pages to list; are easier to understand--only one function is necessary to rebuild a tree from a list; require less stack space--fewer parameters are used for the recursive functions; and execute faster. Bibliography Terry, Bruce. June 1991. "Optimizing Binary Trees." The C Users Journal. 65-74. Timing Comparison In these tests, a small program, which adds the same random numbers to each of two binary trees, optimizes each copy of the tree 30 times. (Since the tree is initially flattened and then rebuilt, rebuilding a balanced tree should take just as long as rebuilding a degenerate one.) Thirty repetitions was deemed to be sufficient to cancel any round-off or timer errors. Table 1 gives the time needed for each routine to complete its optimizations. Figure 3 is a graphical representation of the data in the table. Figure 1 Normal result of a five-node tree Figure 2 Preferred result of a five-node tree Figure 3 Execution time comparison Table 1 Execution times for original and proposed methods Listing 1 Code for optimizing a binary tree #include /* TREE STRUCT: GENERIC BINARY TREE NODE */ typedef struct treenode { struct treenode *left; struct treenode *right; /* Data is unimportant ...*/ } TREENODE; /* STATIC VARIABLES */ static int list_num; static TREENODE list_base; static TREENODE *list_tail; static TREENODE *root; static int left; /*list_num IS SIMPLY A COUNTER OF HOW MANY ITEMS HAVE BEEN ADDED TO THE LIST. list_base IS USED AS A DUMMY NODE AT THE HEAD OF THE LIST TO PREVENT TESTING FOR A NULL ROOT EVERY TIME A NODE IS ADDED TO THE LIST. list_tail IS USED AS A POINTER TO THE LAST NODE IN THE LINKED LIST. root IS USED AS A POINTER TO THE NODE AT THE HEAD OF THE LIST. left IS A BOOLEAN TO KEEP TRACK OF WHICH WAY WE ARE GOING IN THE TREE SO THAT WE DIVIDE UNEVEN LISTS CORRECTLY. ALTHOUGH IT WOULD BE MORE CLEAR TO USE A PARAMETER, THE ROUTINE IS FINISHED WITH left BEFORE IT RECURSES (IT IS NO LONGER USING IT) AND MAKING IT STATIC REDUCES THE STACK SIZE REQUIRED FOR THE RECURSION. */ /* Function prototypes */ void *optree(void *); static void list_build(TREENODE *); static TREENODE *form_tree (int); /* OPTIMIZE A TREE */ void *optree(void *opt_root) { if (opt_root == NULL) return(NULL); list_tail = &list_base; list_num = 0; list_build(opt_root); root = list_base.right; left = 0; return((void *) form_tree(list_num)); } /* FORM AN OPTIMIZED TREE FROM LIST */ TREENODE *form_tree(int num) { int middle; TREENODE *ptr; middle = (num >> 1); /* (num / 2) */ /* SPECIAL 5-NODE CASE */ if(num == 5) middle++; /* LEAN BRANCH TO CENTER OF TREE */ if(left) middle = num - middle - 1; /* REMOVE LEFT SUBTREE FROM LIST */ left = 1; ptr = (middle > 0) ? form_tree(middle) : NULL; root->left = ptr; /* REMOVE THIS NODE FROM LIST */ ptr = root; root = root->right; /* REMOVE RIGHT SUBTREE FROM LIST */ left = 0; middle =num - middle - 1; ptr->right = (middle > 0) ? form_tree(middle) : NULL; return ptr; } /* FLATTEN TREE INTO LINKED LIST */ void list_build(TREENODE *node) { if(node->left) list_build(node->left); list_tail->right = node; list_tail = node; lis_num++; if(node->right) list_build(node->right); } /* End of File */ A Library of Financial Functions William Smith William Smith is the engineering manager at Montana Software, a software development company specializing in custom applications for MS-DOS and Windows. You may contact him by mail at P.O. Box 663, Bozeman, MT 59771-0663. The need in science and engineering for high-speed data processing was the original driving force behind the invention and the later improvement of computer hardware. It did not take long, however, before business became the largest user of computers and the driving force behind the improvement and diversification of computer software. Presently, the largest volume categories of software applications are business-oriented. Business uses of computers and software cover such diverse applications as financial forecasting, economic modeling, accounting, tracking inflation, record keeping, materials planning, inventory, depreciation, document processing, taxes, and bookkeeping, to name just a few. As a whole, business software is concerned with money. A central theme to business-oriented calculations and modeling is the time value of money. Loans, investments, rate-of-return calculations, depreciation, taxes, and cost/benefit analysis all require calculations based upon interest rates over periods of time. Nearly everyone has a loan, a bank account, or credit card. Interest rates and the resulting time value of money directly affects us all. Computations that involve interest rates are tedious by hand. They require using interest rate tables to relieve some of the number pushing. Computer automation can easily handle these types of calculations. More than a couple of times I have had to incorporate the equivalent of interest-rate tables into a software project. The relationships between the three quantities present value, future values, and equivalent uniform series form six basic formulas. These formulas are a function of interest rates and time expressed as number of periods. The theory of compounding, or how often you calculate the interest, further complicates matters. Calculations with continuous compounding requires an additional set of six formulas. In total, there are twelve interest rate formulas that I have utilized when writing business applications. These formulas eventually became a small library of C functions and macros that I have used many times. They are simple, small, and constitute a useful addition to your library and arsenal of programming tools. There are additional interest rate formulas for increasing series of values known as gradients. I have never had to use these formulas, so they have not appeared in my financial function library yet. Numeric Money Types When writing a business application in C, one of the first stumbling blocks you will come up against is which numeric type works best to represent money. Representing money values in C is a challenge. Floating-point and integer types have some contrasting pros and cons. Using a floating-point value to represent money has the problem of requiring a conversion to fixed point. This requires rounding off and can eventually lead to round-off error. Using an integer value has the disadvantage of not covering a large enough range of values under some compiler/operating system combinations. C-library vendors have addressed these problems by offering business-math libraries. There are many business-math libraries available that each vendor claims overcomes the problems of round-off error and range. I have never used any so I cannot make any statements as to whether they overcome some of the problems that developers typically encounter when working with money. For the interest-rate functions, I choose to use the standard floating-point type double. I choose this because the functions work with interest rates and periods. Neither of these is a money type. Interest rate is a true floating-point value. The number of periods is an integer value. The value returned by the functions is also a floating-point value. You use this value on or with money values, but it is not a money value. This gets the interest-rate functions out of the dilemma of having to commit to a best way to represent a money type. It is up to the user of the functions to determine how best to handle rounding and representation of money values. Interest-Rate Formulas The basic requirements of interest calculations involve working with the concepts of present values, future values, and equivalent uniform series of values. The relationships between these three quantities depend upon interest rates and time. For the purpose of the formulas, time is broken up into periods. The interest rate values must apply to the same period. There are six possible relationships between the three values of present value (P), future value (F), and equivalent uniform series of values (A). Table 1 contains some symbols and definitions that the interest rate formulas use. The relationships between A, F, and P are represented by ratios between the values. Each of the six formulas calculate one of the possible six ratios. The relationship or ratio depends only on the interest rate and number of periods. Table 2 lists the six relationships for discrete compounding. The table includes the formula as a function of interest rate (i) and number of periods (n). The formulas and functions require the interest rate as a floating-point number, not a percentage. The table also lists the symbolic representation of the formula as a ratio between A, F, or P. The first column in the table is the name of the C-function rendition of the formula. The formulas in Table 2 assume discrete compounding. They expect the interest rate to apply to the same period as the compounding frequency. As long as this criterion holds, these formulas will work for any interest rate and number of periods. If the compounding frequency is different from the interest period, the number of periods and interest per period must be converted into values adjusted for compounding periods and interest. This is done by dividing the interest rate by the compounding frequency per period and multiplying the number of periods by the compounding frequency. For example, if the interest rate is 0.05 (5.0%) per period, the period length is one year, the number of periods is 10, and the compounding is monthly, you must adjust both the interest rate and the number of periods to use the formulas in Table 2. The interest rate (i) becomes 0.05 / 12 = 0.004167 and the number of periods (n) are 10 * 12 = 120. Continuous compounding requires entirely different formulas. You can think of continuous compounding as increasing the compounding frequency to its absolute limit. The formulas still use an interest rate and the number of periods. Since the interest-rate period is no longer equal to the compounding period, the interest rate is no longer the effective interest rate, but a nominal interest rate (r). The nominal interest rate does not include the effect of compounding. Table 3 lists the six interest-rate formulas and functions for continuous compounding. The symbolic representation of the formulas is the same as in Table 2. The Symbols column is a ratio between A,F, or P. I have changed the C function names by adding _c to the end of the name. This indicates that the function is for continuous compounding. The formulas depend upon the nominal interest rate per period and the number of periods. In addition, the formulas use the constant e (natural log base). Functions and Macros Listing 1, FINANCES.C, contains the twelve functions listed in Table 2 and Table 3. These functions are interdependent. They make calls to one another. I have optimized these functions for size. The functions are as terse as possible. They rely upon the similarities between one another to keep them short and compact. This may take the least amount of overall code space, but is not the fastest rendition possible. For those who need speed, Listing 2, FINANCES.H contains macro equivalents to the functions in Listing 1. I have optimized the macros for speed. The macros avoid calling any of the functions in Listing 1. Listing 2 also contains prototypes for all the functions in Listing 1. If you want to create a hybrid of these two versions you can change the way each function is coded. Just include a call to a macro inside a function wrapper. For example you can rewrite the function a_to_f as double a_to_f( double i, int n ) { return ( A_TO_F( i, n ) ); } This would speed up the functions by avoiding interdependency and calls to other functions in the library. Here is an example of using the functions to calculate the monthly payment required to pay back a $100,000.00 loan in 30 years with a 8% annual interest rate. This is a typical mortgage these days. Payment = 100000.0 * p_to_a( 0.08 / 12.0, 30 * 12 ); (Monthly payment is $733.77) Conclusions Calculations involving interest rates are fundamental to business and financial software. This library of functions presents a set of tools. They are building blocks you can use to add time value of money features to your applications. I have used them in applications ranging from a simple Real Estate loan-calculation program to a sophisticated financial-modeling application. Do not underestimate their utility. The Real Estate loan calculation program was extremely simple. It consisted of nothing more than a very easy to use interface on top of these functions. I created this program as a vertical application for some local realtors who were computer-illiterate and computer-intimidated. For a very small amount of work it garnered a handsome rate of return. Table 1 Symbols and definitions used by the interest rate formulas Symbol Description ----------------------------------------- A uniform series amount or annuity F future value or worth P present value or worth e natural logarithm base (2.718) n number of periods i effective interest rate per period r nominal interest rate per period Table 2 Interest rate functions and formulas for discrete compounding Function Description Symbol Formula Name ---------------------------------------------------------- p_to_f present value to future value F/P (1 + i)n -------- 1 f_to_p future value to present value P/F 1 -------- (1 + i)n f_to_a future value to annuity A/F i ---------- (1 + i)n-1 p_to_a present value to annuity A/P i(1 + i)n ------------ (1 + i)n - 1 a_to_f annuity to future value F/A (1 + i)n - 1 ------------ i a_to_p annuity to present value P/A (1 + i)n - 1 ------------ i(1 + i)n Table 3 Interest rate functions and formulas for continuous compounding Function Description Symbol Formula Name ------------------------------------------------------------ p_to_f_c present value to future value F/P ern --- 1 f_to_p_c future value to present value P/F 1 --- ern f_to_a_c future value to annuity A/F (er - 1) --------- (ern - 1) p_to_a_c present value to annuity A/P (er - 1)ern ----------- (ern - 1) a_to_f_c annuity to future value F/A (ern - 1) --------- (er - 1) a_to_p_c annuity to present value P/A (ern - 1) ----------- (er - 1)ern Listing 1 FINANCES.C -- functions for calculating interest rate formulas /****************************************************** File Name: FINANCES.C Description: Library of functions for calculating interest rate formulas Global Function List: a_to_f a_to_f_c a_to_p a_to_p_c f_to_a f_to_a_c f_to_p f_to_p_c p_to_a p_to_a_c p_to_f p_to_f_c Portability: Standard C ******************************************************/ /* Standard C */ #include /* Own */ #include /***************************************************** Name: a_to_f Description: annuity to future value Parameters: i - effective interest rate per period n - number of periods Return: F/A - annuity to future value factor *****************************************************/ double a_to_f( double i, int n ) { return ( ( p_to_f( i, n ) - 1.0 ) / i ); } /***************************************************** Name: a_to_f_c Description: Annuity to future value continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: F/A - Annuity to future value factor *****************************************************/ double a_to_f_c( double r, int n ) { return ( ( p_to_f_c( r, n ) - 1.0 ) / ( pow( LN BASE, r ) - 1.0 ) ); } /***************************************************** Name: a_to_p Description: annuity to present value Parameters: i - effective interest rate per period n - number of periods Return: P/A - annuity to present value factor *****************************************************/ double a_to_p( double i, int n ) { return ( a_to_f( i, n ) / p_to_f( i, n ) ); } /***************************************************** Name: a_to_p_c Description: annuity to present value continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: P/A - annuity to present value factor *****************************************************/ double a_to_p c( double r, int n ) { return ( a_to_f_c( r, n ) / p_to_f_c( r, n ) ); } /***************************************************** Name: f_to_a Description: future value to annuity Parameters: i - effective interest rate per period n - number of periods Return: A/F - future value to annuity factor *****************************************************/ double f_to_a( double i, int n ) { return ( 1.0 / a_to_f( i, n ) ); } /***************************************************** Name: f_to_a c Description: future value to annuity continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: A/F - future value to annuity factor *****************************************************/ double f_to_a c( double r, int n ) { return ( 1.0 / a_to_f_c( r, n ) ); } /***************************************************** Name: f_to_p Description: future value to present value Parameters: i - effective interest rate per period n - number of periods Return: P/F - future to present value factor *****************************************************/ double f_to_p( double i, int n ) { return ( 1.0 / p_to_f( i, n ) ); } /***************************************************** Name: f_to_p_c Description: future value to present value continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: P/F - future to present value factor *****************************************************/ double f_to_p_c( double r, int n ) { return ( 1.0 / p_to_f_c( r, n ) ); } /***************************************************** Name: p_to_a Description: present value to annuity Parameters: i - effective interest rate per period n - number of periods Return: A/P - present value to annuity factor *****************************************************/ double p_to_a( double i, int n ) { return ( p_to_f( i, n ) / a_to_f( i, n ) ); } /**************************************************** Name: p_to_a_c Description: present value to annuity continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: A/P - present value to annuity factor ****************************************************/ double p_to_a_c( double r, int n ) { return ( p_to_f_c( r, n ) / a_to_f_c( r, n ) ); } /**************************************************** Name: p_to_f Description: present value to future value Parameters: i - effective interest rate per period n - number of periods Return: F/P - present to future value factor ****************************************************/ double p_to_f( double i, int n ) { return ( pow( 1 + i, n ) ); } /**************************************************** Name: p_to_f_c Description: present value to future value continuous compounding Parameters: r - nominal interest rate per period n - number of periods Return: F/P - present to future value factor ****************************************************/ double p_to_f_c( double r, int n ) { return ( pow( LN_BASE, r * n ) ); } /* End of File */ Listing 2 FINANCES.H /**************************************************** File Name: FINANCES.H Description: Include file for FINANCES.C Portability: Standard C ****************************************************/ #if !defined ( FINANCES_DEFINED ) /* Prototypes for function in FINANCES.C */ double a_to_f( double i, int n ); double a_to_f_c( double r, int n ); double a_to_p( double i, int n ); double a_to_p_c( double r, int n ); double f_to_a( double i, int n ); double f_to_a_c( double r, int n ); double f_to_p( double i, int n ); double f_to_p_c( double r, int n ); double p_to_a( double i, int n ); double p_to_a_c( double r, int n ); double p_to_f( double i, int n ); double p_to_f_c( double r, int n ); /* Natural log base */ #define LN_BASE 2.718281828459 /* Macro versions of functions in FINANCES.C */ #define A_TO_F( i, n ) \ ( ( pow( 1.0 + i, n ) - 1.0 ) / i ) #define A_TO_F_C( r, n ) \ ( ( pow( LN_BASE, r * n ) - 1.0 ) / \ ( pow( LN_BASE, r ) - 1.0 ) ) #define A_TO_P( i, n ) \ ( ( pow( 1.0 + i, n ) - 1.0 ) / \ ( i * pow( 1.0 + i, n) ) ) #define A_TO_P_C( r, n ) \ ( ( 1.0 - pow( LN_BASE, -( r * n ) ) ) / \ ( pow( LN_BASE, r ) - 1.0 ) ) #define F_TO_A( i, n ) \ ( i / ( pow( 1.0 + i, n ) - 1.0) ) #define F_TO_A_C( r, n ) \ ( ( pow( LN_BASE, r ) - 1.0 ) / \ ( pow( LN_BASE, r * n ) - 1.0 ) ) #define F_TO_P( i, n ) \ ( 1.0 / pow( 1.0 + i, n ) ) #define F_TO_P_C( r, n ) \ ( 1.0 / pow( LN_BASE, r * n ) ) #define P_TO_A( i, n ) \ ( i * pow ( 1.0 + i, n ) / \ ( pow( 1.0 + i, n ) - 1.0 ) ) #define P_TO_A_C( r, n ) \ ( ( pow( LN_BASE, r ) - 1.0 ) / \ ( 1.0 - pow( LN_BASE, -( r * n ) ) ) ) #define P_TO_F( i, n ) \ (pow( 1.0 + i, n) ) #define P_TO_F_C( r, n ) \ ( pow( LN_BASE, r * n ) ) #endif /* End of File */ Making C++ Safe for Threads Dave Plauger Dave has over 20 years experience with the design and development of operating systems. His main areas of expertise include real time and multi processing. Dave currently serves as Technical Editor of POSIX 1003.14 (Multiprocessing Profile) and has been a member of various POSIX working groups since the days of /usr/group. Dave may be reached at Mercury Computer Systems, Lowell MA. The phone number is (508) 458-3100; email djp@mc.com. Introduction Multithreading is a powerful technique that can speed up application programs on both multiprocessors and uniprocessors. On multi processors there is a potential speedup due to parallel computation. On uni processors there is a potential speedup due to the overlap of I/O with computation. The speedup is potential because, in the real world, there are always trade-offs. In order to have computational parallelism you need synchronization, which adds overhead. In order to overlap I/O with computation you again need synchronization in the form of I/O completion events. Multi threading helps by hiding (but not eliminating) the complexity and the overhead associated with both types of parallelism. In the many cases where parallelism does help, programming with threads is a convenient way to use it. Issues of Thread-Safety The main problem that arises when adding threads to programs is synchronizing updates to shared variables. The problem arises because updates to data structures often require several machine steps. These individual operations may be interleaved at random among multiple processors. And that can cause corrupted data structures, storage overwrites, and even more unpleasant problems. Even on a uni processor threads may be time-shared, which means they effectively run in parallel. High-level languages generally let you ignore the underlying machine, but parallel programming drags you back into worrying about what is really happening. On the other hand, you do have enough information in the declarations to help spot the potential problems. Variables that are declared at file scope and static data within functions clearly need to be protected from uncoordinated updates by multiple threads. Less obviously, control data structures used by library packages need careful attention as well. Examples of these are the FILE structures used in the Standard C library and the dirent structures used in UNIX and POSIX. These data structures hold state information between calls. Problems may arise if they are used in more than one thread at the same time, or even alternately. The same issues arise with member data in C++ classes. In the C library, something has to be done for errno, stdin, stdout, and stderr (at least), to make them thread-safe. In C++, the analogous structures are cin, cout, cerr, and clog. Two Solutions There are two general approaches to dealing with a global shared variable in a multi-threaded program: provide locking in order to synchronize updates eliminate the problem by making the variable thread private There are lots of mechanisms to enforce locking. Some examples are semaphores, monitors, and message queues. The POSIX threads package (IEEE P1003.4a, Draft 6) provides the function pthread_mutex_lock and pthread_mutex_unlock. Locks should be associated with the data, not the code. Locking code that happens to access shared variables may overly limit parallelism. For example, in an operating system one has to lock the global run queue or task data during updates, but one should avoid having a scheduler lock. The latter approach results in a system that may be thread-safe, but it will be slower than the original, single-threaded version. The trade-off one makes is to implement enough fine-grained locking to maximize parallelism without adding too much overhead. Besides adding to the overhead of even a single-threaded program, locking introduces the problem of deadlock. Deadlock can be avoided by recognizing the hierarchical structure of the data, and obtaining locks in a strict order. However, this requires global knowledge of all data, which may be impossible in a modular system, such as a third-party library. Thread-private data may be supported explicitly by the compiler, if you're lucky. But there are routines in each thread package that provide for thread-private data, because it is indispensable. In POSIX threads, the routines are pthread_keycreate, pthread_getspecific, and pthread_setspecific. These routines implement an associative memory, in which a shared key is associated with a thread-specific pointer. The easiest form of thread-private storage to use is the stack. Each thread has its own stack, therefore all automatic variables are thread-safe. You can also place a data structure in thread-private storage by using malloc (in C) or new (in C++) to allocate the storage, then storing the pointer in thread-specific data. When a new thread is created, its specific data pointers are set to null. Knowing this, the program can tell at runtime when to allocate and initialize another instance of the data structure. Therefore, not every thread needs a copy of every data structure. pthread_keycreate registers a destructor function to be called for each key when the thread exits. This function may free the storage and otherwise clean things up. It's best to make variables thread-private whenever possible. The access cost of thread-specific data is much less compared to lock/unlock overhead, and the associated code can be written using the simpler, uni processor model. Other Methods One can dedicate a thread to particular functions, which makes data and code thread-private by convention. For example, one could contrive the program so that all calls to X-Window routines occur in a particular thread. The initial thread--the one which runs main--is the safest one to dedicate to older libraries. For backward compatibility, many threads packages associate the initial thread with global resources, such as errno. Sometimes global shared variables can be used in a multi-threaded program without additional protection. For example, a Boolean variable used as a flag can be set or cleared without synchronization. The code sequence: lock; flag=1; unlock is silly. But, if two or more variables need to change "simultaneously," then locking may be needed to protect the program state, disallowing certain intermediate states. Statistics counters employ another kind of update which may go unprotected. An expression like ++count is a multi-step update of a global variable, and is a candidate for locking. But if all you ever care about are zero/non-zero values of the count, no harm will result even if multiple processors collide on an update. (The value will be at least 1.) This is worth mentioning because test coverage tools often insert such counters. It is seldom worth the trouble to make these tools thread-safe. A Question of Semantics Choosing whether to add locking or to make data thread-private is more than a matter of mechanics. The bigger question concerns the thread model to be used. Are threads to be independent or cooperating? The answer to this question determines the implementation strategy. If the threads are to be cooperating, meaning they produce or consume the same stream of data, then the data must have internal locking and external synchronization. In this model, the POSIX opendir/readdir facility, for example, would operate with a shared data structure. An application would want to have threads alternately read or write directory entries. Each independent thread, on the other hand, would produce or consume its own stream of data. In this case, the buffering would be thread-private, and there would be no need for internal locking. Either approach is viable, it just depends upon the application. But what is the library writer supposed to do? The answer may be to supply both. A Thread-Safe C++ Library If the underlying C implementation is thread-safe, then C++ is very close to it. However, there are a few special considerations. First, the I/O streams cin, cout, cerr, and clog are static objects provided by the library. Nothing prevents two threads from simultaneously writing to the same output stream, and thus corrupting its internal state. Second, C++ uses an internal list to keep track of sizes of arrays of objects. This is due to the fact that C++ lets you say delete [] array-ref which implies that the compiler/library has to determine the number of elements. (For various reasons, most implementations can't store the size in the same chunk of memory as the array itself.) Since an array may be allocated on one thread and freed on another, this internal list has to be global. Therefore it has to be protected during updates. An important issue is whether I/O streams be thread-private or shared. Given the nature of the interface, which is to perform I/O by the character, word, or line, it doesn't do much good to provide locking within the interface functions. For example, one thread may print hello and another print world, but the screen output will be jumbled anyway unless the threads coordinate their outputs in some way outside of the I/O library. Adding locking within each I/O primitive call (inside the interface) will make the I/O library thread-safe. Indeed, this was the approach taken for POSIX threads. The problem is that it introduces a lot of overhead in the interest of safety. Alternatively, you could establish the rule that I/O streams are thread-private. Cooperating threads will need to synchronize outside of the interface. (Maybe they would have to anyway.) Independent threads would read or write streams as efficiently as in the single-threaded case. Given this rule, the library only has to concern itself with the streams it provides initially, namely: cin, cout, cerr, and clog. In C++, it's fairly easy to provide a "wrapper class" that can operate on a thread-private version of the original class. Wrapper-Class Implementation The wrapper class defines a small object that holds the thread-specific data key, initial file descriptor, and other stream-initialization parameters. The declared object has a static initializer which takes care of allocating the key. Each of the pre-defined streams is both declared and replaced with a macro, as follows: extern ostream_withassign cout;// stream declaration extern t_ostream_withassign __tcout;// stream wrapper #define cout (__tcout.ref()) Any textual reference to cout is replaced with a call to the wrapper class, which returns a reference to the thread-private copy of the stream. The thread-private stream is not allocated and initialized until the first call to the ref member function is made. This introduces the limitation that &cout may not be used in an initializer list--because the address is no longer a constant value, but must be computed at runtime. The initial thread points at the pre-defined streams. Old binaries will therefore access the initial thread's I/O streams. This permits backward compatibility with single-threaded code. The wrapper is a very small class that defines a static object (such as __tcout) which is shared by all threads. The static initializer does three things: It allocates a key to access thread-private storage. It stores the stream-initialization parameters it will need later. It points the wrapper at the corresponding global stream. Static initializers are called from the initial thread before main executes. A pointer to the wrapper class is stored in thread-private storage. This pointer is later presented as an argument to the destructor function called by thread-exit processing. Therefore, the thread-private wrapper object and its stream object are deleted when the thread exits. The ref member function simply uses the key stored in the static object and tries to look up the thread-private pointer to the wrapper. The very first time a new thread calls any I/O function, ref will find that its thread-private pointer is null. When the pointer is null, ref allocates a new wrapper object and a new stream, then stores the pointer to its wrapper object in thread-private storage. Subsequently, ref returns a reference to the stream object assocated with the wrapper object, both of which are private to the thread. With this implementation, only threads that use stream I/O get copies of stream objects. Threads can be created at any time, since I/O stream allocation occurs on demand. The main cost added to implement thread-private streams is one lookup in thread-private storage for every I/O call. This is much cheaper than adding lock/unlock to every I/O call. A C++ Wish List It would be nicer if you could have redefined cout as an object of the wrapper class instead of having to use a macro. In other words, you would want to say extern t_ostream_withassign cout Then you could rely on the user-defined type conversion facility to yield a reference to the desired class. That is, you would simply define operator ostream_withassign&() {return ref();} in the wrapper class. I tried this and it mostly works, but the following limitations were found: Invocations of member functions didn't work. For example, cout.flush() had to be transformed to ((ostream_withassign&)(cout)).flush(). Here is a case where it would be nice to be able to overload the dot operator. Then you could say: ostream_withassign& operator.() {return ref();} The result is that the wrapper class could substitute a reference to the desired class that defines the member functions. The compiler complains when the expression in a return statement does not match the function's declaration. For example: ostream_withassign& f() {return cout;} would work only if the compiler applied the user-defined type conversion. There is a similar warning when the type of an initializer does not match the type of the variable being initialized. For example, ostream_withassign &os = cout would work only if the compiler applied the user-defined type conversion. If the user-defined type conversion were applied uniformly and it were possible to overload the dot operator, then it would be easier to implement a very useful form of indirection for any object. Arrays of Objects As I stated above, the other change needed for the C++ library was to add locking around an internal list. This change is not visible to the library's user. The implementation of the locking mechanics used is perhaps worthy of discussion. The list is protected with simple, mutual exclusion, using pthread_mutex_lock and pthread_mutex_unlock in the underlying implementation. The list-handling file has a definition at file scope like static_thread_lock_def list_lock; where thread_lock_def defines a static initializer that allocates and initializes a pthread_mutex_t. The body of the code where mutual exclusion is needed has the syntax: { thread_lock L(list_lock); // do something to the list } //end of critical region The braces are used to control the scope of L, which is a reference to the lock. The class thread_lock acquires the pthread_mutex in its initializer, and releases it in its destructor. Thus, the scope of L determines the bounds of the critical region. This could be considered just another stupid C++ trick except that, should there be a return or an exception within the region, the destructor for L will be called, which will release the lock. Thus, locking is integrated with returns and exceptions because of this method. There is a quiet gotcha in this implementation. Should you forget to supply a variable name (in this case it's L), and write thread_lock(list_lock), C++ will lock then immediately unlock list_lock, leaving the critical region unprotected. The reason there is no warning has something to do with the great debate over the lifetime of temporaries. For example, in a simple expression like X + Y the compiler may have to make temporary copies of intermediate results, which one would expect to get deleted at the end of the expression, not at the end of the current scope. See your local language lawyer for details. Conclusion I successfully converted the Hewlett-Packard C++ library to a thread-safe form. Along the way, I observed: C++ is very close to being thread-safe, except for some potential uses of I/O streams and arrays of objects. Making global data thread-private is more efficient than adding locking. I/O streams should be thread-private, because the shared use of a stream requires external coordination anyway. Uniform application of user-defined type conversion coupled with the ability to overload the dot operator would be useful extensions. This would aid in the creation of wrapper classes for indirect access to any object. Locking based upon C++ scoping rules integrates locking with exceptions. Image Processing, Part 9: Histogram-Based Image Segmentation Dwayne Phillips The author works as a computer and electronics engineer with the U.S. Department of Defense. He has a PhD in Electrical and Computer Engineering from Louisiana State University. His interests include computer vision, artificial intelligence, software engineering; and programming languages. Introduction to the Series of Articles This is the ninth in a series of articles on images and image processing. Previous articles discussed reading, writing, displaying, and printing images (TIFF format), histograms, edge detection, spatial frequency filtering, and sundry image operations. This article will discuss simple image segmentation based on histograms and image thresholding. Image segmentation is the process of dividing an image into regions or objects. It is the first step in the field of image analysis. Image processing displays images and alters them to make them look "better," while image analysis tries to discover what is in the image. The basic idea of image segmentation is to group individual pixels (dots in the image) together into regions if they are similar. Similar can mean they are the same intensity (shade of gray), form a texture, line up in a row, create a shape, etc. There are many techniques available for image segmentation, and they vary in complexity, power, and area of application. Histogram-Based Image Segmentation Histogram-based image segmentation is one of the simplest and most often used segmentation techniques. It uses the histogram to select the gray levels for grouping pixels into regions. In a simple image there are two entities: the background and the object. The background is generally one gray level and occupies most of the image. Therefore, its gray level is a large peak in the histogram. The object or subject of the image is another gray level, and its gray level is another, smaller peak in the histogram. Figure 1 shows an image example and Figure 2 shows its histogram. The tall peak at gray level 2 indicates it is the primary gray level for the background of the image. The secondary peak in the histogram at gray level 8 indicates it is the primary gray level for the object in the image. Figure 3 shows the image of Figure 1 with all the pixels except the 8s blanked out. The object is a happy face. This illustrates histogram-based image segmentation. The histogram will show us the gray levels of the background and the object. The largest peak represents the background and the next largest peak the object. We choose a threshold point in the valley between the two peaks and threshold the image. Thresholding takes any pixel whose value is on the object side of the point and sets it to one; it sets all others to zero. The histogram peaks and the valley between them are the keys. The idea of histogram-based segmentation is simple, but there can be problems. Where should you set the threshold point for the image of Figure 1? If you choose the point mid-way between the two peaks (threshold point=5), you produce the image of Figure 4. This is not the happy face object desired. If you choose the valley floor values of 4 or 5 as the threshold point, you also have a poor result. The best threshold point would be 7, but how could you know that without using trial and error? This example is difficult because there are only ten gray levels and the object (happy face) is small. In practice, the techniques discussed below will perform adequately, but there will be problems. Automatic techniques will fail, and you may have to resort to manual methods. Preprocessing: Histogram Equalization and Histogram Smoothing Histogram-based segmentation depends on the histogram of the image. Therefore, you must prepare the image and its histogram before analyzing it. The first step is histogram equalization (Phillips, August 1991). Histogram equalization attempts to alter an image so its histogram is flat and spreads out over the entire range of gray levels. The result is an image with better contrast. Photograph 1 shows an aerial image of several house trailers with its histogram. The contrast is poor and it would be very difficult to find objects based on its histogram. Photograph 2 shows the result of performing histogram equalization. The contrast is much better and the histogram is spread out over the entire range of gray levels. Photograph 3 shows the result of performing high-pass filtering (Phillips, October 1992) on the image of photograph 2. It is easy to see the house trailers, sidewalks, trees, bushes, gravel roads, and parking lots. The next preprocessing step is histogram smoothing. When examining a histogram, you look at peaks and valleys. Too many tall, thin, peaks and deep valleys will cause problems. Smoothing the histogram removes these spikes and fills in empty canyons while retaining the same basic shape of the histogram. Figure 5 shows the result of smoothing the histogram given in Figure 2. You can still see the peaks corresponding to the object and background, but the peaks are shorter and the valleys are filled. Photograph 4 shows another example of histogram smoothing. The histogram on the left is the original with several spikes and canyons. The histogram on the right has been smoothed. I will analyze this in the segmentation. Smoothing a histogram is an easy operation. It replaces each point with the average of it and its two neighbors. Listing 1 shows the smooth_histogram function that performs this operation. Thresholding and Region Growing There are two more topics common to all the methods of image segmentation: image thresholding and region growing. Image thresholding sets the pixels in the image to one or zero. Listing 2 shows the routine threshold_image_array that accomplishes this task. The difficult task is region growing. Figure 6 shows the result of thresholding Figure 1 correctly. The "object" in Figure 6 is a happy face. It comprises three different regions (two eyes and the smile). Region growing takes this image, groups the pixels in each separate region, and gives them unique labels. Figure 7 shows the result of region growing performed on Figure 6. Region growing grouped and labeled one eye as region one, the other eye as region two, and the smile as region three. Figure 8 shows the algorithm for region growing. It begins with an image array g comprising zeros and pixels set to a value. The algorithm loops through the image array looking for a g(i,j) == value. When it finds such a pixel, it calls the label_and_check_neighbor routine. label_and_check_neighbor sets the pixel to g_label (the region label) and examines the pixel's eight neighbors. If any of the neighbors equal value, they are pushed onto a stack. When control returns to the main algorithm, each pixel on the stack is popped and sent to label_and_check_neighbor. All the points on the stack equaled value, so you set them and check their neighbors. After setting all the pixels in the first region, you increment g_label and move on looking for the next region. Listing 3 shows the functions that implement region growing with grow being the primary routine. grow runs through the region-growing algorithm and calls label_and_check_neighbor shown next in Listing 3. The grow and label_and_check_neighbor functions follow the region-growing algorithm step for step. The only unusual item is the stack. There are several ways to implement a stack. I used a simple array and a file. I did this because a region could be as large as ROWSxCOLS (10,000 in the C Image Processing System), and the stack must be large enough to hold that many points. I push points onto the stack array by putting them in the array and moving the top of stack pointer. When the stack array becomes full, I write it to the stack file. If the array fills again, I push the array onto the top of the stack file. I pop points off the stack array by reading them and decrementing the top of stack pointer. When the stack array is empty, I pop an array off the stack file. The final four functions of Listing 3 (push_data_onto_stack_file, pop_data_off_of_stack_file, append_stack_files, and copy_stack_files) implement the push and pop functions for the stack array and file. Histogram-Based Segmentation Techniques There are four segmentation techniques: the manual technique, histogram peak technique, histogram valley technique, and an adaptive technique. The Manual Technique In the manual technique the user inspects an image and its histogram manually. Trial and error come into play and the result is as good as you want it to be. I used Photograph 4 as the input for all the segmentation examples. Photograph 5, Photograph 6, and Photograph 7 show the result of segmentation using three different thresholds. The result in Photograph 5 used a high of 255 and a low of 125. The segmentation included the white gravel roads as well as the house trailers and sidewalks. The result in Photograph 6 used a high of 255 and a low of 175. The gravel roads begin to disappear, but the house trailers and sidewalks remain. Photograph 7 shows the result using a high of 255 and a low of 225. This segmentation only finds the four dominant house trailers. Which answer is correct? That depends on what you wanted to find. Note, all image segmentations will appear rough. You can perform additional processing to make the result more pleasing to the eye, but that is not the purpose of segmentation. The purpose is to break the image into pieces so later computer processing can interpret their meaning. The output is for computer not human consumption. Also note how difficult it is for the computer, even with manual aid, to find objects that are trivial for humans to see. Anyone could trace over the input image and outline the objects better than the segmentation process. Listing 4 shows the code that implements manual segmentation. The function manual_threshold_segmentation has the same form as the other application functions in this series. It creates the output image file if it does not exist, reads in the input data, thresholds it (Listing 2), grows regions if requested (Listing 3), and writes the output. manual_threshold_segmentation has the usual inputs (image file names, image arrays, etc.) as well as the high and low threshold values, and the value and segment parameters. The value parameter specifies the value at which to set a pixel if it falls between the high and low thresholds. You usually set value equal to 1 since those pixels outside the high-low range are set to zero. The segment parameter specifies whether or not to grow regions after thresholding. Sometimes you only want to threshold an image and not grow regions. The two operations are identical except for the last step. If segment == 1, you call the region-growing routines. Manual segmentation is good for fine tuning and getting a feel for the operation. Its trial-and-error nature, however, makes it time consuming and impractical for many applications. You need techniques that examine the histogram and select threshold values automatically. Histogram Peak Technique The first such technique uses the peaks of the histogram. This technique finds the two peaks in the histogram corresponding to the background and object of the image. It sets the threshold half way between the two peaks. Look back at the smoothed histogram in Figure 5. The background peak is at 2 and the object peak is at 7. The midpoint is 4, so the low threshold value is 4 and the high is 9. The peaks technique is straightforward except for two items. In the histogram in Figure 5, you'll note the peak at 7 is the fourth highest peak. The peaks at 1 and 3 are higher, but they are part of the background mountain of the histogram and do not correspond to the object. When you search the histogram for peaks, you must use a peak spacing to ensure the highest peaks are separated. If you did not, then you would choose 2 as the background peak and 1 as the object peak. Figure 9 shows the disastrous effect of this. The second item to watch carefully is determining which peak corresponds to the background and which corresponds to the object. Suppose an image had the histogram shown in Figure 10. Which peak corresponds to the background? The peak for gray level 8 is the highest, but it corresponds to the object not the background. The reason is the mountain surrounding the peak at gray level 2 has a much greater area than that next to gray level 8. Therefore, gray levels 0 through 6 occupy the vast majority of the image, and they are the background. Listing 5 shows the source code to implement the peaks technique. peak_threshold_segmentation is the primary function. It checks for the existence of the output image, reads the input image, and calculates and smoothes the histogram. Next, it calls new functions to find the histogram peaks and determine the high and low threshold values. Finally, it thresholds the image, performs region growing if desired, and writes the result image to the output file. The functions find_peaks and insert_into_peaks in Listing 5 analyze the histogram to find the peaks for the object and background. These functions build a list of histogram peaks. There are several ways to do this. I used an array of values. find_peaks loops through the histogram and calls insert_into_peaks, which puts the histogram values in the proper place in the array. find_peaks ends by looking at the spacing between the largest peaks to ensure we do not have a disaster such as shown in Figure 9. The function peaks_high_low takes the two peaks from find_peaks and calculates the high- and low-threshold values for segmentation. peaks_high-low examines the mountains next to the peaks as illustrated in Figure 10. It then finds the mid-point between the peaks and sets the high and low threshold values. Photograph 8 shows the result of applying the peaks technique to the image of Photograph 4. The peaks technique found the two peaks at 255 and 77. The mid-point is 166, so the high threshold is 255 and the low threshold is 166. This is a reasonably good segmentation of Photograph 4. Histogram Valley Technique The second automatic technique uses the peaks of the histogram, but concentrates on the valley between them. Instead of setting the mid-point arbitrarily half way between the two peaks, the valley technique searches between the two peaks to find the lowest valley. Look back at the histogram of Figure 10. The peaks are at gray levels 2 and 8 and the peaks technique would set the midpoint at 5. In contrast, the valley technique searches from 2 through 8 to find the lowest valley. In this case, the "valleypoint" is at gray level 7. Listing 6 shows the code that implements the valley technique. The primary function is valley_threshold_segmentation. It checks for the output file, reads the input image, calculates and smoothes the histogram, and finds the peaks as peak_threshold_segmentation did. It finds the valley-point via the functions valley_high_low, find_valley_point, and insert_into_deltas. find_valley_point starts at one peak and goes to the next inserting the histogram values into a deltas array via the insert_into_deltas function. This uses an array to create a list of deltas in the same manner as insert_into_peaks did in Listing 5. Once you have the valleypoint, valley_high_low checks the mountains around the peaks to ensure you associate the peaks with the background and object correctly. Photograph 9 shows the result of applying the valley technique to the image in Photograph 4. It found the peaks at 77 and 255 and went from 77 up to 255 looking for the lowest valley. It pinpointed the lowest valley at gray level 241. Adaptive Histogram Technique The final technique uses the peaks of the histogram in a first pass and adapts itself to the objects found in the image in a second pass (Castleman 1979). In the first pass, the adaptive technique calculates the histogram for the entire image. It smoothes the histogram and uses the peaks technique to find the high and low threshold values. In the second pass, the technique works on each ROWSxCOLS area of the image individually. In each area, it segments using the high and low values found during the first pass. Then, it calculates the mean value for all the pixels segmented into background and object. It uses these means as new peaks and calculates new high and low threshold values for that ROWSxCOLS area. Now, it segments that area again using the new values. Listing 7 shows the code that implements the adaptive technique with adaptive_threshold_segmentation being the primary function. It is very similar to the peak_threshold_segmentation function of Listing 5 in that it uses all that code for its first pass. The second pass starts by calling threshold_and_find_means. This function thresholds the image array into background and object and calculates the mean pixel value for each. The second pass continues by using peaks_high_low to find new threshold values based on the background and object means. Finally, you threshold the image using these new threshold values. Photograph 10 shows the result of applying the adaptive technique to the image of Photograph 4. The first pass found the high- and low-threshold values to be 255 and 166. On the left side of the photograph, the second pass thresholded the image array and found the background mean to be 94 and the object mean to be 205. The new threshold values were 255 and 149. On the right side of the photograph, the background and object means were 84 and 200 and the new threshold values were 255 and 142. Integrating Histogram-Based Segmentation into the C Image Processing System Listing 8 shows the new code for the main routine of CIPS. I've added the options of thresholding and segmentation using the four techniques discussed above in cases 16 and 17. Listed next are the changes to the CIPS main menu and the two functions that interact with the user to obtain the processing options. Listing 9 shows a stand-alone application program for thresholding and segmenting entire image files. It is command- line driven and calls the functions shown in the earlier listings. Conclusions This installment in the series introduced image segmentation. This is the first step in locating and labeling the contents of an image. The techniques discussed work on simple images with good contrast and gray level separation between the object and background. You will need other techniques to attack more complex images. References Castleman, Kenneth R. 1979. Digital Image Processing. Prentice-Hall. Phillips, Dwayne. August 1991. "Image Processing, Part 4: Histograms and Histogram Equalization," The C Users Journal. Phillips, Dwayne. October 1992. "Image Processing, Part 7: Spatial Frequency Filtering," The C Users Journal. Figure 1 An image example 22222232221222212222 32222321250132123132 22588897777788888232 12988877707668882122 22888892326669893213 21278221222666665222 22002222220226660225 21221231223266622321 32238852223266821222 21288888342288882232 22328888899888522121 22123988888889223422 23222278888882022122 22232323883212123234 25221212222222222222 22122222320222202102 20222322412212223221 22221212222222342222 21222222221222222142 Figure 2 A histogram of the image in Figure 1 Figure 3 The image in Figure 1 with all the pixels except the 8's blanked out -------------------- -------------------- ---888------88888--- ---888-------888---- --8888--------8----- ----8--------------- -------------------- -------------------- ----88--------8----- ---88888----8888---- ----88888--888------ ------8888888------- -------888888------- --------88---------- -------------------- -------------------- -------------------- -------------------- -------------------- Figure 4 Figure 1 with athreshold point of 5 00000000000000000000 00000000000000000000 00011111111111111000 00111111101111110000 00111110001111110000 00011000000111110000 00000000000001110000 00000000000011100000 00001100000011100000 00011111000011110000 00001111111111000000 00000111111111000000 00000011111110000000 00000000110000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 Figure 5 The result of smoothing the histogram given in Figure 2 Figure 6 The result of correctly thresholding Figure 1 00000000000000000000 00000000000000000000 00011100000011111000 00011100000001110000 00111100000000100000 00001000000000000000 00000000000000000000 00000000000000000000 00001100000000100000 00011111000011110000 00001111100111000000 00000011111110000000 00000001111110000000 00000000110000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 Figure 7 The result of region growing performed on Figure 6 00000000000000000000 00000000000000000000 00011100000022222000 00011100000002220000 00111100000000200000 00001000000000000000 00000000000000000000 00000000000000000000 00003300000000300000 00033333000033330000 00003333300333000000 00000033333330000000 00000003333330000000 00000000330000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 Figure 8 Pseudocode for region growing 1. Given an image g with m rows and n columns g(i,j) for i=1,m j=1,n g(i,j) = value for object = 0 for background 2. set g_label=2 this is the label value 3. for (i=0; i= low && in_image[i][j] <= hi){ out_image[i][j] = value; counter++; } else out_image[i][j] = 0; } /* ends loop over j */ ) /* ends loop over i */ printf("\n\tTIA> set %d points", counter); } /* ends threshold_image_array */ /* End of File */ Listing 3 Functions that implement region growing /********************************************* * grow(... * This function is an object detector. * Its input is an binary image array * containing 0's and value's. * It searches through the image and connects * the adjacent values. **********************************************/ grow(binary, value) short binary[ROWS][COLS], value; { char name[80]; int first_call, i, j, object_found, pointer, pop_i, pop_j, stack_empty, stack_file_in_use; short g_label, stack[STACK_SIZE][2]; /*********************************** * Now begin the process of growing * regions. *************************************/ g_label = 2; object_found = 0; first_call = 1; for(i=0; i found %d objects", g_label); } /* ends grow */ /******************************************** * label_and_check_neighbors(... * This function labels a pixel with an object * label and then checks the pixel's 8 * neighbors. If any of the neigbors are * set, then they are also labeled. **********************************************/ label_and_check_neighbor(binary_image, stack, g_label, stack_empty, pointer, r, e, value, stack_file_in_use, first_call) int e, *first_call, *pointer, r, *stack_empty, *stack_file_in_use; short binary_image [ROWS] [COLS], g_label, stack[STACK_SIZE][2], value; { int already_labeled = 0, i, j; if (binary_image[r][e] == g_label) already_labeled = 1; binary_image[r][e] = g_label; /************************************* * Look at the 8 neighors of the * point r,e. * Ensure the points you are checking * are in the image, i.e. not less * than zero and not greater than * ROWS-1 or COLS-1. **************************************/ for(i=(r-1); i<=(r+1); i++){ for(j=(e-1); j<=(e+1); j++){ if((i>=0) && (i<=ROWS-1) && (j>=0) && (j<=COLS-1)) { if(binary_image[i][j] == value){ *pointer = *pointer + 1; stack[*pointer][0] = i; /* PUSH */ stack[*pointer][1] = j; /* OPERATION */ *stack_empty = 0; if(*pointer >= (STACK_SIZE - STACK_FILE_LENGTH)){ push_data_onto_stack_file(stack, pointer, first_call); *stack_file_in_use = 1; } /* ends if *pointer >= STACK_SIZE - STACK_FILE_LENGTH*/ } /* end of if binary_image == value */ } /* end if i and j are on the image */ } /* ends loop over i rows */ } /* ends loop over j columns */ } /* ends label_and_check_neighbors */ /*************************************** * push_data_onto_stack_file(... * This function takes the stack array * and pushes it onto the stack file. *****************************************/ push_data_onto_stack_file(stack, pointer, first_call) int *first_call, *pointer; short stack[STACK_SIZE][2]; { char backup_file_name[MAX_NAME_LENGTH]; FILE *backup_file_pointer, *stack_file_pointer; int diff, i; short holder[STACK_FILE_LENGTH][2]; printf("\nSFO> Start of push_data_onto_stack "); diff = STACK_SIZE - STACK_FILE_LENGTH; /***************************************** * Copy the elements to be stored to the * stack file into holder ******************************************/ for(i=0; i Could not open stack file"); else{ /*printf("\n\nSFO> Writing to stack file");*/ fwrite(holder, sizeof(holder), 1, stack_file_pointer); fclose(stack_file_pointer); } /* ends else could not open stack_file */ } /* ends if *first_call == 1 */ else{ /* else stack file has been used already */ strcpy(backup_file_name, STACK_FILE); append_string(".bak\0", backup_file_name); if((backup_file_pointer = fopen(backup_file_name, "wb")) == NULL) printf("\nSFO> Could not open backup file"); else{ /*printf("\n\nSFO> Writing to backup file");*/ fwrite(holder, sizeof(holder), 1, backup_file_pointer); fclose(backup_file_pointer); } /* ends else could not open backup_file */ append_stack_files(backup_file_name, STACK_FILE, holder); copy_stack_files(backup_file_name, STACK_FILE, holder); } /* ends else first_call != 1 */ printf("--- End of push_data_onto_stack"); } /* ends push_data_onto_stack_file */ /*************************************** * pop_data_off_of_stack_file(... * This function pops the stack array * off of the stack file. ****************************************/ pop_data_off_of_stack_file(stack, pointer, stack_file_in_use) int *pointer, *stack_file_in_use; short stack[STACK_SIZE][2]; { char backup_file_name[MAX_NAME_LENGTH]; FILE *backup_file_pointer, *stack_file_pointer; int i; long write_counter; short holder[STACK_FILE_LENGTH][2], holder2[STACK_FILE_LENGTH][2]; /******************************************** * POP - Read 1 time from stack * Copy the remainder of stack to * stack.bak * Copy stack.bak to stack * This has the effect of popping off * of the stack. * Read the holder array from the stack file. * Open the stack file for reading in binary * mode. * If it requires more than one write to * copy the remainder of stack to * stack.bak then there is still data in the * stack file so set stack_file_in_use = 1. * Else set it to 0. ***********************************************/ printf("\nSFO> Start of pop_data_off_of_stack "); write_counter = 0; strcpy(backup_file_name, STACK_FILE); append_string(".bak\0", backup_file_name); if( (stack_file_pointer = fopen(STACK_FILE, "rb")) == NULL) printf("\nSFO> Could not open stack file"); else{ /*printf("\n\nSFO> Reading from stack file");*/ fread(holder, sizeof(holder), 1, stack_file_pointer); backup_file pointer = fopen(backup_file_name, "wb"); while( fread(holder2, sizeof(holder2), 1, stack_file pointer) ){ fwrite(holder2, sizeof(holder2), 1, backup_file_pointer); ++write_counter; } /* ends while reading */ if(write_counter > 0) *stack_file_in_use = 1; else *stack_file_in_use = 0; fclose(backup_file_pointer); fclose(stack_file_pointer); } /* ends else could not open stack file */ copy_stack_files(backup_file_name, STACK_FILE, holder2); for(i=0; i Cannot open file %s", first_file); if((second = fopen(second_file, "rb")) == NULL) printf("\n\nSFO> Cannot open file %s", second_file); /*************************************** * Seek to the end of the first file and * to the beginning of the second file. *****************************************/ fseek(first, OL, 2); fseek(second, OL, 0); while(fread(holder, sizeof(holder), 1, second) ){ fwrite(holder, sizeof(holder), 1, first); } /* ends while reading */ fclose(first); fclose(second); } /* ends append_stack_files */ /****************************************** * copy_stack_files(... * Copy the first file to the second. ********************************************/ copy_stack_files(first_file, second_file, holder) char first_file[], second_file[]; short holder[STACK_FILE_LENGTH][2]; { FILE *first, *second; int i; if( (first = fopen[first_file, "rb")) == NULL) printf("\n\nSFO> Cannot open file %s", first_file); if( (second = fopen(second_file, "wb")) == NULL) printf("\n\nSFO> Cannot open file %s", second_file); /**************************************** * Seek to the beginning of the first file. *****************************************/ fseek(first, 0L, 0); while( fread(holder, sizeof(holder), 1, first) ){ fwrite(holder, sizeof(holder), 1, second); } /* ends while reading */ fclose(first); fclose(second); } /* ends copy_stack_files */ /* End of File */ Listing 4 Code implementing manual segmentation /************************************************** * manual_threshold_segmentation(... * This function segments an image using thresholding * given the hi and low values of the threshold * by the calling routine. It reads in an image * and writes the result to the output image. * If the segment parameter is 0, you only * threshold the array - you do not segment. ***************************************************/ manual_threshold_segmentation(in_name, out_name, the_image, out_image, il, ie, ll, le, hi, low, value, segment) char in_name[], out_name[]; int il, ie, ll, le, segment; short hi, low, the_image[ROWS][COLS], out_image[ROWS][COLS], value; { int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\nMTS> output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); threshold_image_array(the_image, out_image, hi, low, value); if(segment == 1) grow(out_image, value); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends manual_threshold_segmentation */ /* End of File */ Listing 5 Code implementing the histogram peaks technique /********************************************* * peak_threshold_segmentation(... * This function segments an image using * thresholding. It uses the histogram peaks * to find the hi and low values of the * threshold. * If the segment parameter is 0, you only * threshold the array - you do not segment. ***********************************************/ peak_threshold_segmentation(in_name, out_name the_image, out_image, il, ie, ll, le, value, segment) char in name[], out_name[]; int il, ie, ll, le, segment; short the_image[ROWS][COLS], out_image[ROWS] [COLS], value; { int length, peak1, peak2, width; short hi, low; struct tiff_header_struct image_header; unsigned long histogram[GRAY_LEVELS+1]; if(does_not_exist(out_name)){ printf("\n\nPTS> output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does not exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); zero_histogram(histogram); calculate_histogram(the_image, histogram); smooth_histogram(histogram); find_peaks(histogram, &peak1, &peak2); peaks_high_low(histogram, peak1, peak2, &hi, &low); threshold_image_array(the_image, out_image, hi, low, value); if(segment == 1) grow(out_image, value); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends peak_threshold_segmentation */ /******************************************* * find_peaks(... * This function looks through the histogram * array and finds the two highest peaks. * The peaks must be separated, cannot be * next to each other, by a spacing defined * in cips.h. * The peaks array holds the peak value * in the first place and its location in * the second place. *******************************************/ find_peaks(histogram, peak1, peak2) unsigned long histogram[]; int *peak1, *peak2; { int distance[PEAKS], peaks[PEAKS][2]; int i, j=0, max:0, max_place:0; for(i=0; i0; 1--) if(distance[i] > PEAK_SPACE) *peak2 = peaks[i][1]; } /* ends find_peaks */ /******************************************* * insert_into_peaks(... * This function takes a value and its * place in the histogram and inserts them * into a peaks array. This helps us rank * the the peaks in the histogram. * The objective is to build a list of * histogram peaks and thier locations. * The peaks array holds the peak value * in the first place and its location in * the second place. *******************************************/ insert_into_peaks(peaks, max, max_place) int max, max_place, peaks[PEAKS][2]; } int i, j; /* first case */ if(max > peaks[0][0]){ for(i=PEAKS-1; i>0; i--){ peaks[i][0] = peaks[i-1][0]; peaks[il[1] = peaks[i-1[1]; } peaks[0] [0] = max; peaks[0][1] = max_place; } /* ends if */ /* middle cases */ for(j=0; j peaks[j+1][0]){ for(i:PEAKS-1; i>j+1; i--){ peaks[i][0] = peaks[i-1][0]; peaks[i][1] = peaks[i-1][1]; } peaks[j+1][0] = max; peaks[j+1][1] = max_place; } /* ends if */ } /* ends loop over j */ /* last case */ if(max < peaks[PEAKS-2][0] && max > peaks[PEAKS-1] [0]){ peaks[PEAKS-1][0] = max; peaks[PEAKS-1][0] = max_place; } /* ends if */ } /* ends insert into_peaks */ /*********************************************** * peaks_high_low(... * This function uses the histogram array * and the peaks to find the best high and * low threshold values for the threshold * function. You want the hi and low values * so that you will threshold the image around * the smaller of the two "humps" in the * histogram. This is because the smaller * hump represents the objects while the * larger hump represents the background. ***********************************************/ peaks_high_low(histogram, peak1, peak2, hi, low) int peak1, peak2; short *hi, *low; unsigned long histogram[]; { int i, mid_point; unsigned long sum1 = 0, sum2 = 0; if(peak1 > peak2) mid_point = ((peak1 - peak2)/2) + peak2; if(peak1 < peak2) mid_point = ((peak2 - peak1)/2) + peak1; for(i=0; i= sum2){ *low = midpoint; *hi = GRAY_LEVELS; } else{ *low = 0; *hi = mid_point; } } /* ends peaks_high_low */ /* End of File */ Listing 6 Code implementing the histogram valley technique /********************************************* * valley_threshold_segmentation(... * This function segments an image using * thresholdlng. It uses the histogram valleys * to find the hi and low values of the * threshold. * If the segment parameter is 0, you only * threshold the array - you do not segment. ***********************************************/ valley_threshold_segmentation(in_name, out_name, the_image, out_image, il, ie, ll, le, value, segment) char in_name[], out_name[]; int il, ie, ll, ie, segment; short the_image[ROWS] [COLS], out_image[ROWS][COLS], value; { int length, peak1, peak2, width; short hi, low; struct tiff_header_struct image_header; unsigned long-histogram[GRAY_LEVELS+1]; if(does_not_exist(out_name)){ printf{"\n\nVTS> output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, ie); zero_histogram(histogram); calculate_histogram(the_image, histogram); smooth_histogram(histogram); find_peaks(histogram, &peak1, &peak2); valley_high_low(histogram, peak1, peak2, &hi, &low); threshold_image_array(the_image, out_image, hi, low, value); if(segment == 1) grow(out_image, value); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends valley_threshold_segmentation */ /****************************************** * valley_high_low(... * This function uses the histogram array * and the valleys to find the best high and * low threshold values for the threshold * function. You want the hi and low values * so that you will threshold the image around * the smaller of the two "humps" in the * histogram. This is because the smaller * hump represents the objects while the * larger hump represents the background. *******************************************/ valley_high_low(histogram, peak1, peak2, hi, low) int peak1, peak2; short *hi, *low; unsigned long histogram[]; { int i, valley_point; unsigned long sum1 = 0, sum2 = 0; find_valley_point(histogram, peak1, peak2, &valley_point); /*printf("\nVHL> valley point is %d", valley_point);*/ for(i=0; i= sum2){ *low = valley point; *hi = GRAY_LEVELS; } else{ *low = 0; *hi = valley_point; } } /* ends valley_high low */ /****************************************** * find_valley_point(... * This function finds the low point of * the valley between two peaks in a * histogram. It starts at the lowest * peak and works its way up to the * highest peak. Along the way, it looks * at each point in the histogram and inserts * them into a list of points. When done, * it has the location of the smallest histogram * point - that is the valley point. * The deltas array holds the delta value * in the first place and its location in * the second place. *******************************************/ find_valley_point(histogram, peak1, peak2, valley_point) int peak1, peak2, *valley_point; unsigned long histogram[]; { int deltas[PEAKS][2], delta_hist, i; for(i=0; i0; i--){ deltas[i][0] = deltas[i-1][[0]; deltas[i] [1] = deltas[i-1] [1]; } deltas[0] [0] = value; deltas[0][1] = place; } /* ends if */ /* middle cases */ for(j=0; j deltas[j][0] && value < deltas [j+1] [0] ) { for(i=PEAKS-1; i>j+1; i--){ deltas[i][0] = deltas[i-1][0]; deltas[i][1] = deltas[i-1][1]; } deltas[j+1][0] = value; deltas[j+1][1] = place; } /* ends if */ } /* ends loop over j */ /* last case */ if(value > deltas[PEAKS-2][0] && value < deltas[PEAKS-i][o]){ deltas[PEAKS-1][0] = value; deltas[PEAKS-1][1] = place; } /* ends if */ } /* ends insert_into_deltas */ /* End of File */ Listing 7 Code implementing the adaptive technique /************************************************ * * adaptive_threshold_segmentation(... * * This function segments an image using * thresholding. It uses two passes * to find the hi and low values of the * threshold. The first pass uses the peaks * of the histogram to find the hi and low * threshold values. It thresholds the image * using these hi lows and calculates the means * of the object and background. Then we use * these means as new peaks to calculate new * hi and low values. Finally, we threshold * the image again using these second hi low * hi low values. * * If the segment parameter is 0, you only * threshold the array - you do not segment. * **************************************************/ adaptive_threshold_segmentation(in_name, out_name, the_image, out_image, il, ie, ll, le, value, segment) char in_name[], out_name[]; int il, ie, ll, le, segment; short the_image[ROWS] [COLS], out_image[ROWS] [COLS], value; { int length, peak1, peak2, width; short background, hi, low, object; struct tiff_header_struct image_header; unsigned long-histogram[GRAY_LEVELS+1]; if(does_not exist(out_name)){ printf("\n\nATS> output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does not exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); zero_histogram(histogram); calculate_histogram(the_image, histogram); smooth_histogram(histogram); find_peaks(histogram, &peak1, &peak2); peaks_high_low(histogram, peak1, peak2, &hi, &low); threshold_and_find_means(the_image, out_image, hi, low, value, &object, &background); peaks_high_low(histogram, object, background, &hi, &low); threshold_image_array(the_image, out_image, hi, low, value); if(segment == 1) grow(out_image, value); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends adaptive_threshold_segmentation */ /************************************************ * * threshold_and_find_means(... * * This function thresholds an input image array * and produces a binary output image array. * If the pixel in the input array is between * the hi and low values, then it is set to value. * Otherwise, it is set to 0. * ************************************************/ threshold_and_find_means(in_image, out_image, hi, low, value, object_mean, background_mean) short *background_mean, hi, low, in_image[ROWS][COLS], *object_mean, out_image[ROWS][COLS], value; { int counter = 0, i, j; unsigned long object = 0, background = 0; for(i=0; i= low && in_image[i][j] <= hi){ out_image[i][j] = value; counter++; object = object + in_image[i][j]; } else{ out_image[i][j] : 0; background = background + in_image[i][j]; } } /* ends loop over j */ } /* ends loop over i */ object = object/counter; background = background/((ROWS*COLS)-counter); *object_mean = (short)(object); *background mean = (short)(background); printf("\n\tTAFM> set %d points", counter); printf("\n\tTAFM> object=%d background=%d", *object_mean, *background_mean); } /* ends threshold_and_find_means */ /* End of File */ Listing 8 main routine for the C Image Processing System revised to include histogram image segmentation case 16: /* image thresholding */ printf("\nCIPS> Enter input image name\n"); get_image_name(name); printf("\nCIPS> Enter output image name\n"); get_image_name(name2); get_parameters(&il, &ie, &ll, &le); get_threshold_options(ts_method, &hi, &low, &value); if (ts_method[0] == 'm' ts_method[0] == 'M') manual_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, hi, low, value, 0); if(ts_method[0] == 'p' ts_method[0] == 'P') peak_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, value, 0); if(ts_method[0] == 'v' ts_method[0] == 'V') valley_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, value, 0); if(ts_method[0] == 'a' ts_method[0] == 'a') adaptive_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, value, 0); break; case 17: /* image segmentation */ printf("\nCIPS> Enter input image name\n"); get_image_name(name); printf("\nCIPS> Enter output image name\n"); get_image_name(name2); get_parameters(&il, &ie, &ll, &le); get_segmentation_options(ts_method, &hi, &low, &value); if(ts_method[0] == 'm' (ts_method[0] == 'M') manual_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, hi, low, value, 1); if(ts_method[0] == 'p' ts_method[0] == 'P') peak_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, value, 1); if(ts_method[0] == 'v' ts_method[0] == 'V') valley_threshold_segmentation(name, name2, the_image, out_image, il, ie, ll, le, value, 1); if(ts_method[0] == 'a' ts_method[0] == 'a') adaptive_threshold_segmentation(name, name2, the_image. out_image, il, ie, ll, le, value, 1); break; . . . /**************************************************** * show_menu(.. * This function displays the CIPS main menu. *****************************************************/ show_menu() { printf("\n\nWelcome to CIPS"); printf("\nThe C Image Processing System"); printf("\nThese are you choices:"); printf("\n\t1. Display image header"); printf("\n\t2. Show image numbers"); printf("\n\t3. Print image numbers"); printf("\n\t4. Display image (VGA & EGA only)"); printf("\n\t5. Display or print image using halftoning"); printf("\n\t6. Print graphics image using dithering"); printf("\n\t7. Print or display histogram numbers"); printf("\n\t8. Perform edge detection"); printf("\n\t9. Perform edge enhancement"); printf("\n\t10. Perform image filtering"); printf("\n\t11. Perform image addition and subtraction"); printf("\n\t12. Perform image cutting and pasting"); printf("\n\t13. Perform image rotation and flipping"); printf("\n\t14. Perform image scaling"); printf("\n\t15. Create a blank image"); printf("\n\t16. Perform image thresholding"); printf("\n\t17. Perform image segmentation"); printf("\n\t20. Exit system"); printf("\n\nEnter choice _\b"); } /* ends show_menu */ . . . /****************************************** * get_segmentation_options(... * This function interacts with the user * to obtain the options for image * segmentation. *******************************************/ get_segmentation_options(method, hi, low, value) char method[]; short *hi, *low, *value; { int i, not_finished = 1, response; while(not_finished){ printf("\n\nThe image segmentation options are:\n"); printf("\n\t1. Method is %s", method); printf("\n\t (options are manual peaks"); printf( " valleys adapative)"); printf("\n\t2. Value is %d", *value); printf("\n\t3. Hi is %d", *hi); printf("\n\t4. Low is %d", *low); printf("\n\t Hi and Low needed only for"); printf( "manual method"); printf("\n\nEnter choice (0 = no change):_\b"); get_integer(&response); if(response == 0) not_finished == 0; if(response == 1){ printf("\nEnter method (options are:"); printf(" manual peaks valleys adaptive)\n\t"); read_string(method); } if(response == 2){ printf("\nEnter value: __\b\b\b"); get_short(value); } if(response == 3){ printf("\nEnter hi: __\b\b\b"); get_short(hi); } if(response == 4){ printf("\nEnter low: __\b\b\b"); get_short(low); } } /* ends while not_finished */ } /* ends get_segmentation_options */ /********************************************* * get_threshold_options{... * This function interacts with the user * to obtain the options for image * threshold. **********************************************/ get_threshold_options(method, hi, low, value) char method[]; short *hi, *low, *value; { int i, not_finished = 1, response; while(not_finished){ printf("\n\nThe image threshold options are:\n"); printf("\n\tl. Method is %s", method); printf("\n\t (options are manual peaks"); printf( "valleys adapative)"); printf("\n\t2. Value is %d", *value); printf("\n\t3. Hi is %d", *hi); printf("\n\t4. Low is %d", *low); printf("\n\t Hi and Low needed only for"); printf( " manual method"); printf("\n\nEnter choice (0 = no change):_\b"); get_integer(&response); if(response == 0) not_finished = 0; if(response == 1){ printf("\nEnter method (options are:"); printf(" manual peaks valleys adaptive)\n\t"); read_string(method); } if(response == 2){ printf("\nEnter value: __\b\b\b"); get_short(value); } if(response == 3){ printf("\nEnter hi: __\b\b\b"); get_short(hi); } if(response == 4){ printf("\nEnter low: __\b\b\b"); get_short(low); } } /* ends while not_finished */ } /* ends get_threshold_options */ /* End of File */ Listing 9 Application for thresholding and segmenting an image file /*************************************** * * file d:\cips\mainseg.c * * Functions: This file contains * main * * Purpose: * This file contains the main calling * routine in an edge detection program. * * External Calls: * gin.c - get_image_name * numcvrt.c - get_integer * int_convert * tiff.c - read_tiff_header * enhance_edges * hist.c - zero_histogram * smooth_histogram * show_histogram * calculate_histogram * segment.c - threshold_image_array * grow * find_peaks * peaks_high_low * valley_high_low * threshold_and_find_means * * Modifications: * 27 September 1992 - created * ****************************************/ #include "cips.h" short the_image[ROWS][COLS]; short out_image[ROWS][COLS]; unsigned long histogram[GRAY_LEVELS+1]; main(argc, argv) int argc; char *argv[]; { char name[80], name2[80], response[80]; int count, i, ie, i1, j, k, le, length, ll, peakl, peak2, size, t, type, v, width; short background, hi, low, object, value; struct tiff_header_struct image_header; _setvideomode(_TEXTC80); /* MSC 6.0 statements */ _setbkcolor(1); _settextcolor(7); _clearscreen(_GCLEARSCREEN); if(argc < 7){ printf("\n\nmainseg in-file out-file hi low value operation"); printf("\n\t\toperation = threshold grow peaks valleys adaptive"); printf("\n"); exit(0); } strcpy(name, argv[1]); strcpy(name2, argv[2]); hi = atoi(argv[3]); low = atoi(argv[4]); value = atoi(argv[5]); il = 1; ie = 1; ll = ROWS+1; le = COLS+1; read_tiff_header(name, &image_header); length = (90 + image_header.image_length)/ROWS; width = (90 +image_header.image_width)/COLS; count = 1; printf("\nlength=%d width=%d", length, width); if(does_not_exist(name2)){ read_tiff_header(name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header;image_width = width*COLS; create_allocate_tiff_file(name2, &image_header, out_image); } /* ends if does_not_exist */ zero_histogram(histogram); /********************************** * * Manual Threshold operation * ***********************************/ if(argv[6][0] == 't'){ for(i=0; i Calling threshold"); threshold_image_array(the_image, out_image, hi, low, value); write_array_into_tiff_image(name2, out_image, il +i*ROWS, ie+j*COLS, 11+i*ROWS, le+j*COLS); } /* ends loop over i */ } /* ends loop over j */ } /* ends if t */ /********************************** * * Grow region operation * **********************************/ if(argv[6][0] == 'g'){ for(i=0; i Calling grow"); grow(the_image, value); write_array_into_tiff_image(name2, the_image, il+i*ROWS, ie+j*COLS, 11+i*ROWS, le+j*COLS); } /* ends loop over i */ } /* ends loop over j */ } /* ends if g */ /********************************** * * Peak threshold operation * ***********************************/ if(argv[6][0] == 'P'){ /* calculate histogram for the entire image file */ zero_histogram(histogram); for(i=0; i Calling hist functions"); calculate_histogram(the_image, histogram); } /* ends loop over i */ } /* ends loop over j */ smooth_histogram(histogram); show_histogram(histogram); find_peaks(histogram, &peak1, &peak2); printf("\npeakl=%d peak2=%d", peak1, peak2); peaks_high_low(histogram, peak1, peak2, &hi, &low); printf("\nhi=%d low=%d", hi, low); /* now read the image file again and threshold and grow objects. */ count = 1; for(i=0; i Calling hist functions"); calculate_histogram(the_image, histogram); } /* ends loop over i */ } /* ends loop over j */ smooth_histogram(histogram); show_histogram(histogram); find_peaks(histogram, &peak1, &peak2); printf("\npeakl=%d peak2=%d", peak1, peak2); valley_high_low(histogram, peak1, peak2, &hi, &low); printf("\nhi=%d low=%d", hi, low); /* now read the image file again and threshold and grow objects. */ count = 1; for(i=0; i Calling hist functions"); calculate_histogram(the_image, histogram); } /* ends loop over i */ } /* ends loop over j */ /* find the peaks for the entire image file. */ smooth_histogram(histogram); show_histogram(histogram); find_peaks(histogram, &peakl, &peak2); printf("\npeakl=%d peak2=%d", peak1, peak2); peaks_high_low(histogram, peak1, peak2, &hi, &low); printf("\nhi=%d low=%d", hi, low); /* Second Pass */ count = 1; for(i=0; i char *ap[] = { "INTEGER", "PROPORTION", "DEBUGGER", "PORTABLE", "TOWER!" }; char **app[] = { ap + 4, ap + 3, ap + 2, ap + 1, ap }; char ***ppp = app; char ****pppp = &ppp; void main() { printf("%.*s", 2, *--**pppp); printf("%.*s", 3, *(++*pppp[0] - 4)); printf("%s " , ++*--*++pppp[0] + 5); printf("%.*s", 2, *++pppp[0] [3] + 3); printf("%s\n", (*pppp + 2)[-2][2] + 2); } /* End of File */ Solving Linear Equations Using C Matt Weisfeld Matt Weisfeld is currently employed by the Allen-Bradley Company in Highland Heights, Ohio. He is responsible for the design and development of test software on VAX/VMS, UNIX, DOS and other platforms. Matt is currently working on a book entitled Building and Testing Portable Libraries in C to be published by QED later this year. He can be reached on Compuserve at [71620,2171]. Resource allocation, a fundamental issue in any discipline, involves maximizing value and minimizing costs. As early as Junior High School, students solve simultaneous equations to find where these two variables break even. However, solving simultaneous equations by hand works only when the number of unknowns is small. When the problem involves dozens of equations and variables, pencil and paper just aren't enough. Linear Programming (LP) is used to solve these large resource-allocation problems. This article presents a program for solving linear equations using C. Sample Application To illustrate how Linear Programming works in a real life application, consider the following example. The Acme soft drink company markets two different lines of cola: diet and regular. Operating at maximum daily capacity, Acme can produce four tons of diet and six tons of regular. Most ingredients (such as water and sweetener) are obtained upon demand, so they do not affect potential output. However, the secret ingredient that makes Acme cola taste so special is limited to 24 pounds a day. Each ton of diet needs three pounds of the secret ingredient while each ton of regular needs four pounds. Finally, each ton of diet sells for $200,000, while each ton of regular sells for $600,000. The Acme company must maximize income by producing the optimal amount of diet and regular cola. The standard LP problem consists of variables, an objective, and constraints. The variables in this example represent the two types of cola, diet (x1) and regular (x2). The goal is to allocate these variables in a fashion that maximizes the company's income (z). The objective function can be written z = 2x1 + 6x2 (in units of $100,000) To complete the LP model three constraints must be factored in: x1 œ 4 (max diet in tons) x2 œ 6 (max regular in tons) 3x1 + 4x2 œ 24 (in pounds) The final problem is: Maximize z = 2x1 + 6x2 (objective function) such that x1 œ 4 (constraint 1) x2 œ 6 (constraint 2) 3x1 + 4x2 œ 24 (constraint 3) x1, x2 >=0 (it is impossible to have a negative amount of cola) The Graphical Solution Since this problem has only two variables, you can depict the solution graphically. Understanding the problem in terms of a graph makes the transition to the algorithmic solution much easier. However, once a third variable (and thus a third dimension) enters the picture, graphing becomes quite a challenge. Figure 1 shows the graph of the Acme problem with only the constraints plotted. Notice the area contained within the constraint lines and the x1 and x2 axis. This area is called the Feasible Solution Space. Any point within this area will yield a combination of diet and regular cola that can be produced and yield some income. However, the goal is to find the best solution, not simply a feasible one. Notice also that there are five edges that surround the Feasible Solution Space. An edge is where two lines meet to form a single point. The optimal solution will be found at one of these edges. To find the optimal solution, you need the objective function. Figure 2 shows two possible solutions based on a different value for z. Since the ratio of x1 to x2 is known, the slope of the line can be calculated. By varying the position of the line (the slope is always the same so the lines must move in a parallel manner), different solutions to this problem can be explored. The values of x1 and x2 are obtained by examining the points where the objective function intersects the lines bounding the feasible solution space. Line 1 intersects the feasible solution space at edge b (point x1=0, x2=6: z=36). Notice that as the line moves away from the origin, the value of z increases. Also remember that the objective line must intersect the feasible solution space for a valid solution to exist. Looking at the graph, it is apparent that the largest value of z that intersects the Feasible Solution Space is at edge c (point x1=1.5, x2=6, z=39). An optimal solution not falling on an edge represents a situation where the objective function is parallel to one of the constraints and more than one (actually infinite) optimal solution exists. The Standard Form of the LP Model In order to have a mathematical algorithm that can solve general LP models as well as lend itself to a computer implementation, all LP problems must follow the Standard Form of the LP model. The Standard Form includes these features: All constraints are equations with a non-negative Right Hand Side (RHS). If a RHS is negative, simply multiple both sides by --1. All variables are non-negative. In this case there is no choice. There is no way to produce a negative amount of diet cola. The objective function can be maximized or minimized. In the sample problem you are maximizing income. However, the problem could be designed to minimize costs (this will be discussed later). To satisfy the first rule the constraints must be converted to equations. This may seem confusing, but in their present form they are not now equations. The operator used in all the constraints is œ. To make the constraints conform to the Standard Form, equal signs must separate the left-hand-side from the right-hand-side. However, simply replacing the œ with an = is inappropriate since it actually alters the constraint (eg: iœ1 is obviously not the same as i=1). To properly convert the constraint to an equation, a slack variable must be added. The slack variable accounts for the fact that the œ represents a wide range of values and permits the use of the equality. Even though the slack variables are necessary in the conversion of the constraints, they do not contribute to the value of the objective function. Only x1 and x2 affect the income. The slack variables are created to assist in the computations, they do not represent real unknowns. To illustrate this fact, the objective function multiplier for each slack variable is 0. Thus the Acme Cola problem, represented in the Standard Form, is: Maximize z = 2x1 + 6x2 + 0s1 + 0s2 + 0s3 (objective function) such that x1 + s1 = 4 (constraint 1) x2 +s2 = 6 (constraint 2) 3x1 + 4x2 s3 = 24 (constraint 3) x1, x2 >= 0 (it is impossible to have a negative amount of cola) In this case, the slack variables are all positive because all the constraints are of the œ type. If the constraints were >=, the slack variables would be negative. Figure 3 shows how the slack variables are represented graphically. There is a direct relationship between the number of graph sides (5) and the number of variables (x1, x2, s1, s2, s3). The Starting Solution To begin solving the Acme problem you must identify an initial, feasible starting solution. Since the optimal solution must be an edge, this problem has five possible candidates (as can be seen on the graph). Normally, you would begin at the origin when a problem is bounded by the x1 and x2 axis (where x1, x2 > 0), since this is both a feasible solution and an edge. The concept of the algorithm (presented later) is to move from the current edge to the next adjacent edge on a path to the optimal solution. The direction to move is determined by the algorithm. Thus, from the origin (edge a in Figure 2), the algorithm moves to edge b and then stops at edge c, the optimal solution. The algorithm must follow adjacent edges (that is, the solution cannot move directly from edge a to edge c). The Initial Table A table format is used to group all the information needed to solve the problem. The C data structures mimic this table. Table 1 contains the initial table. The top row of the table is simply a header. The second row represents the objective function. The remaining number of rows are variable and depend on the number of constraints. The first column identifies the variables that are currently in the basic solution. As mentioned before, the starting edge is at the origin (x1 and x2 are zero). This leaves the slack variables (s1, s2, s3) as the variables in the starting solution, which is called the basis. Notice that the rows representing the slack variables form an identity matrix (ones down the horizontal). Note, if some of the constraints had been >=, the corresponding slack variables would be negative and thus not a candidate for entry into the basis. In this instance, the algorithm presented would need to be modified to construct a proper basis. Finally, the column at the far right represents the current values for the variables in the basis. The C Data Structures Since the user interface is not a topic of this discussion, all data structures are defined internally. However, in any practical application the input data should not be hard-coded. Listing 1 contains all definitions. The C program includes three arrays that hold all the data necessary to represent a table. The arrays basis and objective hold the names (labels) that represent the variables. The actual computations are performed on the array called table. The dimensions of the table are always rows by columns. In the example table, the dimensions are four rows by seven columns. The only other information needed are the number of variables and constraints, which in this example are two and three respectively. The Simplex Method The algorithm used to solve this problem is called the simplex method. With this method you determine if there are any non-basic variables that, if in the basic solution, would enhance the final result. Based on the initial table, the basis variables are s1, s2, and s3. This is a feasible solution, however, since the variables x1 and x2 are not in the basis, the result is to produce nothing (z=0). To identify any variables that should enter the basis, inspect the objective function. The objective function is z x1 x2 s1 s2 s3 sol --------------------------- 1 -2 -6 0 0 0 0 Any variable that is negative (in a maximization problem) will increase the value of z, and so should enter the basis. If there is more than one negative value, the most negative one is chosen. Thus, there are two candidates for the entering value (--2 and --6). In this case, x2 should enter the basis. It is obvious that if one variable enters the basis, another currently in the basis must leave. Since s1, s2, and s3 are in the basis, one must leave if x2 is to enter. Determining the leaving variable is done by inspecting the numbers in the column under entering variable, x2. They are x2 ----- s1 0.0 s2 1.0 s3 3.0 All numbers that are not positive are not considered, thus s1 cannot leave the basis at this time. To determine which candidate actually leaves the basis, the ratios between the values in the entering column and the values in the solution are taken. For example, since there are two possible candidates to leave the basis, the ratios considered are as follows: sol x2 ratio ----- ----- ----- s2 : 6 / 1 = 6 s3 : 24 / 3 = 8 The lowest value is chosen to leave the basis, thus s2 receives the honor. This makes the value 1.00, at the intersection of column x1 and row s2, the pivot element. Now that the entering and leaving variables have been identified, the next iteration of the table is calculated by using the Gauss-Jordon method. Looking at the table, it is evident that for x2 to become part of the basis, it must replace s2 as part of the identity matrix. To do this the pivot element must be a 1. Thus, all values in the leaving equation are divided by the pivot element. In this instance, the value of the pivot element is already a 1 so the division does not change anything. The row that was s2 and is now replaced by x2 is called the pivot equation. As mentioned above, placing x2 in the basis means that it must replace s2 as part of the identity matrix. Since the pivot element is now a 1, all the other values in the x2 column must be 0. To do this the transformation new equation = old equation - (entering column coefficient new pivot equation) is applied to all elements of all the rows (including the objective function) other than the pivot equation. Thus the objective function transformation is old :1 -2 -6 0 0 0 0 -(-6)0 -(6)(0) -(6)(1) -(6-(0) -(6)(1) -(6)(0) -(6)(6) ------------------------------------------------------------------- new :1 -2 0 0 6 0 36 The s1 equation is unchanged due to the fact that the entering column coefficient is already zero. The s3 equation transformation is as follows: old :0 4 3 0 0 1 24 -(3)0 -(3)(0) -(3)(1) -(3)(0) -(3)(1) -(3)(0) -(3)(6) ------------------------------------------------------------------ new :0 4 0 0 -3 1 6 The first iteration is complete and the second table is presented in Table 2. The important information that can be gathered from inspecting this table is in the solution column. The 36 in the solution column of the objective function indicates that with the current solution the ACME Co. can make $36.00 (in units of 100,000). To determine the product mix, examine the variables in the basis. In this case only x2 is in the basis. Its solution is 6 (if six units of x2 are produced, ACME will make $36.00). However, since the objective function still has a negative value, the optimum solution has not yet been reached. The variable x1, which has a value of -2, is the next to enter the basis. The ratios are: s1: 4/1 = 4 s3: 6/4 = 1.5 Thus s3 is the leaving variable. In this case the pivot element is not one, so all the elements of the pivot equation are divided by the pivot element, which is four. The new pivot equation is as follows: 0 1 0 0 -0.75 0.25 1.5 Applying the Gauss-Jordon technique yields the table in Table 3. The first thing to notice is that there are no negative values in the objective function. This means that the optimal solution has been found (z=39). Also notice that both variables, x1 and x2, are in the basis. Thus, to maximize profits, ACME should produce 1.5 units of diet and 6 units of regular. The C Routines The amount of C code necessary to perform the table operations is surprisingly small. Listing 2 contains the code for the mainline which loops until there are no more entering variables, printing each iteration of the table. The entering variable (enter_pos) and leaving variable (leave_pos) are determined by the code in Listing 3. If enter_pos is 0 then the optimal solution has already been obtained and the loop terminates. The location of the pivot element is placed in the variable pivot_element. The routines new_pivot and new_equation are presented in Listing 4. new_pivot calculates the next pivot equation by dividing all elements in the pivot row by the pivot element, while new_equation performs the Gauss-Jordon algorithm on the entire table and completes the current iteration. To help illustrate the program's performance, and to aid in debugging, two additional routines are included. The first, build_basis, labels all the rows and columns for better readability when the table is printed. The second, print_table, simply prints the contents of the table so that the information it contains can be inspected. These routines are presented in Listing 5. All of the listing files can be combined into one program file for compilation and execution. Conclusion Linear Programming is a powerful tool when attempting to allocate resources. The example used in this article presents a solution for one specific case, a maximization problem with all constraints of the œ variety. As stated before, there are a number of variations to the standard LP model. In practice, there can be a mixture of constraints (>=, œ, or = ) and the final goal may be to minimize the result (as in costs). When the constraints are mixed, extensions to the simplex algorithm are required. Furthermore, the results obtained from the final table are anything but final. All the numbers in the table represent useful information. (The act of gathering this information is called sensitivity analysis.) References Hamdy A. Taha. 1982. Operations Research, An Introduction, 3rd ed. New York, NY: MacMillan Publishing Company, Inc. Hillier, Frederick S. and Gerald J. Lieberman. 1980. Introduction to Operations Research, 3rd ed. San Francisco, CA: Holden-Day Inc. Figure 1 Feasible solution space for ACME problem Figure 2 Two possible solutions based on a different value for z Figure 3 Graphical representation of the slack variables Table 1 Original table basis z x1 x2 s1 s2 s3 sol -------------------------------------------------- z 1.00 -2.00 -6.00 0.00 0.00 0.00 0.00 s1 0.00 1.00 0.00 1.00 0.00 0.00 4.00 s2 0.00 0.00 1.00 0.00 1.00 0.00 6.00 s3 0.00 4.00 3.00 0.00 0.00 1.00 24.00 Table 2 Iteration 1 basis z x1 x2 s1 s2 s3 sol -------------------------------------------------- z 1.00 -2.00 0.00 0.00 6.00 0.00 36.00 s1 0.00 1.00 0.00 1.00 0.00 0.00 4.00 x2 0.00 0.00 1.00 0.00 1.00 0.00 6.00 s3 0.00 4.00 0.00 0.00 -3.00 1.00 6.00 Table 3 Iteration 2 basis z x1 x2 s1 s2 s3 sol -------------------------------------------------- z 1.00 0.00 0.00 0.00 4.50 0.50 39.00 s1 0.00 0.00 0.00 1.00 0.75 -0.25 2.50 x2 0.00 0.00 1.00 0.00 1.00 0.00 6.00 x1 0.00 1.00 0.00 0.00 -0.75 0.25 1.50 Listing 1 Function definitions #include #define LABELSIZE 10 /* define matrix parameters */ #define ROWS 4 #define COLUMNS 7 #define VARIABLES 2 #define EQUATIONS 3 /* initialize matrix */ float table[ROWS][COLUMNS] = { 1,-2,-6,0,0,0,0, 0, 1, 0,1,0,0,4, 0, 0, 1,0,1,0,6, 0, 4, 3,0,0,1,24, }; /* char strings to hold labels */ char basis[LABELSIZE][LABELSIZE]; char objective[LABELSIZE][LABELSIZE]; /* used to build labels */ char var[LABELSIZE]; char num[LABELSIZE]; /* save info for leaving & entering var */ int leave_pos; float leave_holder; int enter_pos; float enter_holder; /* save info for pivot element */ float pivot_element; /* count # of iterations */ int pass; /* prototypes */ int select_entering(void); int select_leaving(void); void new_pivot(void); void new_equation(void); void build_basis(void); void print_table(void); /* End of File */ Listing 2 Main loop of the LP program main(int argc, char **argv) { int i,j; /* loop control */ char temp[10]; /* workspace */ /* build the initial tableau */ build_basis(); printf ("**** ORIGINAL TABLE\n"); print_table(); pass = 1; /* select_entering will return a 0 when there are no more entering VARIABLES, otherwise, the location of the entering variable is returned */ while (enter_pos = select_entering()) { /* return pos for leaving variable */ leave_pos = select_leaving(); /* calculate the pivot element */ pivot_element = table[leave_pos][enter_pos]; /* calculate the new pivot equation */ new_pivot(); /* calculate all the non-pivot EQUATIONS */ new_equation(); /* label the new basis */ strcpy (basis[leave_pos], objective[enter_pos]); printf ("\n**** ITERATION %d\n\n", pass); pass++; print_table(); } } /* End of File */ Listing 3 Code to determine entering and leaving variables int select_entering(void) { int i,j; enter_pos = 0; enter_holder = 0.0; /* determine the most neg value, if any */ for (j=1; j 0) { ratio[i] = table[i][COLUMNS-1]/table[i][enter_pos]; if (ratio[i] < leave_holder) { leave_holder = ratio[i]; leave_pos = i; } } } return (leave_pos); } /* End of File */ Listing 4 Code for finding pivot row and completing current iteration void new_pivot(void) { int i,j; /* calculate the new pivot equation */ for (j=0; j," CUJ January 1993.) With just those facilities, you can obtain time stamps and measure small and large time intervals. But that's not the end of it. The Standard C library also lets you represent times in terms of their more familiar clock and calendar components (type struct tm). Moreover, the component representation can be in terms of either local time or UTC (formerly GMT). It can even keep track of whether Daylight Savings Time is in effect. My goal this month is to lead you through the code that converts between scalar and structured times. If you've ever worked with calendar computations, you know that this is hard code to write and debug. That makes it rather hard to understand as well. Be prepared to take an occasional deep breath. What the C Standard Says 7.12.3 Time conversion functions Except for the strftime function, these functions return values in one of two static objects: a broken-down time structure and an array of char. Execution of any of the functions may overwrite the information returned in either of these objects by any of the other functions. The implementation shall behave as if no other library functions call these functions. 7.12.2.3 The mktime function Synopsis #include time_t mktime (struct tm *timeptr); Description The mktime function converts the broken-down time, expressed as local time, in the structure pointed to by timeptr into a calendar time value with the same encoding as that of the values returned by the time function. The original values of the tm_wday and tm_yday components of the structure are ignored, and the original values of the other components are not restricted to the ranges indicated above.139 On successful completion, the values of the tm_wday and tm_yday components of the structure are set appropriately, and the other components are set to represent the specified calendar time, but with their values forced to the ranges indicated above; the final value of tm_mday is not set until tm_mon and tm_year are determined. Returns The mktime function returns the specified calendar time encoded as a value of type time_t. If the calendar time cannot be represented, the function returns the value (time_t)-1. Example What day of the week is July 4, 2001? #include #include static const cha[ *const wday[] = { "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "-unknown-" }; struct tm time_str; /*...*/ time_str.tm_year = 2001 - 1900; time_str.tm_mon = 7 - 1; time_str.tm_mday = 4; time_str.tm_hour = 0; time_str.tm_min = 0; time_str.tm_sec = 1; time_str.tm_isdst = -1; if (mktime(&time_str) == -1) time_str. tm_wday = 7; printf("%s\n", wday [time_str.tm_wday]); .... 7.12.3.3 The gmtime function Synopsis #include struct tm *gmtime (const time_t *timer); Description The gmtime function converts the calendar time pointed to by timer into a broken-down time, expressed as Coordinated Universal Time (UTC). Returns The gmtime function returns a pointer to that object, or a null pointer if UTC is not available. 7.12.3.4 The local time function Synopsis #include struct tm *localtime (const time_t *timer); Description The local time function converts the calendar time pointed to by timer into a broken-down time, expressed as local time. Returns The localtime function returns a pointer to that object. Footnotes 139. Thus, a positive or zero value for tm_isdst causes the mktime function to presume initially that Daylight Saving Time, respectively, is or is not in effect for the specified time. A negative value causes it to attempt to determine whether Daylight Saving Time is in effect for the specified time. Using the Conversion Functions Note that the two functions that return a value of type pointer to struct tm return a pointer to a static data object of this type. Thus, a call to one of these functions can alter the value stored on behalf of an earlier call to another (or the same) function. Be careful to copy the value stored in this shared data object if you need the value beyond a conflicting function call. gmtime -- (The gm comes from GMT, which is now a slight misnomer.) Use this function to convert a calendar time to a broken-down UTC time. The member tm_isdst should be zero. If you want local time instead, use localtime, below. See the warning about shared data objects, above. localtime -- Use this function to convert a calendar time to a broken-down local time. The member tm_isdst should reflect whatever the system knows about Daylight Savings Time for that particular time and date. If you want UTC time instead, use gmtime, above. See the warning about shared data objects, above. mktime -- This function first puts its argument, a broken-down time, in canonical form. That lets you add seconds, for example, to the member tm_sec of a broken-down time. The function increases tm_min for every 60 seconds it subtracts from tm_sec until tm_sec is in the interval [0, 59]. The function then corrects tm_min in a similar way, then each coarser division of time through tm_year. It determines tm_wday and tm_yday from the other fields. Clearly, you can also alter a broken-down time by minutes, hours, days, months, or years just as easily. mktime then converts the broken-down time to an equivalent calendar time. It assumes the broken-down time represents a local time. If the member tm_isdst is less than zero, the function endeavors to determine whether Daylight Savings Time was in effect for that particular time and date. Otherwise, it honors the original state of the flag. Thus, the only reliable way to modify a calendar time is to convert it to a broken-down time by calling localtime, modify the appropriate members, then convert the result back to a calendar time by calling mktime. Implementing the Conversion Functions Listing 1 shows the file gmtime.c. The function gmtime is the simpler of the two functions that convert a calendar time in seconds (type time_t) to a broken-down time (type struct_tm). It simply calls the internal function _Ttotm. The first argument is a null pointer to tell _Ttotm to store the broken-down time in the communal static data object. The third argument is zero to insist that Daylight Savings Time is not in effect. Listing 2 shows the file xttotm.c. It defines the function _Ttotm that tackles the nasty business of converting seconds to years, months, days, and so forth. The file also defines the function _Daysto that _Ttotm and other functions use for calendar calculations. _Daysto counts the extra days beyond 365 per year. To do so, it must determine how may leap days have occurred between the year you specify and 1900. The function also counts the extra days from the start of the year to the month you specify. To do so, it must sometimes determine whether the current year is a leap year. The function recognizes that 1900 was not a leap year. It doesn't bother to correct for the non-leap years 1800 and earlier, or for 2100 and later. (Other problems arise within just a few decades of those extremes anyway.) _Daysto handles years before 1900 only because the function mktime can develop intermediate dates in that range and still yield a representable time_t value. (You can start with the year 2000, back up 2,000 months, and advance 2 billion seconds, for example.) The logic is carefully crafted to avoid integer overflow regardless of argument values. Also, the function counts excess days rather than total days so that it can cover a broader range of years without fear of having its result overflow. _Ttotm uses _Daysto to determine the year corresponding to its time argument secsarg. Since the inverse of _Daysto is a nuisance to write, _Ttotm guesses and iterates. At worst, it should have to back up one year to correct its guess. Both functions use the macro MONTAB, defined at the top of the file, to determine how many days precede the start of a given month. The macro also assumes that every fourth year is a leap year, except 1900. The isdst (third) argument to _Ttotm follows the convention for the isdst member of struct tm: If isdst is greater than zero, Daylight Savings Time is definitely in effect. _Ttotm assumes that its caller has made any necessary adjustment to the time argument secsarg. If isdst is zero, Daylight Savings Time is definitely not in effect. _Ttotm assumes that no adjustment is necessary to the time argument secsarg. If isdst is less than zero, the caller doesn't know whether Daylight Savings Time is in effect. _Ttotm should endeavor to find out. If the function determines that Daylight Savings Time is in effect, it advances the time by one hour (3,600 seconds) and recomputes the broken-down time. Thus, _Ttotm will loop at most once. It calls the function _Isdst only if it needs to determine whether to loop. Even then, it loops only if _Isdst concludes that Daylight Savings Time is in effect. Listing 3 shows the file xisdst.c. The function _Isdst determines the status of Daylight Savings Time (DST). _Times._Isdst points at a string that spells out the rules. (I'11 show the file asctime.c next month. It contains the definition of _Times.) _Isdst works with the rules in encoded form. Those rules are not current the first time you call the function or if a change of locale alters the last encoded version of the string _Times._Isdst. If that string is empty, _Isdst looks for rules appended to the time-zone information _Times._Tzone. It calls _Getzone as necessary to obtain the time-zone information. It calls _Gettime to locate the start of any rules for DST. The function _Getdst then encodes the current array of rules, if that is possible. Given an encoded array of rules, _Isdst scans the array for rules that cover the relevant year. It adjusts the day specified by the rule for any weekday constraint, then compares the rule time against the time that it is testing. Note that the first rule for a given starting year begins not in DST. Successive rules for the same year go in and out of DST. Listing 4 shows the file xgetdst.c. It defines the function _Getdst that parses the string pointed to by _Times._Isdst to construct the array of rules. The first character of a (non-empty) string serves as a field delimiter, just as with other strings that provide locale-specific time information. The function first counts these delimiters so that it can allocate the array. It then passes over the string once more to parse and check the individual fields. _Getdst calls the internal function getint to convert the integer subfields in a rule. No overflow checks occur because none of the fields can be large enough to cause overflow. The logic here and in _Getdst proper is tedious but straightforward. Local Time Listing 5 shows the file localtim.c. The function localtime calls _Ttotm much like gmtime. Here, however, localtime assumes that it must convert a UTC time to a local time. To do so, the function must determine the time difference, in seconds, between UTC and the local time zone. The file localtim.c also defines the function _Tzoff that endeavors to determine this time difference (tzoff, in minutes). The time difference is not current the first time you call the function or if a change of locale alters the last encoded version of the string _Times._Tzone. If that string is empty, _Tzoff calls the function _Getzone to determine the time difference from environment variables, if that is possible. However obtained, the string _Times._Tzone takes the form :EST:EDT:+0300. _TzoffS calls the function _Gettime to determine the starting position (p) and length (n) of the third field (#2, counting from zero). The function strtol, declared in must parse this field completely in converting it to an encoded integer. Moreover, the magnitude must not be completely insane. (The maximum magnitude is greater than 12*60 because funny time zones exist on either side of the International Date Line.) Listing 6 shows the file xgettime.c. It defines the function _Gettime that locates a field in a string that specifies locale-specific time information. See the description of _Getdst, above, for how _Gettime interprets field delimiters. If _Gettime cannot find the requested field, it returns a pointer to an empty string. Listing 7 shows the file xgetzone.c. The function_Getzone calls getenv, declared in , to determine the value of the environment variable "TIMEZONE". That value should have the same format as the locale-specific time string _Times. _Tzone, described above (possibly with rules for determining Daylight Savings Time bolted on). If no value exists for "TIMEZONE", the function _Getzone then looks for the environment variable "TZ". That value should match the UNIX format ESTO5EDT. The internal function reformat uses the value of "TZ" to develop the preferred form in its static buffer. If _Getzone finds neither of these environment variables, it assumes that the local time zone is UTC. In any event, it stores its decision in the static internal buffer tzone. Subsequent calls to the function return this remembered value. Thus, the environment variables are queried at most once, the first time that _Getzone is called. Converting to Scalar Time Listing 8 shows the file mktime.c. The function mktime computes an integer time_t from a broken-down time struct tm. It takes extreme pains to avoid overflow in doing so. (The function is obliged to return the value --1 if the time cannot be properly represented.) The first part of mktime determines a year and month. If they can be represented as type int, the function calls_Daysto to correct for leap days since 1900. mktime then accumulates the time in seconds as type double, to minimize further fretting about integer overflow. If the final value is representable as type time_t, the function converts it to that type. mktime calls _Ttotm to put the broken-down time in canonical form. Finally, the function corrects the time in seconds for Daylight Savings Time and converts it from local time to UTC. (The resultant code reads much easier than it wrote.) Conclusion Next month, I conclude my discussion of the time functions. I also conclude my guided tour of the Standard C library. The trip is almost over. This article is excerpted in part from P.J. Plauger, The Standard C Library, (Englewood Cliffs, N.J.: Prentice-Hall, 1992). Listing 1 gmtime.c /* gmtime function */ #include "xtime.h" struct tm *(gmtime)(const time_t *tod) { /* convert to Greenwich Mean Time (UTC) */ return (_Ttotm(NULL, *tod, 0)); } /* End of File */ Listing 2 xttotm.c /* _Ttotm and_Daysto functions */ #include "xtime.h" /* macros */ #define MONTAB(year) \ ((year) & 03 (year) == 0 ? mos : lmos) /* static data */ static const short lmos[] = {0, 31, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335}; static const short mos[] = {0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334}; int _Daysto(int year, int mon) { /* compute extra days to start of month */ int days; if (0 < year) /* correct for leap year: 1801-2099 */ days = (year - 1) / 4; else if (year <= -4) days = 1 + (4 - year) / 4; else days = 0; return (days + MONTAB(year)[mon]); } struct tm *_Ttotm(struct tm *t, time_t secsarg, int isdst) { /* convert scalar time to time structure */ int year; long days; time_t secs; static struct tm ts; secsarg += _TBIAS; if (t == NULL) t = &ts; t->tm_isdst = isdst; for (secs = secsarg; ; secs = secsarg + 3600) { /* loop to correct for DST */ days = secs / 86400; t->tm_wday = (days + WDAY) % 7; { /* determine year */ long i; for (year = days / 365; days < (i = _Daysto(year, 0) + 365L * year); ) --year; /* correct guess and recheck */ days -= i; t->tm_year = year; t->tm_yday = days; } { /* determine month */ int mon; const short *pm = MONTAB(year); for (mon = 12; days < pm[--mon]; ) ; t->tm_mon = mon; t->tm_mday = days - pm[mon] + 1; } secs %= 86400; t->tm_hour =secs / 3600; secs %= 3600; t->tm_min = secs / 60; t->tm_sec = secs % 60; if (0 <= t->tm_isdst (t->tm_isdst = _Isdst(t)) <= 0) return (t); /* loop only if <0 => 1 */ } } /* End of File */ Listing 3 xisdst.c /* _Isdst function */ #include #include "xtime.h" int _Isdst(const struct tm *t) { /* test whether Daylight Savings Time in effect */ Dstrule *pr; static const char *olddst = NULL; static Dstrule *rules = NULL; if (olddst != _Times._Isdst) { /* find current dst_rules */ if (_Times._Isdst[0] == '\0') { /* look beyond time zone info */ int n; if (_Times._Tzone[0] == '\0') _Times._Tzone = Getzone(); _Times._Isdst = _Gettime(_Times._Tzone, 3, &n); if (_Times._Isdst[0] != '\0') --_Times._Isdst; /* point to delimiter */ } if ((pr = _Getdst(_Times._Isdst)) == NULL) return (-1); free(rules); rules = pr; olddst = _Times._Isdst; } { /* check time against rules */ int ans = 0; const int d0 = _Daysto(t->tm_year, 0); const int hour = t->tm_hour + 24 * t->tm_yday; const int wd0 = (365L * t->tm_year + d0 + WDAY) % 7 + 14; for (pr = rules; pr->wday != (unsigned char)-1; ++pr) if (pr->year <= t->tm_year) { /* found early enough year */ int rday = _Daysto(t->tm_year, pr->mon) - d0 + pr->day; if (0 < pr->wday) { /* shift to specific weekday */ int wd = (rday + wd0 - pr->wday) % 7; rday += wd == 0 ? 0 : 7 - wd; if (pr->wday <= 7) rday -= 7; /* strictly before */ } if (hour < rday * 24 + pr->hour) return (ans); ans = pr->year == (pr + 1)->year ? !ans : 0; } return (ans); } } /* End of File */ Listing 4 xgetdst.c /* _Getdst function */ #include #include #include #include "xtime.h" static int getint(const char *s, int n) { /* accumulate digits */ int value; for (value = 0; 0 <= --n && isdigit(*s); ++s) value = value * 10 + *s - '0'; return (0 <= n ? -1 : value); } Dstrule *_Getdst(const char *s) { /* parse DST rules */ const char delim = *s++; Dstrule *pr, *rules; if (delim == '\0') return (NULL); { /* buy space for rules */ const char *s1, *s2; int i; for (s1 = s, i = 2; (s2 = strchr(s1, delim)) != NULL; ++i) s1 = s2 + 1; if ((rules = (Dstrule *)malloc(sizeof (Dstrule) * i)) == NULL) return (NULL); } { /* parse rules */ int year = 0; for (pr = rules; ; ++pr, ++s) { /* parse next rule */ if (*s == '(') { /* got a year qualifier */ year = getint(s + 1, 4) - 1900; if (year < 0 s[5] != ')') break; /* invalid year */ s += 6; } pr->year = year; pr->mon = getint(s, 2) - 1; s += 2; pr->day = getint(s, 2) - 1; s += 2; if (isdigit(*s)) { pr->hour = getint(s, 2); s += 2; } else pr->hour = 0; if (12 <= pr->mon 99 < pr->day 99 < pr->hour) break; /* invalid month, day, or hour */ if (*s != '+' && *s != '-') pr->wday = 0; else if (s[1] < '0' '6' < s[1]) break; /* invalid week day */ else { /* compute week day field */ pr->wday = s[1] == '0' ? 7 : s[1] - '0'; if (*s == '+') /* '-': strictly before */ pr->wday += 7; /* '+': on or after */ s += 2; } if (*s == '\0') { /* done, terminate list */ (pr + 1)->wday = (unsigned char)-1; (pr + 1)->year = year; return (rules); } else if (*s != delim) break; } free(rules); return (NULL); } } /* End of File */ Listing 5 localtim.c /* localtime function */ #include #include "xtime.h" time_t _Tzoff(void) { /* determine local time offset */ static const char *oldzone = NULL; static long tzoff = 0; static const long maxtz = 60*13; if (oldzone ! = _Times._Tzone) { /* determine time zone offset (East is +) */ const char *p, *pe; int n; if {_Times._Tzone[0] == '\0') _Times._Tzone = _Getzone(); p = _Gettime(_Times._Tzone, 2, &n); tzoff = strtol(p, (char **)&pe, 10); if (pe - p != n tzoff <= -maxtz maxtz <= tzoff) tzoff = 0; oldzone = _Times._Tzone; } return (-tzoff * 60); } struct tm *(localtime){const time_t *tod) { /* convert to local time structure */ return (_Ttotm(NULL, *tod + _Tzoff(), -1)); } /* End of File */ Listing 6 xgettime.c /* _Gettime function */ #include #include "xtime.h" const char *_Gettime(const char *s, int n, int *len) { /* get time info from environment */ const char delim = *s ? *s++ : '\0'; const char *s1; for (; ; --n, s = s1 + 1) { /* find end of current field */ if ((s1 = strchr(s, delim)) == NULL) s1 = s + strlen(s); if (n <= 0) { /* found proper field */ *len = s1 - s; return (s); } else if (*s1 == '\0') { /* not enough fields */ *len = 1; return (s1); } } } /* End of File */ Listing 7 xgetzone.c /*_Getzone function */ #include #include #include #include "xtime.h" /* static data */ static const char *defzone = ":UTC:UTC:0"; static char *tzone = NULL; static char *reformat(const char *s) { /* reformat TZ */ int i, val; static char tzbuf[] = ":EST:EDT:+0300"; for (i = 1; i <= 3; ++i) if (isalpha(*s)) { tzbuf[i] = *s; tzbuf[i + 4] = *s++; } else return (NULL); tzbuf[9] = *s == '-' *s == '+' ? *s++ : '+'; if (!isdigit(*s)) return (NULL); val = *s++ - '0'; if (isdigit(*s)) val = 10 * val + *s++ - '0'; for (val *= 60, i = 13; 10 <= i; --i, val /= 10) tzbuf[i] = val % 10 + '0'; if (isalpha(*s)) for (i = 5; i <= 7; ++i) if (isalpha(*s)) tzbuf[i] = *s++; else return (NULL); return (*s == '\0' ? tzbuf = NULL); } const char *_Getzone(void) { /* get time zone information */ const char *s; if (tzone) ; else if ((s = getenv("TIMEZONE")) != NULL) { /* copy desired format */ if ((tzone = (char *)malloc(strlen(s) + 1)) != NULL) strcpy(tzone, s); } else if ((s = getenv("TZ")) != NULL) tzone = reformat(s); if (tzone == NULL) tzone = (char *)defzone; return (tzone); } /* End of File */ Listing 8 mktime.c /* mktime function */ #include #include "xtime.h" time_t (mktime)(struct tm *t) { /* convert local time structure to scalar time */ double dsecs; int mon, year, ymon; time_t secs; ymon = t->tm_mon / 12; mon = t->tm_mon - ymon * 12; if (mon < 0) mon += 12, --ymon; if (ymon < 0 && t->tm_year < INT_MIN - ymon 0 < ymon && INT_MAX - ymon < t->tm_year) return ((time_t)(-1)); year = t->tm_year + ymon; dsecs = 86400.0 * (_Daysto(year, mon) - 1) + 31536000.0 * year + 86400.0 * t->tm_mday; dsecs += 3600.0 * t->tm hour + 60.0 * t->tm_min + (double)t->tm-sec; if (dsecs < 0.0 (double)(time_t)(-1) <= dsecs) return ((time_t)(-1)); secs = (time_t)dsecs - _TBIAS; _Ttotm(t, secs, t->tm_isdst); if (0 < t->tm_isdst) secs -= 3600; return (secs - _Tzoff()); } /* End of File */ On the Networks What Happened -- Again? Sydney S. Weinstein Sydney S. Weinstein, CDP, CCP is a consultant, columnist, author, and president of Datacomp Systems, Inc., a consulting and contract programming firm specializing in databases, data presentation and windowing, transaction processing, networking, testing and test suites, and device management for UNIX and MS-DOS. He can be contacted care of Datacomp Systems, Inc., 3837 Byron Road, Huntingdon Valley, PA 19006-2320 or via electronic mail on the Internet/Usenet mailbox syd@DSI.COM (dsinc!syd for those who cannot do Internet addressing). Once again, things have dried up in comp.sources.unix. But this time they have also dried up in comp.sources.x. Both groups have been extremely quiet, with no postings in comp.sources.unix. and only one in comp.sources.x. But what is unusual is that there hasn't been any word from the moderators of each group about any problems. Also different this time is that the other groups have also quieted down. This is unusual. Normally when one of the groups quiets down, the others overflow with the items that would have been posted to that group. This time even the catchall group, comp.sources.misc has been relatively quiet. I can only chalk this up to a combination of three things: Fewer new sources are being written for free distribution. The concept of FTP mirror sites and the vast number of people now able to access them has reduced the importance of having the software posted to a main stream USENET Network News group. CD-ROMs are being widely used as an alternative distribution medium. I can say that the last two are true about the Version 2.4 release of Elm. It was submitted to the moderators of comp.sources.unix two months ago as of this writing, and I haven't pressed as to why it hasn't been distributed yet because we now use a set of FTP mirror sites that include ones able to be accessed via UUCP. In addition we publish it on several of the CD-ROM collections. The single posting in comp.sources.x consisted of wscrawl from Brian Wilson . The word "wscrawl" stands for "window-scrawl." The user may think of wscrawl v2.0 as a paint program shared by any number of people at the same time. When wscrawl is run, it opens up a separate window on each participant's display. From that point onward, each participant sees the actions and ideas of every other participant as they occur. Each individual may simply watch, or participate at any moment. Any individual may exit out of the session at any time without affecting the other participants. wscrawl requires Motif 1.1 to compile but binaries for common architectures are available from 128.109.178.23 via anonymous ftp. Reviews Alive, Barely The only controlled newsgroup that had activity was comp.sources.reviewed, with updates to two packages and two patches. Mike Lijewski updated his Directory Editor, written in C++, to version 1.8. dired was posted as Volume 2, Issues 32-37. dired is a directory editor modeled after Dired Mode of GNU Emacs, but targeted for non-emacs users and designed for the UNIX environment. In addition to fixing bugs, and portability problems, about two pages worth of new features have been added. Version 1.7 of cextract from Adam Bryant was posted in Volume 2, Issues 38-43. cextract is a C prototype extractor. It is designed to generate header files for large multifile C programs, and will provide an automated method for generating all of the prototypes for all of the functions in such a program. It may also function as a rudimentary documentation extractor, generating a sorted list of all functions and their locations. Changes since the last posted version (1.2) include portability fixes, file name and line numbers in error messages, improved documentation, support for C++ // comments, proper support of the __STDC__ #if construct, and support for runtime selection of which C preprocessor to be used. On the patch front, ncompress received patch 1 in Volume 2, Issue 27. Peter Jannesen fixed the -c flag, a utime error, and added support for the AMIGA. ncompress is an enhancement to the USENET de facto standard file-compaction program to increase its speed. Volume 2, Issues 28-31 were patch 2 to mawk from Mike Brennan . This was a bug fix patch to this awk work-alike and also added new configuration support for AIX, Convex, and SVR4/386. Even misc is Updates Most of the postings in comp.sources.misc were also patches and updates to existing packages. Please don't tell me that everything good has already been written. The patches included patch 2 to oraperl v2 from Kevin Stock in Volume 32, Issue 93. oraperl is a set of extensions to perl to access Oracle databases. This patch handles null fields properly and a change to ora_ titles to allow truncating the titles to the data width or retrieving the full titles of the columns. ECU, which was highlighted in the last column, had patch 3 posted by Warren Tucker for Volume 32, Issue 96. Changes include addition of a -l flag to fkmap to load an alternate map from the library directory, changes to how seven-bit mode is reported and recognizing the ,M suffix in the UUCP Devices tables for all platform types. John F. Haugh II's shadow password suite, shadow, was updated to version 3.2.2 by patch 6 posted as Volume 32, Issues 98-100. The most significant change made for this patch is the addition of administrator-defined authentication mechanisms. This allows the system administrator to replace the standard encrypted password with a program which performs the authentication. Any user-written program may be used. This feature may be used to add any of a number of authentication schemes. Haugh has also started to revise the commands to work both with the shadow password and with the standard password file. The Mail Users Shell has been updated with patch 5 for version 7.2 by Bart Schaefer in Volume 32, Issues 101-103. New in this version is POP Support, MIME support, expansion of variables now more closely resembles how the shells handle it, and checking for the UNIX From_ line even if MMDF is being used. Of course there are also numerous bugs fixed. Chip Rosenthal issued a portability patch for SCO XENIX 2.3.3 users to his prtscrn2 screen capture program (for SCO UNIX and XENIX console screens). This very short patch fixes up the doc a bit and adds a missing define for that version of the OS. It appeared as Volume 33, Issue 29. The delete/undelete utilities for UNIX from Jonathan I. Kamens were updated to patchlevel 15 with Volume 33, Issue 70. This is a bug fix patch which corrects malloc problems, POSIX dirent support, AFT mount point problems, some memory allocation problems on nesting, and a bug in symlink support. Leaving the patches and turning to packages, yet another graphical directory tree program was contributed by Tom A. Baker for Volume 32, Issue 97. tbtree allows users to get a visual idea of where things are on their system. It was written on SunOS, and with a little work should port to other flavors of UNIX. A device driver for UNIX System V.3 to implement the poll and select system calls was submitted for Volume 32, Issue 105 by Bruce Momjian .poll and select are UNIX C library functions that allow programs to determine if a number of file descriptors are ready for reading or writing. In addition, his pol package includes a modified version of the public domain System V pty device driver written by Jens-Uwe Mager, with changes for System V by Michael Bloom. Guido Gronek had the need for a fast substring search routine. He created qsearch, an ANSI-C implementation that searches for the leftmost or rightmost occurrence of a pattern string in a text string. The algorithm used is "quick search" by D. M. Sunday, which is a simple but fast practical method. It's supposed to be even faster than the well known Boyer-Moore algorithm. (See Sunday's original paper, CACM 33.8, page 132 for several improvements of the basic method as well.) He implemented the reverse text scanning by a rather simple variation of the original algorithm. qsearch was posted as Volume 32, Issue 106. A newer derivative to an older vi (a UNIX text editor) clone package called stevie, was contributed by John Downey . Xvi was posted as Volume 33, Issues 10-27. While the name starts with X, there is as of yet no specific X-window version of the editor. Xvi is a portable multi-window version of vi that uses text windows separated by horizontal status lines on character-mode displays. The windows may represent different files being edited, or different views on to the same file. For those just learning vi, Wes Craig contributed vilearn for Volume 33, Issue 35. There are five short tutorials, each a text file intended to be edited with vi. The first, "Basic Editing," covers the handful of commands required to both navigate all five tutorials and do basic editing. The second tutorial, "Moving Efficiently," covers all of the cursor positioning commands. These are the commands used later as arguments to editing commands. Tutorial three, "Cutting and Pasting," introduces the first compound commands, numbering, and copy buffers. The "Inserting Techniques" tutorial continues the discussion of compound commands, while completing the list of insertion commands first discussed in tutorial one. The final tutorial, "Tricks and Timesavers," is less a tutorial than a description of common vi commands which don't fit correctly into normal vi logic. An interpreter for a superset of the ANSI Standard for Minimal BASIC (X3.60-1978) was contributed by Ted A. Campbell for Volume 33, Issues 37-47. bwbasic is implemented in ANSI C, and offers a simple interactive environment including some shell program facilities as an extension of BASIC. The interpreter has been compiled successfully on a range of ANSI C compilers on varying platforms with no alterations to source code necessary. An update to tarmail v2.3 was contributed for Volume 33, Issue 36 by Paul Lew . tarmail and untarmail provide a reliable way to send files through electronic mail systems. Large files will be divided into smaller chunks for transmission. Unlike the previous version of tarmail, the new tarmail will attach a CRC checksum on an individual chunk instead of on the entire file(s). The ability to retransmit only the faulty chunks make tarmail the idea tool for sending files by electronic mail. Brendan Kehoe has updated his archie client in Volume 33, Issues 50-56. archie is a system to locate programs stored in the various anonymous ftp archives on the Internet. This patch takes archie to 1.4.1. David F. Skoll has released an update to his remind package. Version 3.0.0 was contributed for Volume 33, Issues 58-69. remind is a sophisticated reminder program. It has a flexible and powerful script language, and allows you to easily specify reminders for most occasions including: particular dates (birthdays, etc.), holidays like Labor day, which occur on a particular weekday of a month, dates which repeat with an arbitrary period, meetings which are automatically moved in the event of holidays among others. remind also includes a feature to activate timed alarms in the background. remind should work on most UNIX systems, as well as MS-DOS. This allows you to use the same script on your UNIX and MS-DOS systems. A program to calculate dates in the Jewish calendar for a given gregorian year was contributed by Danny Sadinoff . hebcal v1.2, Volume 33, Issue 71, is fairly flexible in terms of which events in the Jewish calendar it displays. Each of the following can be individually turned on or off: the Hebrew date, Jewish holidays (including Yom Ha'atzmaut and Yom HaShoah etc.), the weekly Sedrah, the day of the week, and the days of the Omer. Mike Lijewski contributed problem v1.1, a database manager for bug reports and such, meant to be used in a UNIX environment. It is written in C++, uses the GNU Database Management Library (GDBM) for low-level database operations, and uses the termcap(3) library for screen control. The basic idea is to provide a central front-end for managing the various databases of bugs and miscreant behaviour that a large UNIX site might be interested in tracking, and facilitating the sharing of this information amongst all interested parties. Version 1.1 was posted in Volume 33, Issues 72-78. Twice, Twice The trend in comp.sources.games is to post things twice. It happened a bit this time, where the same game got posted and then updated with a newer version. A version of the popular game scrabble was contributed as scrabble2 by James A. Cherry for Volume 14, Issues 93-110 with patch1 in Volume 15, Issue 9. This version of scrabble is curses-based and uses a dictionary to allow play against one or more computer opponents. It only requires a text-based screen. A generic tetris game for X11R4/5 was submitted by Qiang Alex Zhao for Volume 15, Issues 5 and 6. gtetris2 is a generic version that uses no toolkit, only Xlib, and is highly portable. Thomas Grennefors contributed xminesweeper for Volume 15, Issue 3. It is a game where your task is to find the hidden mines in a minefield. Play is with the mouse and buttons control marking where the mines are, or marking safe squares. One wrong move and the game is over. Patch1 appeared in Volume 15, Issue 10. Xstratego is a X-window-based stratego interface for two players submitted by Henk-Jan Visscher for Volume 15, Issues 11-14. You can either play against another player (on the same or different host) or create a board setup for later use. It uses the Xaw toolkit. Two different curses-based versions of reversi were posted. The first, reversi, contributed by Elias Martensson was posted in Volume 15, Issues 7 and 8. Cursor movement is via the usual h, j, k, and l keys. The second, reversi2, was contributed, but not written, by Eric Safern and was posted in Volume 15, Issues 18 and 19. This version supports both human vs. computer and two human player mode. A restructuring of bt4, the broken throne multiplayer real-time conquest game was submitted by Tom Boutell in Volume 15, Issues 15-18. Added are some new features and support for AIX. Note, this game requires INET sockets (the BSD socket interface) as well as curses. Previews from alt. sources At least this hasn't been quiet. It sounds like only the moderated groups are being shunned. Here are the highlights of what I hope will reappear on the moderated source groups. Version 1.2 of uustatus, a real-time UUCP status monitor for BSD and System V was posted on July 17, 1992 by Ed Carp in one part. This simple program dynamically displays the status of your UUCP connections for you, without your having to cd into all of those pesky directories. It's also faster than uustat -m, and it's real-time! Bug fixes were posted in a one-part patch on August 5, 1992. A program to convert xwd (X-window dump) files into color or gray-scale PostScript was posted on August 30, 1992 in two parts by Brian Totty . Colors can be stored in an array to allow for mapping to be modified after the file is generated. Steve Cole contributed xtpanel, a program to build an interactive X program from the command line using a simple scripting language. It is not intended as a replacement for a full-featured interface-programming toolkit or as a replacement for a simple menu builder. It falls somewhere in the gap between the two. It is intended as an easy to use tool that can be used to add an interactive wrapper to all those old programs and shells that you have lying around. It was posted on September 3, 1992 in eight parts. The tin newsreader was updated with two patches by Iain Lea on September 14, 1992 in 15 parts, a complete report at patchlevel 6, and on November 15, 1992 in 10 parts, a patch to PL6 taking it to PL7. Many new features have been added to the threader news reader including CD-ROM support, INN support, support for many more OS types and of course more bug fixes. If you find your cursor on the X display is always in the way, unclutter, posted in one part on September 28, 1992 by Charles Hannum will help. unclutter is a program which runs permanently in the background of an X11 session. It checks on the X11 pointer (cursor) position every few seconds, and when it finds it has not moved (and no buttons are pressed on the mouse, and the cursor is not in the root window) it creates a small sub-window as a child of the window the cursor is in. The new window installs a cursor of size 1x1 but a mask of all 0, i.e. an invisible cursor. This allows you to see all the text in an xterm or xedit, for example. The human-factors crowd would agree it should make things less distracting. Once created, the program waits for the pointer to leave the window and then destroys it, restoring the original situation and thereby redisplaying the cursor. Last is the beta release of version 3.0 of Steven Grimm's workman v3.0. Posted on November 16, 1992 in six parts, its an X CD player program. It requires XView 3.0 or higher (XView source is available with the X11R5 distribution) and runs under both SunOS 4.x and Solaris 2. The major new feature is that tracks may now be split into sections at arbitrary locations. You may split a track while the CD is playing, useful for marking off particular sections of a song. Sections may be named and selected just like tracks. Of course, if the CD-ROM drive doesn't support data across the SCSI bus (and Sun's doesn't) you must use external speakers or a patch cable to take the analog sound from the CD-ROM. Questions & Answers Macros and Debugging Ken Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member on the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. Q It seems sort of ironic that the issue devoted to debugging contains an example of using macros to solve the "Fixed Field Files." (This arrangement produces a set of macro invocations.) The article, "Glass-Box Testing" hints at how to use debugging to break code and verify coverage. Unfortunately macros can't be debugged. After reviewing, understanding, and maintaining the code in various products, I've come to the conclusion that C programmers abuse the macros and the preprocessor in general. I am a big convert to C++ because the macro preprocessor hardly needs to be used. With true consts and inline functions there is no need to write nasty macros. The following code is written in C-style #define MAX_LINE 80 getline( char ** line ) { static char myline[MAX_LINE]; } Compare it to const int MAX_LINE=80; getline( char ** line ) { static char myline[MAX_LINE]; } which is written in C++-style. Now with a debugger you can print MAX_LINE without saying find /usr/project/x400 -name *.h -name *.c \ -exec grep '#define MAX_LINE' to search through 200 header files to find the correct macro definition. As for C++ I'm just praying that programmers will not go overboard and use overloaded functions in C++ too much. Because of the inheritance/dynamic binding we can have multiple overloaded function(s) in each class, and the superclass (base class) can have those same overloaded classes defined. I guess it would look something like Listing 1. What's SIZE? The problem gets worse as the program gets larger. I was working on debugging my project and reading the C Users Journal in between the compiles. I hope in the future programmers will recognize that in the world of "reuse" that readability must come before performance and space considerations. I know a programmer who does the following because he's too lazy to write the whole line: #define ptr a.get.line.alpha.beta ptr->data Now when I try and print ptr, the debugger says, ptr not defined. I guess you probably get the message. By the way. David Staker's book, C: Style Standard & Guidelines is quite informative. David A. Dennerline A I definitely agree with you that C++ can help a programmer write better code, if it is utilized properly. But basic design rules need to be followed, especially in using meaningful names. I cannot emphasize too much that when one writes a program, it should be written in English or in whatever native language code set is supported by one's compiler. Meaningful variable, member, parameter, and function names go a long way to writing maintainable code. Making too many assumptions about what the reader knows can result in programs that are difficult to understand. The member function names are particularly important in C++ as one is not supposed to look at the implementation to determine what the function is doing. Let me take your example functions, which were created for example purposes, rather than as actual code. Looking at the parameter names gives no clue as to what is really required. virtual int get_size( char * s ); virtual int get_size( int idx ); In the first case, the question is whether s is the name of the car or the model of the car or a combination of both. Whichever it is should be demonstrated by the name, as virtual int get_size( char * model_name); In the second case, the parameter name is not an English word. It ought to be replaced by a real word, such as "index." I do not think that even that is enough, as some indication should be given as to what the index is used for. The question is whether the index is based on passenger compartment size, horsepower, consumer rating, or something else. virtual int get_size( int index_of_consumer_rating); We are still left with two overloaded functions. I would eliminate the overloading by modifying the name of each member function. In addition, there is no indication in the name of what size is really being gotten. So the function might really be called virtual int get_engine_size_by_model_name ( char * model_name); virtual int get_engine_size_by_consumer_rating (int index_of_consumer_rating); Then the code might read get_engine_size_by_consumer_rating(SIZE); Now the name SIZE looks rather funny here. That's because it is not long enough to describe what it really is. If you are just the class provider, you do not have any control over the names by which your classes will be invoked. At least you have done your best to provide the means to write readable code. One objection to requiring meaningful names is that the source code is longer. Sometimes it will take two lines to write an expression that could have fit onto one. My response is that statements are like sentences. Look at the last letter or article that you wrote. See how many sentences fit onto a single line. Another objection is that the names can be too long and can exceed the 31-character limitation of significance. The words in the name do not have to all be in the dictionary. Abbreviations are okay, especially if they appear in an industry dictionary or glossary. Proprietary abbreviations are okay, if the company has a standard glossary. Abbreviations that are local to your programs should be avoided, unless they are consistently applied and a glossary is included. Every time you use your own abbreviation, you add an information element that has to be absorbed by the reader. I remember reading David Copperfield in my youth. It was a bit advanced for me at the time, so I had it in one hand with a dictionary in the other. Every time I came across an unknown word, I looked it up in the dictionary. You can imagine how that affected following the plot. Using too many of your own abbreviations can make the reader miss the flow of the code. Let me point out that overloading constructors is almost unavoidable, since there is only one name for the constructor. However, as pointed out in Tom Cargill's C++ Programming Style, many overloaded constructors can be eliminated by specifying default arguments in the parameter list for the constructor. Your other instance of bad programming shows that the designer did not properly modularize the code in C. One has to learn to do this in C, as that is a design technique, not a feature of the language. C++ makes it easier to design modular code as many of the bookkeeping aspects are taken care of, but it does not demand it. My basic rule of thumb for structure access is that anytime you use more than one member operator, you should examine how to redesign your code. If you use more than two, you should start the redesign immediately. A program that quires a reference as a.get.line.alpha.beta is going to be difficult to maintain. External Declarations Q I had a long debugging session recently when I ran across the following problem (see Listing 2). I've cut it down to the essentials. The code appears all in the same source file. The value of cnt is incremented by both functions. The correct code for the second function should have been: static int cnt1; ... cnt1++; printf("cnt is %d", cnt1); This code was only a small portion of a large file with lots of small functions, so it was not readily apparent that it was incorrect. However I would have thought the compiler would have told me that cnt was declared twice. No error message was generated, so the bad code was hard to find. What gives? Sam Mellon Boston, MA A You have been exposed to the new semantics of external declarations in ANSI C. This is a major change from K&R C, but it is not noted as such in many texts. Let me review how external variables are linked together, as well as the differences in K&R and ANSI. In this explanation of externals, K&R refers to the Appendix A in the Kernighan & Ritchie The C Programming Language Book. Some compilers implemented externals slightly differently. The linker matches external references (REFs) to external definitions (DEFs). There are REFs and DEFs for both functions and data, though they are generated slightly differently. Take functions first. Their operation is the same in both K&R and ANSI. Each call to a function in your code generates a REF for that function. The code for the function itself is the DEF for the function. For example, this code segment: void calling_function(void) { called_function(); .. } generates a DEF for calling_function and a REF for called_function. You can have one and only one DEF for a function in all the files that are linked together. Of course, you can and probably will have lots of REFs for a function, but you don't even have to have any. You may have gotten messages from the linker regarding duplicate definitions or the failure to provide a definition. Depending on the vendor, it may or may not allow you to create an executable file if those errors occur. External REFs and DEFs for data work slightly differently. Under K&R C (Appendix A), the same rules applied as to having one and only one DEF and allowing zero or more REFs. A DEF was generated by a variable declaration outside of any function, such as: int global_i; calling_function () { global_i = 5; ... } This declaration could appear in one and only one source file. Optionally, you could initialize the variable as: int global_i = 5; and the initial value would be set to 5. If you did not specify an initializer, its value would default to 0. To generate a REF, the declaration with the keyword extern was used: extern int global_i; another_calling_function() { global_i = 7; ... } It was permissible to have both a DEF and a REF in the same source file, such as: int global_i = 5; extern int global_i; This would not normally be done directly, but indirectly, as the REF or a set of REFs would be placed into a header file, such as: "external.h" extern int global_i; /* Other extern declarations */ The DEFs could be in individual files or a single file. The header file would be included in this file, such as: #include "external.h" int global_i = 5; If the declaration of the DEF does not match the declaration of the REF, the compiler generates an error. Although K&R was pretty straightforward on how to handle DEFs and REFs, vendors developed their own styles, some of which were adapted from other languages. I won't go into all the possibilities. ANSI C adopted an amalgamation of the styles. It also added the concept of tentative DEF. The first declaration of an external without an explicit initializer is considered to be a tentative DEF. All similar declarations that follow in the same file are also tentative DEFs. For example, suppose your file contained: int global_i; /* Tentative DEF */ ... int global_i; /* Tentative DEF */ These are not double definitions of global_i, but simply a set of tentative definitions of the same variable. One tentative DEF will be turned into a true DEF by the compiler. Now if you declare an initializer in the declaration, it becomes a true DEF. The compiler will accept one true DEF and ignore all the other tentative DEFs. int global_i = 5; /* True DEF */ ... int global_i; /* Tentative DEF - refers to previous*/ The compiler will complain if two true DEFs appear in the same source file, as: int global_i = 5; /* True DEF */ int global_i = 6; /* Compiler error - double DEF */ Just as with K&R, you can only have a single true DEF in all the linked source files. The linker should complain if two true DEFs appear in multiple linked source files. However, the way many compilers/linkers work extends the concept of tentative DEFs to multiple source files. A tentative DEF in a source file continues to act as a tentative DEF, rather than being turned into a true DEF. If there is no true DEF in any source file, then the linker turns a tentative DEF into a true DEF with an initializer of zero. The linker will accept one true DEF and ignore all the other tentative DEFs. If you include the keyword static, the scope of the variable is only within the source file. However the rules for tentative DEFs still apply: static int cnt; /* Tentative DEF */ static int cnt; /* Tentative DEF */ Both tentative DEF's refer to the same variable. This is why your code had a problem. However, it is acceptable ANSI C, so the compiler did not complain. If you used initializations, you should get an error, such as: static int cnt = 0; /* True DEF */ static int cnt = 0; /* 2nd True DEF - compiler error */ If you had simply initialized one, the other would have been treated as a tentative DEF for the same variable: static int cnt = 0; /* True DEF */ static int cnt; /* Tentative DEF */ To complete the external puzzle, the keyword extern has been altered from K&R. It now can be used with an initializer, as: extern int global_i = 5; /* True DEF */ extern int global_i; /* REF */ If used without an initializer, it still means REF. I should note that the concept of tentative DEF's does not exist in C++. A set of declarations as: int global_i; int global_i; is a compiler error due to the double definition. I've covered my guidelines for using global variables long ago. It can be summed up in a word--Don't. However, if you must, I suggested the style that follows that as shown for K&R above. Use extern declarations (REF's) in a header file that gets included in every source file. Then in a single file initialize all the externals. For static externals, place all declarations together near the top of the source file. P.J. Plauger long ago pioneered a similar style with Whitesmith's C, using the new ANSI standard for externals. Initialized declarations (extern int i = 1;) in one file are marked true DEF's. Uninitialized declarations (extern int i;) in a header file were used as REF's. Social Security Feedback I received a note from Bruce Bogert regarding the conventions on Social Security numbers. The first three digits are assigned on a state by state basis. For example 212 through 220 are for Maryland. The next two digits may be used for individual divisions within a state and the last four digits are sequentially assigned. If you do need to make up a Social Security number for some outfit that requires it but is not legally obligated to have it, you might want to know that numbers beginning with 900 to 999 are not used for the most part. (KP) Listing 1 A C++ base class with overloaded functions defined class Car { public: virtual int get_size( char * s ); virtual int get_size( int idx ); } class SportsCar :Car { public: virtual int get_size( char * s ); virtual int get_size( int idx ); } SportsCar sportsCar; main() { sportsCar.get_size( SIZE ); } /* End of File */ Listing 2 The value of cnt is incremented by both functions. static int cnt; void test() { cnt++; printf("Cnt is %d\n", cnt); } static int cnt; void test1() { cnt++; printf("cnt is %d", cnt); } /* End of File */ Code Capsules A C++ Date Class, Part 1 Chuck Allison Chuck Allison is a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C ++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. Last month's column introduced a function, date_interval, that calculates the number of years, months, and days between two arbitrary dates. This month's column presents a C++ solution to the same problem. The essence of this approach is to create a new data type, which behaves much the same as built-in types do. In other words, you shift from a function-based approach ("How do I want to do things?") to an object-based one ("What are the elements, the objects, of my problem?"). Using C++ effectively requires a different way of thinking about problem solving. To make that shift, it helps to know why C++ exists in the first place. A Tale of Two Languages C++ had its beginnings at AT&T in the early 1980's with Bjarne Stroustrup's "C with Classes". He was seeking a way to speed up simulations written in Simula-67. "Class" is the Simula term for a user-defined type, and being able to define objects that mirror reality is a key to good simulations. What better way to get fast simulations than to add classes to C, the fastest procedural language? Choosing C not only provided an efficient vehicle for classes, but a portable one as well. Although other languages supported data abstraction through classes long before C++, it is now the most widespread. Almost every major platform that has a C compiler also supports C++. The last I heard, the C++ user base was doubling every seven months. A first look at C++ can be overwhelming. If you're coming from C, you need to add the following (and then some) to your vocabulary:abstract class, access specifier, base class, catch clause, class, class scope, constructor, copy constructor, default argument, default constructor, delete operator, derived class, destructor, exception, exception handler, friend, inheritance, inline function, manipulator, member function, multiple inheritance, nested class, new handler, new operator, overloading, pointer to member, polymorphism, private, protected, public, pure virtual function, reference, static member, stream, template, this pointer, try block, type-safe linkage, virtual base class, virtual function. The good news is that C++ is a powerful, efficient, object-oriented language able to handle complex applications. The bad news is that the language itself must therefore be somewhat complex, and is more difficult to master than C. And C itself is part of the problem. C++ is a hybrid, a blending of object-oriented features with a popular systems programming language. It is impossible to introduce such a rich set of new features without the host language having to bend a little. Yet compatibility with C is a major goal of the design of C++. As Bjarne stated in his keynote address to the ANSI C++ committee, C++ is an "engineering compromise," and must be kept "as close as possible to C, but no closer." How close is still being determined. But there is more good news. An Incremental Journey You can use C++ effectively without having to master all of it. In fact, object-oriented technology promises that if vendors do their job (providing well-designed class libraries, crafted for reuse and extensibility), then your job of building applications will be easier. Current products, such as Borland's Application Frameworks, are proving this to be true in many cases. If you feel you must master the language, you can do it in steps, and still be productive on the way. Three plateaus have emerged: 1. A Better C 2. Data Abstraction 3. Object-oriented programming You can use C++ as a better C because it is safer and more expressive than C. Features on this plateau include type-safe linkage, mandatory function prototypes, inline functions, the const qualifier (yes, ANSI C borrowed it from C++), function overloading, default arguments, references, and direct language support for dynamic memory management. You will also need to be aware of the incompatibilities that exist between the two languages. There is a robust subset common to both which Plum and Saks call "Type-safe C" (see C++ Programming Guidelines, Plum and Saks, Plum-Hall, 1992). As I will illustrate in this article and the next, C++ supports data abstraction--user-defined types that behave essentially as built-in types. The data abstraction mechanisms are classes, access specifiers, constructors and destructors, operator over-loading, templates, and exceptions. Object-oriented programming takes data abstraction a step further by making the relationships between classes explicit. The two key concepts are inheritance (defining a new class by stating how it is the same and how it is different from another, reusing the sameness) and polymorphism (providing a single interface to a family of related operations, transparently resolved at runtime). C+ + supports inheritance and polymorphism through class derivation and virtual functions, respectively. Classes A class is just an extended struct. In addition to data members, you define member functions that act upon objects of the class. The definition of the Date class is in the file date.h in Listing 1. It differs from last month's C version because interval is a member function instead of a global function. The implementation of Date::interval() is in Listing 2. The double colon is called the scope resolution operator. It tells the compiler that interval is a member of the Date class. The ampersand in the prototype for Date::interval() indicates that its parameter is passed by reference. (See the sidebar on references.) The program in Listing 3 shows how to use the Date class. You must use structure member syntax to call Date:: interval(): result = d1.interval (d2); The structure tag Date serves as a type specifier, just like built-in types do (i.e., you can define a Date object without using the struct keyword). There is never a need to define typedef struct Date Date; In fact, the concept of class is so fundamental that C++ has merged the separate name space for structure tags that exists in C with that of ordinary identifiers. Note that I have defined isleap as an inline function in Listing 2 (it was a macro in the C version). Inline functions get expanded "in-line" like macros do, but also perform all the scope and type checking that normal functions do. Unless you need to use the stringizing or token-pasting operations of the preprocessor, you don't need function-like macros in C++. Now consider the statement years = d2.year - year; in Listing 2. What object does year refer to? In the C version, this statement appeared as years = d2.year - d1.year; Since a member function is always called in association with an object (e.g., d1. interval (d2)), then un-prefixed occurrences of data members naturally refer those of the associated object (in this case, year refers to d1.year). The this keyword represents a pointer to the underlying object, so I could make the more explicit statement: years = d2.year - this->year but this is rarely done. In Listing 4 I have added the following statements to the class definition: Date(); Date(int,int,int); These are special member functions called constructors. Constructors allow you to specify how to initialize an object when it is created. The first, called the default constructor (because it takes no arguments), is used when you define a date object without any initializers: Date d; The following statement invokes the second: Date d(10,1,51); When the implementation of member functions is trivial, you can make them inline by moving the implementation inside the class definition itself. (See Listing 7--don't forget to mentally remove them from Listing 5.) The test program in Listing 6 puts off constructing the objects d1, d2, and result until they are needed. (Object definitions can appear anywhere a statement can in C++.) I have almost illustrated the key feature of data abstraction, namely encapsulation. Encapsulation occurs when a userdefined type has a well-defined boundary between its internal representation and its public interface. To truly say that I have created a new type that acts like a built-in type, I must disallow any inadvertent access to its internal representation. For example, at this point, the user could execute a statement such as d1.month = 20; A well-behaved object controls the access to its data members. In a practical date class, I want to allow the user to query the month, day, and year, but not to set them directly, so I make them private, and provide accessor functions to retrieve their values (see Listing 8). Since having private members is the more common situation, I usually replace the struct keyword with the keyword class, whose members are private by default (see Listing 9). Accessor functions like get_month don't alter the private part of a Date object, so I declare them as const member functions. (Date::interval() is also const--don't forget to add const to its definition in the implementation file date3.cpp also.) I must now replace references to data members with calls to the accessor functions in tdate3.cpp (see Listing 10). We are only about half way to a complete C++-style approach to a date class. Next month we will incorporate stream I/O, static members, and operator overloading. References in C++ A reference in C++ is an alias for another object. It can appear wherever the object it refers to can. The following program uses the reference iref as a substitute for i: /* ref1.c: Illustrate references */ #include main() { int i = 10; int &iref = i; // An alias for i printf("%d\n",iref); iref = 20; printf("%d\n",i); return 0; } /* Output: 10 20 */ You can think of a reference as a "smart" pointer, since it refers to another object without requiring explicit addressing and dereferencing like pointers do: /* ptr.c: Do the same thing with pointers */ #include main() { int i = 10; int *iref = &i; printf("%d\n" ,*iref); *iref = 20; printf("%d\n",i); return 0; } The major differences between a pointer and a reference are: You must initialize a reference with the object it refers to. A declaration such as int &iref; is meaningless (except as a function parameter). Once initialized, you can't modify a reference to refer to a different object. Since a reference always refers to something, you don't need to check for NULL as you do with pointers. References neither require nor allow the use of the & or * operators. All addressing and dereferencing are automatic. You can think of a reference as a const pointer that is derefenced each time it is used. Like pointers, however, references can be function return values. Since a reference is by definition an lvalue, this allows the unusual practice of placing a function call on the left-hand side of an assignment: /* ref2.cpp: Returning a reference */ #include int & current(); // Returns a reference int a[4] = {0,1,2,3}; int index = 0; main() { current() = 10; index = 3; current() = 20; for (int i = 0; i < 4; ++i) printf("%d ",a[i]); putchar('\n'); return 0; } int & current() { return a[index]; } /* Output: 10 1 2 20 */ Another use for references is to implement pass-by-reference semantics, meaning that changes to function parameters persist in the calling procedure after the called function returns. You can do this with pointers, but references are cleaner: /* ref3.cpp: Swap via references */ #include void swap(int &, int &); main() { int i = 1, j = 2; swap(i,j); printf("i == %d, j == %d\n",i,j); return 0; } void swap(int &x, int &y) { int temp = x; x = y; y = temp; } /* Output: i==2, j == 1 */ Even if you aren't going to modify a function parameter, it is a good idea to pass large objects by reference for efficiency. For example, if the data type X is large: struct X { // lotsa stuff }; then a function f that takes a parameter of type X, but doesn't modify it, should have a prototype similar to void f(const X&); For a more complete treatment of references, see Dan Saks' column in the September 1991 issue ("Reference Types", CUJ Vol. 9, No. 9). Listing 1 Member definitions for the Date class // date.h: A simple date class struct Date { int month; int day; int year; Date * interval(const Date&); }; /* End of File */ LISTING 2 Implementation of Date::interval() //date.cpp: Implement the Date class #include "date.h" inline int isleap(int y) {return y%4 == 0 && y%100 != 0 y%400 == 0;} static int Dtab[2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date * Date::interval(const Date& d2) { static Date result; int months, days, years, prev_month; // Compute the interval - assume d1 precedes d2 years = d2.year - year; months = d2.month - month; days = d2.day - day; // Do obvious corrections (days before months!) // // This is a loop in case the previous month is // February, and days < -28. prev_month = d2.month - 1; while (days < 0) { // Borrow from the previous month if (prev_month == 0) prev_month = 12; --months; days += Dtab[isleap(d2.year)][prev_month--]; } if {months < 0) { // Borrow from the previous year --years; months += 12; } /* Prepare output */ result.month = months; result.day = days; result.year = years; return &result; } /* End of File */ Listing 3 A program that shows how to use the Date class // tdate.cpp: Test the Date class #include #include #include "date.h" main() { Date d1, d2, *result; int nargs; // Read in two dates - assume 1st precedes 2nd fputs("Enter a date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &d1.month, &d1.day, &d1.year); if (nargs != 3) return EXIT_FAILURE; fputs("Enter a later date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &d2.month, &d2.day, &d2.year); if (nargs != 3) return EXIT_FAILURE; // Compute interval in years, months, and days result = d1.interval(d2); printf("years: %d, months: %d, days: %d\n", result->year, result->month, result->day); return EXIT_SUCCESS; } /* Sample Execution: Enter a date, MM/DD/YY> 10/1/51 Enter a later date, MM/DD/YY> 11/14/92 years: 41, months: 1, days: 13 */ /* End of File */ Listing 4 Definitions for a variation of the Date class shown in Listing 5 // date2.h struct Date { int month; int day; int year; // Constructors Date(); Date(int, int, int); Date * interval (const Date&); }; /* End of File */ Listing 5 A variation on the Date class // date2.cpp #include "date2.h" inline int isleap(int y) {return y%4 == O && y%100 != 0 y%400 == 0;} static int Dtab [2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date * Date::interval (const Date& d2) { (same as in Listing 2) } Date::Date() { month = day = year = 0; } Date::Date(int m, int d, int y) { month = m; day = d; year = y; } /* End of File */ Listing 6 A program for testing the Date class // tdate2.cpp #include #include #include "date2.h" main() { int m, d, y, nargs; // Read in two dates - assume 1st precedes 2nd fputs("Enter a date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &m,&d,&y); if (nargs != 3) return EXIT_FAILURE; Date d1(m,d,y); fputs("Enter a later date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &m,&d,&y); if (nargs != 3) return EXIT_FAILURE; Date d2(m,d,y); // Compute interval in years, months, and days Date *result = d1.interval(d2); printf("years: %d, months: %d, days: %d\n", result->year, result->month, result->day); return EXIT_SUCCESS; } /* End of File */ Listing 7 Moving implementation of member functions into the class definition // date2.h struct Date { int month; int day; int year; // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} Date * interval(const Date&); }; /* End of File */ Listing 8 Definitions for a class that uses accessor functions to retrieve values // date3.h struct Date { private: int month; int day; int year; public: // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date * interval(const Date&) const; }; /* End of File */ Listing 9 Definitions for a class that replaces struct with class // date3.h class Date { int month; int day; int year; public: // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date * interval(const Date&) const; }; /* End of File */ Listing 10 A test for version 3 of the Date class // tdate3. cpp #include #include #include "date3.h" main() { int m, d, y, nargs; // Read in two dates - assume 1st precedes 2nd fputs("Enter a date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &m,&d,&y); if (nargs != 3) return EXIT_FAILURE; Date d1(m,d,y); fputs("Enter a later date, MM/DD/YY> ",stderr); nargs = scanf("%d/%d/%d%*c", &m,&d,&y); if (nargs != 3) return EXIT_FAILURE; Date d2(m,d,y); // Compute interval in years, months, and days Date *result = d1.interval(d2); printf("years: %d, months: %d, days: %d\n", result->get_year (), result->get_month (), result->get_day ()); return EXIT_SUCCESS; } /* End of File */ C++ Components and Algorithms Steve Halladay Steve Halladay owns Creative C Corp. in Louisville, CO. He has over 10 years experience developing products like CSIM and Multi-C in C and C++. Steve received his MS in Computer Science in 1982 from Brigham Young University. You can reach Steve at Creative C Corp. (303) 673-6683. C++ Components and Algorithms is not just another text on how to use C++. This book focuses on a few useful software components and shows how to implement them in C++. In addition, Scott Ladd provides the kind of insight associated with experienced software developers. Ladd expects the reader to have a working knowledge of C++. Organization and Contents C++ Components and Algorithms includes C++ source code that shows the implementation of strings, arrays, sorting algorithms, statistical functions, persistent objects, hash tables, binary trees, and B-trees. The 779 pages of the book are divided into four major sections entitled "Groundwork," "Arrays," "Indexing, Hashing and Filing," and "Appendices." In his first section, entitled "Groundwork", Ladd describes the philosophy that guides the development of the objects he presents. As he points out, it is important to understand the developer's general philosophy to fully understand the software. Ladd briefly describes how object-oriented software differs from traditional software and defines some of its associated terminology. Ladd also takes time to describe some pitfalls associated with "trendy" C++ such as the additional opportunity the language gives you to make mistakes. In this first section Ladd also introduces some basic types such as booleans, error reporters, random-number generators, and strings. Ladd mentions that the string class he presents in the third chapter is very similar to string classes he has presented in previous books, but notes that he has included the string class in this book for completeness. This is certainly useful for those who do not own his previous books. The second section of the book, entitled "Arrays," introduces a couple different types of arrays. The inherent implementation of arrays in C and C++ gives programmers enough rope with which to hang themselves. Ladd shows how to build an array class that helps programmers avoid common array pitfalls by supplying enhancements like index bounds checking. The array types he describes include integer and floating-point arrays. Ladd avoids the use of templates for his implementation of arrays because many current compilers have template implementation deficiencies. But since Ladd gives both integer and floating-point examples of arrays, it is easy to see how to create additional types of arrays using his base classes. In the section on arrays, Ladd also takes advantage of array constructs to discuss some related topics. For example, Ladd presents one chapter on sorting in which he describes the quicksoft algorithm and shows how to implement it in C++. He also has a chapter about statistics in which he discusses some basic statistical concepts like mean, variance, and correlation. Ladd shows how both sorting and statistics relate to his array classes. The third section of the book, entitled "Indexing, Hashing and Filing," discusses some basic database concepts. Ladd starts the third section by talking about persistent objects. He shows how to store and retrieve objects with files. The third section also discusses hash tables and shows how to use them as an indexing mechanism for a simple database. Ladd ends the third section of the book by discussing tree structures as database indexing mechanisms. In the chapters on tree structures, Ladd shows a binary tree and a B-tree implementation. He also explains the strengths and weaknesses of each structure. Ladd's implementation of the B-tree is thorough enough that he includes code to perform not only key insertion and searching, but also key deletion. The fourth section of the book is an appendix that includes the complete source code for all the classes presented in the earlier parts of the book. The appendix is 279 pages of listings. The book comes with an accompanying disk that also contains the complete source listings as contained in the appendix. Commentary C++ Components and Algorithms is Ladd's third book. His comfortable writing style attests to his experience as a writer. The explanations and examples are clear and concise. Most of the book is written in the first person as though you were looking over his shoulder as he guides you through his code. This book is not for the passive reader. Ladd includes lots of source code. The average page probably has more lines of source code than accompanying textual explanation. However, the code is commented where necessary and easy to understand. If you like to see examples of good clean code, this book is full of them. Ladd's source code also is reasonably compiler and system independent. There is nothing more frustrating than getting a book of programming examples only to find that they will not work on your system. The only system specific code handles the different PC compilers' file opening command modes. You might think that a book with so much source code is bound to have many errors. On the contrary, Ladd seems to have taken meticulous care to make sure his code compiles under at least Borland's and Microsoft's compilers. The very few errors I noticed in the book were not in the source code at all. The most interesting part of the book to me was Ladd's use of sound development techniques throughout the book. Ladd's class designs are elegant and simple. He avoids the convoluted inheritance hierarchies that are often prevalent in many current C++ programming examples. He also uses a fundamental programming style that avoids the many confusing extensions that C++ offers. But Ladd's avoidance of the exotic features of C++ does not prevent him from showing how to use the major valuable aspects of the language. It was refreshing to see code that did not try to impress you with its extreme use of the language. Conclusion C++ Components and Algorithms fills a niche for the intermediate software developer. The book focuses on useful, sound implementations of data structures and algorithms in C++. Ladd's down-to-earth approach and clear explanations make both his descriptions and examples useful to students of C++ and software engineering. Title: C++ Components and Algorithms Author: Scott Robert Ladd Publisher: M&T Books, 411 Borel Avenue, San Mateo, CA 94402 Price: $39.95 (disk included) ISBN: 1-55851-227-6 Editor's Forum Here's an update on the name game. Back in the May 1990 Editor's Forum, I discussed an offhand suggestion I'd heard. Someone felt we should change the name of this magazine to The C++ Users Journal. After all, C++ is where all the action is these days, nicht wahr? Why not change with the times? I declined to make such a recommendation to Robert Ward at the time. I felt then that C still had a lot of life left in it. And legions of programmers still needed to read about style, techniques, and esoterica peculiar to the C language proper. We were running articles on C++ (and other dialects) on a regular basis. That was enough. I predicted then that our C++ coverage would increase if the need arose. And indeed the need has arisen. Lots of programmers are finding reasons to try C++. Some are grappling with new coding styles. Some with new techniques. Many are finding lots more esoterica to master. You'll now find that a typical issue of The C Users Journal devotes substantial editorial real estate to articles on C++. We still differ from the new "object-oriented" publications in one important regard. Few of those articles we now include are preachy. Rather, they are written by C/C++ programmers to help others use this new dialect to advantage. That's been our strength in the C world for years. It will continue to be our focus in the emerging era of C++. And we will continue to cover the huge, and growing, C market for the foreseeable future. So should we change our name? I cite two other organizations who kept their old names even as they changed with the times. One is the Association for Computing Machinery (ACM). Many members felt that the focus of that organization had shifted from mere machinery to vastly more important topics. They called for a referendum to keep the initials ACM, but have them in future stand for Association for CoMputing. It was defeated. (I voted against the change on the grounds that computer science would be just an esoteric branch of mathematics were there no fast, reliable, and inexpensive computing machines.) The other group is the League of Women Voters. When they began allowing men to join, someone suggested that they should henceforth be the League of Voters (or the League of People Voters). My favorite argument against the change was made by one of the leaders. She suggested that they simply explain to men that "women" in this context was a generic term that also included men. The men would understand. If we don't change our name, I'm sure the C++ programmers will understand too. P.J. Plauger pjp@plauger.com New Products Industry-Related News & Announcements Micro-Processor Services Releases MASM to C Translator Micro-Processor Services, Inc., has released MASM2C, a tool designed to translate MASM code generated by a disassembler into C source code. Working with a disassembler, MASM2C can be used to translate from an .EXE file to intelligible C source. MASM2C contains a syntax analyzer, a MASM-to-tertiary converter, and a tertiary-to-C converter. The tertiary language is used to maintain logical equivalence between source and target languages, and is used in the entire Micro-Processor Services family of translators. MASM2C is compatible with V Communications' Sourcer and other disassemblers, and is error message compatible with "Brief" (or other editors supporting Microsoft C) permitting translation within an editor environment. MASM2C supports 8086, 80186, 80286, 80286P, 80386, and 80386P instructions. The translation includes MASM error bypass and C beautifier. MASM2C is available in both a standard version ($475) and a protected-mode version ($850). The standard version can handle files of 2000 to 3000 lines, while the protected-mode version can handle up to 40,000 lines at 16MB. Translation speed is 400 to 500 lines per minute. For information, contact Micro-Processor Services, Inc., 92 Stone Hurst Lane, Dix Hills, NY 11746, (516) 499-4461; FAX: (516) 499-4727. Compass Point Software Introduces application::ctor Compass Point Software, Inc., has announced their application::ctor (pronounced "application constructor") product, a GUI application-development tool for Microsoft Windows. application::ctor includes an object-oriented "view Editor", a user-interface class library containing over 100 classes, and a C++ class browser. Application::ctor can be used to build Multiple Document Interface (MDI) windows and provide attachment and gravity allowing automatic rearranging and resizing of controls when the window is resized. application::ctor can also subdivide a window into regions called "hot areas." Protoypers can use the View Editor to construct an application's user-interface and to attach pre-defined prototyping functions from the class library. Once the prototype's functionality and appearance are established, developers can attach C++ code to the events in the View Editor to build a deliverable application. application::ctor requires Windows 3.1 and a C++ compiler. The price is $99. Contact Compass Point Software, 332A Hungerford Drive, Rockvill, MD 20850, (301) 738-9109. Intel's Tunes C Compiler for Superscalar Microprocessor Intel Corporation has introduced CTOOLS960 Release 4.0, a C compiler which exploits the superscalar architectures of their i960R CA and i960 CF microprocessors. Intel's announcement claimed improvements of up to 40% in application program performance. The CTOOLS960 compiler collects information about an application program during execution of the program to drive its optimization. The CTOOLS960 compiler was designed to bring optimization to both superscalar and non-superscalar, and can optimize programs written for any i960 microprocessor, including the original Kx and the Sx series. The CTOOLS960 package includes the compiler, the assembler, and utilities designed for appliciation development and performance optimization on the i960 microprocessor family. CTOOLS960 Release 4.0 is priced at $2000 for MS-DOS hosts, $3500 for HP9000 hosts, and $4300 for SUN4 or IBM RS6000. For more information, contact the Intel Literature Center at (800) 548-4725, or write for: Intel Literature Pack #A9P71, P.O. Box 7641, Mt. Prospect, IL 60056-7641. Microtech Research Supports Cross Development for Intel processors on IBM RS/6000 Microtec Research, Inc., has announced it's Intel i960 and 8086-family microprocessor development tools for the IBM RISC System/6000. These tools complement the Motorola 68000-family tools already provided by Microtec Research, and were ported under an agreement with IBM. On the RS/6000, the cross development tools include an ANSI C cross compiler, a Macro Cross Assembler, and the XRAY Debugger. A C++ cross compiler is available for 68000 targets. The XRAY Debugger debugs optimized C code, allowing engineers to work with production-quality code. The XRAY Debugger provides an X Window System Motif interface. The XRAY Debugger on the RS/6000 supports instruction set simulation and debug monitor as execution environments. With instruction set simulation, software development can begin before the target hardware is available, while with monitor-based debugging, developers can perform real-time debugging while software runs on the target hardware. Microtec Research C compilers comply with ANSI C, but also support K&R C. The C++ compilers comply with version 2.1 of the AT&T specification. For information, contact Microtec Research Inc., 2350 Mission College Boulevard, Santa Clara, CA 95054, (800) 950-5554; FAX: (408) 982-8266. TauMetric Announces C++/C Debugger/Browser for OpenWindows TauMetric Corporation has announced a Windowed Debugger/Browser (WDB) for C++ and C programs on Sun SPARCs. WDB provides an OPEN LOOK interface to a source-level debugger which includes source, data, and class browsing facilities. WDB runs under OpenWindows 2.0 or 3.0, and debugs programs compiled with TauMetric's Oregon C++/C compiler. WDB includes both a Class Hierarchy Browser and Class Member Browsers. The Class Hierarchy Browser graphically depicts inheritance relationships between all classes in a program. The Class Member Browsers displays the members of a class (both data members and member functions), including inherited members, and allows access to the source code defining a class and its members. WDB is sold with the Oregon C++ Development System for SPARC, and is available at a discount for current Oregon C++ users. Oregon C++ is also available for VAX/VMS and DECstation/Ultrix. TauMetric plans calls for ports of WDB to the other platforms along with expansion into a complete object-oriented programming environment. For information, contact TauMetric Corporation, 8765 Fletcher Parkway, Suite 301, La Mesa, CA 91942, (61) 697-7607, or (800) 874-8501; FAX: (619) 697-1140. SunPro Licenses MetaWare Code Generation Technology MetaWare Incorporated and SunPro, a Sun Microsystems, Inc. business, have announced a licensing agreement in which MetaWare's x86 code generation technology will be used in a new family of SunPro compilers for Intel-based personal computers. SunPro and MetaWare, in their release, project the new family of compilers, ProCompilers, to be the first compiler suite available for Solaris 2.0 for x86. ProCompilers will provide source compatibility across SPARC and Intel x86. ProCompilers are now available in SunSoft's Solaris 2.0 for x86 early access kit, the developers version of the new operating environment for Intel 32-bit platforms. SunSoft has announced that the Intel version of Solaris 2.0 willbe available in volume toward the end of 1992. Contact MetaWare (408) 429-6832 or SunPro (415) 336-4638. Sierra Systems Upgrades Sierra C Compiler Sierra Systems has upgraded their C cross compiler and development system for the Motorola 68000 family. Sierra C 3.0 was designed specifically for the embedded systems engineer, and generates position independent, re-entrant, and ROMable code for any memory configuration. Version 3.0 incorporates a "Hole Compression" Optimization Utility. Sierra C's Linker can determine on the first pass when a more compact addressing mode can be used, relays the information to the assembler, which makes a second pass and adjusts the code improving size (4-7% on average) and performance. Version 3.0 also includes extended-memory support, floating-point enhancements, enhanced multiplication by constant, improved function-call stack cleanup, and inline strcpy and unrollable structure copy code. Sierra C is priced at $2,000 for PCs or $3,500 for UNIX workstations. Contact Sierra Systems, 6728 Evergreen Avenue, Oakland, CA 94611, (800) 776-4888 or (510) 339-8200; FAX: (510) 339-3844. SLR Systems Releases OPTLINK v3.0 for Windows SLR Systems, Inc. has released OPTLINK v3.0 for Windows. OPTLINK v3.0 for Windows adds smart-linking technology and p-code support for Microsoft's C/C++ v7.0, and supports elimination of all unreferenced functions. OPTLINK v3.0 allows up to 64K each of libraries, symbols, segments, modules, and other link specific information. OPTLINK v3.0 for Windows replaces LINK, and supports all compilers and assemblers that generate standard object files. OPTLINK v3.0 also supports CodeView 3.x and 4.x. Operating from MS-DOS, OPTLINK v3.0 for Windows generates 16-bit .EXEs and .DLLs for both Windows and OS/2. OPTLINK v3.0 Segmented accomodates MS-DOS and OS/2 hosted linking for Windows and OS/2 executable. OPTLINK v3.0 for Windows and OPTLINK v3.0 Segmented are priced at $350 and $499, respectively. Contact SLR Systems, Inc., 1622 N. Main Street, Butler, PA 16001, (412) 282-0864; FAX: (412) 282-7965; BBS: (412) 282-2799. Magna Carta Moves Communications Library to Windows Magna Carta Software has released their C Communications Toolkit/Windows, a C developer's library for serial communications under Windows. The library supports single or multiport communications under Windows and works with all Hayes-compatible modems. The library supports data transmission and reception, flow control, and a variety of file transfer protocols, including XMODEM, YMODEM, ZMODEM, and Kermit. C Communications Toolkit/Windows costs $299. The library supports Borland C++, Turbo C++, Microsoft C/C++, and Microsoft Quick C for WIndows. The package comes with full, commented source code, 600 pages of documentation, an da 30-day money-back guarantee. For more information, contact Magna Carta Software, P.O. Box 475594, Garland, TX 75047-5594, (214) 226-6900. NeruoDynamX Provides Embeddable Neural Network Software NeuroDynamX, Inc., has released DynaMind Developer neural network software for implementing neural networks. The package bundles DynaMind neural network training software (3.0) with several runtime options, including linkable C-code routines and neural network hardware simulation. Networks trained with DynaMind can be run standalone, embedded into C applications, or used in simulations of neural network hardware. The package includes NeuroLink v2.0, a library of C routinges for linking networks into applications. The library can link multiple networks serially or in parallel. The package includes a simulator for Intel's 80170NX ETANN chip. NeuroLink compiles under Borland Turbo C 2.0, Turbo C++, or Borland C++ and carries no run-time license fee. DynaMind Developer is priced at $495, while DynaMind 3.0 software is available separately for $145. Contact NeuroDymanX, Inc., P.O. Box 323, Boulder, CO 80306, (800) 747-3531; FAX: (303)442-2854. Tom Sawyer Software Introduces Graph Layout Toolkit Tom Sawyer Software has introduced the Graph Layout Toolkit, an automated object positioning tool which helps application programs provide visual representations of objects and their connections. The Graph Layout Toolkit provides real-time graph layout services for both directed and undirected graphs through a set of extensible class libraries written in C++. An ANSI C API is furnished as well. The package can handle cyclic graphs, multiple disconnected subgraphs, multigraphs, and reflexive edges. The toolkit supports crossing minimization, graph reduction, hierarchical layout, and subgraph position balancing. The software is available for PC, Mac, Sun, HP 9000, RS/6000, and NeXT platforms. Tom Sawyer is seeking to license this graph layout technology for embedding/repackaging by product-oriented companies. Price is determined by contract. Contact Tom Sawyer Software, 1824B Fourth Street, Berkeley, CA 94720, (510) 848-0853; FAX: (510) 848-0854. XVT Announces Developer Toolkit for Windows NT XVT Software, Inc., has announced support for Windows NT with its latest version of the XVT Portability Toolkit. The XVT Portability Toolkit allows programmers to build a C/C++ application, then recompile to other GUIs without rewriting code. Applications built with XVT/NT will use a 32-bit linear memory model. With NT's pre-emptive multitasking system, XVT/NT will offer memory protection for each application. XVT/NT supports all Portability Toolkit Release 3.x functionality. The Portability Toolkit for NT is priced at $1,450 on Intel 486 systems, and at $4,400 for DEC and MIPS. XVT does not charge royalties. Contact XVT Software Inc., 4900 Pearl East Circle, Boulder, CO 80301, (303) 443-4223. IDE Introduces C++ Tools and Support Interactive Development Environments, Inc., has added Object-Oriented Structured Design/C++ (OOSD/C++) to its Software through Pictures family of CASE tools. OOSD/C++ provides developers with a graphical design tool, a Reuse Browser, and a Code Generator for C++. IDE is also marketing a coordinated program (the Success Package for C++) of tools, training, consulting, maintenance, and support that includes OOSD/C++ and is based on the premise that in order to exploit object-oriented design, comprehensive training and support are essential. Contact Interactive Development Environments, Inc., 595 Market Street, 10th Floor, San Francisco, CA 94105, (415) 543-0900; FAX: (415) 543-0145. Vitamin C/CE Released for COHERENT Creative Programming Consultants, Inc., and Mark Williams Company have announced the release of Creative Programming's Vitamin C/CE, an advanced user interface library for COHERENT. An alternative to curses, Vitamin C/CE enables programmers to create portable, multiwindowed, terminal independent applications with full data entry, validation, help, and menuing capabilities. Vitamin C/CE provides source level compatibility over a variety of platforms, including MS-DOS, SYSV, BSD, SUN/OS, HP/UX, AIX, Ultrix, and VMS. Vitamin C/CE is available for $99 from Creative Programming Consultants, 2013 N. Broadway, Carrollton, TX 75006, (214) 245-9139; FAX: (214) 245-9717. Symantec Announces MultiScope Debuggers for Borland and Microsoft C++ Symantec Corporation has announced version 2.0 of its MultiScope Debuggers. Version 2.0 supports Borland C++ and Microsoft C 6.0 and C/C++ 8.0 languages for programming Windows and MS-DOS applications. The MultiScope Debuggers provide a Windows-hosted user interface, with a "CommandBar" which allows quick access to frequently-used commands. MultiScope debuggers are designed specifically for C++ code and include C++ specific features. New features include a "point-and-shoot" collapse and expand C++ class hierarchy browser, C++ object browsing, automatics C++ object mapping for all C++ inheritance types, alternative C++ class information member scope display, object-oriented breakpoints directly on object methods, direct browsing of member pointers, complete C++ expression evaluation, name unmangling, and the ability to update the source window to the actual function by selecting the method while browsing. MultiScope includes a Crash Analyzer System, with a Monitor Execution and Dump (MED) utility, which allows analysis of program crashes. MultiScope Debuggers for Windows are priced at #379. MS-DOS products are priced at $179. Upgrades for registered users are available. For information contact Symantec Corporation, 10201 Torre Avenue, Cupertino, CA 95014-2132, (800) 441-7234 or (408) 252-3570; FAX: (408) 253-4092 or 252-4694. Blaise Computing Announces C ASYNC MANAGER Version 4.0 Blaise Computing Inc., has announced version 4.0 of their C ASYNCH MANAGER, for applications in Microsoft and Borland C/C++, Quick C, and Turbo C and C++. Version 4.0 supports XMODEM, YMODEM, ZMODEM, and Kermit file transfer protocols; background transfers; and multiple simultaneous transfers. Other features include new modem control functions, enhanced support for 16550A UART, and support for multiport boards such as Digiboard PC/X. Contact Blaise Computing Inc., 819 Bancroft Way, Berkeley, CA 94710, (510) 540-5441; FAX: (510) 540-1938. GUI Computer Releases ObjectTable C/C++ V1.5 GUI Computer has released version 1.5 of their ObjectTable C/C++, a Windows-oriented library that implements a programmable multicolumn table object. This release supports protected column, draggable column width, international currency, different true-type font and color for column, and various title options. ObjectTable C/C++ can work as a custom control and is compatible with Borland Resource Workshop and Microsoft Dialog Editor. Contact GUI Computer, Inc., P. O. Box 795908, Dallas, TX 75379, (800) 800-9010; FAX: (214) 250-1355; BBS: (214) 250-2077. Lucid's Energize 1.1 Expands Tools and Utilities Lucid, Inc., has announced Energize 1.1, with support for more tools and utilities and improved performance in its native code C++ and C compiler. New releases of Lucid C and Lucid C++ have begun shipping. Energize 1.1 supports tools and libraries for C and C++, including system math libraries and the latest version (3.01) of the InterViews library from Stanford University. Energize support what Lucid calls Computer-Aided Programming (CAP). CAP offloads routine and repetitive tasks to the computer. Energize also supports incremental compilation and linking. Contact Lucid, Inc., 707 Laurel Street, Menlo Park, CA 94025, (415) 329-8400; FAX: (415) 329-8480. Non Standard Logics Introduces Version 2.0 of XFaceMaker Non Standard Logics has introduced version 2.0 of XFaceMaker (XFM), its X/Motif application interface management system. XFM's new WidgetMaker feature allows designers to build customized widgets that can be used with Motif, Open Look and other toolkits. XFM's templates can be used to assemble widgets into objects with assigned behavior, inheritance, and open-ended reusability. XFM includes a resource editor, a C-like scripting language (FACE) and a test mode. Contact Non Standard Logics, 99 Bedford Street, Boston, MA 02111, (617) 482-6393; (617) 482-9707. Microware Announces ANSI C Compiler for Intel and Motorola Processors Microware Systems Corporation has announced their Ultra C compiler, an ANSI C compatible compiler with an architecture and optimizing algorithms designed to maximize performance in real-time applications. Ultra C has been tested for compliance with the Plum Hall C Validation Suite. Ultra C is closely tied to Microware's OS-9 and OS-9000 real-time operating systems, including I/O and system calls. The Ultra C compiler is compatible with any processor running a Microware operating system. With Ultra C, Microware provides a tightly-coupled compiler and operating system, and a single source of technical support. The Ultra C architecture divides compilation into four processes: a front-end language process, an optimization and linking function, a target processor back end, and the assembly and linking function. Ultra C makes extensive use of an intermediate code (I-Code), performing linking and optimization on the I-Code, in addition to the other three stages of compilation. The language processor is compatible with source for previous Microware compilers, ANSI C source, and ANSI-extendable code. Ultra C can generate object code for any of its target processors, facilitating application development for multiple target systems. For information, contact Microware Systems Corp., 1900 NW 114th Street, Des Moines, IA 50325-7077, (800) 475-9000; FAX: (515) 224-1352; Telex: (910) 520-2535. Phar Lap's 286 DOS-Extender Supports Borland C++ 3.1 and Microsoft C/C++ 7.0 Phar Lap Software, Inc., has announced version 2.5 of their 286 DOS-Extender which is compatible with Borland C++ 3.1 and Microsoft C/C++ 7.0, as well as Turbo Debugger and CodeView for Windows. Version 2.5 includes faster floating point for Microsoft C/C++ users and huge memory model support for Borland C++ users. 286 DOS-Extender allows programs to access up to 16MB of memory on any DOS-based 80286, 386, or i486 PC. 286 DOS-Extender is priced at $495, upgrade discounts are available. Contact Phar Lap Software, Inc., 60 Aberdeen Avenue, Cambridge, MA 02138, (617) 661-1510; FAX: (617) 876-2972. Sequiter Software Releases CodeTranslator 2.0 Sequiter Software, Inc., has released CodeTranslator 2.0, which translates Clipper '87 and dBASE III PLUS applications into C. This tool can be used for performance enhancement, porting to UNIX, Microsoft Windows, or OS/2, and teaching xBASE programmers the C language. CodeTranslator generates ANSI compliant C code and can be combined with a portable library, CodeBase (currently version 4.5, but with version 5.0 planned for release in November 1992). CodeTranslator 2.0 is priced at $245, and CodeBase 4.5 retails for $395. Contact Sequiter Software, Inc., #209, 9644 - 54 Avenue, Edmonton, Alberta, Canada, T6E 5V1, (403) 437-2410; FAX: (403) 436-2999. Pinnacle Publishing Upgrades Graphics Server SDK to Version 2.0 Pinnacle Publishing, Inc. has announce Version 2.0 of the Graphics Server Software Development Kit (SDK), a dynamic link library (DLL) for adding graphing and charting capabilities to Windows applications. Graphics Server SDK supports C/C++, Turbo Pascal, Visual Basic, SQL Windows, Superbase, PowerBuilder, and Actor. Version 2.0 includes three new graph types--bubble graph, tape graph, and 3d area graph, automatic "hot graphs", improved axis and font control, multiple graphs, a print cancel option, and curve fitting capabilities. Contact Pinnacle Publishing, Inc., P.O. Box 1088, Kent, WA 98035, (206) 251-1900; FAX: (206) 251-5057. JMI Announces C EXECUTIVE for AMD Am29205 MMI Software Consultants, Inc., has announced C EXECUTIVE support for AMD's new Am29205 microcontroller. JMI's C EXECUTIVE is a real-time, multitasking, ROMable operating system kernel used in embedded systems. C EXECUTIVE is written in C for portability and is available for 80186, R3000, 29000, i960, 80386, 68000 and other processors. C EXECUTIVE also offers a file system, CE-DOSFILE, and a debugger, CE-VIEW. For some processors, C EXECUTIVE will be available packaged with FUSION TCP/IP from Network Research Corp. Contact JMI Software Consultants, Inc., 904 Sheble Lane, P.O. Box 481, Spring House, PA 19477, (215) 628-0840. Intermetrics Releases Three Debuggers for 68HC16 Processor Intermetrics Microsystems Software, Inc., has announced CXDBmon, CXDBsim, and CXDBice, C source-level debuggers for the Motorola 68HC16 processor. CXDBice is pre-integrated with in-circuit emulators from Motorola, Pentica, Nohau, EST, and Huntsville Microsystems; CXDBsim provides an instruction set simulator; and CXDBmon provides a monitor target program and supports the Motorola 68HC16 EVB development board. Contact Intermetrics Microsystems Software, Inc., 733 Concord Avenue, Cambridge, MA 02138-1002, (617) 661-0072; FAX: (617) 868-2843. TeleSoft Announces TeleUSE Version 2.1 TeleSoft has announced version 2.1 of TeleUSE, a user interface management system (UIMS) for developing GUIs based on OSF/Motif. The new version includes a session manager, an improved interface, a more powerful Dialog Manager with expanded C++ support, and third-party tool integration. TeleUSE is priced at $7,500 and supports UNIX platforms including: SPARC, HP, and IBM. Contact TeleSoft, (619) 457-2700. Electronic Imagery's ImageScale Plus Adds JPEG Compression Electronic Imagery has announced that its ImageScale Plus Developer's Toolkit for MS-DOS and UNIX applications, now includes the JPEG compression algorithm. The toolkit provides command-line executable programs, and graphics, text, and cursor routines in an image processing library written in C. Contact Electronic Imagery, Inc, 1100 Park Central Boulevard South, Suite 3400, Pompano Beach, FL 33064, (305) 968-7100; FAX: (305) 968-7319. EMS Professional Shareware Ships dBUtilily and C Utilily Libraries on CD-ROM EMS Professional Shareware has begun shipping two new CD-ROMs: the Library of PD/Shareware C Utilities, with 722 programs, and the dBUtility Library, with 2238 different public domain and shareware products. The disks are priced at $125, are updated quarterly, and include a substantial database of PC products (35,000) and vendors (13,000). Contact EMS Professional Shareware, 4505 Buckhurst Ct., Olney, MD 20832, (301) 924-3594; FAX: (301) 963-2708. We Have Mail Dear Mr. Plauger, For some years, I wrote most of my code in assembler, first on a CP\M system, and later, under MS-DOS. A friend introduced me to C just over a year ago, and I could kick myself for waiting so long to discover it. What used to take days in MASM can be written in hours (or minutes) with C, especially the input/output functions. When I recently upgraded to QuickC v2.5, I returned the CUG membership card in the package, and began receiving the C Users Journal. I've enjoyed the magazine almost as much as I've enjoyed working with the language. Your column and comments are always informative, and I appreciate the magazine's coverage of specific issues--C++, Liana, Windows, etc. These articles are so useful because they represent real-world programming, where I might spend one evening coding a device controller, then spend three months coding the user interface (even in robotics and automation, they won't buy it unless it has pull-down menus and bells and whistles). That brings me to my point. A reader recently complained that C doesn't have a cls function. You answered that the ANSI committee was reluctant to set any standards of that type, because they insisted on viewing all input and output as streams. That letter struck a chord with me. By being so fanatical about portability, and by insisting on viewing I/O from a single standpoint--that of streams--the ANSI committee may have ended up making the language less portable. Simply put: 1. Few people will be impressed with and/or use (or more importantly, buy) a program that has a dumb-terminal display. Programs that scroll one line at a time are almost as exciting as watching mold grow. 2. Because the ANSI refused to specify standards for these basic display functions, the programmer is forced to sacrifice portability with the first clearscreen or locate cursor function. Sure, I could write a very portable program that uses ANSI.SYS to control the screen. The problem is, it would run slower than itch, and no one would buy it. Now, I realize that setting a GUI standard might be beyond the scope of an ANSI committee. But after giving it a lot of thought, I believe that the current standard is too limited. More simply put: I respect ANSI's viewpoint, but in this day and age of 486 PCs and VGA monitors and Visual BASIC and MacIntoshes, can't I expect my programming language to have a standard function for wiping a screen or locating a text cursor? A number of objections are commonly raised. The first one goes, "You're opening Pandora's box; where do we stop? Next, they'll want functions like SET_CIRCLE_COLOR ..." I agree that you have to know where to stop. Graphics functions, in particular, might not be easily standardized. But couldn't we agree on a couple of functions that would at least bring C up to par with BASIC for text display? My list would include cls, locate_cursor, and set_text_color. These could be implemented on virtually all modern computer systems. The next objection would be that many computers don't support text positioning, color and/or graphics. So? If the computer won't support color, the system just ignores set_color and displays in monochrome text. If the computer won't support hardware functions such as moving the cursor, I'll just have to work around it. That's precisely what I'm doing now, so what have I lost? In closing, I don't want to criticize the ANSI committee unfairly. I respect what they've done, and agree that a standard was needed. All I'm saying is that they could have looked a bit more at the real world, rather than just at a theoretical stream concept, before setting that standard. As computer hardware becomes more sophisticated, we're actually going to pay the price down the road in reduced portability, as programmers are increasingly forced to resort to their own--and quite non-ANSI--solutions to each user interface. Yours truly, Stephen M. Poole 122 N. Main Street Raeford, NC 28376 You missed the point. We were not in love with a "theoretical stream concept" to the exclusion of all reason. Rather, we did nearly all our work at a time when streams were very real and screens were very new. It wasn't clear that your particular set of primitives were worth standardizing. Now the issues are more clear cut. You can still isolate the disturbance in a few system-dependent functions. Remember, portability is a statement about relative cost, not a true or false condition. -- pjp Dear CUJ: Mr. Wilbon Davis' article on time complexity in the September 1992 issue was quite interesting. However, it gave only a single sentence to a very serious problem concerning many quicksort implementations; that the use a fixed pivot point can result in O(n2) time, the same as a bubble sort. qsort's run time is very much probabilistic. There are a great many programmers who believe otherwise. In the Febrary 1992 issue of Unix Review, Nr. John Bently explained the reason for this and showed that the problem is present in many of the popular qsort implementations. Since any fixed pivot will result in quadratic run time he suggests adding a bit of randomness to the pivot selection. The possibility of quadratic run time still exists, but only if the data to be sorted is in cahoots with the random number generator. In the August 1992 issue of Unix Review. Mr. John Bently also examines the heapsort and its problems. Sincerely, Mr. Carey Bloodworth 1601 North Hills Blvd. Van Bure, AR 72956 Your point is well taken. Quicksort can have nasty time complexity for some patterns of input. -- pjp Dear Sir/Madam: I would like to know if you could help me with the following. I would like to create computer games and animation (musical animation). I have some programming background, and I have Borland's Turbo C++ and Assembler. I have a 386SX IBM compatible (16 MHz) with a VGA monitor. I use v3.3 MS-DOS. I have 1 Meg of RAM, a 60 Meg hard drive (Drive C), two high density drives 5 1/4" (Drive A) and a 3 1/2" (Drive B), a 24 pin dot matrix printer (Roland Raven 2417), and use a keyboard for everything. I am presently upgrading to the following: MS-DOS 5.0; Windows 3.1; Soundblaster Pro; 10 Megs of RAM; 245 Meg hard drive; math co-processor; and a mouse. I am considering at the end of this year looking into a 33 MHz or higher motherboard and next year a CD-ROM, but I am considering waiting a little longer for the CD-ROM as I want to read and write it and the ones available now are read only. I would like to create computer games like Conquest of Longbow, Wolfpack, Indiana Jones Last Crusade, Fate of Atlantis, Prince of Persia etc. I would like to create my games as life like as the ones mentioned. I have a game in mind that deals with ancient Egypt. It needs to be able to create pictures such as pyramids, coffins, King Tut's gold mask, hieroglyphics, etc. I would also like to create animated films such as Walt Disney's Beauty and the Beast and 101 Dalmations, etc. What I want to do (if possible) is to create the animation, with the music from the CD-player, and record it on diskette. Once this done I would like to transfer it on a VCR tape and play it on my T. V. Can this be done? Can you also provide me with the following: 1. Can you recommend books that show how to create games and animation? If so, please give title of book, author's name, and if possible where to purchase. 2. Call you recommend any schools that provide correspondence learning in this matter. Please give name and address. For the programs listed in your magazine, can diskette with the programs already in the magazines be purchased. If so please provide cost, shipping and handling, and method of payment. Also can they be used with Borland's Turbo C++? If you can kindly provide me with any other information that is not requested in this letter, please do so as I have written letters to schools here in Canada, magazines, and computer companies. But all say they cannot help me. Please understand that I would like very much to learn as I enjoy playing the games and am fascinated by them. I would really like to learn how I can create my own. Thank you for your cooperation in this matter. Hope to hear from you soon. Yours truly, Victoria Ceolin 510 Acadia Drive Hamilton, Ontario Canada L8W 3A4 You've laid out a very ambitious program. You have much to learn to get where you want to go. You'll also have to wait another year or two, as you've already guessed, for inexpensive hardware to come within reach of your dreams. But please don't stop dreaming. Your fascination with what can be done with computers can see you through the tough learning. My recommendation is that you get one of the simpler animation programs currently available and start practicing your craft with it. Reading CUJ will help you learn whatever programming you may need, but I suspect you want to keep that to a minimum. Also read PC Magazine from time to time to keep track of the latest advances in PC hardware and software. Sorry I can't be more specific, but I encourage any readers who share your interests to contact you directly. Good luck. -- pjp Dear Dr. Plauger: One of the few things more humbling than admitting to bugs in your carefully-crafted software is the introduction of new bugs when attempting to squash the old ones! Or, as Charlie Brown would say, ARRRRRRRRRRRGH! I am referring to your article in the September, 1992 issue of the C Users Journal, in which you attempt to fix a bug in which a failed memory allocation with malloc is ignored and data are copied to an address specified by a null pointer. (The code in question can be found on page 12.) Either the replacement code has fallen victim to a typesetting error, or else it must not been successfully passed through a conforming C compiler. This is a very real risk that we expose ourselves to whenever a code change seems so trivial that we only test it with the C compiler located between our ears! Have you noticed the error, now that I've directed your attention to the code fragment? Maybe it's true that the only time we can say with assurance that all the bugs are gone from a program is when the very last copy of the program in existence has been incinerated! I feel that this is true for most non-trivial programs I've written! Yours truly, John P. Toscano PharmData Systems P.O. Box 11537 St. Paul, MN 55111-0537 Error noted. It was indeed a transcription error. Thanks for reporting it. -- pjp Mr. Plauger, I thoroughly enjoyed Dwayne Phillips's, "The Foundation of Neural Networks: The Adaline and Madaline" (September 1992). It very clearly presented the fundamentals of neural network programming. I feel that a few small changes in the C code provided will yield improved performance. I am referring specifically to Listing 4, which makes the final AND/OR/MAJORITY decision. In the first two cases AND and OR), basic logical principles allow the for loops to be terminated immediately upon finding the first of the appropriate values. For example, in the AND decision, the loop may be terminated upon finding the first FALSE value (--1). There is no need to check any further inputs, as only one FALSE input is necessary to ensure a FALSE output in an AND condition. Similarly, the OR decision can be terminated as soon as the first TRUE value (1) is found. This is the same method used by C compilers to "short-circuit" logical tests. In both cases, inserting a break statement into the loop should do the trick. [Code available on monthly code disk.] The MAJORITY decision code can also be improved by noting that the outputs being checked conveniently have the values 1 and --1. All that is necessary is to sum up all outputs. If the sum is positive, the 1 values are in the majority; if the sum is negative, the --1 values are in the majority. To be consistent with the original code, a sum of O should also yield a result of --1. Refer to the listing for the exact method. Admittedly, with the small size of the networks presented in the article, the effects of these changes will probably be negligible. However, the performance improvement from these modifications could be important if this (or similar) code is used as the basis of larger, more complex networks. Yours very truly, Eric B. Schuyler 81 Yorktown Road Snyder, NY 14226 Dear Dr. Plauger, Thanks for your encouraging response to my earlier letter (CUJ Sep. 1992) concerning a proposed article on fail/fool-proof data input functions in C. At your request, I will expand a bit further on my suggestion. I see two aspects of the problem: input of individual data fields of various types, and combination of these fields in a data input screen. Concerning the first aspect, an input function for a given data type, e.g. currency, date, string of limited length, and other specialized types, should validate input immediately during data entry and alert the operator on errors. It should also allow correction of mistakes made during entry. Enclosed is an example of such a function for input of currency data that I wrote some time ago. It uses non-portable, unbuffered input functions, getch and putch, supported by Borland and Microsoft compilers, but not necessarily by others. Possibly, getchar could be made to work as well. Of more importance, I suspect that this function could be implemented more succinctly by a better programmer--hence my suggestion for the article! As to the second aspect, a collection of data input functions should be combined in such a manner that the operator can return to an earlier data field to make last-minute corrections, e.g. through use of the PgUp and PgDn keys. This may be the easier of the two aspects, but is certainly not trivial. After my letter appeared (the first time!) in the August issue of CUJ, I received a response from Robert A. Radcliffe (Philadelphia, PA), author of Data Handling Utilities in Microsoft C. Unfortunately, that text is now out of print, although the author still has some of the code disks. Maybe you could convince him to write the kind of article I have in mind! Thanks for your interest, W.F.H. Borman 209 Logwood Drive Evansville, IN 47710 Tel: 812 464-5435 Dear Mr. Plauger: I was much impressed by your mea culpa contained in the "Standard C: Bugs" column in the September, 1992 issue. It was a pleasure to find someone noble enough to admit mistakes can occur and that he, too, is a mere mortal. Unfortunately, character as fine as yours is a rare commodity! I do think, however, that you were a little too hard on yourself. There is another certainty in life besides Death and Taxes for anyone who writes (or uses) software: bugs. A bug in the vaunted Kernighan and Ritchie getline function on page 26 in the first edition and page 29 in the second edition of The C Programming Language causes the example to fail, as do dozens of other examples throughout both editions of "the book." They use getline expecting it to return 0 (zero) for a blank line when in fact it always returns 1 (one) or greater. The precedence table on page 49 of the first edition has errors in it. Borland's C compilers happily fopen files whose names have spaces in them (which of course can't be deleted by normal means). In Turbo C v2.0 and older versions, free failed if gets and related functions were used for keyboard input. If the authors and all those people involved at Bell Labs and Prentice Hall couldn't get a 13-line function right after ten years and two editions, why should anyone expect you to write perfect software? A true cynic sees known bugs as job insurance! If even the originators of such a great lean and mean language as C can't avoid a few bugs shouldn't one place simplicity and reliability at the top of one's priorities and always expect bugs? Ronnie R's favorite Russian proverb, "Trust but verify," should hang on every programmer's wall. Kindest regards, Elliott K. Rand President, Keep It Simple Systems P.O. Box 510093 Melbourne Beach, Florida 32951 (407) 729-0187 Thanks, I needed that. -- pjp Mr. Plauger, Jodi Leonard informs me that you doubled the price for your C library code disk since it was "updated and expanded." Humpf! Other than fix bugs, what does "expanded" include? I read your article describing the bugs found, it was somewhat amusing as a programmer's "confession", if not very scientific.... Care to expand in your next column what was changed? Does the current disk also come with a list of known bugs? Thanks, Mike MacFaden Group Leader Unix NMS Development Premisys Communications, Inc 1032 Elwell Court Suite 111 Palo Alto, CA 94303 email: ...!fernwood!premisys!mike mike@premisys.com Compuserve: 72711.2060 voice: 415-940-4787 fax: 415-940-7713 The new code disk contains no bug list because I fixed all known bugs at the time I released it. I have since accumulated two (small) bug fixes, but I'm not about to thaw a frozen release whenever I find yet another bug. -- pjp SSX -- Stack Swap eXecutive Tom Green and Dennis Cronin Tom Green is a UNIX Software engineer who specializes in UNIX driver development. He also writes MS-DOS, Windows and embedded 80x86 applications. He may be reached via electronic mail at tomg@cd.com. Dennis Cronin almost completed an EE degree but got lured into the sordid world of fast computers, easy money, and loose connections. Specialties: UNIX driver development and embedded systems. He may be reached via electronic mail at denny@cd.com. Introduction A project we worked on recently required us to port existing disk-controller software from an 80186-based board to a new 80386-based product. The total port required that we port not only the actual controller software, but also our real-time executive and a debugger as well. Our existing real-time executive was written in C and assembly language. We simply set out to directly port it to the 386. Oh yeah, we also committed to a pretty aggressive (translation: insane) schedule. The existing executive used a software-interrupt entry point with parameters passed in registers--pretty standard stuff. Task switches were implemented in assembly language, swapping in and out registers for the new task and the running task. We used a similar interrupt interface in the protected-mode version with 80386 task gates used to swap in and out registers for tasks. After we ported the disk controller software to protected mode, we tied the whole thing together and tried to run the new software. The new software worked fine. There was one problem though--it was slower than the old version running on the 80186! We were quite surprised (the words dismayed and panicked come to mind as well) by the results of our first test. We set out to find the problem. We discovered that our disk-controller software was doing a rather large number of task switches per second and making a pile of calls into the real-time executive. Referring to our trusty Intel black book, we then noted that calling an 80386 task gate takes 309 clock cycles--not good. But after looking over the executive, we decided that calling 80386 task gates was not our only speed problem. The whole process of calling our executive took a lot of time. First an assembly-language routine was called from the disk-controller C-code software. The assembly-language routine would get all of the parameters passed C-style and then stuff them in registers. Next a software interrupt was executed, followed by getting the registers back onto the stack so we could call into the C portion of the executive. We also had to preserve the state of the registers so that, on return from the executive, the registers would be restored for the calling software. It should be noted that this basic approach resulted from our early experiences with a particular commercial real-time executive and its C support libraries. After studying things, we realized we could streamline the whole process dramatically. Why not simply take advantage of the fact that a C compiler saves and restores as much context as necessary between C function calls? The compiler knows when making a call to another C routine which registers will be saved across the call. This meant that we could ditch swapping tasks with an 80386 task gate and skip the 309 cycles used to do a task switch. And switching tasks was now a simple C-language call into the linked executive--a small set of subroutines linked with the main application, instead of a remote set of services accessible only through software interrupts. Since the basic mechanism of the task switch is now a rather trivial stack frame switch, we called the design the Stack Swap eXecutive, or SSX (Listing 1, Listing 2, and Listing 3). Pros and Cons of SSX There are many things to recommend SSX. The native C-language interface and frisky stack-swap task switch make it very fast and efficient. And it is very small. The small size can save precious RAM or EPROM space that a larger executive might take up. SSX is also very portable. This article uses the executive in the MS-DOS environment but it is flexible enough to use in embedded applications or on different processor families. And since SSX is a very minimal executive, it is also easy to understand, port, modify, and extend. There are, of course, a few limitations to using SSX. SSX is written in C and works best with an application written in C or C++, although it's not too hard to call into SSX from assembly after saving a few registers. SSX must be linked with your application. If you have a number of separate programs which much share CPU time and resources, they must all be linked together. Another disadvantage with SSX is its lack of features compared with many commercially-available executives. Features of SSX SSX is a real-time, preemptive, multitasking executive. Tasks are allowed to run until: The task readies another task of equal or higher priority A time slice (a clock tick) passes and there are other ready tasks of equal priority An interrupt readies a task of equal or higher priority The task explicitly gives up the CPU by calling the ssx_delay routine Synchronization of tasks is accomplished via shared data structures of type wait_q. A task that needs to wait for an event calls ssx_wait with a wait_q as an argument. If the event has already occurred, as indicated by a message flag at the queue, the task does not get suspended but remains ready. When another task (or interrupt) wishes to ready a sleeping task, it issues an ssx_alert to the queue. The highest priority task waiting is readied. If no task is waiting, the message flag is set. Out of these two basic primitives you can build just about anything you could possibly need! This model of synchronization has the advantage that it is quite efficient to implement. There is no hashing of addresses onto sleep queues or any of that kind of messiness. Moreover, it seems reasonable to assume in today's object-oriented society that if tasks are cozy enough to be synchronizing with each other, they should be well enough acquainted to share the wait_q data object. At the risk of making our lean mean executive seem feature-laden, we also provided a primitive for waiting at a wait_q with an alarm timer set. If the event does not occur within the specified period of time, the alarm goes off and the task becomes ready again with a return value indicating the reason for the resumption. And to round out our real-time toolkit, we have the ssx_task_delay call which simply excuses the task from execution for a specified number of clock ticks. A crucial aspect of any real-time executive is how it is accessed from an interrupt to signal events to task-level code. SSX provides two calls which are used to frame the interrupt handler code. ssx_lock is called right after the interrupt handler saves the necessary CPU register. ssx_lock disables task switching so that the executive doesn't try to switch out the interrupt handler before it has completed its processing. ssx_unlock is called at the end of the interrupt handler right before it restores the CPU registers. This allows task switching to take place again. It should be noted that for extra efficiency you can skip both these calls if the interrupt handler meets the following criteria: It only makes one ssx_alert call. All hardware processing is done before the call to ssx_alert (e.g. the interrupt controller has been reset and any other steps necessary to clear the interrupt at the requestor have been taken). The idea is that at that point, the interrupt thread has just become a continuation of the task that was running. The actual interrupt return can now happen at any time. The ssx_lock and ssx_unlock calls can also be used from the task level to temporarily disable rescheduling. If a task is going to do something time-consuming enough that it doesn't want to risk masking interrupts, but can't afford to be switched out while performing the specific activities, it can call ssx_lock to lock control of the CPU. Interrupts can still be handled, but any changes they make to the task state will not be examined until the ssx_unlock call is invoked. For a complete list of function calls see Table 1. Porting SSX SSX currently works with Borland C and C++ compilers in the small model. SSX will need porting to a different memory model or an environment other that MS-DOS. To port SSX from the MS-DOS environment to a new one you must port four functions. They are: ssx_task_create stack_swap disable_ints enable_ints A task is created by a call to ssx_task_create before or after ssx_run is called. You must have at least one task created before calling ssx_run. When a task is created a stack space is allocated. This stack is then set up so that when the task's stack pointer is swapped in, a simple return from stack_swap will start the task off. Figure 1 shows what the newly-created task's stack looks like after being created for the MS-DOS executive. This sets the task up to be run for the first time. The code /* stack_swap - switch from stack of current task * to stack of new task */ LOCAL void stack_swap(unsigned int **old_stack_ptr, unsigned int **new_stack_ptr) { asm or di,di /* fool compiler into saving */ asm or si,si /* di and si registers */ /* save stack ptr of old task */ *old_stack_ptr = (unsigned int *)_SP; /* load stack ptr register with stack ptr */ /* of new task */ _SP=(unsigned int)*new_stack_ptr; } shows the C listing of stack_swap. On entry, the code executes two lines of in-line assembly language. These instructions make sure the compiler saves and restores the two register variables, 80X86 registers di and si. The last two instructions use the pseudo-variable _SP. With Borland C compilers _SP allows you to directly access the 80X86 sp (stack pointer) register. This part of the code stores the stack pointer of the old task and puts the stack pointer of the new task in the sp register. This is all the code has to do to switch tasks. The code stack_swap proc near ; prologue for a C function push bp mov bp,sp push si push di ; this fools compiler into saving ; si and di because ; they are register variables or di,di or si,si ; save stack pointer for old task mov bx,word ptr [bp+4] mov word ptr [bx],sp ; load stack ptr register with stack ; ptr of new task mov bx,word ptr [bp+6] mov sp,word ptr [bx] ; epilogue for a C function pop di pop si pop bp ret stack_swap endp shows an 80X86 assembly-language listing of the C function stack_swap. In the epilogue of the function stack_swap, registers di, si, and bp are popped off the stack. We have placed these values on the task's stack during ssx_task_create. The last instruction is ret which will pop the return address off the stack and execute the function run_new_task. run_new_task enables interrupts and runs the new task by calling a function pointer. The code /* run_new_task - starts up a new * task making sure interrupts are * enabled */ LOCAL void run_new_task(void) { ints_on(); (t_current-task_ptr)(); } shows the function run_new_task. disable_ints is a routine that gets the current state of interrupts and then disables interrupts and returns the previous state of interrupts. In the MS-DOS version of SSX this function is coded as in-line assembly language code. The pseudocode disable_ints(void) { save current state of interrupts (enabled or disabled); disable interrupts; returned saved state of interrupts (positive integer if they were enabled 0 is disabled) } indicates what is necessary to port disable_ints to other environments. enable_ints enables interrupts. In the MS-DOS version of this function we have used a Turbo C macro that places an 80X86 sti instruction in the code. SSX Demo Code The file demo.c (Listing 4) demonstrates many of the calls in SSX. This MS-DOS demo program sets up a timer interrupt and then creates several tasks. The timer interrupt handler uses vector 8 on the MS-DOS PC. This vector is called 18.2 times a second. Each time the interrupt routine is called ssx_clock_tick is called. This is one area of the code that is not portable. On an MS-DOS AT PC you could also use the interrupt that is called 1,024 times a second if you need more resolution than this timer supplies. After the timer handler is setup the demo program sets up several tasks. Here is a description of the tasks that are made. Print queue tasks--These tasks increment a counter (one for each task) and print the value of the counter. These tasks use a semaphore that allows only one task at a time to call cprintf since it is not reentrant. The code /* initialize semaphore to having * waiting message */ #define SET_SEMAPHORE(wqptr) \ (wqptr)-mesg_flg=1; \ (wqptr)-task_ptr=NULL /* initialize wait_q to NULL task_ptr * and no */ /* message waiting */ #define INIT_WAIT_Q(wqptr) \ (wqptr)-mesg_flg=0; \ (wqptr)-task_ptr=NULL contains macros to setup semaphores and wait queues. Many Standard C functions are not reentrant, so keep that in mind when using C library functions. Five of these tasks are created. Time-slice tasks--These tasks get a full time slice to increment a counter (one for each task). Since print-queue tasks have to get permission to print each time through, their counters will increment slower than these time-slice tasks. Five of these tasks are created. Print-time-slice task--This task prints the counter values for each time-slice task once every three seconds. This task must also use our print semaphore each time it needs to print. Keypress task--This task checks for a keypress every two seconds and calls ssx_stop if it finds one. System-time task--This prints the system time in seconds every second. This task must also use our print semaphore each time it needs to print. This code has been tested with Turbo C 2.0, Turbo C++ 1.0, and Borland C 3.1. To make the demo program type: tcc demo.c ssx.c or bcc demo.c ssx.c Since this code uses in-line assembly language you will also need tasm if you are using Turbo C 2.0 or Turbo C++ 1.0. We hope you will find this executive portable and easy to use. For many applications this will have more than enough features. If you find it is missing something you need, no big deal. Change it. It's easy. It's all in C. Figure 1 Table 1 SSX Function Calls ----------------------------------------------------------------------- int ssx_alert(wait_q ssx_alert sends a message to a task waiting *wqptr) on the specified wait_q. If there is already a message waiting on the specified wait_q, then a MW_ERR (see ssx.h) is returned. If there are any tasks waiting on the wait_q, the first task is scheduled if it has a high enoughpriority. If no task is waiting then a message is left on the wait_q. int ssx_init(void) ssx_init is called before creating tasks or running the SSX executive. This initializes SSX executivedata. int ssx_task_create ssx_task_create is called to create a task to (unsigned char task_pri, run with the SSX executive. Following is list unsigned char task_id, of parameters passed to ssx_task_create: fptr task_ptr, unsigned name -- an eight-character name for the int stack_size, char task. This is useful in debugging SSX *name) applications. The size of name is setup in ssx_conf.h. stack_size -- the size of the stack for this task in bytes. In our demo application we are using 0x200. This will depend on your application. Start high if you think you will be deeply nested or you are using a lot of automatic variables. task_id -- the ID for the task being created. IDs range from 1 to 0xfe. ID 0xff is reserved for a background task that SSX creates. ID 0 is used in some SSX calls when a task wants to refer to itself. task_pri -- the task priority. Tasks with the highest priority are scheduled to run first. Priorities range from 0 (highest priority) to 0xff (lowest priority). task_ptr -- a pointer to a function. (Its typedef is in ssx.h.) Pass a pointer to the C function that is to be used as a task. A function used as a task must have no parameters passed and return no value. A task function is generally a loop without exit. int ssx_task_delete ssx_task_delete deletes the task with the (unsigned char task_id) task_id passed. If you pass in ID 0, the calling task is deleted. int ssx_wait_with_alarm ssx_wait_with_alarm causes a task to wait for (wait_q *wqptr, long a message for the number of ticks passed. If timeout) no message is received within the timeout number of ticks, a TO_ERR is returned (see ssx.h). ssx_wait_with_alarm returns SUCCESS (see ssx.h) if the task gets a message on the wait_q. long ssx_get_time(void) ssx_get_time gets the current clock ticks in SSX. unsigned char ssx_change_priority changes the priority of ssx_change_priority the calling task, but does not immediately (unsigned char) reschedule tasks. new_priority void ssx_clock_tick(void) ssx_clock_tick is called to inject a clock tick into SSX. This is usually called by a timer interrupt handler. void ssx_lock(void) ssx_lock disables task switching. Called either at the entry to an interrupt handler to prevent the interrupt handler from getting switched out or from task code to reserve control of the CPU for a while. Note: a task cannot wait, delay, or otherwise cause itself to become unrunnable while it has task switching locked. void ssx_run(void) ssx_run is called after you have created at least one task and started the executive running. void ssx_set_time(long ssx_set_time sets the number of clock ticks ticks) in SSX to the number passed in. void ssx_stop(void) ssx_stop is called to shutdown the SSX executive. It returns where ssx_run was called. void ssx_switch(void) ssx_switch forces rescheduling of tasks by SSX. ssx_switch is usually called from within the executive. It schedules the highest priority task in the front of the ready queue. void ssx_task_delay(long ssx_task_delay delays the calling task for ticks) the number of ticks passed. It will be scheduled back in after the number of ticks have passed and when it is highest priority on the ready queue. void ssx_unlock(void) ssx_unlock re-enables task switching. A matching call to ssx_lock must already have been made. Called at the exit of an interrupt handler, or from task level code to release the CPU to other tasks. void ssx_wait(wait_q ssx_wait causes a task to wait for a message *wqptr) on the passed wait_q. The typedef for a wait_q data structure is in the file ssx.h. If there is a message waiting, ssx_wait returns immediately. Ifthere is no message waiting, ssx_wait schedules a new task while the calling task waits for a message. Listing 1 SSX.C -- Stack Swap eXecutive /****************************************************/ /* By Tom Green and Dennis Cronin */ /* 10/19/92 */ /****************************************************/ /* turn on inline asm */ #pragma inline #include #include #include #include #include "ssx.h" #include "ssx_conf.h" /* Task Control Block */ typedef struct tcb { /* task chain forward ptr */ struct tcb *forw; /* task chain backward ptr */ struct tcb *back; /* delay chain forward ptr */ struct tcb *dforw; /* delay chain backward ptr */ struct tcb *dback; /* pointer to task code */ fptr task_ptr; /* pointer to start of allocated stack */ unsigned int *stack; /* task's current stack pointer */ unsigned int *stack_ptr; /* delay counter */ long timeout; /* flag for task timed out */ unsigned char timedout; /* flag for TCB in use */ unsigned char active; /* status flags */ unsigned char status; /* task priority */ unsigned char priority; /* task ID */ unsigned char id; /* for storing extra task context */ char context[CNTXT_SZ]; } tcb; /* misc. defines */ #define TRUE 1 #define FALSE 0 /* background task defines */ #define BG_TASK_ID 0xff #define BG_TASK_PRI 0xff /* make data and code local to this file */ #define LOCAL static /* flags for the TCB status word */ #define T_READY 0 /* ready to run */ #define T_WAITING 1 /* waiting on wait_q */ #define T_DELAYED 2 /* delay timer running */ /* local function prototypes */ LOCAL tcb *get_tcb(void); LOCAL void free_tcb(tcb *tbp); LOCAL void put_ready(tcb *tbp); LOCAL void rotate_tasks(tcb *tbp); LOCAL void put_delay(long timeout); LOCAL void run_new_task(void); LOCAL void bg_task(void); LOCAL void stack_swap(unsigned int **old_stack_ptr, unsigned int **new_stack_ptr); LOCAL int disable_ints(void); /* local variables */ LOCAL long sys_time; /* system timer */ LOCAL unsigned char slice_cnt; LOCAL int running; LOCAL int initd; LOCAL jmp_buf jbuf; LOCAL int switch_lock; /* task control */ LOCAL tcb t_pool[MAX_TASKS + 1]; /* pool of TCBs */ LOCAL tcb t_ready; /* head of ready task queue */ LOCAL tcb t_null; /* the NULL task */ LOCAL tcb *t_free; /* head of free queue */ LOCAL tcb *t_current; /* pointer to current task */ /* delay control */ LOCAL tcb d_chain; /* q head for delayed tasks */ /* MACROS to unlink from task and delay queues */ /* t_unlink - must be used w/ interrupts off */ #define t_unlink(tbp) \ { \ (tbp)->back->forw = (tbp)->forw; \ if((tbp)->forw != NULL) \ (tbp)->forw->back = (tbp)->back; \ } /* d_unlink - must be used w/ interrupts off */ #define d_unlink(tbp) \ { \ (tbp)->dback->dforw = (tbp)->dforw; \ if((tbp)->dforw != NULL) \ (tbp)->dforw->dback = (tbp)->dback; \ } /* ssx_init - init ssx data */ int ssx_init(void) { int i; tcb *tcbp; if(initd) return(INIT_ERROR); memset(&d_chain,0,sizeof(d_chain)); /* init TCB free queue links */ for(i=0,tcbp=t_pool;i < MAX_TASKS-1;i++,tcbp++){ tcbp->forw = &t_pool[i+1]; } t_pool[i].forw = NULL; for(i = 0;i < MAX_TASKS;i++){ t_pool[i].active=FALSE; } t_current = NULL; t_free = t_pool; t_ready.forw = NULL; switch_lack = 0; /* set up background task */ if((ssx_task_create(BG_TASK_PRI,BG_TASK_ID, bg_task,0x200, "bg_task")) != SUCCESS) return(INIT_ERROR); initd = TRUE; return(SUCCESS); } /* sx_run - this starts executive */ void ssx_run(void) { int val; val = setjmp(jbuf); if(val != 0) return; slice_cnt = 0; sys_time = 0; /* make current task ptr point to dummy tcb so * beginning of time stack pointer save will * have a safe place to save to. */ t_current = &t_null; /* mark SSX as active */ running = TRUE; /* this will start the first task rolling */ ssx_switch( ); } /* sx_stop - this stops executive */ void ssx_stop(void) { int i; int_state_var istate; ints_off(istate); /* free any allocated stacks */ for(i = 0; i < MAX_TASKS; i++){ if(t_pool[i].stack != NULL){ free(t_pool[i].stack); t_pool [i].stack = NULL; } } initd = FALSE; running = FALSE; restore_ints(istate); longjmp(jbuf,1); } /* ssx_task_create - create task and set up tcb */ int ssx_task_create(unsigned char task_pri, unsigned char task_id,fptr task_ptr, unsigned int stack_size,char *name) { unsigned int i; tcb *tbp; int_state_var istate; ints_off(istate); if(task_id == 0) { restore_ints(istate); return(TID_ERR); } /* check for TID already in use */ for(i = 0,tbp = t_pool;i < MAX_TASKS;i++,tbp++) { if(tbp->active && tbp->id == task_id) { restore_ints(istate); return(TID_ERR); } } if((tbp = get_tcb()) == NULL) { /* get a tcb */ restore_ints(istate); return(TCB_ERR); } /* allocate stack for this task */ if((tbp->stack = (unsigned int *) malloc(stack_size)) == NULL){ restore_ints(istate); return(STACK_ERR); } /* fill in the blanks */ strncpy(tbp->context,name,CNTXT_SZ); tbp->priority = task_pri; tbp->id = task_id; tbp->status = T_READY; tbp->timedout = FALSE; tbp->timeout = OL; tbp->forw = tbp->back = tbp->dforw = tbp->dback = NULL; tbp->task_ptr = task_ptr; tbp->stack_ptr = (unsigned int *)(tbp->stack + (stack_size / 2)); /* setup task stack to have address of start up * routine and fake di, si, bp registers to pop. * This part is not portable. the stack looks * like this: * * - high * address of run_new_task * - * bp * --------------------- * si * - * di * - low */ *(-tbp->stack_ptr) = (unsigned int)run_new_task; *(-tbp->stack_ptr) = 0; /* fake BP,SI,DI */ *(-tbp->stack_ptr) = 0; /* on stack */ *(--tbp->stack_ptr) = 0; /* put on active chain */ rotate_tasks(tbp); ssx_switch(); restore_ints(istate); return(SUCCESS); } /* ssx_task_delay - cause task to delay for number of ticks */ void ssx_task_delay(long timeout) { int_state_var istate; ints_off(istate); if(timeout == 0) { ssx_switch(); restore_ints(istate); return; } put_delay(timeout); /* put current task on */ /* delay queue */ t_unlink(t_current); /* take off ready queue */ ssx_switch(); restore_ints(istate); } /* ssx_task_delete - delete a task and remove from queues */ int ssx_task_delete(unsigned char task_id) { unsigned int i; tcb *tp; int_state_var istate; ints_off(istate); /* look for background task ID */ if(task_id == BG_TASK_ID){ restore_ints(istate); return(TID_ERR); } /* look for 'self' form */ if(task_id == 0) { if(t_current) { /* get current task's id */ task_id = t_current->id; } } /* brute force, search all TCBs for matching ID */ for(i = 0,tp = t_pool;i < MAX_TASKS;i++,tp++) { if(tp->active && tp->id == task_id) { break; } } /* see if found match */ if(i == MAX_TASKS){ restore_ints(istate); return(TID_ERR); } switch(tp->status & (T_DELAYED T_WAITING)) { case T_DELAYED: d_unlink(tp); /* remove from delay q */ break; case T_WAITING: t_unlink(tp); /* remove from some */ /* wait_q */ break; case T_DELAYED T_WAITING: t_unlink(tp); /* remove from some */ /* wait_q */ d_unlink(tp); /* remove from delay q */ break; case T_READY: t_unlink(tp); /* remove from ready q */ break; } free(tp->stack); /* free allocated stack */ tp->stack = NULL; free_tcb(tp); /* free up the TCB */ ssx_switch(); restore_ints(istate); return(SUCCESS); } /* ssx_change_priority - change priority for currently * running task, * don't immediately reschedule * returns old priority */ unsigned char ssx_change_priority(unsigned char new_priority) { unsigned char old_priority; int_state_var istate; ints_off(istate); old_priority = t_current->priority; t_current->priority = new_priority; t_unlink(t_current); rotate_tasks(t_current); restore_ints(istate); return(old_priority); } /* ssx_wait - wait on wait_q. reschedule later */ void ssx_wait(wait_q *wqptr) { tcb *tp; tcb *t_cur; int_state_var istate; ints_off(istate); /* check for message flag already set */ if(wqptr->mesg_flg){ wqptr->mesg_flg = FALSE; restore_ints(istate); return; } t_cur = t_current; t_unlink(t_cur); /* take off ready queue */ tp = (tcb *)&wqptr->task_ptr; /* find where to insert waiting task into * wait queue */ while((tp->forw) != NULL) { if(t_cur->priority <= tp->forw->priority) break; tp = tp->forw; } /* insert into queue */ if((t_cur->forw = tp->forw)!= NULL) t_cur->forw->back = t_cur; tp->forw = t_cur; t_cur->back = tp; t_cur->status = T_WAITING; ssx_switch(); restore_ints(istate); } /* ssx_wait_with alarm - wait on wait_q with alarm. * reschedule now */ int ssx_wait_with_alarm(wait_q *wqptr,long timeout) { tcb *tp; tcb *t_cur; int_state_var istate; ints_off(istate); /* check for message flag already set */ if(wqptr->mesg_flg){ wqptr->mesg_flg = FALSE; restore_ints(istate); return(SUCCESS); } t_cur = t_current; t_unlink(t_cur); /* take off ready queue */ tp = (tcb *)&wqptr->task_ptr; /* find where to insert waiting task into wait queue */ while((tp->forw) != NULL) { if(t_cur->priority <= tp->forw->priority) break; tp = tp->forw; } /* insert into queue */ if((t_cur->forw = tp->forw) != NULL) t_cur->forw->back = t_cur; tp->forw = t_cur; t_cur->back = tp; t_cur->status = T_WAITING; /* if there is timeout value, put task on delay queue */ if(timeout) put_delay(timeout); ssx_switch(); /* we were scheduled back in so * see if task timed out and return error */ if(t_cur->timedout){ t_cur->timedout = FALSE; restore_ints(istate); return(TO_ERR); } restore_ints(istate); /* task did not time out, so return success */ return(SUCCESS); } /* ssx_alert - alert wait_q. reshedule if task * alerted is equal or higher * priority than current task */ int ssx_alert(wait_q *wqptr) { tcb *np; tcb *oldtcb; int_state_var istate; ints_off(istate); /* check for message waiting */ if(wqptr->mesg_flg){ restore_ints(istate); /* cannot alert if */ return(MW_ERR); /* messgae is waiting */ } np=(tcb *)wqptr->task_ptr; /* check if there is a task waiting on wait_q */ if(np != NULL){ t_unlink(np); if(np->status & T_DELAYED) d_unlink(np); np->status &= ~(T_WAITING T_DELAYED); put_ready(np); /* switch to waiting task if it is equal * or higher priority */ if(np->priority <= t_current->priority){ oldtcb = t_current; t_current = np; /* check and see if scheduling is disabled */ if(switch_lock == 0) stack_swap(&oldtcb->stack_ptr ,&np->stack_ptr); } restore_ints(istate); return(SUCCESS); } /* fell thru, simply leave message in wait_q */ wqptr->mesg_flg = TRUE; restore_ints(istate); return(SUCCESS); } /* ssx_clock_tick - call this to update ssx clock from * timer interrupt handler */ void ssx_clock_tick(void) { tcb *tp; int_state_var istate; ints_off(istate); if(running == FALSE){ restore_ints(istate); return; } sys_time++; /* do time updates */ tp = (tcb *)&d_chain; /* check for timed out tasks */ while((tp = tp->dforw) != NULL) { if((sys_time - tp->timeout) >= 0) { d_unlink(tp); /* this one's ready */ tp->timedout = TRUE; if(tp->status & T_WAITING) t_unlink(tp); tp->status = T_READY; /* put task on ready queue */ rotate_tasks(tp); } else break; /* passed the ready ones */ } /* round robin rotation */ if((++slice_cnt) == TIME_SLICE){ slice_cnt = 0; /* if task is running and was left ready */ if(t_current && t_current->status == T_READY){ /* remove from ready queue */ t_unlink(t_current); /* puts at back of pri group */ rotate_tasks(t_current); } } ssx_switch(); restore_ints(istate); } /* ssx_set_time - this sets SSX system time */ void ssx_set_time(long time) { int_state_var istate; ints_off(istate); sys_time = time; restore_ints(istate); } /* ssx_get_time - this returns SSX system time */ long ssx_get_time(void) { return(sys_time); } /* ssx_lock - disable task switching */ void ssx_lock(void) { int_state_var istate; ints_off(istate); switch_lock++; restore_ints(istate); } /* ssx_unlock - enable task switching */ void ssx_unlock(void) { int_state_var istate; ints_off(istate); /* call ssx_switch if we are not nested */ if(-switch_lock == 0) ssx_switch(); restore_ints(istate); } /* ssx_switch - run next ready task * notes: there must always be a runnable task w/ * SSX. a background task is created by ssx_init * and this task can never wait or do anything * that would remove it from the active queue. * this saves checks in this routine, making * it more efficient. */ void ssx_switch(void) { tcb *oldtcb; if(!running) return; oldtcb = t_current; /* switch tasks */ t_current = t_ready.forw; /* get next */ /* ready task */ /* * if new task is same as old, do not * bother with switch */ if(t_current == oldtcb) return; /* check and see if scheduling is disabled */ if(switch_lock == 0){ /* we have a new task so do a task switch */ stack_swap(&oldtcb->stack_ptr, &t_current->stack_ptr); } } /* get_tcb - get task control block */ LOCAL tcb * get_tcb(void) { tcb *tbp; if(t_free == NULL) return(NULL); tbp = t_free; t_free = tbp->forw; tbp->active = TRUE; return(tbp); } /* free_tcb - free task control block */ LOCAL void free_tcb(tcb *tbp) { /* '****' for debug TCB not in use */ tbp->timeout = 0x2a2a2a2aL; tbp->active = FALSE; tbp->forw = t_free; t_free = tbp; } /* put_ready - put task at head of ready queue */ LOCAL void put_ready(tcb *tbp) { tcb *tp,*np; unsigned int priority; /* get priority of task to be inserted in chain */ priority = tbp->priority; /* put on the active chain */ tp = (tcb *)&t_ready; /* sort in order of decreasing priority, put at * head of pri group */ while((np = tp->forw) != NULL && np->priority < priority) tp = np; /* link in */ tbp->forw = np; tbp->back = tp; tp->forw = tbp; if(np != NULL) np->back = tbp; } /* rotate_tasks - put task at back of ready queue */ LOCAL void rotate_tasks(tcb *tbp) { tcb *tp,*np; unsigned int priority; /* get priority of task to be inserted in chain */ priority = tbp->priority; /* put on the active chain */ tp = (tcb *)&t_ready; /* sort in order of decreasing priority, * put at back of pri group */ while((np = tp->forw) != NULL && np->priority <= priority) tp = np; /* link in */ tbp->forw = np; tbp->back = tp; tp->forw = tbp; if(np != NULL) np->back = tbp; } /* put_delay - put task on delay queue */ LOCAL void put_delay(long timeout) { tcb *tp,*np; t_current->timeout = timeout + sys_time; /* actual time ready */ t_current->timedout=FALSE; t_current->status = T_DELAYED; tp = (tcb *)&d_chain; /* sort in order increasing target time */ /* trick to solve wrap of sys_time */ while((np = tp->dforw) != NULL) { if(timeout <= np->timeout - sys_time) /* hit a more future one */ break; tp = np; } /* link in */ t_current->dforw = np; t_current->dback = tp; tp->dforw = t_current; if(np != NULL) np->dback = t_current; } /* run_new_task - starts up a new task making sure * interrupts are enabled */ LOCAL void run_new_task(void) { ints_on(); (t_current->task_ptr) (); } /* bg_task - must have a background task */ LOCAL void bg_task(void) { while(1); } /* WARNING - routines from here on are not portable */ /* stack_swap - switch from stack of curent task to * stack of new task */ LOCAL void stack_swap(unsigned int **old_stack_ptr, unsigned int **new_stack_ptr) { asm or di,di /* fool compiler into saving */ asm or si,si /* di and si registers */ /* save stack pointer of old task from sp reg */ *old_stack_ptr = (unsigned int *)_SP; /* load sp reg with stack pointer of new task */ _SP=(unsigned int)*new_stack_ptr; } /* disable_ints - disable interrupts and return state * of interrupts before before they * were disabled. Returns positive * integer if they were enabled */ LOCAL int disable_ints(void) { asm pushf /* save flags to get */ /* interupt status */ asm cli /* interrupts off */ asm pop ax /* get interrupt state from */ /* flags that were pushed */ asm and ax,0200h /* and flags to get interrupt */ /* status */ return(_AX); } /* End of File */ Listing 2 SSX.H -- definitions for SSX /**************************************************** /* By Tom Green and Dennis Cronin */ /* 10/19/92 */ /****************************************************/ /* function pointer */ typedef void (*fptr)(void); /* this is a wait_q structure */ typedef struct wait_q{ void *task_ptr; int mesg_flg; }wait_q; /* SSX prototypes */ int ssx_init(void); void ssx_run(void); void ssx_stop(void); int ssx_task_create(unsigned char task_pri, unsigned char task_id,fptr task_ptr, unsigned int stack_size,char *name); void ssx_task_delay(long ticks); int ssx_task_delete(unsigned char task_id); unsigned char ssx_change_priority(unsigned char new_priority); void ssx_wait(wait_q *wqptr); int ssx_wait_with_alarm(wait_q *wqptr,long timeout); int ssx_alert(wait_q *wqptr); void ssx_clock_tick(void); void ssx_set_time(long ticks); long ssx_get_time(void); void ssx_lock(void); void ssx_unlock(void); void ssx_switch(void); /* SSX status codes */ #define SUCCESS 0 /* task ID error */ #define TID_ERR 1 /* message waiting error */ #define MW_ERR 2 /* no TCBs error */ #define TCB_ERR 3 /* could not allocate stack for task */ #define STACK_ERR 4 /* task timed out (wait_with_alarm) */ #define TO_ERR 5 /* error initializing SSX */ #define INIT_ERROR 6 /* initialize semaphore to having waiting message */ #define SET_SEMAPHORE(wqptr) (wqptr)->mesg_flg=1; \ (wqptr)->task_ptr=NULL /* initialize wait_q to NULL task_ptr and no * message waiting */ #define INIT_WAIT_Q(wqptr) (wqptr)->mesg_flg=0; \ (wqptr)->task_ptr=NULL /* End of File */ Listing 3 SSX_CONF.H -- configuration info for SSX /*****************************************************/ /* By Tom Green and Dennis Cronin */ /* 10/19/92 */ /*****************************************************/ /* main config params */ /* maximum number of tasks */ #define MAX_TASKS 24 /* 1 tick per slice */ #define TIME_SLICE 1 /* number of bytes of context info */ #define CNTXT_SZ 8 /* enable interrupts - this will have to be ported * in other environments */ #define enable_ints enable /* variable type to keep interrupt status in */ typedef int int_state_var; /* save off current state, disable all interrupts */ #define ints_off(isv) isv=disable_ints() /* explicitly enable interrupts */ #define ints_on() enable_ints() /* reload previous interrupt enables */ #define restore_ints(isv) { if(isv) enable(); } /* End of File */ Listing 4 DEMO.C -- demo code for SSX *****************************************************/ /* By Tom Green and Dennis Cronin */ /* 10/19/92 */ /****************************************************/ #include #include #include #include #include #include "ssx.h" /* vector for PC timer interrupt */ #define TIMER_INT_VEC 8 /* 18.2 clock ticks per second on PC */ #define TICKS_PER_SECOND 18L #define NUM_PQ_TASKS 5 #define NUM_TS_TASKS 5 void key_task(void); void print_ts_count_task(void); void sys_time_task(void); void ts_task(void); void print_q_task(void); void interrupt tick_handler(void); unsigned int pq_task_counter=0; unsigned int ts_task_counter=0; wait_q print_q; void interrupt (*old_timer_int)(void); unsigned long pq_count[NUM_PQ_TASKS]; unsigned long ts_count[NUM_TS_TASKS]; void main(void) { unsigned int i; clrscr(); pq_task_counter=0; ts_task_counter=0; if((ssx_init()) == INIT_ERROR){ printf("\nError initting SSX"); exit(1); } /* set up timer tick interrupt handler */ old_timer_int = getvect(TIMER_INT_VEC); setvect(TIMER_INT_VEC,tick_handler); /* install print q tasks */ for(i = 1;i <= NUM_PQ_TASKS;i++){ if((ssx_task_create(1,i,print_q_task,0x200, "task"))!=SUCCESS) cprintf("\nError creating task %d",i); } /* install time slice tasks */ for(i = NUM_PQ_TASKS + 1;i <= (NUM_TS_TASKS + NUM_PQ_TASKS);i++){ if((ssx_task_create(1,i,ts_task,0x200, "ts_task"))!=SUCCESS) cprintf("\nError creating ts task"); } /* install print ts count task */ if((ssx_task_create(1,22,print_ts_count_task,0x200, "pts_task"))!=SUCCESS) cprintf("\nError creating ts count task"); /* install keypress task */ if((ssx_task_create(1,21,key_task,0x200, "key_task"))!=SUCCESS) cprintf("\nError creating keypress task"); /* install system time task */ if((ssx_task_create(1,23,sys_time_task,0x200, "st_task"))!=SUCCESS) cprintf("\nError creating system time task"); cprintf("\r\nDone creating tasks"); cprintf("\r\nPress key to start SSX"); bioskey(0); clrscr(); SET_SEMAPHORE(&print_q); ssx_run(); /* reset to old timer int handler */ setvect(TIMER_INT_VEC,old_timer_int); clrscr(); } /* ts_task - time slice task. this task gets a full * time slice to increment counter. * print_ts_count_task prints counters. */ void ts_task(void) { int task_num; task_num = ts_task_counter++; ts_count[task_num] = 0L; while(1){ ts_count[task_num]++; } } /* print_q_task - this task waits for print q we have * set up and then increments counter * and prints and then releases print q */ void print_q_task(void) { unsigned int y,task_num; task_num = pq_task_counter++; y = task_num + 1; pq_count[task_num] = 0L; while(1){ ssx_wait(&print_q); pq_count[task_num]++; gotoxy(1,y); cprintf("Task %02d %08lx",task_num, pq_count[task_num]); ssx_alert(&print_q); } } /* print_ts_count_task - task that prints counts * from time sliced tasks * once every 3 seconds */ void print_ts_count_task(void) { int task_num; while(1){ ssx_wait(&print_q); for(task_num = 0;task_num < NUM_TS_TASKS; task_num++){ gotoxy(1,task_num + NUM_TS_TASKS + 1); cprintf("TS Task %02d %08lx",task_num, ts_count[task_num]); } ssx_alert(&print_q); ssx_task_delay(3 * TICKS_PER_SECOND); } } /* key_task - task that checks for keypress every * 2 seconds. calls ssx_stop when key * is pressed */ void key_task(void) { while(1){ if((bioskey(1)) != 0){ bioskey(0); ssx_stop(); } ssx_task_delay(2 * TICKS_PER_SECOND); } } /* sys_time_task - prints how long it has been * running once every minute */ void sys_time_task(void) { int task_num; while(1){ ssx_wait(&print_q); gotoxy(1,15); cprintf("Sytem time = %08lx seconds", ssx_get_time()/ TICKS_PER_SECOND); ssx_alert(&print_q); ssx_task_delay(1 * TICKS_PER_SECOND); } } /* tick_handler - handles 18.2 per second timer * interrupts on PC */ void interrupt tick_handler(void) { /* call original PC timer handler */ (*old_timer_int)(); /* inform SSX of clock tick */ ssx_clock_tick(); } /* End of File */ Developing 80x86-Based Embedded Systems Andrew P. Beck Andrew Beck is a systems engineer specializing in the design of computers for industrial and scientific applications. He has been designing embedded systems more than 15 years. Andrew can be reached at (908) 806-8262. Introduction Embedded systems were once programmed only by those brave souls who to dared write in assembly language and who knew enough about the hardware to program to the chip level. But as the embedded systems market continues to expand, and the need for embedded systems programmers grows, all of that has changed. Most manufacturers are reducing the size of their products. With the shortage of experienced embedded systems programmers, more MS-DOS programmers will be required to develop embedded programs. In this article, I will try to ease the path for those of you who have never developed an embedded system. I address the fundamental issues of developing a program to run on an 80x86-based system. I explain the steps necessary to convert your C programs into embedded code, And I give you some hints on how to avoid some of the common pitfalls of embedded systems design. A Little History In 1978 Intel developed the 8086 microprocessor. Due primarily to IBM's use of it in their first PC, the 80x86 family has become the industry's most popular microprocessor. There are literally millions of 80x86-based computer systems in operation today. Because of the PC's popularity, scores of lowcost, high-quality software development tools have been created for it. Soon after the introduction of the 8086, embedded systems designers started developing new products around it. These early designers had to rely on high-priced development systems to help them refine their designs and develop their software. This, along with the large number of support chips required for even a modest 8086 design, prevented many designers from using the 8086 in their embedded designs. Only those who could justify the high initial development cost and long development cycle could afford to use it. Intel soon realized that embedded systems offered a potentially large market, so in 1982 they announced the 80186 microcontroller. The 80186 is a highly-integrated controller that includes some of the most often used peripherals on the same chip as the microprocessor. In the past couple of years there has been a dramatic increase in the number of new designs utilizing this controller. In addition to Intel, NEC and AMD currently produce microcontrollers that are code compatible with the 8086. Users of all types of equipment, from consumer electronics to high-end instrumentation, are demanding ever more intelligent devices. Manufacturers are continuing to put more powerful systems into ever smaller boxes. One market that has seen a tremendous growth in the last few years is the field of industrial instrumentation and automation. The typical handheld instrument of today is required to outperform desktop systems produced just a few years ago. In addition, manufacturers are striving to make these systems easier to use, while introducing new products faster than ever. All of these factors have combined to force the embedded systems designer to look for tools to help produce better products in a shorter period of time. One of the most powerful tools currently available is the C programming language. C allows the designer to develop programs faster, and with fewer bugs, than assembly language. C programs can be better structured, and therefore more portable and maintainable, than their assembly-language counterparts. A few years ago, many designers were reluctant to use C in their designs. They felt the language used too much memory. It was often difficult to write interrupt handlers in C, and the designer had little control over placement of data or code. All of this has changed. Today's C compilers offer a high level of optimization. In large complex programs, the C implementation is often no larger than the assembly version. Most compilers provide support for generating interrupt function completely in C. What is an Embedded System? An embedded system generally consists of a microprocessor and a few peripherals that run a dedicated program. That program is usually stored in non-volatile memory, such as PROM. Embedded systems are not programmable by the end user, and usually have very limited I/O resources. While many embedded systems are designed with off-the-shelf components, the majority of them are based on custom-designed hardware. Since many embedded systems do not run under an operating system, the programmer is responsible for supplying all of the low-level I/O functions. And unlike MS-DOS-based programs that can be loaded from disk, embedded systems require you to load the program into PROM. Consequently, developing and debugging these systems presents some unique challenges. Embedded Systems Development Tools Due to the popularity of the PC, there are a number of low-cost, high-quality development tools available for the 80x86 processors. Two of the most popular C compilers for the PC are produced by Borland and Microsoft. Because of their popularity, these compilers have led to the development of many third-party design tools that support embedded systems. To create an 80x86-based embedded system, you will need a standard MS-DOS-based assembler, compiler, and linker. You will also need a locate program. This program converts a standard MS-DOS .exe file into a form that can be burned into PROM, typically a hex or binary file. If this is your first embedded system design, I'd recommend that you purchase a third-party embedded-systems development package. In addition to the locate program, the vendor will supply the requisite startup code and runtime support functions for your compiler. Most vendors will also supply their own libraries, or a set of utilities to remove non-ROMable functions from your compiler's library. Basic Principles Programs written for MS-DOS-based systems run in the system's RAM. When a program is executed, the program loader locates an available block of memory and loads the program into it. MS-DOS .exe files are simply relocatable files. As the program is loaded, the program loader uses a table stored in the file to resolve the relocatable segment addresses. Embedded systems require two types of memory, volatile RAM and non-volatile PROM (or ROM). All of the program's variables, as well as the stack, are located in RAM. The executable portion of the program is stored in PROM. Constants, such as string data and initialized variables, are initially stored in the PROM. However, before the program can be executed, this data must be copied to RAM. This is the responsibility of the system's startup code. This code is also responsible for initializing the segment registers, clearing the uninitiliazed RAM, and setting up the heap. Finally the startup code must call the program's main function. The startup code is the most important piece of software in the system. It is also the most difficult for the first-time embedded systems programmer to develop. If you purchase a third-party embedded development package, the vendor will supply the requisite startup code. When the 80x86 processor begins running after reset, it executes the code located at 0xffff:0xfff0. Normally the only instruction at this location is a jump to a lower address in PROM, where the actual program is stored. The first piece of code to be executed after the initial jump is the startup code. The startup code is responsible for establishing an environment that the C program can run in, consequently it is always written in assembly. All embedded C programs require that at least three segments be defined. The code segment contains executable instructions--your program. The data segment contains all of the program's static variables. Finally, the stack segment is where non-static variables, passed arguments, and function return addresses are stored. Depending on the memory model used, some programs will also require a far data segment. Some advanced applications may make use of multiple code, data, or stack segments. Since your programs, constants, and initialized data will be stored in PROM, the startup code must copy them into RAM. It must also zero out the uninitialized data area so that all other static variables will be initialized to zero. Finally it has to setup all of the segment registers and the stack pointer. Most of the time the startup code will also be responsible for setting the system's interrupt vectors. Interrupts 0 through 6 are special interrupts generated by the CPU. Interrupts 0, 4, 5, 6 and 7 are processor exception interrupts. They should point to an exception-handler function. Interrupt 2 is the nonmaskable interrupt (NMI). If your hardware utilizes the NMI, its vector must be initialized to your NMI handler's address. If you are using a microcontroller, its on-board peripherals will use several other interrupts. If you operate them in the interrupt mode, you must initialize their vectors. The remaining interrupts are available for your application. All unused interrupts should be initialized to point to a dummy function that simply performs a return. This prevents an errant interrupt from crashing the system. In some systems, the startup code may also be responsible for initializing some, or all, of the system's hardware. Once the initialization is complete, the startup code calls the main function. Command-line arguments can be emulated by pushing data onto the stack before main is called. These arguments can then be accessed via the standard argv and argc variables. Typically, these would be used to indicate the status of the system's setup switches, or some other hardware interface. Memory Considerations Since constants are stored in PROM and then copied to RAM, you pay a penalty in terms of memory usage. Constants require twice as much memory in embedded systems as they do in MS-DOS-based system. Careful design can reduce this penalty. One way to reduce memory usage is by eliminating storage of duplicate constants. Both the Turbo-C and Microsoft compilers can be instructed to merge duplicate string data. However, they can eliminate duplicate strings only within a single program module. If you use a lot of strings, careful attention to where and how you display them can help you conserve your system's precious memory. You can place all of your display functions, along with their associated strings, in one module. Or you can simply place all of your text in one module and reference it via pointers. Consider how your strings are constructed. If you use a lot of common substrings, you may want to break them into separate strings so that the compiler can eliminate duplicates for you. For example, consider an application in which you need to display three labels across the bottom of your screen. The text of each label varies depending on which mode the system is in. You could simply generate separate print statements for each set of labels as shown below: printf("DOWN RUN UP"); printf("LEFT RUN RIGHT"); printf(" RUN STOP"); However, if you had several dozen statements like these, you would find that you had a lot of duplicate substrings. In that case a better approach would be to create pointers to each substring as depicted below. You could then direct the compiler to merge duplicate strings, and save a considerable amount of memory. printf("%s %s %s", "DOWN ", " RUN ", " UP"); printf("%s %s %s", "LEFT ", " RUN ", "RIGHT"); printf("%s %s %s", " ", " RUN ", " STOP"); Another technique for conserving memory is to place all of your strings into their own far data segment and reference them via far pointers. This technique does not need the data to be copied to RAM. Unfortunately the Turbo-C compiler does not support the printf far string pointer argument (%Fs), so this technique is only valid if you are using the Microsoft compiler. printf("%Fs %Fs %Fs", down_text, run_text, up_text); printf("%Fs %Fs %Fs", left_text, run_text, right_text); printf("%Fs %Fs %Fs", blank_text, run_text, stop_text); Building an Application You can program the bulk of your embedded program just as you would an MS-DOS-based program. However, you must avoid using any library functions that are not ROMable. The ROMability of functions varies tremendously from library to library. If you are using an embedded-system development package, the vendor will indicate which functions can be ROMed. Many of the low-level I/O functions in MS-DOS, such as putch and getch, interface to the hardware via MS-DOS's interrupt 21 handler. High-level functions, such as printf and scanf, use this same interrupt. If you emulate the MS-DOS interrupt 21 handler you can use most of your library's standard I/O functions. Except for the disk-I/O functions, all of Turbo C's I/O functions are fully-ROMable if you supply an interrupt 21 handler. If you use your library's standard functions, instead of developing your own, your development time will be reduced and your program will be more portable. If your development package supports it, you can also include floating-point math in your application. You will have to supply a runtime interface to the floating-point emulator, or coprocessor, as well as a floating-point exception handler. Most embedded-systems development packages supply the requisite code. If you are developing your own embedded support functions, you have to make sure you don't inadvertently use a library function that is MS-DOS dependent. The library that comes with Borland's Turbo C compiler tends to be more independent of MS-DOS than does Microsoft's. For example, if you supply emulation for MS-DOS's interrupt 21 console I/O functions, you can use Turbo C's printf function without any other support. On the other hand, Microsoft's printf depends on several low-level MS-DOS functions and cannot be embedded unless you supply emulations of them. If you are using Microsoft's C, refer to their C Compiler User's Guide for a list of the functions that are MS-DOS independent. Both Microsoft and Borland will sell you the source code for most of their library functions, the exceptions being their math and graphics libraries. Compiling, Linking, and Locating Once you've written your program, you can compile and link it just as you would any MS-DOS-based program. The only difference is that you will have to link in your startup code in place of the compiler's. You will also have to link in any special runtime support functions that are required by your system. Make sure that your startup code is the first module listed in the link list, this ensures that it is the first code executed. You must also instruct your linker to create a map file. This will be used by the locate program to aid in determining where code and data are to be placed in your target system. The output file produced by your linker will be an .exe file, a relocatable image of your program. This file contains two parts, a relocation header and the actual program data and code. The relocation header consists of a series of offsets into the program that point to segment references. The locate program must modify these references so that they match the memory layout of your target system. The locate program will use the link map to determine the length and location of each of the segments contained in the .exe file. As a final step the locate program must output your program in a form that can be downloaded into a test system or burned into PROM. Usually this will be a hex or binary file as dictated by the needs of your PROM programmer, emulator, or debugger. The actual locate procedure will vary somewhat depending on your locate program. Most of them require you to create a file that contains a memory map of the target system, along with directives indicating where each of the segments is to be located. You must also indicate which segments are to be duplicated in PROM. Typically this will consist of only the segment that contains your initialized data. Debugging Embedded systems present their own unique problems when it comes to debugging. If you simply burned your program into PROM and ran it, it would be virtually impossible to determine where a problem existed when it didn't run properly. Most embedded systems have very limited I/O capability--often nothing more than a DIP switch and a few LEDs. These would prove useless if you were trying to debug anything more than the simplest of programs. Traditionally embedded systems designers have used an incircuit emulator (ICE), or a dedicated development system to debug their designs. While development systems provide the most integrated development environment, they tend to be very expensive, often costing upwards of $20,000. Many lowcost emulators are available, and they are fine for developing hardware, but most fall short when it comes to debugging complicated programs. If you've opted to use your compiler's libraries, and you've carefully designed and structured your program, you can test a large percentage of your code right on your PC. You can generate test functions that emulate your target system's low-level I/O routines. These routines can make calls to MS-DOS to display data on the PC's screen and get input from the keyboard. If you've emulated the MS-DOS interrupt 21 handler in your program you may not need to generate any low-level test functions. Perhaps's the best debugging environment for embedded systems is one with which most programmer's are already familiar--the source-level debugger. Most MS-DOS programmers have, at one time or another, debugged their applications with either Borland's Turbo Debugger or Microsoft's Codeview. Both of these products provide a host of options for debugging your program. They allow you view your source code in its native form, including comments. Both support multiple breakpoints and allow you to single step your program. They will also display the contents of memory as well as the CPU registers. However, their most powerful feature is their ability to display the values of complex data types such as arrays, structures, and unions. Several vendors offer a version of Borland's Turbo Debugger for use on embedded systems. By utilizing the debugger's remote mode you can download and debug your program on your target system. Some vendors offer their own remote debuggers. Like Turbo Debugger, most of these communicate with your target system via an RS232 serial link. Some, such as Paradigm's DEBUG/RT, can be configured to communicate with any type of interface, including parallel ports and PROM emulators. With the power of these debuggers available to you, there is no reason to even test your code on the PC. You can perform all of your testing and debugging right on your target system. This will help you uncover subtle problems, especially timing-related ones, early in the development cycle. Special Considerations Embedded systems are often required to maintain some of their data even when their power has been turned off. This may be data collected and stored for later retrieval, or simply setup information such as the configuration of its peripherals. It's relatively easy to provide this ability in hardware. The most popular way is by equipping the system with battery-backed RAM. While system power is turned off, a small battery applies enough voltage to the RAM for it to maintain its data. From a programmer's standpoint, non-volatile data presents something of a problem. You already know that at reset the startup code copies over any initialized variables and zeros out the uninitialized area. This will reinitialize all data stored in your data segment. So you have to find a way to protect your non-volatile data from the startup code. The simplest method would be eliminate the code that zeros out the uninitialized data area. This presents its own set of problems. If you didn't zero out this area you couldn't depend on uninitialized static variables to be zero. An unwary maintenance programmer who wasn't aware of this particular peculiarity could easily be tripped up by this. The best way to solve this problem is to create a segment for all of your non-volatile variables. Since they're not included in either the initialized or uninitialized data areas, the startup code will not modify them. You still face one other problem though. If your program depends on any of them being in a known state at reset time, and the startup code isn't modifying them, how do they every get initialized in the first place? Again a simple solution is at hand. You must provide a function that sets all of these variables to default values. Then in the startup code, or at the beginning of main, you call this function if some predefined condition exists. For example, you could generate a checksum for all of the non-volatile variables, and call your reset function if the checksum is invalid. Or you can test for some user-generated condition such as a certain switch setting or keyboard input. Performance Issues The first time you run your embedded application you may be disappointed with its performance. Often your program will simply not run as fast as expected. This problem is particularly prevalent if you test and debug your program on the PC. Most embedded systems run much slower than today's PCs. Testing your program on the target system is one way to identify performance problems early, before they overwhelm the project. Most PCs are so fast that many MS-DOS programmers do not pay much attention to the speed issues that were a common concern just a couple of years ago. Embedded systems programming still requires that extra attention. Careful attention to selection and implementation of algorithms is paramount. One of the simplest ways to get a little more performance out of your system is to set a couple of your compiler's command-line switches. If you're using the Intel 80186 microprocessor, set the switch that enables code generation for it. The 80186 includes several instructions, not included in the 8086, that can increase the speed of interrupts and function calls. If you have plenty of PROM in your system, set your compiler to optimize for speed, instead of for code size. One area that causes a lot of problems for embedded systems programmers is the use of floating-point math. Most embedded systems lack a math coprocessor, so you'll have to depend on the floating-point emulator in your math library. Even with a highly-optimized, floating-point library, floating-point math is much slower than integer math. Many embedded applications don't require the range of numbers that floating-point supports. Often fixed-point numbers will offer more range and precision than you need. If addition to the improvement in speed, switching from floating-point to fixed-point math can save a considerable amount of RAM. One of the biggest issues in the design of embedded systems is the tradeoff of hardware versus software. Often hardware designers will opt to eliminate a simple circuit that can be emulated in software. With the rising cost of software development, these tradeoffs have to be considered carefully. In low-volume applications, the software development cost may far exceed the savings in hardware cost. If your application makes heavy demands on the processor, one or two simple circuits may significantly improve overall system performance. One example of a common input device that is often emulated in software is the keyboard controller. Often it will be implemented by simply connecting a few switches to one of the system's input ports. The software must then periodically poll the status of the switches to determine if any has been pressed. The software must also determine if it is a valid switch closure or simply "switch bounce" from the last key press. In simple systems this rarely taxes the processor. In many real-time systems, however, the processor may service several I/O devices. It may not be able to spend the time necessary to poll and debounce the keyboard. In this case a simple hardware circuit can be used to decode and debounce the keyboard and interrupt the processor only when a valid key press occurs. Obviously, the better your understanding of the hardware, the easier it will be for you to spot potential problems and inform the hardware designers before the design is finalized. Conclusion Developing software for an embedded system is different than for a MS-DOS-based system, but it isn't necessarily harder. In this article I've only skimmed the surface of embedded systems programming. There are many issues that are beyond the scope of an introductory text. Before embarking on your first design, I'd recommend that you develop a good understanding of your target hardware platform. Also look carefully at the development tools that are available, and consider which best meet your needs and budget. Three-Dimensional Modeling Under Windows 3.1 Thomas W. Olsen Thomas writes a variety of Windows and DOS software for a major insurance company. He can be reached on the CompuServe Information Service at (76450,1767). Nothing communicates an idea better than a picture. It's the defining principle behind the popularity of Microsoft Windows and other graphical environments. Not long ago, though, you might recall serious debate over whether Windows would ever meet the performance demands of a graphical user interface. Much of the trepidation resulted, no doubt, from the tremendous freedom of MS-DOS programs to directly access video hardware. However, faster CPUs, accelerated video adapters, local bus connections, multimedia, and a mature Graphical Display Interface (GDI) have since conspired in favor of Windows. These changes pose significant opportunities in 3-D modeling software development. Three-dimensional modeling has become increasingly important over time because it mimics elements in the real world. This article focuses on simple strategies for 3-D modeling. The accompanying source code was compiled and linked with the Microsoft C/C++ Optimizing Compiler Version 7.0 and Microsoft Windows 3.1 Software Developer Kit (SDK). Every effort has been made to ensure compatibility with other compilers. Compile instructions are provided in comments at the top of each listing. The bitmaps were created with Microsoft Paintbrush. This article barely scratches the surface of 3-D graphics. Rendering, shading, ray tracing, and texture mapping get a more thorough treatment from the references in the bibliography. This article will present a building block for such advanced features. Do not confuse 3-D modeling with Windows metafiles. A metafile is a binary-encoded collection of GDI function calls. You create one by sending the output from various GDI function calls to a special device context. A recorded metafile can be played back by passing its associated resource handle to PlayMetaFile. The resulting picture may look three-dimensional but it's just a two-dimensional facade. Vector Graphics Microsoft Windows derives its characteristic look and feel primarily from raster-based or bitmapped graphics technology. Most fonts, controls, and icons are nothing more than bitmaps--two-dimensional blocks of pixels--that appear three-dimensional due to varying color gradients. Bitmaps look great but generally experience some kind of image distortion when rotated, stretched, or scaled onto devices of varying resolutions. To address such deficiencies Microsoft incorporated so-called TrueType font technology into Windows 3.1. TrueType is a form of vector-based graphics, in which images are constructed with individual graphical primitives such as lines, points, rectangles, ellipses, and curves rather than bitmaps. Vector graphics are scalable, require less memory than bitmaps, and enable us to model three-dimensional objects with relative ease. Coordinate Systems The Windows GDI does not contain explicit support for 3-D modeling, but it is not difficult to build a suitable framework atop existing primitive functions. Nearly any 3-D object can be drawn with a set of interconnected points or vertices. For example, a simple cube has eight vertices (one for each corner) while the General Dynamics F16 Falcon aircraft in 3D.C in Listing 1 (along with the files in Listing 2 and Listing 3) contains literally hundreds of vertices. Of course, a collection of points is worthless without some frame of reference, so they are placed in a domain called the World Coordinate System (WCS). You can change the object's orientation in WCS by multiplying each vertex by a series of transformation matrices. There are distinct transformation matrices for rotation, reflection, shearing, scaling, and translation operations, respectively. Try to imagine yourself floating in space around a motionless object. As you change position, each WCS vertex is transformed with respect to an Eye Coordinate System (ECS) that emanates from your eye and points toward the object (Figure 1). To complicate matters even more, the resulting ECS vertices must be transformed from three-dimensional to two-dimensional screen coordinates (SCS) before the object can be displayed. Once these screen positions are known, you can connect-the-dots to produce a transparent wireframe model, or use filled polygons for a more realistic solid model (Figure 2). Three-dimensional objects are drawn one surface at a time, so vertices are generally grouped in that order. Hidden-Surface Removal Hidden-surface removal is one of the more complicated and computation-in-tensive facets of 3-D modeling. The most popular method of hidden-surface removal is called backplane elimination. Basically, it involves computing whether a normal (perpendicular) vector emanating from a given surface points away from or toward the viewer's line of sight. Those surfaces facing away from the viewer cannot be seen and, therefore, need not be drawn. It does have its drawbacks, too. Objects with irregular or overlapping surfaces will not be drawn properly. One quick-and-dirty solution is to sort the surfaces in terms of decreasing distance from the viewer (ECS z-coordinate). Surfaces lying farther away from the viewer are drawn first and subsequently masked by closer surfaces. Depth sorting has its flaws but performance-conscious applications usually don't mind. Constructing Models 3D.C (Listing 1) supports both wireframe and solid models. It first creates a window with horizontal and vertical scroll bars, and uses SetScrollRange to lock the thumb between 0 and 360 degrees. Depressing the scroll bars changes the angular position of the viewer relative to the object. This is especially convenient because the viewer's position is given in spherical coordinates (distance, theta, phi). All measurements are given in device units. The F16 object "database" has been optimized to keep code size to a minimum. 3D.C calls DrawObject whenever the viewer depresses the scroll bars or resizes the window. DrawObject determines the center point of the window and creates a compatible work bitmap. Using an intermediate bitmap prevents the flashing effects that occur while drawing straight to a display context. DrawObject also precalculates sine and cosine values for global variables theta and phi. These values are needed when f16.vertex[].world vertices are transformed to f16.vertex[].eye vertices and, finally, to f16.vertex[].screen coordinates. DrawObject clears the work bitmap and loops through each surface in the f16.info[] array. For solid models, DrawObject performs a depth sort on the f16.vertex[].eye vertices, removes hidden surfaces with backplane elimination, selects a brush color from f16.info[].brushColor, and calls Polygon; otherwise, it calls PolyLine for a wire-frame surface. DrawObject then updates the client area with the completed work bitmap and deletes unused resources. Computations Floating-point computations incur a considerable amount of overhead--even with a math coprocessor installed--but you can improve performance significantly by substituting fixed-point integers for floating-point numbers. Fixed-point integers incorporate both the whole and fractional components of floating-point numbers but can be manipulated in single arithmetic operations, such as IMUL and IDIV. For example, it is possible to represent the floating-point number 12.345 with integer 12345 by shifting the decimal place three positions. There are certain problems with this technique, as well. Integers can only represent so many digits before overflowing, so you must strike a balance between the scale and precision of the 3-D model. NASA's Jet Propulsion Laboratories unveiled a computer-generated film several months ago that depicted the surface topography of some distant planet. JPL had downloaded countless bits of radar imaging data from a distant probe into three Cray super computers and rendered the surreal landscape frame-by-frame over the course of three solid weeks. The resulting bitmaps were finally transferred to videotape and made available for public consumption. There's a lesson hiding behind this madness. Even though real-time 3-D graphics were out of the question, JPL eventually got what it paid for by blending vector and raster technologies. References Adams, Lee. 1986. High-Performance CAD Graphics in C. Blue Ridge Summit, PA: Windcrest/Tab. Microsoft Corp. 1991. Microsoft Windows Multimedia Authoring and Tools Guide. Redmond, WA: Microsoft Press. Microsoft Corp. 1991. Microsoft Windows Multimedia Programmer's Reference. Redmond, WA: Microsoft Press. Microsoft Corp. 1991. Microsoft Windows Multimedia Programmer's Workbook. Redmond, WA: Microsoft Press. Park, Chan S. 1985. Interactive Microcomputer Graphics. Reading, MA: Addison-Wesley Publishing Company. Petzold, Charles. 1990. Programming Windows. Redmond, WA: Microsoft Press. Wilton, Richard. 1987. Programmer's Guide to PC & PS/2 Video Systems. Redmond, WA: Microsoft Press. Figure 1 The Viewing Transformation Figure 2 Wire-Frame and Solid Models Listing 1 3d.c -- creates a General Dynamics F16 Falcon aircraft #include "windows.h" #include #include //****************************************************************** //Title: 3D.C //Author: Thomas W. Olsen //Version: 1.0 //Compiler: Microsoft C/C++ 7.0 // rc /r 3d. rc // cl /c /AL /Gsw /W3 /Oas /Zpe /Zi /FPi 3d.c // link /CO /NOD 3d,,, libw llibcew win87em, 3d.def // rc 3d.exe //***************************************************************** //********************** Various Constants ************************* #define CENTER 0 #define PI 3.141593 #define RADIANS(a) (a * (PI / 180.0)) #define DEGREES(a) (a * (180.0 / PI)) #define NO_OF_VERTICES 66 #define NO_OF_SURFACES 45 #define NO_OF_SURFACE_VERTICES 220 #define VIEW_RATIO (40 / 10) #define MIN_DEGREES 0 #define MAX_DEGREES 359 #define ROTATE_DEGREES 5 //********************* Structure Definitions ********************** typedef struct tagPOINT3D { double x; double y; double z; } POINT3D; typedef struct tagVERTEX { POINT3D world; // World Coordinates POINT3D eye; // Eye Coordinates POINT screen; // Screen Coordinates } VERTEX; typedef struct tagSURFACE { int mapIndex; // Index Into the Surface Map Array int noOfVertices; // No of Vertices Forming the Surface int depthIndex; // Used in Determining Depth of Surface int brushColor; } SURFACE; typedef struct tagOBJECT { VERTEX vertex [NO_OF_VERTICES]; SURFACE info[NO._OF_SURFACES]; int map[NO_OF_SURFACE_VERTICES]; } OBJECT; //************************* Static Data *************************** OBJECT f16 = { { // Vertex Array { 0.00, 0.00, 0.00 }, //Center { 0.00, -17.00, 9.0 }, { 0.00, -13.00, 9.00 }, //Tail { 0.00, -14.50, 3.00 }, { 0.00, -8.00, 3.00 }, { 0.00, -14.50, 2.00 }, { 0.00, -5.00, 2.00 }, { -0.50, -16.00, 1.00 }, { -1.00, -16.00, 0.00 }, //Exhaust { -0.50, -16.00, -0.50 }, { 0.00, -16.00, -1.00 }, { 0.50, -16.00, -0.50 }, { 1.00, -16.00, 0.00}, { 0.50, -16.00, 1.00 }, { -1.00, -13.00, 2.00 }, { -2.00, -13.00, 0.00 }, //Fuse { -1.50, -13.00, -1.50 }, { 0.00, -13.00, -2.00 }, { 1.50, -13.00, -1.50 }, { 2.00, -13.00, 0.00 }, { 1.00, -13.00, 2.00 }, { -1.00, 7.00, 2.00 }, { -2.00, 7.00, 0.00 }, { -1.50, 7.00, -1.50 }, { 0.00, 7.00, -2.00 }, { 1.50, 7.00, -1.50 }, { 2.00, 7.00, 0.00 }, { 1.00, 7.00, 2.00 }, { 2.00, -8.00, 0.00 }, { 11.00, -8.00, 0.00 }, //Wings { 11.00,-5.00,0.00 }, { 5.00, 0.00, 0.00 }, { 2.00, 7.00, 0.00 }, { -2.00, -8.00, 0.00 }, { -11.00, -8.00, 0.00 }, { -11.00, -5.00, 0.00 }, { -5.00, 0.00, 0.00 }, { -2.00, 7.00, 0.00 }, { 0.50, 2.00, 2.00 }, { -0.50, 2.00, 2.00 }, //Cockpit { 1.00, 5.00, 2.00 }, { 0.50, 5.00, 3.50 }, { -0.50, 5.00, 3.50 }, { -1.00, 5.00, 2.00 }, { 1.00, 8.00, 2.00 }, { 0.50, 8.00, 3.50 }, { -0.50, 8.00, 3.50 }, { -1.00, 8.00, 2.00 ), { 0.50, 11.00, 2.00 }, { -0.50, 11.00, 2.00 }, { 0.00, 7.00, -1.00 }, { -1.00, 11.00, 2.00 }, //Subfuse { -2.00, 11.00, 0.00 }, { 0.00, 11.00, -1.00 }, { 2.00, 11.00, 0.00 }, { 1.00, 11.00, 2.00 }, { 0.00, 11.00, -1.00 }, { 0.00, 17.00, 0.00 }, //Nose { -2.00, -16.00, 0.00 }, { -8.00, -16.00, 0.00 }, //Elevators { -8.00, -14.00, 0.00 }, { -2.00, -10.00, 0.00 }, { 2.00, -16.00, 0.00 }, { 8.00, -16.00, 0.00 }, { 8.00, -14.00, 0.00 }, { 2.00, -10.00, 0.00 } }, { // Surface Info ... (Points to Surface Map) { 0, 5, 60, DKGRAY_BRUSH }, { 5, 5, 64, DKGRAY_BRUSH }, //Elevators { 10, 5, 60, DKGRAY_BRUSH }, { 15, 5, 64, DKGRAY_BRUSH }, { 20, 5, 1, GRAY_BRUSH }, { 25, 5, 4, LTGRAY_BRUSH }, //Tail { 30, 5, 1, GRAY_BRUSH }, { 35, 5, 4, LTGRAY_BRUSH }, { 40, 5, 14, BLACK_BRUSH }, ( 45, 5, 15, DKGRAY_BRUSH }, //Exhaust { 50, 5, 16, BLACK_BRUSH }, ( 55, 5, 17, BLACK_BRUSH }, { 60, 5, 18, DKGRAY_BRUSH }, { 65, 5, 19, BLACK_BRUSH }, { 70, 5, 20, DKGRAY_BRUSH }, { 75, 5, 22, DKGRAY_BRUSH }, { 80, 5, 23, GRAY_BRUSH }, //Fuse { 85, 5, 24, DKGRAY_BRUSH }, { 90, 5, 25, DKGRAY_BRUSH }, { 95, 5, 26, GRAY_BRUSH }, { 100, 5, 27, DKGRAY_BRUSH }, { 105, 5, 20, GRAY_BRUSH, }, { 110, 6, 30, DKGRAY_BRUSH }, ( 116, 6, 35, DKGRAY_BRUSH }, //Wings { 122, 6, 30, DKGRAY_BRUSH }, { 128, 6, 35, DKGRAY_BRUSH }, { 134, 5, 21, LTGRAY_BRUSH }, { 139, 5, 22, LTGRAY_BRUSH }, //Subfuse { 144, 5, 50, LTGRAY_BRUSH }, { 149, 5, 26, LTGRAY_BRUSH }, { 154, 5, 27, LTGRAY_BRUSH }, { 159, 4, 55, GRAY_BRUSH }, { 163, 4, 54, GRAY_BRUSH }, //Nose { 167, 4, 56, GRAY_BRUSH }, { 171, 4, 52, GRAY_BRUSH }, { 175, 4, 51, LTGRAY_BRUSH }, { 179, 5, 41, WHITE_BRUSH }, { 184, 5, 45, WHITE_BRUSH }, //Cockpit { 189, 5, 46, WHITE_BRUSH }, { 194, 4, 41, WHITE_BRUSH }, { 198, 4, 42, WHITE_BRUSH }, { 202, 5, 41, WHITE_BRUSH }, { 207, 5, 46, WHITE_BRUSH }, { 212, 4, 45, WHITE_BRUSH }, { 216, 4, 46, WHITE_BRUSH } }, { // Surface Map ... (Points to Vertex Array) 58, 61, 60, 59, 58, 62, 63, 64, 65, 62, 58, 59, 60, 61, 58, // Elevators 62, 65, 64, 63, 62, 1, 3, 4, 2, 1, 3, 5, 6, 4, 3, 1, 2, 4, 3, 1, // Tail 3, 4, 6, 5, 3, 14, 15, 8, 7, 14, 15, 16, 9, 8, 15, 16, 17, 10, 9, 16, // Exhaust 17, 18, 11, 10, 17, 18, 19, 12, 11, 18, 19, 20, 13, 12, 19, 20, 14, 7, 13, 20, 14, 21, 22, 15, 14, 15, 22, 23, 16, 15, 16, 23, 24, 17, 16, // Fuse 17, 24, 25, 18, 17, 18, 25, 26, 19, 18, 19, 26, 27, 20, 19, 20, 27, 21, 14, 20, 28, 29, 30, 31, 32, 28, 33, 37, 36, 35, 34, 33, 28, 32, 31, // Wings 30, 29, 28, 33, 34, 35, 36, 37, 33, 21, 51, 52, 22, 21, 22, 52, 53, 50, 22, 50, 53, 54, 26, 50, // Subfuse 26, 54, 55, 27, 26, 27, 55, 51, 21, 27, 55, 54, 57, 55, 54, 56, 57, 54, 56, 52, 57, 56, 52, 51, 57, 52, // Nose 51, 55, 57, 51, 41, 42, 39, 38, 41, 45, 46, 42, 41, 45, 48, 49, 46, 45, 48, // Cockpit 40, 41, 38, 40, 42, 43, 39, 42, 44, 45, 41, 40, 44, 46, 47, 43, 42, 46, 48, 45, 44, 48, 49, 47, 46, 49 } }; BOOL wireFrame = FALSE; double distance = 75, thetaDegrees = 90, phiDegrees = 60; //************************** Static Data *************************** LONG FAR PASCAL WindowProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam); void DrawObject(HWND hWnd, HDC hDC, OBJECT *object ); int __cdecl compareProc(const void *elem1, const void *elem2 ); //************************** Program Begin ************************* int PASCAL WinMain(HANDLE hInst, HANDLE hPrevInst, LPSTR lpCmdLine, int numCmdShow ) { MSG msg; HWND hWnd; WNDCLASS wc; //************************ Setup Window ************************ wc.style = (UINT) NULL; wc.lpfnWndProc = WindowProc; wc.cbClsExtra = 0; wc.cbWndExtra = 0; wc.hInstance = hInst; wc.hIcon = LoadIcon( NULL, IDI_APPLICATION); wc.hCursor = LoadCursor( NULL, IDC_ARROW); wc.hbrBackground = GetStockObject(BLACK_BRUSH); wc.lpszMenuName = (LPSTR) "Menu"; wc.lpszClassName = (LPSTR) "3DClass"; if (!RegisterClass(&wc)) return(FALSE); hWnd = CreateWindow( "3DClass", "3D Modeling Example", WS_OVERLAPPEDWINDOW WS_SCROLL WS_HSCROLL, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, NULL, NULL, hInst, NULL ); if (!hWnd) return (FALSE); SetScrollRange( hWnd, SB_HORZ, MIN_DEGREES, MAX_DEGREES, TRUE ); SetScrollRange( hWnd, SB_VERT, MIN_DEGREES, MAX_DEGREES, TRUE ); SetScrollPos( hWnd, SB_HORZ, (int) thetaDegrees, TRUE ); SetScrollPos( hWnd, SB_VERT, (int) phiDegrees, TRUE ); ShowWindow( hWnd, numCmdShow ); while (GetMessage(&msg, NULL, NULL, NULL)) /* Typical Mossage Loop */ { TranslateMessage(&msg); DispatchMessage(&msg); } return (msg.wParam); } LONG FAR PASCAL WindowProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam) { PAINTSTRUCT paint; HDC hDC; HMENU hMenu; int vPos, hPos; switch (message) { case WM_COMMAND: hMenu = GetMenu( hWnd ); CheckMenuItem( hMenu, wParam, MF_CHECKED); if (wParam == 1) { CheckMenuItem( hMenu, 2, MF_UNCHECKED); wireFrame = TRUE; } else { CheckMenuItem( hMenu, 1, MF_UNCHECKED); wireFrame = FALSE; } InvalidateRect( hWnd, NULL, TRUE ); break; case WM_DESTROY: PostQuitMessage( NULL ); break; case WM_HSCROLL: if (wParam == SB_THUMBTRACK) break; hPos = GetScrollPos( hWnd, SB_HORZ); switch( wParam ) { case SB_TOP: hPos = MIN_DEGREES; break; case SB_BOTTOM: hPos = MAX_DEGREES; break; case SB_LINEUP: case SB_PAGEUP: hPos -= ROTATE_DEGREES; break; case SB_PAGEDOWN: case SB_LINEDOWN: hPos += ROTATE_DEGREES; break; case SB_THUMBPOSITION: hPos = LOWORD(lParam); break; } if (hPos < MIN_DEGREES) hPos = MAX_DEGREES; else if (hPos > MAX_DEGREES) hPos = MIN_DEGREES; SetScrollPos( hWnd, SB_HORZ, hPos, TRUE ); thetaDegrees = (double) hPos; InvalidateRect( hWnd, NULL, TRUE ); break; case WM_VSCROLL: if (wParam == SB_THUMBTRACK) break; vPos = GetScrollPos( hWnd, SB_VERT ); switch( wParam ) { case SB_TOP: vPos = MIN_DEGREES; break; case SB_BOTTOM: vPos = MAX_DEGREES; break; case SB_LINEUP: case SB-PAGEUP: vPos -= ROTATE_DEGREES; break; case SB_PAGEDOWN: case SB_LINEDOWN: vPos += ROTATE_DEGREES; break; case SB_THUMBPOSITION: vPos = LOWORD(lParam); break; } if (vPos < MIN_DEGREES) vPos = MAX_DEGREES; else if (vPos > MAX_DEGREES) vPos = MIN_DEGREES; SetScrollPos( hWnd, SB_VERT, vPos, TRUE ); phiDegrees = (double) vPos; InvalidateRect( hWnd, NULL, TRUE ); break; case WM_SIZE: InvalidateRect( hWnd, NULL, TRUE ); break; case WM_PAINT: hDC = BeginPaint( hWnd, &paint ); DrawObject(hWnd, hDC, &f16); ReleaseDC( hWnd, hDC ); EndPaint( hWnd, &paint ); break; default: return (DefWindowProc(hWnd, message, wParam, lParam)); } } void DrawObject(HWND hWnd, HDC hDC, OBJECT *object) { double sinTheta, cosTheta, sinPhi, cosPhi; double s1, s2, s3; POINT3D *v1, *v2, *v3; POINT center; RECT rect; POINT points[10]; HBITMAP hBitmap, hOldBitmap; HRGN hRgn; HDC hMemDC; int surface, vertex, mapIndex, vertexIndex, loop; GetClientRect( hWnd, &rect); // Determine Size of Client Area center.x = (rect.right / 2); // Calculate X-Centerpoint center.y = (rect.bottom / 2); // Calculate Y-Centerpoint hRgn = CreateRectRgn( rect.left, rect.top, rect.right, rect.bottom ); hMemDC = CreateCompatibleDC( hDC ); hBitmap = CreateCompatibleBitmap( hDC, rect.right, rect.bottom); hOldBitmap = SelectObject( hMemDC, hBitmap ); //********************************************************************** //* Precalculate SIN(x) and COS(x) * //********************************************************************** cosTheta = cos( RADIANS(thetaDegrees) ); sinTheta = sin( RADIANS(thetaDegrees) ); cosPhi = cos( RADIANS(phiDegrees) ); sinPhi = sin( RADIANS(phiDegrees) ); //****************************************************************** //* Calculate Eye and Screen Coordinates * //****************************************************************** for (loop = 1; loop < NO_OF_VERTICES; loop++) { object->vertex[loop].eye.x = (-object->vertex[loop].world.x * sinTheta) + (object->vertex[loop].world.y * cosTheta); object->vertex[loop].eye.y = (-object->vertex[loop].world.x * cosTheta * cosPhi) - (object->vertex[loop].world.y * sinTheta * cosPhi) + (object->vertex[loop].world.z * sinPhi); object->vertex[loop].eye.z = (-object->vertex[loop].world.x * sinPhi * cosTheta) - (object->vertex[loop].world.y * sinTheta * sinPhi) - (object->vertex[loop].world.z * cosPhi) + distance; object->vertex[loop].screen.x = (int) (VIEW_RATIO * (object->vertex[loop].eye.x / object->vertex[loop].eye.z) * center.y + center.x); object->vertex[loop].screen.y = (int) (-VIEW_RATIO * (object->vertex[loop].eye.y / object->vertex[loop].eye.z) * center.y + center.y); } //****************************************************************** //* Draw Object * //****************************************************************** FillRgn( hMemDC, hRgn, GetStockObject(BLACK_BRUSH)); SelectObject( hMemDC, GetStockObject(wireFrame == TRUE ? WHITE_PEN:BLACK_PEN) ); if (wireFrame == FALSE) qsort( object->info, NO_OF_SURFACES, sizeof(SURFACE), compareProc ); for (surface = 0; surface < NO_OF_SURFACES; surface++) { mapIndex = object->info[surface].mapIndex; if (wireFrame == FALSE) // No Hidden Surface Removal For Wire Frame { v1 = &object->vertex[ object->map[ mapIndex ] ].eye; // Setup pointers to three v2 = &object->vertex[ object->map[ mapIndex+1 ] ].eye; // surface vertices v3 = &object->vertex[ object->map[ mapIndex+2 ] ].eye; s1 = v1->x * (v2->y * v3->z - v3->y * v2->z); s1 = (-1) * s1; s2 = v2->x * (v3->y * v1->z - v1->y * v3->z); // Perform dot product on surface s3 = v3->x * (v1->y * v2->z - v2->y * v1->z); // vectors to find normal vector } if (wireFrame == TRUE s1 - s2 - s3 <= 0.0) { for (vertex = 0; vertex < object->info[surface].noOfVertices; vertex++, mapIndex++) { vertexIndex = object->map[mapIndex]; points[vertex].x = object->vertex[vertexIndex].screen.x; points[vertex].y = object->vertex[vertexIndex].screen.y; } if (wireFrame == TRUE) Polyline( hMemDC, &points[0], vertex; else { SelectObject( hMemDC, GetStockObject(object->info[surface].brushColor) ); Polygon( hMemDC, &points[0], vertex); } } } BitBlt( hDC, 0, 0, rect.right, rect.bottom, hMemDC, 0, 0, SRCCOPY); SelectObject( hMemDC, hOldBitmap ); DeleteObject( hBitmap ); DeleteObject( hRgn ); DeleteDC( hMemDC ); } int _cdecl compareProc( const void *elem1, const void *elem2 ) { if( f16.vertex[((SURFACE *) elem1)->depthIndex].eye.z > f16.vertex[((SURFACE *) elem2)->depthIndex].eye.z) return -1; if( f16.vertex[((SURFACE *) elem1)->depthIndex].eye.z < f16.vertex[((SURFACE *) elem2)->depthIndex].eye.z) return 1; else return 0; } /* End of File*/ Listing 2 3d.def NAME 3D DESCRIPTION '3D Modeling Example' EXETYPE WINDOWS STUB 'WINSTUB.EXE' CODE PRELOAD MOVEABLE DATA PRELOAD MOVEABLE MULTIPLE HEAPSIZE 2048 STACKSIZE 4096 EXPORTS WindowProc @1 Listing 3 3d.rc #include "windows.h" Menu MENU { POPUP "&Draw" { MENUITEM "&Wire Frame", 1 MENUITEM "&Solid", 2, CHECKED } } Pointer Power in C and C++, Part 2 Christopher Skelly Christopher Skelly has been a teacher of C and C++ for the past ten years, first for Plum Hall Inc., and then for his own company, Insight Resource Inc. Insight Resource also developed the best-selling help utility, "KO-PILOT for WordPerfect," which Brit Hume called "the best add-in ever written." Chris has served on both the C and C++ ANSI committees, and was the Technical Chairman for this year's "CPlusC++" and "C++ in Action" conferences, presented by Boston University. He writes regularly for the C User's Journal and the C++ Journal, and can be reached at Insight Resource Inc., 914-631-5032, or at 71005.771@compuserve.com. This article extends and continues the techniques presented last month in Part 1 of "Pointer Power in C and C++." At the end of this article I will repeat and then solve the pointer puzzle presented in Part 1. Just in case you don't happen to have last month's issue immediately available, here are the eight Key Facts from Part 1: 1. A pointer is a variable whose contents is an address. 2. A pointer always "knows" the type of thing it addresses. It can be properly used only to access something of the correct type. 3. Pointer values are address/type pairs, just like pointer variables. However, pointer values are not storable lvalues. 4. Every pointer has three fundamental attributes. These attributes are the location, the contents, and the indirect value of the pointer. 5. The three attributes of a pointer represent three distinct address levels. These address levels can also be called levels of indirection. 6. Pointer space is organized into a series of planes or levels. Every pointer expression can be assigned to one of these planes. The plane of a pointer expression is a measure of how much potential for indirection there is in that pointer expression. 7. The name of an array usually behaves as if the array name were a pointer value. 8. The name of an array, in almost every context, evaluates to the address of the array's own "zeroth" element. Armed with these Key Facts, you are ready to learn the next set of techniques, a game informally called Pointer Dominos. Pointer Dominoes The key to mastering pointers is to learn to play Pointer Dominos. The game of pointer dominos is simply the game of using operators in expressions involving pointers. Each of the allowable operators does something very specific, and the operators are always to be played, or really evaluated, in a very precise order. If you know exactly what each operator does, and if you know how to determine the order of evaluation, you can play pointer dominos. Key Fact #9 -- Only a small number of operations are ever performed on pointers. If you know exactly what each operation does, and what the right order to apply the operations is, you can understand and create any pointer expression in C. Each pointer has three attributes and each attribute is at a different level of indirection. You know that certain operators actually change the level of indirection of an expression using a pointer. Specifically, you have already seen that & takes you up one level and * takes us down a level when applied to a pointer in an expression. Note that in declarations the * builds in one level of indirection, as does the []. The rules of pointer dominos apply only to expressions, not to declarations. int i = 0; /* i is declared at level 0 */ int *p; /* p is declared at level 1 */ p = &i; /* &i is level 1, so is p */ x = *p; /* * on p takes us down from level 1 to level 0 */ This leads to the first two rules of Pointer Dominos. When used in pointer expressions: & takes the type of the expression up one level of indirection. * takes the type of the expression down one level of indirection. Several other operators affect the overall level of indirection of an expression. But all the operators do one of a small number of things. They take us up, down, or sometimes even sideways on the Ladder. Fortunately, each allowable operation has a precise, well-defined meaning. Only six moves appear in expressions involving pointers. These moves form our next key fact. Key Fact #10 -- The six moves of pointer dominos are: 1. Go up one level of indirection using & 2. Go down one level of indirection using * 3. Go down one level of indirection using [] 4. Increase an address using + or ++ 5. Decrease an address using - or --n 6. Change the type of the pointer's window on memory with a cast. Structures and their members are not included here, but are easy to add to the fundamentals of the model. Each move corresponds to one or two C language operators. The first move to consider is the &, the unary address-of operator. & always lifts you up one level of indirection. Generating the address of something is the equivalent of moving up to the next plane in pointer space. As I've said, this process is known as referencing, and involves a relative move up to the next higher address plane. Two operators move you down one level of indirection. First the *, or unary indirection operator, means go down to the plane immediately below the plane you start on. If x lives on plane 5, *x is an expression that lives on plane 4. Most typically, if p lives on plane 1, *p is a non-pointer living on plane 0. This move is called dereferencing, and it is a relative move. *x is one level below x. You can't say anything about what level this actually is, until you know what level x itself resides on. The [] operator is also a dereferencing operator. a[n] lives on the plane below a, where a is any address expression. An important principle of pointer dominos is the fact that both * and [] bring an expression down one level of indirection from what they are applied to. The formula which relates how * and [] are related is the important formula: a[n] == *(a + n) This master formula in C and C++ shows that a subscript is syntactically equivalent to dereferencing an offset from a pointer. The a in the formula represents any address. The address may come from a pointer, or from an array name, or from a casted expression. It doesn't matter. The subscript can always be applied to an address, just like the *, and the relation between the two forms comes from this formula. The fourth and fifth moves involve operations of addition and substraction, including +, ++, -, and --. These operators produce no change in level of indirection. They move you sideways on the same plane, either toward higher or lower memory and always by a scaled offset. p + 1 is the address of one object higher in memory than the object p points to. p - 2 is the address of an object two objects below the object addressed by p. p, p + 1, and p - 2, however, all exist on the same plane and have the same level of indirection. The final pointer domino move is the cast. Casting a pointer usually produces no change on the level of indirection. Casting a pointer to int to a pointer to double, for example, does not change the level of the pointer. What does change is the size and format of the "window on memory" that this pointer accesses. A pointer to char accesses one byte of integer data. A pointer to double accesses eight bytes of floating-point formatted data. There are special cases, however, where a cast does affect the level of indirection of an expression. Consider: char *p = malloc(1000); char **p2; p2 = (char **)p; Here the cast, (char **), does indeed produce a level change from level one up to level two. The real rule is this: the level of an expression with a cast is the level of indirection of the cast. Casts are the wild-cards of pointer dominos. Any expression can be cast to have some new level and type. The declaration inside the cast determines the level of the newly-casted expression. Summarizing the rules of pointer dominos in terms of operators, you have: & -- move up one level * -- move down one level [n] -- move down one level, with an offset of n elements + ++ -- add a scaled offset on the same level - -- -- subtract a scaled offset on the same level (type) -- change the size, format, and possible level of the expression to that of the type in the cast This set of six rules forms the guts of the game. All you need now is one additional rule, which tells you which order to apply the moves when more than one operator are present in the same expression. For instance, in the expression: *++p two operations, * (indirection) and ++ (pre-increment) are being applied to the pointer p. Which operation should you do first, the increment or the indirection? The resulting value will be very different depending on what you decide. If you did the * first, you would take the indirect value of p, and then increment that indirect value. In fact, this is just the opposite of what you are really supposed to do. The rules of precedence state that unary operators, like both pre-increment ++ and *, group right to left. This means that the ++ binds with p before the * is even considered. You must increment the pointer and then take the indirect value. The point here is that the rules of precedence always determine the proper order of evaluation. This is our last key fact. Key Fact #11 -- If more than one operator is applied in an expression, apply the operators in order of precedence. A quick glance at the precedence table reveals that primary operators include both [] and (), while the * and & are weaker-binding unary operators. Furthermore, primary operators group left to right and unaries group right to left. In effect, this means that you will deal with the primary [] and () first in left to right order, and then handle the unary *s or &s in right to left order. For example, the expression *p[n] contains two operators, one primary [] and one unary *. The [] binds first followed by the *, so the interpretation is "locate the array element p[n], then dereference this element." A more complex expression may have lots of operators to consider: *(char *)p2[n][m] Here both subscripts bind first, in left to right order. Then comes the cast, (char *), which changes the type of the value in p2[n][m] to be a pointer to char. Finally, the * on the left dereferences this casted pointer. Showing each step in order: 1. p2--starting with p2 2. p2[n]--access the nth element offset from p2 3. p2[n] [m]--access the mth element offset from p2[n] 4. (char *)p2[n][m]--cast to the type pointer to char 5. *(char *)p2[n][m]--dereference the resulting char pointer The rules are simple. Apply each pointer move in the proper order of precedence. Keep track of levels as you go. Now you are thinking just like the C compiler! Solving the Puzzle It's time to solve the puzzle presented at the beginning of this article. Though the puzzle has inordinately complex expressions, the rules of pointer dominos will make short work of the task. Listing 1 contains the puzzle again. What kind of data structures are you working with in this puzzle? Figure 1 contains a picture of the data. As you can see in Figure 1, ap is an array of pointers to chars, each pointer aimed at one of five character strings. ap is a level two object. Why? First, because ap is an array, it has intrinsically one level of indirection. But the elements of ap are all pointers, each holding their own level one address. So ap evaluates to the address of a pointer, hence ap is a level two expression. app is similarly a level three expression. An array of level two pointers evaluates to the address of a level two pointer, hence app lives on level three. ppp is a level 3 pointer, and pppp is a level four pointer. The relationships between the pointers are illustrated in Figure 1. Here is the first expression to be printed: printf("%.*s", 2, *--**pppp); Do the easy part first. The %.*s format specifier means to fill in the * with the first argument in the argument list following the format string. So you are really asking for %.2s, that is, print the first two characters of the string *--**pppp. How do you unravel *--**pppp? With the rules of pointer dominos! Start at the identifier, pppp, and apply the operators in order of precedence. Both * and -- are unary operators so they group right to left, as follows: 1. pppp--first the identifier pppp 2. *pppp--right-most * dereference pppp 3. **pppp--second right-most * dereference *pppp 4. --**pppp--unary -- pre-decrement the result 5. *--**pppp left-most * dereference again Reading off the quoted strings gives a comprehensible formula for solving this part of the puzzle. To find the answer, start at the top of the diagram of the puzzle's data, at pppp, and move down two levels, following the arrows. You should be at app[0], the zeroth element in the app array of char ** pointers. app[0] holds the address of ap[4], the last element in the array of char * pointers. But now, the rules say you must apply the unary -- operator to app[0]. Instead of pointing at ap[4], app[0] will now hold the address of ap[3]! This change, by the way, persists, and changes the diagram slightly from that shown in Figure 1. After decrementing app[O], you apply the final dereference or *, and arrive at the contents of ap[3]. This is what you will print with the first expression. Actually, the program prints only the first two characters of the string PORTABLE. So PO appears on the output. What does the second expression print? printf("%.*s", 3, *(++*pppp[0] - 4)); This complex expression again uses the %.*s mechanism to pick up the 3 as the number of chars to be printed. In effect, you will print three chars from the address given by the complex expression *(++*pppp [0] - 4)). This one breaks down as follows: 1. pppp--start at pppp 2. pppp[0]--dereference to access [0] th element 3. *pppp[0]--dereference pppp[0] 4. ++*pppp[0]--pre-increment the result 5. ++*pppp[0] - 4--subtract 4 6. *(++*pppp[0] - 4)--dereference again Handling each operator one step at a time gives you the solution. The only new trick here is the translation between [] and *. Remember the all-important a[n] == *(a + n) formula and you'll zip through the steps. Accessing the zeroth element of pppp is the same as dereferencing pppp. a[0] is always the same object as *a. So the subscript [0] and the right-most * bring you down two levels, just as before. Only now you are told to increment the result, namely app[0]. So app[0] now pops back right back to where it started in the first place, namely to point to ap[4], the string TOWER!. Now what? Now, you have to subtract 4 from this pointer. This is where it can get tough. But remember, app is an array of char ** pointers, so app[0] acts like a pointer to a pointer to a char. Decrementing means subtracting the space for four char * pointers. So app[0] - 4 points to ap[0], the very first string in the puzzle, INTEGER. You have to print three characters from this string. So INT appears on the display right after the PO. The screen says POINT. The third expression prints the whole of the expression: ++*--*++pppp[0] + 5 This whopper breaks down according to the precedence rules as follows: 1. pppp 2. pppp[0] 3. ++pppp[0] 4. *++pppp[0] 5. -- *++pppp[0] 6. *--*++pppp[0] 7. ++*--*++pppp[0] 8. ++*--*++pppp[0] + 5 Here's the explanation. Again [0] means move down one level, to ppp. Now increment ppp, so ppp points to ppp[1], not ppp[0] anymore. Dereference with * means move down to app[1]. Now decrement app[1], so app[1] points to ap[2] from now on, rather than ap[3]. Dereferencing with * brings you down to ap[2]. Now increment ap[2], so the pointer to char, ap[2], points to the E rather than the D of DEBUGGER. Finally, add 5. Since you are now down at level one, the contents of ap[2], adding 5 means add the size of five chars to the pointer. Hence you are finally left pointing at the second E in DEBUGGER. Printing the resulting string, along with a space as the puzzle requires, gives us ER. The screen now says POINTER. Only two more. By the time you're done you'll never forget how to do this! The fourth printf statement looks like this: printf("%.*s", 2, *++pppp[0][3] + 3); You are asked to print two characters from the expression: *++pppp[0][3] + 3 Here's the break down of the steps. 1. pppp 2. pppp[0] 3. pppp[0][3] 4. ++pppp[0][3] 5. *++pppp[0][3] 6. *++pppp[0][3] + 3 Now just read through this break down, supplying the interpretation for each step. [0] again moves down to ppp. Adding a [3] to ppp means two things; go down to the next level, and offset by three elements. So while pppp[0][0] is app[1] (remember, you incremented ppp in the last step), pppp[0][3] is app[4]! Now increment this result, app[4], so app[4] points at ap[1], rather than ap[0]. Dereferencing with the * brings you down to the contents of ap[1]. Adding 3, skips the first three characters of PROPORTION so you print the PO substring from the middle of PROPORTION. Now the screen reads POINTER PO. Time for the last one! See if you can get this one without an explanation. If you get to WER!, you're right. Remember to consider the changes that -- and ++ created in the earlier expressions. Here's how. printf("%s\n", (*pppp + 2)[-2][2] + 2); 1. pppp 2. *pppp 3. *pppp + 2 4. (*pppp + 2)[-2] 5. (*pppp + 2)[-2][2] 6. (*pppp + 2)[-2][2] + 2 Dereference pppp to ppp, now still pointing at app[1]. Adding 2 creates a pointer value pointing at app[3]. What does the [-2] subscript do? First it moves you down to the level of app[3], but the -2 means two objects lower in memory, so you move down to app[1], not app[3]. app[1] is still pointing at ap[2], where the -- in the third expression left app[1]. Applying the [2] subscript to this value of app[1] moves us down and over to ap[4], two pointers offset from ap[2]. The last + 2 skips over the first two chars in TOWER!, so you see the last part of the puzzle, WER! on the display. The screen says POINTER POWER! and you had better believe it! You may want to work the steps of this puzzle over several times, perhaps drawing some intermediate diagrams as the pointers change. If you can work this puzzle correctly, you have indeed acquired pointer power as a long-term resource for your future C and C++ programs! Remembering to carefully distinguish the Three Attributes, following the levels on the Ladder of Indirection, and applying the rules of Pointer Dominos will get you to the solution every time. Furthermore, understanding complex pointer behavior will give you the confidence to create your own sophisticated data-handling mechanisms, when and where appropriate. The examples in the puzzle, of course, are not designed to be "good code." They are designed to show that this most powerful part of C is indeed governed by straight-forward rules which can always be applied to understand and work with complex pointer declarations and expressions in C. Now it's on to C++, where void isn't quite so void anymore, where references are not pointers, but sometimes act like them, and where pointers to members are not even addresses in the standard C sense at all! Figure 1 Listing 1 What does this program print? #include char *ap[] = { "INTEGER", "PROPORTION", "DEBUGGER", "PORTABLE", "TOWER!" }; char **app[] = { ap + 4, ap + 3, ap + 2, ap + 1, ap }; char ***ppp = app; char ****pppp = &ppp; void main() { printf("%.*s", 2, *--**pppp); printf("%.*s", 3, *(++*pppp[0] - 4)); printf("%s " , ++*--*++pppp[0] + 5); printf("%.*s" , 2, *++pppp[0][3] + 3); printf("%s\n", (*pppp + 2) [-2][2] + 2); } /* End of File */ C++, Coroutines, and Simulation Trond Akerbaek Trond Akerbaek is teaching at a college in Halden, Norway. Current interests include computer communications, neural networks, and fuzzy systems. He can be contacted at Ostfold DH; N-1757 Halden; Norway; e-mail:tronda@dhhalden.no. Introduction Object-oriented programming is not a new concept. The Simula67 Language, developed in the late sixties, contains all the basic concepts found in modern object-oriented languages such as C++. An important part of Simula not found in C++ is the support for quasi-parallel processes and simulation. Very few programmers ever use parallel functions or coroutines in their programs, even if it will lead to more elegant designs. There are several reasons: few languages have ample support, coroutines may otherwise be cumbersome to implement, and it is possible to solve most problems without. C programmers may use setjmp and longjmp and functions based on these to implement quasi-parallell routines. With C++ and some assembly instructions it is possible to create a set of functions which makes coroutines easier to comprehend and use. Using coroutines, implementing simple tools for simulation modelled after the Simula67 tools, is a fairly easy matter. In this article I will give a brief introduction to coroutines and describe an implementation in Borland C++ 2.0, with a single assembly function in Turbo Assembler. Based on the coroutine concept, the article will discuss simulation and how to implement it in C++. Quasi-Parallel Functions and Coroutines Due to the single-processor constraint, computer programming has traditionally been sequentially-oriented with a single thread of operations. Multitasking operating systems give us the opportunity to create programs consisting of several sequential processes operating in parallel on a timesharing basis. The operating system controls the task switching, with little or no influence from the programmer. The object-oriented approach is still basically sequential, even if it makes it easier to describe real processes which are interacting and operating in parallel. Parallel processes cannot directly be modelled in a sequential language. The process descriptions are running on a single processor, and must switch back and forth, imitating real parallel behavior, hence the expression quasi-parallel. It would be another situation if the model were running on a multiprocessor machine, with one processor for each process. In that case the model would be described in a language designed for such environments. The Coroutine Extensions In order to support the coroutine concept I have created a few extensions to C++: A base-class coroutine for all classes describing coroutines. A virtual function main in coroutine subclasses containing the actions of the coroutine. The primitive resume(coroutine*) which freezes the current coroutines actions, and transfers control to the indicated coroutine, resuming its actions at the freesing point. A new coroutine instance is initially frozen at the start of its main function. A primitive detach is available, which freezes the current coroutine actions and transfers control to the main program. A coroutine is terminated when its main function is terminated normally (not frozen by resume or detach). Resuming a terminated object leads to a runtime error. Automatic variables declared in a class method and alllocated on the stack are not availablee outside the couroutine instance. A variable declared in the main function of the program, is not available within the other coroutines. The main program is in fact a coroutine with the main function defining its actions. There must be no register variables, because registers will not be preserved changing coroutines. Implementation The implementation consists of the coroutine class, the two primitives resume and detach, and a few functions for internal use. Each coroutine, as the main program, has its own stack. Switching between coroutines is done by switching stacks. Whenever a coroutine is activated, the activating coroutine's stack is saved in memory, and the activated coroutine's previously stored stack is copied to the stack segment. Listing 1 contains the definitions. The variables of the coroutine class are used for keeping track of the passive stack. stkSegment and stkOffset are the segment and offset address of the area where the stack is stored, while stkSize holds the size of the used part of the stack. Listing 2 shows the implementation. There are three internal pointers in the implementation file: PREVIOUS referencing the coroutine to switch from, CURRENT referencing the new and active coroutine, and MAINC referencing the main program. When a new process is created as a coroutine subclass instance, the stack of the new process is initialized by coroutine::coroutine(). Space for the initial stack is allocated on the heap, and the addresses and the size are stored in the base class variables. This stack is initialized, so that on activation, the stack is copied from the heap, and control is passed to the superMain function. The trick is to store the address of the starting function (startProcess) on the stack as a return address. When the stack-switching function terminates, control returns to this address as if it was calling the function. This function in turn starts the superMain function, which calls the main function of the subclass. The superMain function also takes care of necessary terminating actions. A coroutine may be deleted. In that case the coroutine::~coroutine() method releases the stack area. The resume primitive switches control from the active coroutine to the new current specified as parameter to the primitve. At first, the size of the current stack is computed, and space for storage is allocated on the heap. The PREVIOUS and CURRENT variables are set, pointing to the two coroutines involved. The assembler function ctxtswc is callled to do the actual stack chnge. On return, the new coroutine is CURRENT and active, and the area for stack storoage is released. The detach primitive resumes MAINC, the main program. You find ctxtswc in Listing 3. The global variables CURRENT and PREVIOUS are imported by ctxtswc. Through these pointers, the stack storage area of the two coroutines involved, are accessible. ctxtswc works by copying the current system stack to the area referenced in the PREVIOUS coroutine instance. This copy of the stack now contains the return address from the ctxtswc call. Then the stack of the new CURRENT coroutine is copied from the storage area to the system stack area, and the SP register is reset. This stack now contains the return address from the ctxtswc call, last time it was called by the coroutine. Therefore, when ctxtswc returns, program execution continues where the resumed coroutine was stopped. This version is written for the compact memory model. Minor changes in ctxtswc and some of the C++ functions are necessary for other memory models. Simulation With the tools described in the previous sections it is possible to model interacting real-world processes in a quasi-parallel program. In such a model you can experiment. By changing the number of processes, the behavior of each, and the input data you can test the model to gain insight into how the real system would perform under different circumstances. Simulating real processes in a computer model is often cheaper than doing real life experiments. In some cases it is not even possible to experiment with the real system. Consider the case of a nuclear reactor. In order to study the behavior of the reactor operators in stress situations, e.g. when serious problems occur, it would be necessary to induce problems in the reactor. We cannot do that, for obvious reasons. Instead a control room connected to a computer model, simulating the reactor, is used in experiments and training. Real systems are often very complex. When creating a model, it is necessary to include those features that are crucial to the model's operation, and avoid insignificant details. Otherwise the simulation results may be of no value. The time aspect of simulation, which is very important in most real systems, is not handled within the coroutine concept. In the next sections, I will introduce a set of tools for modelling processes in simulated time with C++. An Example This is a hypothetical system consisting of a number of physicians working together in a clinic receiving patients. The system is simplified in order to keep the example reasonably small in size. The physicians receive patients from 8 A.M. until 4 P.M. Then the physicians work until no more patients are waiting. The patients arrive one at a time. On arrival the patient goes to the waiting room. If there is a physician in the lunchroom, he is summoned at once. Otherwise the patient waits for his turn. When a patient leaves the clinic, the physician will see the next patient. If nobody is waiting, he will go to the lunchroom for a coffee break. The physicians, observing that most of the day the waiting room is full, want to know if another physician should be invited to join the group. They are losing business because the patients must wait too long, but they are not sure if this might lead to an increasing number of coffee breaks. The model of this system consists of a clinic object, a number of physician objects, and the patients. For each of these objects, there is a set of actions controlling the objects and their interactions. As the objects operate in simulated time, their classes are derived from the class process, a subclass of the class coroutine, which contains the information necessary to support the time concept. Instead of the resume and detach primitives, a small set of more advanced primitives will be used. These are based on resume and detach, but are more suitable for simulations in time. Most simulation models depend on some kind of input data that specifies when the significant events occur. In this case, the important events are the arrivals of patients, and the duration of a consultation. It is possible to collect real data and use the information directly as input to the simulation model. Another solution is to analyze the input data and find the events' probability distributions. Input data may then be generated using random generators. The physicians have observed the arrival pattern of patients and found that patients arrive according to a Poisson distribution, and they have found the expected number of patients each minute. They have also found that the duration of each consultation is uniformly distributed over an interval. A simulation model should always be validated by comparing the results of a simulation with real input data to real system results. If there are discrepancies, the model must be modified. The implementation of the clinic system is found in Listing 4. The implementation consist of three kinds of processes, defined by the classes clinic, physician, and patient derived from the process base class. Each class has a constructor, and a main function describing the process actions. There is a main program which creates the process instances, starts the simulation and presents the results. This example just finds the average waiting time of the patients. Two functions are used to generate input to the model. The treatmentPeriod function returns a randomly-chosen value indicating the duration of a consultation, while periodBeforeNextArrival returns the randomly-chosen period between two arrivals. The clinic class references two FIFO queues, chain lunchRoom and waitingRoom, where the physicians and patients are kept while waiting. In addition, there are variables used for collecting statistical data. The physician and patient classes have a pointer to the clinic, and variables for statistical data. The physician and patient constructors set the clinic pointer. The clinic constructor creates the physician processes and schedules them to run at once. The constructors also initialize statistical variables. The clinic main function describes the clinic actions. The main task is generating new patients. Its main loop will run until simulated time (currentTime) reaches the clinic's closing time. In the loop a new patient is created, sent to the waiting room, and activated. If there is a physician in the lunchRoom she is summoned and activated. Then the clinic process reschedules itself and waits until the next patient arrives, with the hold primitive. The physician process is an eternal loop. Each time the physician process checks if the waitingRoom is empty. If so, she goes to the lunchRoom and has a break by calling the passivate primitive. This primitive will suspend the process until some other process activates it. If there are patients in the waitingRoom, the physician gets the next patient, activates him and suspends herself with the hold primitive while helping the patient. When the consultation ends, she will see the patient out, activating him, and repeat the loop. The patient process is simple. It is activated three times, first by the clinic process when the patient is sent to the waitingRoom, then by the physician on leaving the waitingRoom, and last by the physician when the consultation is over. Each time, the patient process records the time of the event for statistical purposes before passivating itself. (When the patient main function terminates, there is an implicit passivate call.) The main program of the system first initializes the random generator functions and the process library by calling initProcesses. Then it suspends itself until opening hours when a clinic process is created and activated. The main program again suspends itself for a long period, until the clinic is closed and all patients have left. At last statistical data are processed and displayed. In this model, each process acts on its own, performing actions. Between events, the processes are suspended waiting to be restarted, either automatically as with the hold primitive, or by another process with the activate primitive. It is much easier to understand the sequence of events for each process and to describe each process separately as in this example, than to describe the whole system in one piece. Besides, you can modify the model to include other objects such as secretaries, surgeons, several patient types, and other action sequences. After validating the model, you can simulate and investigate different scenarios by changing the number of physicians, the arrival pattern of patients, consultation time etc. It is easy to include statements to collect other types of statistical data, such as the number of minutes spent in the lunchroom, and the distribution of patient waiting times. The Process Extensions I created the following extensions supporting processes: Each process is defined as a subclass of the process base class. A function initProcesses initializes the process library. A virtual function main in descendants of the process class contains the actions of the process. activate(process*,float) schedules the indicated process to run at the specified point of time. If it is already scheduled to run, it is rescheduled. If the current process is activated, it is suspended and rescheduled. hold(float) suspends and reschedules the current process to run after the indicated period of time. passivate suspends the current process without rescheduling. currentTime returns current simulated time. Whenever the current process is suspended, simulated time is increased to the time of the next scheduled process, and this process is restarted. If no process is scheduled to run, a runtime error occurs. In this way simulated time grows from 0 in varying steps as controls are switched from process to process. In addition, a few primitives not used in this example, are available cancel(process *) deschedules the process indicated. Cancelling the current process is the same as passivate. currentProcess returns a pointer to the current process object. mainProcess returns a pointer to the main program process object. The main program in itself is a process, and can be treated as such. resume(coroutine*) and detach may be used with processes as the process class is derived from the coroutine class. Avoid this, it might lead to unexpected activation sequences. Implementation In Listing 5 you find the process class definition. The process class is a coroutine subclass with a single time variable which contains scheduled time to run. It is initially set by the process constructor. Listing 6 contains the implementation of the process library. The data structure of the process module is a priority queue SOS (The Sequence Set) where the scheduled processes are kept sorted in ascending order on the time variable. The current executing process is always the first object in the SOS. There is a process* MAINP which is used for referencing the main program as a process. CURRENT and MAINC are imported from the coroutine module. The initProcesses function initializes the data structure. A process object referencing the main program is created and inserted at the head of the SQS. MAINC and CURRENT used for referencing the main program and the current coroutine, are set to reference this main program process. The four scheduling primitives activate, hold, passivate, and cancel are implemented as functions manipulating the SQS. activate inserts a process in the priority queue, hold removes and reinserts the current process, cancel removes any process, and passivate removes the current process. Whenever the current process is suspended, control is transferred to the next. The three information primitives mainProcess, currentProcess, and currentTime just return state information: which is the main program process, which is currently executing, and what is the current simulated time. The internal superMain function is called from startProcess to manage the user-defined main function. On termination, the process is removed from the SQS and cannot be activated again. Some Comments on the Code listings As shown, the implementations of the process primitives are very simple, once the coroutine primitives are in place. If the need arise, you can create other scheduling primitives of your own design. The code shown in the listings has been compiled and tested using Borland's Turbo C++ 2.0 and Turbo Assembler 2.5, compact memory model. It is easy to modify the system for other memory models. I have removed some include statements referring to system functions and the description of the priority type class chain. You may implement this yourself, otherwise a complete system is available from the author on request. References Birtwistle, Dahl, Myhrhaug, Nygaard: "SIMULA BEGIN," Auerbach, Phil., 1973. (contains an introduction to the SIMULA programming language) Listing 1 The base class coroutine definition // COR.HPP typedef unsigned int word; class coroutine { friend void startProcess(void); friend void resume(coroutine*); word stkSegment,stkOffset,stkSize; virtual void main() {} virtual void superMain() {} public: coroutine(void); ~coroutine(void); }; void resume(coroutine*); void detach(void); // End of File Listing 2 The base-class coroutine implementation // COR.CPP #include "cor.hpp" #include ... extern void ctxtswc(void); coroutine *MAINC=new coroutine(); coroutine *PREVIOUS,*CURRENT=MAINC; coroutine::coroutine() { word *sp,*stkBase; stkSize= 2 * sizeof(word); stkBase=(word *) farmalloc(stkSize); stkSegment=FP_SEG(stkBase); stkOffset=FP_OFF(stkBase); sp=stkBase + 2; *--sp=(word) startProcess;} coroutine::~coroutine() { delete(MK_FP(stkSegment,stkOffset));} void coroutine::superMain(void) { main(); resume(MAINC); FATAL("terminated coroutine resumed");} void startProcess(void) { CURRENT->superMain();} void resume(coroutine* rc) { word sp,*stkBase; if ((CURRENT != NULL) && (rc != CURRENT) && (rc != NULL)) { sp=_SP; CURRENT->stkSize= _stklen - sp + 4; stkBase= (word*)farmalloc(CURRENT->stkSize); CURRENT->stkSegment=FP_SEG(stkBase); CURRENT->stkOffset=FP_OFF(stkBase); PREVIOUS=CURRENT; CURRENT=rc; ctxtswc(); delete(MK_FP(CURRENT->stkSegment, CURRENT->stkOffset)); } } void detach(void) { resume(MAINC);} // End of File Listing 3 The assembler routine for changing stacks ; ctxtswc will do a contextswitch by changing stacks. ; the stack in use will be stored on the heap, and the ; new stack which is previously stored on the heap, ; will be loaded. .MODEL COMPACT .CODE EXTRN _stklen:word EXTRN _PREVIOUS:dword EXTRN _CURRENT:dword PUBLIC @ctxtswc$qv @ctxtswc$qv PROC NEAR push bp ; save bp mov cx,_stklen ; put nmbr of bytes sub cx,sp ; in cx les bx,dword ptr DGROUP:_PREVIOUS mov word ptr es:[bx+4],cx ; save sz of usedstk mov di,word ptr es:[bx+2] ; set adr for stk mov es,word ptr es:[bx] ; copy in di and es mov ax,ds ; save ds mov bx,ss ; set adr of stk in mov ds,bx ; ds mov si,sp ; and si rep movsb ; do copy mov ds,ax ; reset ds les bx,dword ptr DGROUP:_CURRENT mov cx,word ptr es:[bx+4] ; get sz of stk copy mov ax,ds ; save ds mov sp,_stklen sub sp,cx ; set stkPtr mov si,word ptr es:[bx+2] ; set adr of stored mov ds,word ptr es:[bx] ; stk in si and ds mov di,sp ; set dest for copy mov dx,ss ; di=stkPtr mov es,dx ; and es=stkSeg rep movsb ; do copy mov ds,ax ; reset ds pop bp ; reset bp ret @ctxtswc$qv ENDP END ; End of File Listing 4 Simulation of a medical clinic #include "sim.hpp" #include ... #define nmbrOfPhysicians 4 #define openingHours 480.0 #define openPeriod 480.0 #define consultationLow 5.0 #define consultationHigh 20.0 #define arrivalsExpected 0.25 float treatmentPeriod(void) { return(fUniform(consultationLow,consultationHigh)); } float periodBeforeNextArrival(void) { return(fNegexp(arrivalsExpected)); } class clinic : public process {public:chain lunchRoom,waitingRoom; float closingTime,totalWaitingTime; int totalNmbrOfPatients; clinic(int nrOfPhysicians,float Period); void main(void); }; class physician : public process { clinic* theClinic; public: physician(clinic* cl) {theClinic=cl;} void main(void); }; class patient : public process { clinic *theClinic; public: float startWaitingAt,startTreatmentAt, finishedAt; patient(clinic* cl) {theClinic=cl;} void main(void); }; clinic::clinic(int nrOfPhysicians, float Period) { int i; totalWaitingTime=0.0; totalNmbrOfPatients=0; closingTime=currentTime() + Period; for (i=0;iwaitingRoom.empty()) { theClinic->lunchRoom.append(this); passivate(); } else { ptnt=(patient*)theClinic->waitingRoom. getfirst(); activate(ptnt,currentTime()); hold(treatmentPeriod()); activate(ptnt,currentTime()); } } } void patient::main(void) { startWaitingAt=currentTime(); passivate(); startTreatmentAt=currentTime(); passivate(); finishedAt=currentTime(); theClinic->totalWaitingTime+= (startTreatmentAt-startWaitingAt); } void main(void) { clinic *clnc; initRandom(); initProcesses(); hold(openingHours - currentTime()); activate(clnc=new clinic( nmbrOfPhysicians,openPeriod), currentTime()); hold(10000.0); cout << "Total number of patients =" << clnc->totalNmbrOfPatients << end1; cout << "Average waiting time =" << clnc->totalWaitingTime/ clnc->totalNmbrOfPatients << end1; } // End of File Listing 5 The class process definition // SIM.HPP #include "cor.hpp" class process : private coroutine { friend void startProcess(void); friend float currentTime(void); friend void hold(float interval); friend void passivate(void); friend void activate(process *p,float time); friend void cancel (process *p); float time; virtual void superMain(void); virtual void main(void) {} public: process (void) {time=0.0;} ~process (void) {} }; void initProcesses(void); void activate(process *p,float time); void hold(float interval); void passivate(void); void cancel (process *p); process* mainProcess(void); process* currentProcess (void); float currentTime(void); // End of File Listing 6 A library implementation for the clinic simulation // SIM.CPP #include "sim.hpp" #include ... extern coroutine* CURRENT; extern coroutine* MAINC; process* MAINP=NULL; chain SQS; void initProcesses(void) { if (MAINP == NULL) { MAINP=newprocess(); SQS.put(MAINP,0); delete MAINC; MAINC=CURRENT=(coroutine*)MAINP; } } process* mainProcess(void) { return(MAINP); } process* currentProcess(void) { return((process *) CURRENT); } float currentTime(void) { return( ((process*)CURRENT)->time); } void hold(float interval) { if (interval > 0) { SQS.get(CURRENT); ((process *)CURRENT)->time+=interval; SQS.put(CURRENT, ((process *)CURRENT)->time); resume((coroutine*)SQS.first()); } } void passivate(void) { SQS.get(CURRENT); if (SQS.first() == NULL) FATAL("Sequence Set is empty"); resume((coroutine *)SQS.first()); } void activate(process *p,float time) { if (p == CURRENT) hold(time - currentTime()); else { SQS.get(p); p->time= ((time > currentTime())? time : currentTime()); SQS.put(p,p->time); } } void cancel(process *p) { SQS.get(p); if (p == CURRENT) { if (SQS.first() == NULL) FATAL("Sequence Set is empty"); resume((coroutine *)SQS.first()); } } void process::superMain(void) { main(); passivate(); FATAL("terminated process activated");} // End of File Standard C Time Formatting Functions P.J. Plauger P.J. Plauger is senior editor of The C Users Journal. He is convenor of the ISO C standards committee, WG14, and active on the C++ committee, WG21. His latest books are The Standard C Library, published by Prentice-Hall, and ANSI and ISO Standard C (with Jim Brodie), published by Microsoft Press. You can reach him at pjp@plauger.com. This is the last of three installments on the functions declared in . (See "The Header ," CUJ January 1993 and "Time Conversion Functions ," CUJ February 1993.) I conclude by describing the functions that convert encoded times to human-readable text strings. Standard C provides an extraordinary ability to capture times and dates. It also lets you convert freely between scalar and component forms. That lets you do arithmetic on components such as days or months, leaving to the library the hard work of correcting for calendar irregularities. The Standard C library even lets you switch between local and universal (UTC or GMT) time zones, correcting as needed for Daylight Savings Time. (All that assumes that the library has some way of learning time zone information from the environment, of course.) The icing on the cake is the ability to convert times to human-readable form. C has long had the functions ctime and asctime to perform this useful service. Committee X3J11 added the more general function strftime. It lets you pick the time and date components you wish to include. It also adapts to the current locale. Thus Standard C now encourages programs that print time information in the most useful form for each culture. (Again, that assumes that the library has a nontrivial implementation of the locale machinery.) The topic for this month is the remaining functions declared in --those that convert times to text strings. They are not as hairy as the conversion functions I described last month, but they have their own complexities. What the C Standard Says 7.12.3 Time conversion functions Except for the strftime function, these functions return values in one of two static objects: a broken-down time structure and an array of char. Execution of any of the functions may overwrite the information returned in either of these objects by any of the other functions. The implementation shall behave as if no other library functions call these functions. 7.12.3.1 The asctime function Synopsis #include char *asctime (const struct tm *timeptr); Description The asctime function converts the broken-down time in the structure pointed to by timeptr into a string in the form Sun Sep 16 01:03:52 1973\n\0 using the equivalent of the following algorithm. char *asctime (const struct tm *timeptr) { static const char wday_name[7][3] = { "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" }; static const char mon_name[12][3] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" }; static char result[26]; sprintf(result, "%.3s %.3s%3d %.2d:%.2d:%.2d %d\n", wday_name[timeptr->tm_wday], mon_name[timeptr->tm_mon], timeptr->tm_mday, timeptr->tm_hour, timeptr->tm_min, timeptr->tm_sec, 1900 + timeptr->tm_year); return result; } Returns The asctime function returns a pointer to the string. 7.12.3.2 The ctime function Synopsis #include char *ctime(const time_t *timer); Description The ctime function converts the calendar time pointed to by timer to local time in the form of a string. It is equivalent to asctime (localtime(timer)) Returns The ctime function returns the pointer returned by the asctime function with that broken-down time as argument. Forward references: the localtime function (7.12.3.4). .... 7.12.3.5 The strftime function Synopsis #include size_t strftime(char *s, size_t maxsize, const char *format, const struct tm *timeptr); Description The strftime function places characters into the array pointed to by s as controlled by the string pointed to by format. The format shall be a multibyte character sequence, beginning and ending in its initial shift state. The format consists of zero or more conversion specifiers and ordinary multibyte characters. A conversion specifier consists of a % character followed by a character that determines the behavior of the conversion specifier. All ordinary multibyte characters (including the terminating null character) are copied unchanged into the array. If copying takes place between objects that overlap, the behavior is undefined. No more than maxsize characters are placed into the array. Each conversion specifier is replaced by appropriate characters as described in the following list. The appropriate characters are determined by the LC_TIME category of the current locale and by the values contained in the structure pointed to by timeptr. "%a" is replaced by the locale's abbreviated weekday name. "%A" is replaced by the locale's full weekday name. "%b" is replaced by the locale's abbreviated month name. "%B" is replaced by the locale's full month name. "%c" is replaced by the locale's appropriate date and time representation. "%d" is replaced by the day of the month as a decimal number (01-31). "%H" is replaced by the hour (24-hour clock) as a decimal number (00-23). "%I" is replaced by the hour (12-hour clock) as a decimal number (01-12). "%j" is replaced by the day of the year as a decimal number (001-366). "%m is replaced by the month as a decimal number (01-12). "%M" is replaced by the minute as a decimal number (00-59). "%p" is replaced by the locale's equivalent of the AM/PM designations associated with a 12-hour clock. "%S" is replaced by the second as a decimal number (00-61). "%U" is replaced by the week number of the year (the first Sunday as the first day of week 1) as a decimal number (00-53). "%w" is replaced by the weekday as a decimal number (0-6), where Sunday is 0. "%W" is replaced by the week number of the year (the first Monday as the first day of week 1) as a decimal number (00-53). "%x" is replaced by the locale's appropriate date representation. "%X" is replaced by the locale's appropriate time representation. "%y" is replaced by the year without century as a decimal number (00-99). "%Y" is replaced by the year with century as a decimal number. "%Z" is replaced by the time zone name or abbreviation, or by no characters if no time zone is determinable. "%%" is replaced by %. If a conversion specifier is not one of the above, the behavior is undefined. Returns If the total number of resulting characters including the terminating null character is not more than maxsize, the strftime function returns the number of characters placed into the array pointed to by s not including the terminating null character. Otherwise, zero is returned and the contents of the array are indeterminate. The Function strftime The one complicated function declared in (from the outside, at least) is strftime. You use it to generate a text representation of a time and date from a struct tm under control of a format string. In this sense, it is modeled after the print functions declared in . It differs in two important ways: strftime does not accept a variable argument list. It obtains all time and date information from one argument. The behavior of strftime can vary considerably among locales. The locale category LC_TIME can, for example, specify that the text form of all dates follow the conventions of the French culture. For example, the code fragment: char buf[100]; strftime(buf, sizeof buf, "%A, %x", localtime(&t0)); might store in buf any of: Sunday, 02 Dec 1979 dimanche, le 2 decembre 1979 Weekday 0, 02/12/79 If your goal is to display times and dates in accordance with local custom, then strftime gives you just the flexibility you need. You can even write multi-byte-character sequences between the conversion specifiers. That lets you convert dates to Kanji and other large character sets. Here are the conversion specifiers defined for strftime. I follow each with an example of the text it produces. The examples are all from P.J. Plauger and Jim Brodie, ANSI and ISO Standard C, Microsoft Press, 1992. All assume the "C" locale and the date and time Sunday, 2 December 1979 at 06:55:15 AM EST: %a -- the abbreviated weekday name (Sun) %A -- the full weekday name (Sunday) %b -- the abbreviated month name (Dec) %B -- the full month name (December) %c -- the date and time (Dec 2 06:55:15 1979) %d -- the day of the month (02) %H -- the hour of the 24-hour day (06) %I -- the hour of the 12-hour day (06) %j -- the day of the year, from 001 (335) %m -- the month of the year, from 01 (12) %M -- the minutes after the hour (55) %p -- the AM/PM indicator (AM) %S -- the seconds after the minute (15) %U -- the Sunday week of the year, from 00 (48) %w -- the day of the week, from 0 for Sunday (0) %W -- the Monday week of the year, from 00 (47) %x -- the date (Dec 2 1979) %X -- the time (06:55:15) %y -- the year of the century, from 00 (79) %Y -- the year (1979) %Z -- the time zone name, if any (EST) %% -- the per cent character (%) Using the Formatting Functions Note that the two functions that return a value of type pointer to char return a pointer to a static data object. Thus, a call to one of these functions can alter the value stored on behalf of an earlier call to another (or the same) function. Be careful to copy the value stored in one of these shared data objects if you need the value beyond a conflicting function call. asctime--(The asc comes from ASCII, which is now a misnomer.) Use this function to generate the text form of the date represented by the argument (which points to a broken-down time). The function returns a pointer to a null-terminated string that looks like "Sun Dec 2 06:55:15 1979\n". This is equivalent to calling strftime with the format "%a %c\n" in the "C" locale. Call asctime if you want the English-language form regardless of the current locale. Call strftime if you want a form that changes with locale. See the warning about shared data objects, above. ctime--ctime(pt) is equivalent to the expression asctime(localtime(pt)). You use it to convert a calendar time directly to a text form that is independent of the current locale. See the warning about shared data objects, above. strftime--This function generates a null-terminated text string containing the time and date information that you specify. You write a format string argument to specify a mixture of literal text and converted time and date information. You specify a broken-down time to supply the encoded time and date information. The category LC_TIME in the current locale determines the behavior of each conversion. Implementing the Conversion Functions These functions convert encoded times to text strings in various ways. All depend, in the end, on the internal function _Strftime to do the actual conversion. What varies is the choice of locale. The function asctime (and, by extension, the function ctime) convert times by a fixed format, following the conventions of the "C" locale regardless of the current state of the locale category LC_TIME. The function strftime, on the other hand, lets you specify a format that directs the conversion of a broken-down time. It follows the conventions of the current locale. Thus, one of the arguments to _Strftime specifies the locale-specific time information (of type_Tinfo) to use. Listing 1 shows the file asctime.c. It defines the function asctime that formats a broken-down time the same way irrespective of the current locale. The file also defines the data object _Times that specifies the locale-specific time information. And it defines the internal data object ctinfo, which replicates the time information for the "C" locale. Listing 2 shows the file ctime.c. The function ctime simply calls localtime, then asctime, to convert its time_t argument. Thus, it always follows the conventions of the "C" locale. Listing 3 shows the file strftime.c. The function strftime calls _Strftime, using the locale-specific time information stored in_Times. Thus, its behavior changes with locale. Listing 4 shows the file xstrftim.c. It defines the internal function _Strftime that does all the work of formatting time information. _Strftime uses the macro PUT, defined at the top of the file xstrftim.c, to deliver characters. The macro encapsulates the logic needed to copy generated characters, count them, and limit the number delivered. The internal function _Mbtowc, declared in , parses the format as a multibyte string using state memory of type _Mbstate that you provide on each call. Listing 5 shows the file xgentime.c It defines the function _Gentime that performs the actual conversions for _Strftime. The function _Gentime consists primarily of a large switch statement that processes each conversion separately. Each conversion determines a pointer p to a sequence of characters that gives the result of the conversion. It also stores a signed integer count at *pn. A positive count instructs _Strftime to generate the designated sequence of characters. One source of generated characters is the function _Get_time, which selects a field from one of the strings in the locale-specific time information. Another is the internal function getval, also defined in the file xgentime.c, which generates decimal integers. getval stores characters in the accumulator provided by _Strftime. Note that _Gentime includes a nonstandard addition. The conversion specifier %D converts the day of the month with a leading space in place of a leading 0. That's what asctime insists on. _Gentime returns a negative count to instruct _Strftime to "push down" a format string for a locale-specific conversion. Three conversions change with locale: %c, %x, and %X. (The conversion %x, for example, becomes the format string "%b %d %Y" in the "C" locale.) You express these conversions as format strings that invoke the other conversions. Note that the function_Strftime supports only one level of format stacking. The other internal function in the file xgentime.c is wkyr. It counts weeks from the start of the year for a given day of the year. The week can begin on Sunday (wstart is 0) or Monday (wstart is 1). The peculiar logic avoids negative arguments for the modulus and divide operators. Conclusion For over two and a half years, I have used this column for a single purpose--to provide a guided tour of the Standard C library. Along the way, I got out a book of the same name. About half these installments have been excerpts from the completed book. With this installment, I bring to a close that guided tour. Starting next month, I'll pick up on other standards activities in the C community. That's more than you might think. When I started this column, I was a member of one ANSI standards committee. Now I find myself attending meetings for two ANSI committees and three ISO committees. Don't think I do it because it's unmitigated fun. I just think standards activities are too important these days to ignore. I'll do my best to keep you informed too. This article is excerpted in part from P.J. Plauger, The Standard C Library, (Englewood Cliffs, N.J.: Prentice-Hall, 1992). Listing 1 asctime.c /* asctime function */ #include "xtime.h" /* static data */ static const char ampm[] = {":AM:PM"}; static const char days[] = { ":Sun:Sunday:Mon:Monday:Tue:Tuesday:Wed:Wednesday" ":Thu:Thursday:Fri:Friday:Sat:Saturday"}; static const char fmts[] = { "%b %D %H:%M:%S %Y%b %D %Y%H:%M:%S"}; static const char isdst[] = {""}; static const char mons[] = { ":Jan:January:Feb:February:Mar:March" ":Apr:April:May:May:Jun:June" ":Jul:July:Aug:August:Sep:September" ":Oct:October:Nov:November:Dec:December"}; static const char zone[] = {""}; /* adapt by default */ static_Tinfo ctinfo = {ampm, days, fmts, isdst, mons, zone}; _Tinfo_Times = {ampm, days, fmts, isdst, mons, zone}; char *(asctime)(const struct tm *t) { /* format time as "Day Mon dd hh:mm:ss yyyy\n" */ static char tbuf[] = "Day Mon dd hh:mm:ss yyyy\n"; _Strftime(tbuf, sizeof (tbuf), "%a %c\n", t, &ctinfo); return (tbuf); } /* End of File */ Listing 2 ctime.c /* ctime function */ #include char *(ctime)(const time_t *tod) { /* convert calendar time to local text */ return (asctime(localtime(tod))); } /* End of File */ Listing 3 strftime.c /* strftime function */ #include "xtime.h" size_t (strftime)(char *s, size_t n, const char *fmt, const struct tm *t) { /* format time to string */ return (_Strftime(s, n, fmt, t, &_Times)); } /* End of File */ Listing 4 xstrftim.c /* _Strftime function */ #include #include #include "xtime.h" /* macros */ #define PUT(s, na) (void)(nput = (na), \ 0 < nput && (nchar += nput) <= bufsize ? \ (memcpy(buf, s, nput), buf += nput) : 0) size_t_Strftime(char *buf, size_t bufsize, const char *fmt, const struct tm *t, _Tinfo *tin) { /* format time information */ const char *fmtsav, *s; size_t len, lensav, nput; size_t nchar = 0; for (s = fmt, len = strlen(fmt), fmtsav = NULL; ; fmt = s) { /* parse format string */ int n; wchar_t wc; _Mbsave state = {0}; while (0 < (n = _Mbtowc(&wc, s, len, &state))) { /* scan for '%' or '\0' */ s += n, len -= n; if (wc == '%') break; } if (fmt < s) /* copy any literal text */ PUT(fmt, s - fmt - (0 < n ? 1 : 0)); if (0 < n) { /* do the conversion */ char ac[20]; int m; const char *p = _Gentime(t, tin, s++, &m, ac); --len; if (0 <= m) PUT(p, m); else if (fmtsav == NULL) fmtsav = s, s = p, lensav = len, len = -m; } if (0 == len && fmtsav == NULL n < 0) { /* format end or bad multibyte char */ PUT("", 1); /* null termination */ return (nchar <= bufsize ? nchar - 1 : 0); } else if (0 == len) s = fmtsav, fmtsav = NULL, len = lensav; } } /* End of File */ Listing 5 xgentime.c /* _Gentime function */ #include "xtime.h" /* macros */ #define SUNDAY 0 /* codes for tm_wday */ #define MONDAY 1 static char *getval(char *s, int val, int n) { /* convert a decimal value */ if (val < 0) val = 0; for (s += n, *s = '\0'; 0 <= --n; val /= 10) *--s = val % 10 + '0'; return (s); } static int wkyr(int wstart, int wday, int yday) { /* find week of year */ wday = (wday + 7 - wstart) % 7; return (yday - wday + 12) / 7 - 1; } const char *_Gentime(const struct tm *t, _Tinfo *tin, const char *s, int *pn, char *ac) { /* format a time field */ const char *p; switch (*s++) { /* switch on conversion specifier */ case 'a': /* put short weekday name */ p = _Gettime(tin->_Days, t->tm_wday << 1, pn); break; case 'A': /* put full weekday name */ p = _Gettime(tin->_Days, (t->tm_wday << 1) + 1, pn); break; case 'b': /* put short month name */ p = _Gettime(tin->_Months, t->tm_mon << 1, pn); break; case 'B': /* put full month name */ p = _Gettime(tin->_Months, (t->tm_mon << 1) + 1, pn); break; case 'c': /* put date and time */ p = _Gettime(tin->_Formats, 0, pn), *pn = -*pn; break; case 'd': /* put day of month, from 01 */ p = getval(ac, t->tm_mday, *pn = 2); break; case 'D': /* put day of month, from 1 */ p = getval(ac, t->tm_mday, *pn = 2); if (ac[0] == '0') ac [0] = ' '; break; case 'H': /* put hour of 24-hour day */ p = getval(ac, t->tm_hour, *pn = 2); break; case 'I': /* put hour of 12-hour day */ p = getval(ac, t->tm_hour % 12, *pn = 2); break; case 'j': /* put day of year, from 001 */ p = getval(ac, t->tm_yday + 1, *pn = 3); break; case 'm': /* put month of year, from 01 */ p = getval(ac, t->tm_mon + 1, *pn = 2); break; case 'M': /* put minutes after the hour */ p = getval(ac, t->tm_min, *pn = 2); break; case 'p': /* put AM/PM */ p = _Gettime(tin->_Ampm, 12 <= t->tm_hour, pn); break; case 'S': /* put seconds after the minute */ p = getval(ac, t->tm_sec, *pn = 2); break; case 'U': /* put Sunday week of the year */ p = getval(ac, wkyr(SUNDAY, t->tm_wday, t->tm_yday), *pn = 2); break; case 'w': /* put day of week, from Sunday */ p = getval(ac, t->tm_wday, *pn = 1); break; case 'W': /* put Monday week of the year */ p = getval(ac, wkyr(MONDAY, t->tm_wday, t->tm_yday), *pn = 2); break; case 'x': /* put date */ p =_Gettime(tin->_Formats, 1, pn), *pn = -*pn; break; case 'X': /* put time */ p = _Gettime(tin->_Formats, 2, pn), *pn = -*pn; break; case 'y': /* put year of the century */ p = getval(ac, t->tm_year % 100, *pn = 2); break; case 'Y': /* put year */ p = getval(ac, t->tm_year + 1900, *pn = 4); break; case 'Z': /* put time zone name */ if (tin->_Tzone[0] == '\0') tin->_Tzone = _Getzone(); /* adapt zone */ p = _Gettime(tin->_Tzone, 0 < t->tm_isdst, pn); break; case '%': /* put "%" */ p = "%", *pn = 1; break; default: /* unknown field, print it */ p = s - 1, *pn = 2; } return (p); } /* End of File */ Questions & Answers Function Return Values Kenneth Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C+ + language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member on the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. Q First of all I want to thank you for a wonderful magazine and informative answers to a lot of good programming questions. I have two easy ones for you. Why do most of the Standard C library string functions (strcpy, strcat) return a char * when they already modify the string through formal parameters? I have seen: s2 = strcpy(s2, s1); a lot less than I have seen: strcpy(s2, s1); I remember your saying something about return values screwing up the stack if they are not captured, and this seems like a major offender. The second question is geared towards C++. I have never seen a constructor or destructor declared with a return value. Are we assuming they return void or int, either of which would be better style to specifically declare? Thanks in advance for your time and useful information, and keep up the great column. Andrew Tucker Seattle Pacific University A Let's take your questions in order. I can't ever remember saying anything about not using the return value. It usually gets passed back in a register and if you do not use it, the value simply disappears when the register is reused. One reason for the char * return value is to allow the functions to be nested. You could specify a series of operations as: strcpy(a,""); strcat(a, b); strcat(a, c); or as: strcat(strcat(strcpy (a,""), b), c); I cannot say I particularly prefer the latter. Of course, if you were using C++ and a String class, the point becomes moot. String classes overload the operator + to mean concatenation. The assignment operator is also usually overloaded to mean string copying. So the set of operations can be expressed as: a = "" + b + c; or a = b + c; Functions such as fgets also return a character pointer, probably for the same reason. Although this is consistent with the str__ functions, it is inconsistent with the return type of most of the other file functions. As for your second question, constructors and destructors do not return any value. They are implicitly called in declarations, so there is no opportunity for them to return a value. You might call this a void return value, but that implies to me that the function actually is called normally. For example, suppose you coded: #include "string.hpp" a_function() { String a_string; ... } The constructor String(void) is implicitly called when a_string is declared. At the terminating brace (when a_string goes out of scope), the destructor ~String(void) is implicitly called. The lack of return value relates to the problem of what to do if there is an error in the constructor or destruct. For example, the initialization values might be out of range. You cannot return an error code. You have several choices. They include aborting the program, issuing a message alternative to the standard error output, or simply substituting in valid values. Just to be complete, let me note that a constructor can be called explicitly as a normal function. For example, if you had a string = string ("abc"), the constructor string (char*) is called and creates an object of class string. However, there is still no way to return an error code. A destructor can also be called explicitly, but the use for that is fiarly obscure. (KP) Check Digits This is in partial response to your answer to a question by Brendan O'Haire in the November 1992 issue of The C Users Journal. I came across this in reading sometime early in my career, 1958-1964 perhaps. If you would like proofs of the mathematical assertions, I will supply them on request. I remember the source where I read this claimed it was a technique widely-used in the business world. I cannot argue for or against that nor can I cite the original source. This procedure inserts a check "digit" into a number in such a way that all single transpositions of adjacent digits can be detected. It can actually detect single transposition at odd distance, but the adjacent transposition is the more probable keying error. The mathematical algorithm used to compute the check digit is actually an algorithm for computing the remainder upon division by 11. I'll use an example. Suppose the number to be encoded is 34781 and you wish to make the check digit the third digit from the right. Let Y represent the check digit. The new number will have the form 347Y81. Form a sum alternately adding and subtracting the digits in the number: 1 - 8 + Y - 7 + 4 - 3 = -13 + Y If the numeric part of this, -13 in this case, is not between -10 and 10 repeat the process: -13 + Y -> -(3 - 1) + Y = -2 + Y At this point pick a single digit for Y so the sum is either zero or eleven. Y is 2 in this case. It is possible Y might be 10 in some cases. The Roman numeral X is used as the "digit" in this case. The original number with the check digit inserted is now 347281. When the alternate addition and subtraction is performed: 1 - 8 + 2 - 7 + 4 - 3 = -11 -> -(1 - 1) = 0 then the ultimate value should be zero. If two digits are transposed, 342781 for example, the algorithm produces -1 for the example. A non-zero value indicates the number is not correct. The mathematical argument depends on the fact that any integer, in decimal notation, can be represented as a sum of two terms, A+B, where A is the alternating sum and difference of the digits and B is an exact multiple of 11. This latter fact is derived from two assertions which can be proved by mathematical induction. (10 ** (2k + 1)) + 1 and (10**2k) - 1, k = 0, 1, 2,... are each divisible by 11. All the terms in B have a factor of one or the other of these forms. Walter Beck I read with some interest the letter from Brendan O'Haire in your column in the November CUJ regarding check digits. I'm no mathematician nor a theorist, so I don't know the reasons why these things are done the way they are. However, I have recently run into a couple of algorithms for calculation of check digits in real-world programs, which differ significantly from the simple algorithm you gave in your response. The first one I tripped over was in doing some database conversion for a client. Part of the data were bank routing numbers, used for Electronic Funds Transfers (these numbers are, apparently, assigned by the Federal Reserve Bank). They are eight-digit numbers with a ninth digit appended, which is the check digit. The algorithm I was given to use for calculating these numbers is shown in the code in Listing 1. Also, I am currently involved in attempting to understand the specifications from HL-7. (HL-7 is a standard for data messages to be used in Health Care settings.) Some of the data types specified in the standard are short text strings with a check digit calculated either as mod 10 or mod 11. After some digging I was given the following description of the algorithm: Assume you have an identifier equal to 12345. Take the odd digit positions, counting from the right, i.e., 531, multiply this number by 2 to get 1062. Take the even digit positions, starting from the right, i.e., 42, append these to the 1062 to get 421062. Add all of these six digits together to get 15. Subtract this number from the next highest multiple of 10, i.e., 20 - 15 to get 5. The Mod10 check digit is 5. The Mod10 check digit for 401 is 0, for 9999, it's 4, for 9999999, it's 7. Listing 2 is my (possibly simplistic, and certainly not terribly efficient) implementation of this algorithm. I always enjoy reading your column. Keep up the good work! Fred Smith Thank you both for your contribution. I had an encoding course about twenty years ago, but the algorithms I recall were mainly for correction of binary values. They help in detecting and/or correcting binary digit errors (1 for 0 or 0 for 1). The parity bit on memory bytes is an example of a single error detection code. It could not detect a transposition error (switching two binary digits). Cyclic redundancy codes (CRC) can be used to correct multiple errors, such as might be found on magnetic disks. A single check digit for a decimal number can be used to detect single transposition errors or a single digit error. Since the digit is usually included as part of the number, it would be interesting to see if any of these algorithms fail if the check digit is transposed with a normal digit. When I used the check digit, it was set off as a separate character with a hyphen, so that mistake was less likely. Too bad telephone numbers don't include a check digit. The old rotary switches couldn't handle them, but computerized switches should be able to. I wonder if the saving in the amount of time spent dialing wrong numbers would compensate for the time spent dialing an extra digit. (KP) gotos In response to the letter from Raymond Lutz in the November CUJ I would like to point out the following useful feature of goto. I use gotos to consolidate the error-handling code near the bottom of the routine. This can be particularly useful when the normal path also includes most (or all) of the error-handling code as well (as in the case of cleanup code). See Listing 3 for an example. If I had used return instead of goto I would have had three copies of the free code for buf1, two copies of the free code for buf2, and four different exit points in the routine. This way I only have one copy of the free code and one exit point which makes maintaining the code easier. In brief, I believe that gotos, just like handguns, can be misused/abused but that is not a reason to avoid them altogether. You just have to make sure you use the right tool for the right job. James Brown Orem, UT I agree with your sentiments regarding gotos, as I have stated before in this column. Your particular example of memory allocations is of particular interest. A major program that I am helping out with has rather large dynamic memory requirements. It also has the need for exiting gracefully if memory runs out. The compiler vendor's allocation algorithms have created some interesting problems in this application. Memory was getting fragmented more than expected, so some tracing capabilities were needed. Instead of using malloc and free, I made up two wrapper functions called memory_allocate(unsigned int size, char * name) and memory_free(void *address, char * name). The name parameter is a string that describes the variable that is being allocated or freed. It is used for debugging and error reporting purposes. I can't give you the exact code, as it is for a proprietary project, but the pseudo-code looks something like Figure 1. (KP) Figure 1 Pseudocode for memory-tracing capability used for debugging and error reporting static char *names_allocated[MAX_ALLOCATIONS]; static unsigned int sizes_allocated[MAX_ALLOCATIONS]; static unsigned long addresses_allocated[MAX_ALLOCATIONS]; void *memory_allocate(unsigned int size, char *name) { // Call malloc with size // if okay, then store name, size, address in arrays // return address // else, print error message with name // optionally, print all names, sizes, addresses } void memory_free(void *address, char *name) { // If address == NULL // print error message with name // else find address in array // if not found // print error message with name // else // clear array elements } Listing 1 Algorithm for calculating bank routing numbers #define NUMVAL(x) (x - '0') #define TONUM(x) NUMVAL(x) #define TODIGIT(x) (x + '0') static char trconst[] = {'3', '7', '1', '3', '7', '1', '3', '7'}; char calc_check_digit (char * trnum) { int sum, val; int i; for ( sum = i = 0 ; i < 8 ; i++) { sum += TONUM (trnum[i]) * TONUM (trconst[i]); } val = 10 - (sum % 10); if (val == 10) val = 0; return (TODIGIT (val)); } /* End of File */ Listing 2 Algorithm for calculating HL-7 standard check digits /* * This implementation of the above algorithm has some distinct * limitations, although I believe they are not a serious problem. * This is intended for processing short strings of digits (10 or so), * not arbitrarily long digit strings. Each of the substrings that are * broken out by the code below are assumed to be a value that can * be assigned to a long when converted to an integral type. This * limitation requires that the input string not be more than about 20 * characters in length, which is more than adequate for the intended * application. I'm assuming that a long is at least 32 bits. * Further, it assumes a character set where the digits' collating * sequence mimics that of ASCII, so that things like: * x = val - '0'; * will evaluate to an integer value of 3 in the case where val == '3'. */ #include #define MOD10 10 #define MOD11 11 /* * -------------- - check_digit() -------- - * * Calculates a check digit according to the algorithm given above. * NOTE * that there are no checks for array bounds overflow. Beware. */ check_digit (string, mod) char * string; int mod; { char tmpbuf[20]; int i, j, sum; unsigned long odd, even; /* extract the odd digits */ j = strlen (string); for ( j-, i = 0 ; j >= 0 ; j -= 2, i++ ) { tmpbuf[i] = string[j]; } tmpbuf[i] = '\0'; odd = atol (tmpbuf); /* extract the even digits */ j = strlen (string); for ( j -= 2, i = 0 ; j >= 0 ; j -= 2, i++ ) { tmpbuf[i] = string[j]; } tmpbuf[i] = '\0'; even = atol (tmpbuf); /* now make the composite string & sum the digits */ sprintf (tmpbuf, "%lu%lu", even, 2L * odd); i = strlen (tmpbuf); for ( sum = j = 0 ; j < i ; j++) { sum += tmpbuf[j] - '0'; } /* now do the mod10 or mod11 operation */ i = (mod - (sum % mod)); if (i == mod) i = 0; return (i); } #ifdef TEST main () { char buf[30]; while (1) { gets (buf); if (strlen (buf) == 0) break; printf ("string: %s, mod10 checkdigit: %d," "mod11 checkdigit %d:n", buf, check_digit (buf, MOD10), check_digit (buf, MOD11)); } } #endif /* TEST */ /* End of File */ Listing 3 Code using goto to consolidate error-handling code int SomeFunc(void) { char *buf1, *buf2, *buf3; int ccode; if ((buf1 = malloc(1024)) == NULL) { ccode = -1; goto End1; } if ((buf2 = malloc(1024)) == NULL) { ccode = -1; goto End2; } if ((buf3 = malloc(1024)) == NULL) { ccode = -1; goto End3; } // More code here End3: free(buf3); End2: free(buf2); End1: free(buf1); return(ccode); } /* End of File */ Code Capsule A C++ Date Class, Part 2 Chuck Allison Chuck Allison is a software architect for the Family History Department of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest object-oriented technology and education. He is a member of X3J16, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801) 240-4510. In last month's capsule I presented the beginnings of a simple date class. In addition to providing a member function to calculate the interval between two dates, this class illustrated the following features of C++: inline functions references constructors controlled access to private data members In this month's installment I will add relational operators, input/output operations, and the capability to get the current date, while demonstrating these features: operator overloading streams friend functions static members When using dates you often need to determine if one precedes another. I will add the member function int compare(const Date& d2) const; to the date class (see Listing 1). Date::compare behaves like strcmp--it returns a negative integer if the current object (*this) precedes d2, 0 if both represent the same date, and a positive integer otherwise (see Listing 2 for the implementation and Listing 3 for a sample program). For those of you familiar with qsort from the Standard C library, you can use Date::compare to sort dates just like you use strcmp to sort strings. Here is a compare function to pass to qsort: #include "date.h" int datecmp(const void *p1, const void *p2) { const Date *d1p = (const Date *) p1, *d2p = (const Date *) p2; return d1p->compare(*d2p); } (Next month's CODE CAPSULE will cover qsort). Operator Overloading Most of the time, it is more convenient to have relational operators, for example if (d1 < d2) // do something appropriate.. Adding a less-than operator is trivial using Date::compare--just insert the following in-line member function into the class definition: int operator<(const Date& d2) const {return compare(d2) < 0); The compiler translates each occurrence of an expression such as d1 < d2 into the function call d1.operator<(d2) Listing 4 has the class definition with all six relational operators and the updated sample program is in Listing 5. Since the functionality of Date::interval is like a subtraction (it gives us the difference between two dates), it would seem natural to rename it as Date::operator-. Before doing this, take a closer look at the semantics of the statement a = b - c; No matter the type of the variables, the following should hold: a is a distinct object created by the subtraction, and b - c == - (c - b) I will use the convention that a "positive" Date object has all positive data members, and conversely for a "negative" date (mixed signs are not allowed). In Listing 7, I have replaced Date::interval with Date::operator- (const Date&), which affixes the proper signs to the data members and returns a newly-constructed Date object. The new class definition in Listing 6 also contains a unary minus operator, which also has the name Date::operator-, but takes no arguments. The compiler will transform the statements d1 - d2; -d1; respectively into d1.operator-(d2); // Calls Date::operator-(const Date&) d1.operator-(); // Calls Date::operator-() A sample program using these new member functions is in Listing 8. Stream I/O One thing remains before I can say that a Date object has the look and feel of a built-in type--input/output support. C++ supplies stream objects that handle I/O for standard types. For example, the output from the program #include main() { int i; cout << "Enter an integer: "; cin >> i; cout << "You typed " << i << endl; return 0; } will be something like Enter an integer: 5 You typed 5 cout is an output stream (class ostreom) supplied by the C++ streams library and cin is an input stream (class istream), which are associated by default with standard output and standard input, respectively. When the compiler sees the expression cout << "Enter an integer: " it replaces it with cout.operator<<("Enter an integer: ") which calls the member function ostream::operator<<(const char *). Likewise, the expression cout << i where i is an integer calls the function ostream::-operator<<(int). endl is a special stream directive (called a manipulator) which outputs a newline and flushes the output buffer. Output items can be chained together: cout << "You typed " << i because ostream::operator<< returns a reference to the stream itself. The above statement becomes (cout.operator<<("You typed ")).operator<<(i) To accommodate output of Date objects, then, you will need a global function that sends the printed representation to a given output stream, and then returns a reference to that stream: ostream& operator<<(ostream& os, const Date& d) { os << d.get_month() << '/' << d.get_day() << '/' << d.get_year(); return os; } This of course can't be a member function, since the stream (not the object being output) always appears to the left of the stream insertion operator. Friends For efficiency, it is customary to give operator<< access to the private data members of an object. (Most implementations of a class provide the associated I/O operators too, so it seems safe to break the encapsulation boundary in this case.) To bypass the restriction on access to private members, you need to declare operator<< to be a friend of the Date class by adding the following statement to the class definition: friend ostream& operator<<(ostream&, const Date&); This declaration appears in the new class definition in Listing 9, along with the corresponding friend declaration for the input function, operator>>. The implementation for these functions is in Listing 10, and Listing 11 has a sample program. Static Members A class in C++ defines a scope. This is why the function Date::compare would not conflict with a global function named compare (even if the arguments and return value were of the same type). Now consider the array dtab[] in the implementation file. dtab's static storage class makes it private to the file, but it really belongs to the class, not to the file. If I chose to spread the Date member functions across multiple files, I would be forced to group those that needed access to dtab together in the same file. A better solution is to make dtab a static member of the Date class. Static members belong to the whole class, not to a single object. This means that only one copy of dtab exists, and is shared by all objects of the class. Making the function isleap static allows you to call it without an associated object, i.e., you just need to say isleap(y); instead of d.isleap(y); To make isleap available to any caller, place it in the public section of the class definition, and call it like this: Date::isleap(y); As a finishing touch, I will redefine the default constructor to initialize its object with the current date. The final class definition, implementation and sample program are in Listing 12 - Listing 14 respectively. Summary In these last two installments I've tried to illustrate how C++ supports data abstraction--the creation of user-defined types. Constructors cause objects to be initialized automatically when you declare them. You can protect class members from inadvertent access by making them private. Overloading common operators makes your objects look and feel like built-in types--a plus for readability and maintenance. Listing 1 Introduces a function to compare dates // date4.h class Date { int month; int day; int year; public: // Constructors Date() {month = day = year= 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date * interval(const Date&) const; int compare(const Date&) const; }; // End of File Listing 2 Implements the interval and compare member functions // date4.cpp #include "date4.h" inline int isleap(int y) {return y%4 == 0 && y%100 != 0 y%400 == 0;} static int dtab[2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date * Date::interval(const Date& d2) const { static Date result; int months, days, years, prev_month; // Compute the interval - assume d1 precedes d2 years = d2.year - year; months = d2.month - month; days = d2.day - day; // Do obvious corrections (days before months!) // // This is a loop in case the previous month is // February, and days < -28. prev_month = d2.month - 1; while (days < 0) { // Borrow from the previous month if (prev_month == 0) prev_month = 12; -months; days += dtab[isleap(d2.year)][prev_month-]; } if (months < 0) { // Borrow from the previous year -years; months += 12; } // Prepare output result.month = months; result.day = days; result.year = years; return &result; } int Date::compare(const Date& d2) const { int months, days, years, order; years = year - d2.year; months = month - d2.month; days = day - d2.day; // return <0, 0, or >0, like strcmp() if (years == 0 && months == 0 && days == 0) return 0; else if (years == 0 && months == 0) return days; else if (years == 0) return months; else return years; } // End of File Listing 3 Tests the compare member function // tdate4.cpp #include #include "date4.h" void compare_dates(const Date& d1, const Date& d2) { int compval = d1.compare(d2); char *compstr - (compval < 0) ? "precedes" : ((compval > 0) ? "follows" : "equals"}; printf("%d/%d/%d %s %d/%d/%d\n", d1.get_month(),d1.get_day(0),d1.get_year(), compstr, d2.get_month(),d2.get_day(),d2.get_year()); } main() { Date d1(1,1,1970); compare dates(d1,Date(10,1,1951)); compare_dates{d1,Date(1,1,1970)); compare_dates(d1,Date(12,31,1992)); return 0; } /* OUTPUT 1/1/1970 follows 10/1/1951 1/1/1970 equals 1/1/1970 1/1/1970 precedes 12/31/1992 */ // End of File Listing 4 Defines relational operators for the Date class // date5.h class Date { int month; int day; int year; public: // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date * interval(const Date&) const; int compare(const Date&) const; // Relational operators int operator<(const Date& d2) const {return compare(d2) < 0;} int operator<=(const Date& d2) const {return compare(d2) <= 0;} int operator>(const Date& d2) const {return compare(d2) > 0;} int operator>=(const Date& d2) const {return compare(d2) >= 0;} int operator!=(const Date& d2) const {return compare(d2) != 0;} int operator!=(const Date& d2) const {return compare(d2) !=0;} }; // End of File Listing 5 Uses the Date relational operators // tdate5.cpp #include #include #include "date5.h" void compare_dates(const Date& d1, const Date& d2) { char *compstr = (d1 < d2) ? "precedes" : ((d1 > d2) ? "follows" : "equals"); printf("%d/%d/%d %s %d/%d/%d\n", d1.get_month(),d1.get_day(),d1.get_year(), compstr, d2.get_month(),d2.get_day(),d2.get_year()); } main() { Date d1(1,1,1970); compare_dates(d1,Date(10,1,1951)); compare_dates(d1,Date(1,1,1970)); compare_dates(d1,Date(12,31,1992)); return 0; } /* OUTPUT 1/1/1970 follows 10/1/1951 1/1/1970 equals 1/1/1970 1/1/1970 precedes 12/31/1992 */ // End of File Listing 6 Adds binary and unary minus to the Date class // date6.h class Date { int month; int day; int year; public: // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date operator-(const Date& d2) const; Date& operator-() {month = -month; day = -day; year = -year; return *this;} int compare(const Date&) const; // Relational operators int operator<(const Date& d2) const {return compare(d2) < 0;} int operator<=(const Date& d2) const {return compare(d2) <= 0;} int operator>(const Date& d2) const {return compare(d2) > 0;} int operator>=(const Date& d2) const {return compare(d2) >= 0;} int operator==(const Date& d2) const {return compare(d2) == 0;} int operator!=(const Date& d2) const {return compare(d2) != 0;} }; // End of File Listing 7 Implements the binary minus operator // date6.cpp #include #include "date6.h" inline int isleap(int y) {return y%4 == 0 && y%100 != 0 y%400 == 0;} static int dtab[2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date Date::operator-(const Date& d2) const { int months, days, years, prev_month, order; const Date * first, * last; // Must know which date is first if (compare(d2) <= 0) { // this <= d2 order = -1; first = this; last = &d2; } else { order = 1; first = &d2; last = this; } // Compute the interval; first <= last years = last->year - first->year; months = last->month - first->month; days = last->day - first->day; assert(years >= 0 && months >= 0 && days >= 0); // Do obvious corrections (days before months!) // This is a loop in case the previous month is // February, and days < -28. prev_month = last->month - 1; while (days < 0) { // Borrow from the previous month if (prev_month == 0) prev_month = 12; --months; days += dtab[isleap(last->year)][prey_month--]; } if {months < 0) { // Borrow from the previous year --years; months += 12; } // Return a date object with the interval if (order == -1) return Date(-months,-days,-years); else return Date(months, days, years); } int Date::compare(const Date& d2) const { // same as in Listing 2 } // End of File Listing 8 Subtracts two dates // tdate6.cpp: #include #include "date6.h" main() { Date d1(1,1,1970), d2(12,8,1992); Date result = d1 - d2; printf("years: %d, months: %d, days: %d\n", result.get_year(), result.get_month(), result.get_day()); result = d2 - d1; printf("years: %d, months: %d, days: %d\n", result.get_year(), result.get_month(), result.get_day()); int test = d1 - d2 == -(d2 - d1); printf("d1 - d2 == -(d2 - d1)? %s\n", test ? "yes" : "no"); return 0; } /* OUTPUT years: -22, months: -11, days: -7 years: 22, months: 11, days: 7 d1 - d2 == -(d2 - d1)? yes */ // End of File Listing 9 Adds stream I/O to the Date class // date7.h class ostream; class Date { int month; int day; int year; public: // Constructors Date() {month = day = year = 0;} Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date operator-(const Date& d2) const; Date& operator-() {month= -month; day = -day; year = -year; return *this;} int compare(const Date&) const; // Relational operators int operator<(const Date& d2) const {return compare(d2) < 0;} int operator<=(const Date& d2) const {return compare(d2) <= 0;) int operator>(const Date& d2) const {return compare(d2) > 0;} int operator>=(const Date& d2) const {return compare(d2) >= 0;} int operator==(const Date& d2) const {return compare(d2) == 0;} int operator!=(const Date& d2) const {return compare(d2) != 0) // I/O operators friend ostream& operator<<(ostream&, const Date&); friend istream& operator>>(istream&, Date&); }; // End of File Listing 10 Implements Date stream I/O functions #include #include "date7.h" ostream& operator<<(ostream& os, const Date& d) { os << d.month << '/' << d.day << '/' << d.year; return os; } istream& operator>>(istream& is, Date& d) { char slash; is >> d.month >> slash >> d.day >> slash >> d.year; return is; } // End of File Listing 11 Illustrates stream I/O of Date objects // tdate7.cpp: #include #include "date7.h" main() { Date d1, d2; cout << "Enter a date: "; cin >> d1; cout << "Enter another date: "; cin >> d2; cout << "d1 - d2 = "<< d1 - d2 << endl; cout << "d2 - d1 = "<< d2 - d1 << endl; return 0; } /* OUTPUT Enter a date: 10/1/1951 Enter another date: 5/1/1954 d1 - d2 = -7/0/-2 d2 - d1 = 7/0/2 */ // End of File Listing 12 Defines static members // date8.h // Forward declarations class istream; class ostream; class Date { int month; int day; int year; static int dtab[2][13]; public: // Constructors Date(); // Get today's date (see .cpp file) Date(int m, int d, int y) {month = m; day = d; year = y;} // Accessor Functions int get_month() const {return month;} int get_day() const {return day;} int get_year() const {return year;} Date operator-(const Date& d2) const; Date& operator-() {month = -month; day = -day; year = -year; return *this;} int compare(const Date&) const; // Relational operators int operator<(const Date& d2) const {return compare{d2) < 0;} int operator<=(const Date& d2) const {return compare(d2) <= 0;} int operator>(const Date& d2) const {return compare(d2) > 0;} int operator>=(const Date& d2) const {return compare(d2) >= 0;} int operator==(const Date& d2) const {return compare(d2) == 0;} int operator!=(const Date& d2) const {return compare(d2) != 0;} // Stream I/O operators friend ostream& operator<<(ostream&, const Date&); friend istream& operator>>(istream&, Date&); static int isleap(int y) {return y%4 == 0 && y%100 != 0 y%400 == 0;} }; // End of File Listing 13 Final implementation of the Date class // date8.cpp #include #include #include #include "date8.h" // Must initialize statics outside the class definition int Date::dtab[2][13] = { {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31} }; Date Date::operator-(const Date& d2) const { int months, days, years, prev_month, order; const Date * first, * last; // Must know which date is first if (compare(d2) <= 0) { // this <= d2 order= -1; first = this; last = &d2; } else { order = 1; first = &d2; last = this; } // Compute the interval; first <= last years = last->year - first->year; months = last->month - first->month; days = last->day - first->day; assert(years >= 0 && months >= 0 && days >= 0); // Do obvious corrections (days before months!) // // This is a loop in case the previous month is // February, and days < -28. prev_month = last->month - 1; while (days < 0) { // Borrow from the previous month if (prev_month == 0) prev_month = 12; -months; days += dtab[isleap(last->year)][prev_month-]; } if (months < 0) { // Borrow from the previous year -years; months += 12; } // Return a date object with the interval if (order == 1) return Date(-months,-days,-years); else return Date(months,days,years); } int Date::compare(const Date& d2) const { int months, days, years, order; years = year - d2.year; months = month - d2.month; days = day - d2.day; // return <0, 0, or >0, like strcmp() if (years == 0 && months == 0 && days == 0) return 0; else if (years == 0 && months == 0) return days; else if (years == 0) return months; else return years; } ostream& operator<<(ostream& os, const Date& d) { os << d.month << '/' << d.day << '/' << d.year; return os; } istream& operator>>(istream& is, Date& d) { char slash; is >> d.month >> slash >> d.day >> slash >> d.year; return is; } Date::Date() ( // Get today's date time_t tval = time(0); struct tm *tmp= localtime(&tval); month = tmp->tm_mon+1; day = tmp->tm_mday; year = tmp->tm_year + 1900; } // End of File Listing 14 Gets today's date // tdate8.cpp: #include #include "date8.h" main() { Date today, d2; cout << "Today's date is "<< today << endl; cout << "Enter another date: "; cin >> d2; cout << "today - d2 = "<< today - d2 << endl; cout << "d2 - today = "<< d2 - today << endl; return 0; } /* OUTPUT Today's date is 12/12/1992 Enter another date: 1/1/1970 today - d2 = 11/11/22 d2 - today = -11/-11/-22 */ // End of File Stepping Up To C++ Inheritance, Part 1 Dan Saks Dan Saks is the founder and principal of Saks & Associates, which offers consulting and training in C++ and C. He is secretary of the ANSI and ISO C++ committees, and contributing editor for the Windows/DOS Developer's Journal. Dan is coauthor of C++ Programming Guidelines, and codeveloper of the Plum Hall Validation Suite for C++ (both with Thomas Plum). You can reach him at 393 Leander Dr., Springfield OH, 45504,4906, by phone at (513)324-3601, or electronically at dsaks@wittenberg.edu. Although they don't always agree on the exact meaning of the terms, most people who know something about object-oriented programming agree that it employs at least three techniques: Data abstraction Inheritance Polymorphism Thus far in my column, I've only covered C++ features that support data abstraction--namely, classes, access specifiers, constructors and destructors, and features such as references and operator overloading that help you build more intuitive abstractions. In this article, I'll introduce inheritance. Some of you may be wondering why I've waited so long to deal with inheritance. I think inheritance is useful, but not nearly as useful as classes and access specifiers. Many programmers overrate its value and consequently misuse it. I use inheritance sparingly in my own work. The other reason I've postponed discussing inheritance is that inheritance is a technique for defining new classes from existing classes. As such, understanding inheritance requires an understanding of all of those features I listed above. Now that I've covered them, I'll take on inheritance. The Basics Inheritance is a technique for creating a new class, called the derived class, from an existing class, called the base class. The derived class definition looks like any other base class definition, except for the presence of a base class specifier appearing immediately after the derived class name. For example, the definition class D : public B { //... }; includes the base class specifier : public B, so that class D is derived from (a previously defined) base class B. The keyword public after the colon (:) indicates that B is a public base class of D, or conversely, that D is publicly-derived from B. Base classes can also be private or protected. For the moment, I will only consider public base classes. The derived class inherits nearly all the members (data and functions) of the base class even though the derived class definition doesn't even mention the inherited members. You add more members to a derived class by simply declaring them inside the class body. For example, if you define class B as class B { public: int f(); void g(int); private: int i, j; }; then class D : public B { public: double h(double); private: double x; }; defines class D inheriting the members of B aing a public member function h(double) and a private data member x. The inherited public members--functions f() and g(int)--become public members of D. The inherited private members--data members i and j--become part of class D, but remain private to the B part of D. A D member function, like h(double), can only access i and j via public members (and friends, if any) of B. You may be surprised by this restriction, but if derived classes could directly access private base class members, then anyone could violate the encapsulation of a class by simply deriving another class from it. A derived class can be a base class for further derived classes. For example, class F : public D { public: char *p; }; derives class F from D, adding a public data member p to F. Is-A Relationships Inheritance provides a simple, explicit notation for creating specialized versions of broader classes. Inheritance defines an Is-A relationship (some people prefer Is-A-Kind-Of for added clarity): an object of a derived class is an object of the base class. A derived class object has everything that a base class object has, and usually more. It never has less. For example, Stroustrup (1991) introduces inheritance by sketching an example dealing with employees and their managers. Class employee defines the representation for each employee, with data such as name, age, and salary. Since a manager is an employee (with additional powers and responsibilities), Stroustrup derives class manager from class employee: class manager : public employee { // additional members that // distinguish managers from // other employees }; Since a manager is an employee, any function f with a formal parameter of type employee or employee & will accept an actual argument of type manager. For example, given void f(employee &e); manager m; the call f(m) binds the formal parameter e to actual argument m. Inside f, e appears to refer to an employee, which it does. It just happens that in this particular call, e refers to an employee that is also a manager. But f can't tell that e actually refers to a manager; it can only access the employee part of the object. As a general rule, whenever class B is a public base of class D, you can: 1. Convert a D * to a B * (a pointer to a derived object to a pointer to a base object) 2. Convert a D & to a B & (a reference to a derived object to a reference to a base object) 3. Initialize a B & to refer to a D These conversions are transitive. That is, if class D is a public base of class F, then you can convert an F * to either a D *or a B *, or an F & to either a D & or a B &. For example, given class B {...}; class D : public B {...}; class F : public D {...}; B b; D d; F f; then all of the following operations are valid: B *pb = &d; // ok, a D* is a B* D *pd = &f; // ok, an F* is a D* pb = &f; // ok, an F* is a D*, // which is a B* D &rd = f; // ok, an F is a D B &rb1 = rd; // ok, a D is a B B &rb2 = f; // ok, an F is a D, // which is a B Although an object of a derived class is an object of its base class, the opposite is not true. Thus, you cannot convert a pointer (or reference) to a base object to a pointer (or reference) to a derived object, unless you use a cast. For example, given the immediately preceding declarations, then pd = pb; // error, a B* is // not a D* pd = (D *)pb; // ok, but suspect F &f = b; // error, a B is // not an F In general, C++ also lets you initialize a base class object with a derived class object, as in D d; //... B b(d); or simply assign a derived class object to a base class object, as in b = (d); (I detailed the differences between initialization and assignment in "Initialization vs. Assignment," CUJ, September 1992.) Both the initialization and the assignment copy only the inherited B members from d to b. The ARM (Ellis & Stroustrup 1990) doesn't specifically permit conversion from a derived class object to a base class object, but it falls out from the reference conversions. When you write B b(d); C++ translates this to a call to B's copy constructor, typically declared as B::B(const B &br); The constructor call binds formal parameter br (a reference to a base object) to actual argument d (a derived class object), as permitted by in rule 3 previously mentioned. Similarly, when you write the assignment b = d; C++ translates this to a call to B's assignment operator: B &B::operator=(const B &br); Again, calling this assignment operator binds br to d, employing the same reference conversion. How It Works Some insight into how C++ implements inheritance may help you understand and remember the conversion rules a bit better. Although C++ imposes some restrictions on the storage layout for derived class objects, it doesn't require that an implementation use any particular layout strategy. Figure 1 shows a simple base class B and a typical storage layout for an object of class B. Notice that only the data fields occupy storage in the object. C++ resolves B's member function calls at translation time, so it need not store any information about the member functions in the object. Figure 2 shows a typical layout for a class D derived from class B in Figure 1. The B sub-object (the B part) occupies the beginning (the lowest addresses) of a D object. Thus, converting a pointer (or reference) to a D into a pointer (or reference) to a B doesn't require any generated code; it's strictly a compile-time transformation. Calling a function, void f(B &br); with an actual argument d of type D binds br to the B part of d. Typical generated code simply assigns the address of d to the pointer that implements br. No pointer arithmetic or indirection is needed. The body of the function can't tell whether br is bound to a B or to an object of a class derived from B. But, since all classes derived from B have at least everything that B has, f can safely access all members of br. That there might be more members beyond the fringes of B is not a concern. This model also illustrates why you generally can't convert in the opposite direction, that is, from pointer (or reference) to base to pointer (or reference) to derived. For example, calling a function, void g(D &dr); with an actual argument b of type B attempts to bind dr to b. But, referring to Figure 1 and Figure 2, a B object doesn't have an x member, so accessing dr.x inside g would reach beyond the end of b into uncharted territory. Thus, converting from base to derived violates the type safety rules in C++. You can't make the call without casting the actual argument b to type D. Overriding A derived class can redefine a function inherited from a base case, as shown in Listing 1. Here, the base class B defines two functions, f and g. The derived class D inherits both functions from B, but then replaces g with a definition of its own. This replacement is called overriding. Figure 3 shows the output from the program in Listing 1. Calling d.f calls the f inherited from B. The translator need not generate any new code for D's f; it can simply invoke the code already generated for B's f, passing the address of d's B sub-object as the value of this. However, since D's g overrides the g inherited from B, calling d.g calls a different function than calling b.g. When a derived class overrides an inherited public function, that function is hidden, but not completely inaccessible. The derived class can still access the hidden member by using the scope resolution operator, ::, and explicitly qualifying the member name with the base class name, as shown in Listing 2. In this example, derived class D overrides inherited functions and g and h. The call to B::g inside D's g calls B's g, as shown in the program output in Figure 4. Without the qualifier B::, a call to g inside D's g would be a recursive call to D's g. The body of D::h in Listing 2 shows another technique for calling an overridden member function using a cast. Remember that, inside the body of a member function, a call to a member function like h is actually a call to this->h. In a D member function, the type of is D const *. By casting this to B * (which is a valid conversion from pointer to derived to pointer to base) inside D::h, I forced the translator to look for h in the scope of B, and bypass looking in the scope of B. It works, but I recommend avoiding the cast and using the scope resolution operator as I did in D::g. A derived class can override inherited data members as well as function members. For example, consider class B { public: int n; // ... }; class D : public B { public: long n; void f(); // ... }; An object d of class D has storage for both an int n and a long n. Inside D::f, an unqualified reference to n refers to D::n (the long); B::n refers to the inherited int n. A derived class cannot delete inherited members. An Example In my last two articles I described and implemented several versions of class float_array, an array of float for which you can set the number of elements at runtime. (See "Dynamic Arrays," CUJ, November 1992, and "The Function operator[]", CUJ, January 1993.) Listing 3 shows the class definition for the climactic version, which grows automatically to keep subscript references in bounds. float_arrays, like all other arrays in C and C++, have the lowest subscript fixed at zero. This is less than ideal for some applications. Programming languages like Pascal, and its descendents Modula-2 and Ada, let you declare arrays with low bounds other than zero. In C++, you can fill the need by creating a class that I'll call float_vector, for which you specify not the number of elements but the low and high bounds of the subscript. For example, float_vector fv(1, 10); declares fv as an array of float whose subscript range is 1 to 10, inclusive. float_array already embodies much of the functionality for float_vector, so let's consider implementing float_vector by deriving it from float_array. Listing 4 shows the class definition for float_vector. float_vector adds a new private data member, _low, that records the vector's low bound. A float_vector need not store the high bound as a data member because it can determine the high bound from the low bound and length (inherited from float_array). float_vector defines a constructor, float_vector(int lo, int hi), that builds a vector with subscripts from lo to hi, inclusive. It also adds two query functions, low and high, that return the values of the current low and high subscripts, respectively. The float_vector constructor is extremely terse, so I defined it as an inline function in the header, as inline float_vector::float_vector(int lo, int hi) : _low(lo), float_array(hi - lo + 1) { } The constructor's member-initializers do all the work. The first initializer, _low(lo), fills in the private data member _low. The second initializer, float_array(hi - lo + 1), invokes the base class constructor to initialize the inherited data members array and len.hi - lo + 1 is the number of elements in a float_vector whose subscript range is from lo to hi. Remember, inherited private members are not directly accessible in the derived class. The derived class must use the public interface provided by the base class, which in this case, is a public constructor. Unlike member initializers that I've used in the past, the leading identifier in the initializer float_array(hi - lo + 1) is the name of a type, not the name of a data member. There's no named member for the float_array sub-object in the float_vector, so you must refer to it using its type name. float_vector overrides both inherited operator[] functions with new implementations that work when the low subscript bound is nonzero. Notice that the formal parameters for both the const and non-const float_vector::operator[] are int, and not size_t, as they are in class float_array, because a float_vector's low bound may be negative. Listing 5 contains the non-inline float_vector member functions, namely, both forms of operator[]. The function bodies are identical; they both rely on the inherited (overridden) versions of operator[] to do most of the work. The expression i - low shifts the subscript i into a subscript range whose low bound is zero. The statement return float_array::operator[] (i - low); calls the inherited operator[] to select the desired element from the inherited float_array sub-object, and extend the array if necessary. Listing 6 shows a test program for float_vectors. Notice that the display function (brought over from my previous two articles) still accepts a second argument of type const float_array &. I did not change it to const float_vector &. I also left a float_array in the test program to show that the display function accepts arguments of both the base and derived types. A sample output from the program appears in Figure 5. The abnormal termination is intentional. I planted a subscripting error just to show that the assertions work. Food for Thought I was forced to make a simplifying assumption in my float_vector class, namely, that you can only extend the high bound of a vector. The low bound must remain fixed. That's why both float_vector::operator[] functions include the assertion assert(i >= low()); The problem is that my design for float_arrays did not anticipate that I might want to extend the arrays in both directions. I'm not sure that I want to extend the low bound, but given this design, it's out of the question. Inheritance is often advertised as a wonderful technique for reusing existing code. But the reuse doesn't always work out the way you want. The reality is that you must decide for each class that you build whether you intend to use it as a base class for further derivation, and if so, how you or others might wish to use it. My design also raises another question. I said earlier that public inheritance defines Is-A relationships. You might reasonably ask if float_vector really is a float_array, or if I just used a convenient implementation trick. I'll ponder this and other questions in the next part of this series. References Ellis, Margaret A. and Bjarne Stroustrup. 1990. The Annotated C++ Reference Manual. Reading, MA: Addison-Wesley. Stroustrup, Bjarne. 1991. The C++ Programming Language, 2nd. ed. Reading, MA: Addison-Wesley. Figure 1 Base class B and typical storage layout for derived class objects Figure 2 Typical layout for a class D drived from B Figure 3 Output from Listing 1 B::f() B::g() B::f() D::g() Figure 4 Output from Listing 2 B::f() B::g() B::h() B::f() B::g() ... called from D::g() B::h() ... called from D::h() Figure 5 Sample output from the program in Listing 6 low? 3 high? 6 fa = { 3 4 5 6 } fb = { 3 4 5 6 } fb = { 9 4 5 6 } fb = { 9 16 5 6 } fb = { 9 16 25 6 } fb = { 9 16 25 36 } fb = { 9 16 25 36 49 } fb = { 9 16 25 36 49 64 } fb = { 9 16 25 36 49 64 81 } fb = { 9 16 25 36 49 64 81 100 } fb.low() = 3 fb.high() = 10 fc = { 3 4 5 6 } fc = { 3 4 5 123 } fc = Assertion failed: i >= low, file fv1.cpp, line 19 Abnormal program termination Listing 1 Overriding an inherited function #include class B { public: void f(); void g(); }; void B::f() { cout << "B::f()\n"; } void B::g() { cout << "B::g()\n"; } class D : public B { public: void g(); }; void D::g() { cout << "D::g()\n"; } int main() { B b; b.f(); b.g(); D d; d.f(); d.g(); return 0; } // End of File Listing 2 Calling an overridden inherited function #include class B { public: void f(); void g(); void h(); }; void B::f() { cout << "B::f()\n"; } void B::g() { cout << "B::g()\n"; } void B::h() { cout << "B::h()\n"; } class D : public B { public: void g(); void h(); }; void D::g() { B::g(); cout << "... called from D::g()\n"; } void D::h() { ((B *)(this))->h(); cout << "... called from D::h()\n"; } int main() { B b; b.f(); b.g(); b.h(); D d; d.f(); d.g(); d.h(); return 0; } // End of File Listing 3 Class definition for float_array // fa1.h - a dynamic array of float using a subscripting // object #include class fa_index { friend class float_array; public: fa_index &operator=(float f); operator float(); private: fa_index(float_array *f, size_t i); float_array *fa; size_t ix; }; class float_array { friend class fa_index; public: float_array(size_t n = 0); float_array(const float_array &fa); -float_array(); float_array &operator=(const float_array &fa); float operator[](size_t i) const; fa_index operator[](size_t i); inline size_t length() const; private: void extend(size_t i); float *array; size_t len; }; ostream &operator<<(ostream &os, const float_array &fa); inline size_t float_array::length() const { return len; } // End of File Listing 4 Class definition for float_vector // fv1.h - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object #include #include "fa1.h" class float_vector : public float_array { public: float_vector(int lo = 0, int hi = 0); float operator[](int i) const; fa_index operator[](int i); int low() const; int high() const; private: int _low; }; inline float_vector::float_vector(int lo, int hi) : low(lo), float_array(hi - lo + 1) { } inline float_vector::low() const { return_low; } inline float_vector::high() const { return_low + length() - 1; } // End of File Listing 5 Non-inline float_vector member functions. // fv1.cp- a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object #include "fv1.h" #include float float_vector::operator[](int i) const { assert(i >= low()); return float_array::operator[](i - low()); } fa_index float_vector::operator[] (int i) { assert(i >= low()); return float_array::operator[](i - low()); } // End of File Listing 6 Test program for float_vector // tv1.cpp - a test program for float_vectors and // float_arrays #include #include "fv1.h" void display(const char *s, const float_array &fa) { cout << s << " = " << fa << endl; } int main() { int i, low, high; cout << "low? "; cin >> low; cout << "high? "; cin >> high; float_vector fa(low, high); for (i = fa.low(); i <= fa.high(); ++i) fa[i] = i; display("fa", fa); float_vector fb = fa; display("fb", fb); for (i = low; i < low + 2 * fa.length(); ++i) { fb[i] = i * i; display("fb", fb); } cout << "fb.low() = " << fb.low() << '\n'; cout << "fb.high() = " << fb.high() << '\n'; float_array fc = fa; display("fc", fc); i = fc.length(); fc[i - 1] = 123; display("fc", fc); cout << "fa[" << low - 1 << "] = "; cout << fa[low - 1] << '\n'; return 0; } // End of File C-Clearly William Smith William Smith is the engineering manager at Montana Software, a sorftware development company specializing in custom applications for MS-DOS and Windows. You may contact him by mail at P.O. Box 663, Bozeman, MT 59771-0663. C is a free format language. It opens the door to an infinite variation of appearances based on the programmers personal preference. Nearly every programmer has opinions, sometimes very strong, of how C code should be formatted. Arguments over curly brace placement, indentation, white space, and comment placement abound. Over time, programmers first become comfortable with a certain style, then attached and eventually religiously committed. Style, especially the appearance aspect of style is a matter of personal taste. You can radically change the appearance of C without changing what the code does. Depending on the placement of indentation and white space you can make the code very readable or nearly unreadable. Obviously the goal is to use appearance to enhance the visibility of programming structures such as functions, blocks, loops, etc. Unfortunately what is readable and visible to one programmer is obfuscated to another. C-Clearly by V Communications is a C & C++ source code formatting utility that claims to be able to give you complete control over code appearance. It is a "you can have it any way you like it" utility. C-Clearly attempts to be the ultimate code beautifier/formatter. If true, it ends the argument on what appearance is appropriate. Every programmer can have it his or her own way. In a work group situation, programmers can use the style they are comfortable with and later reformat their code to the standards specified by the group. Alternatively, a programmer can take someone else's code and reformat it into the style that he or she is most comfortable with and efficient at reading. Sounds good in theory, but there are limitations. First of all, style is not just where the curly braces go. Style consists of both structure and appearance. Structure involves programming constructs, variable naming, and the type of techniques to execute a certain process. C-Clearly cannot do anything about this aspect of coding style. What it can do is alter the appearance of code. And even this has its limits. What C-Clearly Does There are infinite combinations of appearance options for C and C++ code. Instead of a long list of command-line options or an input screen containing a myriad of check boxes and radio buttons, C-Clearly's user interface for selecting a specific combination of appearance options is a template file. The template file is a recipe of pseudo-C commands that set the formatting definitions. C-Clearly comes with six template files, three each for C and C++. Listing 1, knr. ccl, is the template file for K&R-style C code. You can create your own template file by starting with one of the delivered template files and editing it. You have to be careful when editing a template file. You can only modify the white space in the template file. In addition to the this restriction, the fact that some constructs are defined in more than one place creates another restriction. The constructs that are defined in more than one place must be consistent or C-Clearly will warn you when you have conflicting appearance constraints. C-Clearly supports both a command-line and an interactive user interface. The command-line interface is a subset of the interactive user interface. You cannot specify any style formatting options on the command line. You must use the interactive user interface to specify formatting options and then save them as the default. C-Clearly stores the default settings directly in the executable file. This means it writes to and modifies the executable file. Most Anti-Virus software will detect this. So you will have to tell your Anti-Virus program to ignore this situation. Besides the template file there are four categories of additional appearance choices. The first category involves margins, tabs, and nesting lines. Nesting lines involve adding lines to show the connection between the beginning and end of logical blocks. The second category involves comments. You can select how to align and group comments. C-Clearly will make attempts to beautify comments, but I have never found what it does to my liking. I usually turn most of the comment formatting options off. The third category applies to white space. You can tell C-Clearly to retrain existing blank lines depending upon their placement. The fourth category is for hardcopy output only. You can send the output directly to a printer or to a file that is ready to print. When generating hardcopy output, C-Clearly can add line numbers, printer escape codes, and additional information such as headers, footers, and code metrics information. By inserting printer escape codes around different code constructs you can select printing styles or fonts. For example, you can print all comments in italics. There are no printer drivers so you will have to determine the escape codes for the printer you are using. The combination of code templates, an interactive user interface for additional style selection, and a terse commandline interface works well for C-Clearly. I find it quite easy to use. The user manual is perfect bound and about 55 pages in length. Chapter 2 covers template files. It is only a couple of pages in length, but if you are going to create your own template file it is must reading. This all sounds good until you have to start formatting some code. Even though C-Clearly has a lot going for it, it has some severe limitations. As soon as you are faced with the chore of formatting files, you will most certainly bump up against C-Clearly's unfortunate limitations. What C-Clearly Doesn't Do C-Clearly requires syntactically correct source code. This in itself is not that big a drawback. If your code complies it is probably syntactically correct. Well, not always, according to C-Clearly. C-Clearly does not like some C constructs that are rigorously correct in syntax and compile just fine. The ability of C-Clearly to have so much control and flexibility in formatting code means that it has to parse and understand code nearly to the same level as a compiler. Unfortunately, it chokes on many complex but entirely valid C constructs. When C-Clearly finds a syntax it does not like, it quits and makes no further attempt to format the file. This problem is a serious flaw in C-Clearly. It truly cripples an otherwise fine product. From experience, an important area of weakness that I have identified is in handling types and macros that deal with types. C-Clearly has trouble dealing with typedefs, structures, defines, and variables that happen to have the same name. This may not be a good idea to have the same name for all these different things, but it is valid. C-Clearly also has problems with macros that expand to structure member references. For example macros similar to the following have caused me trouble with C-Clearly. #define MEMBER StructInstance->Member The token pasting preprocessor operator, ##, can also cause C-Clearly to fail on perfectly correct code. Sometimes you can coax C-Clearly to still format a file by turning off formatting or syntax checking for the sections it can not handle. This has the drawback of requiring you to insert C-Clearly instructions in the form of specialized comments into your code. It may still not get some files to work. C-Clearly ignores the code that you have flagged and then expects the unflagged code to still be syntactically correct. Since C-Clearly has the bad habit of not liking certain code structures and flagging them as syntax errors, I have found it useful to monitor the program's return code from within a batch file. Although not documented in the users manual, C-Clearly returns 1 if it fails. You can access this in a batch file as the MS-DOS errorlevel. I use this technique to conditionally execute further processing on the source file after C-Clearly. I have not had success getting C-Clearly to consistently output tabs for indentation. Consequently, I process the file after C-Clearly to convert groups of leading spaces to tabs. The following is an MS-DOS batch file listing. CCL %1 OUTPUT.CCL IF ERRORLEVEL 1 GOTO :EXITERROR ENTAB OUTPUT.OUT 8 COPY OUTPUT.OUT %1 :EXITERROR ERASE OUTPUT.CCL I also add the comment, /* End of File */ to the end of every source file. Although C-Clearly can add comments to closing braces, it does not add comments (footers) to file output. You can add footers to hard-copy (printer-ready) output. I have an additional major complaint. Even with all of its flexibility, I cannot get C-Clearly to format code exactly the way I want. The problem is with long expressions that must be continued on multiple lines. With very long source lines, I like to find a convenient place to make a new line and continue the expression on the next line indented two levels from the indent level of the expression. Try as I might, I have not been able to get C-Clearly to do this exactly the way I want. I cannot even get it to leave continued lines alone. Since C-Clearly comes so close to giving you what you want and your expectations are high, this is especially annoying. The other complaints stem from the limitations of memory imposed by MS-DOS. I have started to run out of memory with long listings, especially code for Microsoft Windows that include WINDOWS.H. Since WINDOWS.H is more than 5,000 lines in length, this is not surprising. As file length goes up, C-Clearly gets significantly slower. A protected-mode version may remedy these problems. User Support -- Upgrade Policy I started using C-Clearly when it first came out about three years ago. I expected a lot from the product. I was not shy in calling V Communications and expressing my desires for C-Clearly to do something that it presently could not do or pointing out limitations and bugs in the product. V Communications responded with a closely-spaced series of upgrades. Many of these upgrades came free of charge. This experience and the ease of access to the technical people involved with the product impressed me. V Communications seemed committed to a quality product and committed to user support. C-Clearly has evolved to a state that works for me. I have found work arounds for most of the limitations and rarely contact the company anymore. The last upgrade cost me $45.00. Conclusions C-Clearly ambitiously sets out to be the ultimate C and C++ source code reformatting utility. It comes close. It fails in two critical areas. The first is that I cannot get C-Clearly to format code exactly the way I want it. The other serious flaw is that it will not format some perfectly good code. It stops when it thinks it has found a syntax error or runs low on memory. If you are willing to live with these limitations, C-Clearly is a powerful source code maintenance tool. I have found it useful and use it frequently. I have also found it frustrating and nearly always do additional processing to files after C-Clearly is done with them. When I have to read someone else's code that is in a style I am not comfortable with, C-Clearly has come to my rescue many times. C-Clearly is not a replacement for good coding style rules such as choice of variable names, consistency, and commenting, but it sure helps put to rest some of the code appearance arguments. Note from the editor: V Communications has released version 2.1 of C-Clearly which addresses some of the limitations outlined in this review. For specific differences please contact V Communications. C-Clearly, Version 2.0 V Communications, Inc. 4320 Stevens Creek Blvd., Suite 275 San Jose, CA 95129 Phone: 800-648-8266 Price: $129.95 Hardware Requirements: IBM PC Compatible, MS-DOS, and 512K RAM Listing 1 C-Clearly template file for K&R code for C code #include "lib.h" #define Macro( Param1, Param2 ) Param1 + Param2 static int i, (far *pfi)(); int v = 0; int ArrayName[XSize][YSize] = {{Init1, Init2}, {Init3}}; enum EnumName {Enum=1, Enum2}; struct StructName { int Field, Field; int Field, Width; }; void ANSIProtoFunc1 (); void ANSIProtoFunc2 (); static int ANSIFunction (char Param[], int *PtrParam) { long int Variable; int Variable; if (Expression) { Statement; } else if (Expression) { Statement; } else { Statement; } if (Expression) { int LocalVariable; int LocalVariable; Statement; Statement; } else if (Expression) { int LocalVariable; int LocalVariable; Statement; Statement; } else { int LocalVariable; int LocalVariable; Statement; Statement; } Array [Index].Field = (int *)PostOp++ * ++PreOp; Struct->Field = -UnaryOp * sizeof Variable; if ((Value = FunctionName ()) == sizeof (Type)) Statement; else if (Expression) Statement; else Statement; { Statement; } { int LocalVariable; int LocalVariable; Statement; Statement; } while (Expression) Statement; while (Expression) { Statement; } while (Expression) { int LocalVariable; int LocalVariable; Statement; Statement; } switch (Expression) { case Value: { Statement; break; } default: Statement; break; } return Expression; } void KnRFunction (Param1, Param2) int Param1; int Param2; { FunctionCall (Param, TestExpr ? ThenExpr : ElseExpr); goto Label; for (Expression; Expression; Expression, CommaExpression) Statement; for (Expression; Expression; Expression, CommaExpression) { Statement; } for (Expression; Expression; Expression, CommaExpression) { int LocalVariable; int LocalVariable; Statement; Statement; } do Statement; while (Expression); do { Statement; } while (Expression); do { int LocalVariable; int LocalVariable; Statement; Statement; } while (Expression); return (Expression); } /* End of File */ The Art of Programming Embedded Systems Mark Gingrich Mark Gingrich has been employed in the medical device industry for the past nine years. Presently he serves as a software engineer for Baxter Healthcare Corporation's Novacor division. He can be reached at 355 Estabrook St., Apt. 403, San Leandro, CA 94577. The series of books by Donald Knuth called The Art of Computer Programming may be the archetype of its genre. So I approach recent epics leading with "The Art of" title with elevated expectation. The Art of Programming Embedded Systems is among the latest (though unrelated to Knuth's trilogy). Its author, Jack Ganssle, checks in with good credentials: a contributing editor for Embedded Systems Programming magazine, a designer and purveyor of in-circuit emulators, and a veteran practitioner of said "art." But why a book on coding embedded systems? Perhaps because the topic is so woefully treated in the engineering/computer science curricula. More often it's a trade learned on the job--the hard way--at no small expense to our employers. And many of us drift into this sea having cast off on the purely hardware or purely software oceans. The luckiest benefit from a mentor helping to steer around obstacles: how to coerce the compiler to accept writeable variables distinct from RAM; how to debug optimized code sans print statements; how to configure the emulator to trap bugs which occur only during neap tides on Groundhog Day; how to do this; how not to do that. Don't have a mentor as such? Well, then, this book may be a reasonable alternative. Only don't expect a tutorial from square zero. Gannsle's approach is more casual--rather like talking shop with colleagues. He assumes that you've already served your software apprenticeship; now your goal is to fill those gaps of wisdom which have postponed your transition to true embedded systems programming enlightenment. The parallel path to this state of being entails time-consuming and costly mistakes (the euphemism is called "experience"). And experience is seldom acquired in each and every aspect of embedded design. For example, chief among my own gaps of wisdom is one in memory management techniques, having never employed bank switching on a project. Ganssle comes through in chapter six with a clear depiction of the camouflaged snake pits lurking in this area. Reading this chapter made it plainly apparent that I would have pathetically underestimated the time required to implement a bank-switching scheme. Likewise, a good introduction to real-time operating systems is found in chapter nine. Although not the be-all, end-all word on the subject, it's an appropriate diving-in point for the novice before swimming through the voluminous sales literature and spec sheets from the umpteen RTOS-to-go vendors. Of particular value is the small--but functional--real-time executive supplied in source listing form. Ever need a lone transcendental function in your system? Instead of calling the compiler's bloated, glacier-speed floating-point math library routine (which returns a result with three digits of precision more than you require), why not roll your own? Ganssle shows how--illustrated with C--in chapter seven. In addition, there are chapters on interrupt management; on signal smoothing and curve fitting (especially intriguing is the Savitsky and Golay technique); on software design which allows for civilized debugging; on designing to permit simplified production test--always guaranteed to endear you with the harried, under-appreciated manufacturing folk. And interspersed with the lucid, here's-the-way-it-is writing style are snippets of reality--flashbacks from Ganssle's eventful past: "It always seems that just before a demo everything falls apart. After a late night of removing the final bugs from microcontroller-based design, I unplugged the emulator and installed the computer chip. On power up the unit did nothing -- it was completely dead. Fortunately the code had a simple test routine that blinked an LED before any other initialization took place. Since the LED didn't blink, I knew immediately that the code was not starting and indeed found that a floating DMA request line was keeping the processor idle. The emulator's slightly different DC characteristics masked the problem during weeks of code development." Such anecdotal digressions in the prose are welcome. They add realism. And they underscore that the proffered advice is not rarefied academic theory; these are eyewitness war stories from the front. Occasionally, too, Ganssle opines on the softer issues of software development: programming style and professional improvement. And he confronts business issues so often avoided like the plague by the technical staff. This holistic approach is commendable. The still-too-pervasive image of "proglodytes" (wearing pocket protectors, of course) hacking away in the back room, oblivious to the rest of the world, has been a hindrance to our collective professional advancement. There is a bottom line, and Ganssle steps back to point out our role and responsibilities within the big picture. Reading widely is among our responsibilities, we're admonished. So Gannsle supplies an eclectic bibliography: from techy Intel application notes to Alvin Toffler's Powershift. (Though I would have preferred a more exhaustive reference section--pointers to the richest embedded systems lore. Indigenous software types, for instance, may need to "speak" electronic more proficiently; another "art of" book, The Art of Electronics, by Horowitz and Hill, is an appropriate text. Those of the hardware stripe would benefit from, say, Kernighan and Plauger's The Elements of Programming Style.) An appendix with recommended periodicals for the cognizant embedded programmer is also offered. (The C Users Journal makes the list; but somehow Dr. Dobb's Journal is omitted, a conspicuous oversight considering it is cited elsewhere in the book.) Be advised, however, that the "art" presented is not the state of the art. Embedded systems are described as they've existed over the past few years, with 4-, 8-, and 16-bit processors. There are no visits from the ghost of Embedded-Programming-Yet-To-Be. One must look elsewhere for coverage of fuzzy logic, neural nets, and DSP chips as embedded controllers. Mind you, I heartily recommend this book, but there are a few too many warts, most of which should have been removed with scrupulous copy editing. On page 152 the definitions of accuracy and precision are confused, as is the described behavior of the sine function, and the constant pi/2 is termed a "round" number. (Ironically, these blunders occur on a page with the subhead "Errors.") Elsewhere, the repeated misspelling of "kernel," the missing arrowhead in the state diagram in Figure 3.2, and the interchanged x-y coordinates in Figure 7.4 are annoying flaws. The state diagram in Figure 9.1 is simple, but it could have been drawn without the confusion-adding crossed lines. Then there are the source listings. Yes, there's enough source in this book (but, alas, no companion disk) to satisfy your minimum daily requirement of real code: much of it in C, a few in sundry assembly languages, and one listing in Basic. But the art of software book publishing demands faithful reproduction of listings. Just a cursory scan caught a missing right parenthesis in the for loop on page 31; the phex routine on page 95 lost a curly bracket somewhere; page 96 contains a commented-out source line, which is somewhat disconcerting. These typos along with the schizophrenic indentation style hint of manually-typeset code listings--a dangerous practice. My overall impression: Academic Press skimped on (or rushed) the proofreading and the illustrations. These are nitpicking complaints. I'm being a bit harsh because such a valuable work deserves better handling. And pricey books with lofty titles justifiably receive more intense scrutiny. But I'll apply a more pragmatic rule of thumb: If a book's cost and the invested reading time is more than compensated by the added quality and productivity of my work, or to the improved quality of my company's product, it's an unequivocable bargain. Without question, The Art of Programming Embedded Systems hits this critical breakpoint. Title: The Art of Programming Embedded Systems Author: Jack G. Ganssle Publisher: Academic Press Price: $49.00 ISBN: 0-12-274880-8 Editor's Forum I'm back on the subject of standards again. (See the Editor's Forum, CUJ November 1992 and January 1993.) The most interesting news is that the ISO C standards committee WG14 voted out an amendment to the C language at its last meeting, back in December. It includes a (much modified) set of alternate spellings for all those operators and punctuators that use funny characters not widely available. It also includes lots more functions for manipulating the large character sets used by the Japanese, Chinese, and several other cultures. Future editions of my column, "Standard C," will discuss the technical details in greater depth. They are of interest mostly to people who write for international markets. My experience is that more and more of you will fall into that category as time goes by. On the subject of time, however, don't feel too rushed. The amendment still faces at least two votes within ISO SC22, the parent committee. Don't look for an official standard for many months to come. The next most interesting news is that SC22 has finally given us clear guidance for both interpreting and patching the C Standard. WG14 has begun by picking up all the ANSI Requests for Interpretation. They should finally see the light of day as an ISO Record of Response. WG14 will continue to ask X3J11, the original authors of the ANSI C Standard, for assistance in forming responses. But we can now use a more streamlined ISO channel for closing the loop. My job as Convenor of WG14 effectively makes me caretaker for the Standard C programming language. Besides convening WG14 meetings on a regular basis, I am now the keeper of what SC22 calls the Defect Report Log -- the formal requests for interpretation of (or correction to) the C Standard. By an administrative quirk, I can also personally expedite the filing of Defect Reports. Please don't take this admission as an invitation to send in all your random queries about the C Standard. I reserve the right not to sponsor any Defect Report that I choose. But if your organization needs a technical clarification, sending the request straight to me just might lob a month or two off of going through ANSI or another ISO member body. I did make good on my threat to resign all my posts within ANSI. That saves me a lot of money and a bit of time attending meetings. It costs me the right to vote on how C and C++ evolve, but what the heck. I hope to use the extra money and time to get more book writing done in 1993. That assumes, of course, that you don't all deluge me with interpretation requests. P.J. Plauger pjp@plauger.com New Products Industry-Related News & Announcements ParcPlace Introduces VisualWorks ParcPlace Systems, Inc., has introduced VisualWorks, an application development environment (ADE) for corporate developers creating graphical, client/server applications that are portable across PC, Macintosh, and UNIX platforms (Sun, IBM, HP, DEC, Sequent). VisualWorks includes a GUI builder, database access capabilities, and a reusable application framework. The GUI builder provides a point-and-click palette and canvas, with layout tools that include a menu builder, an icon painter, and a color tool. ChamelionView, a component of the GUI builder illustrates portability of a new GUI across various front-ends, including Windows, Motif, OS/2 Presentation Manager, Macintosh, and OPENLOOK. ChameleonView allows developers to preview the new interface in the native application look of these platforms. VisualWorks provides direct access to Oracle and Sybase databases. Through Information Builders, Inc.'s EDA/SQL gateway, more than 48 different databases can be accessed. VisualWorks is priced at $2,995 for Windows, OS/2, and Macintosh, and at $4,995 for UNIX. Database drivers cost $495 for Oracle or Sybase and $995 for EDA/SQL. Contact ParcPlace Systems, 999 E. Arques Avenue, Sunnyvale, CA 94086, (408) 481-9090; FAX: (408) 481-9095. Xionics Announces PowerTools and ImageSoft 2.0 for Image Processing Xionics, a developer of image acceleration technology, has announced PowerTools application programming interface for MS-Windows image processing, and ImageSoft 2.0, an update of their C library for image applications software. PowerTools consists of a Windows Dynamic Link Library (DLL) with ten high-level commands to control the aspects of monochrome document image processing. The PowerTools commands each operate by issuing a string of calls to several of the underlying C routines in Xionics ImageSoft Libraries. PowerTools Release 1.0 is available as object code, and is compatible with standard C compilers, such as Microsoft C and Turbo C. ImageSoft 2.0 encompasses over 80 routines for image processing: scanning, printing, compression/decompression, display, and enhancement. Xionics maintains a policy of "assuring complete backward compatibility" in its API. PowerTools is priced at $895, royalty-free. The price includes one year of technical support and software updates. Contact Xionics Inc., Two Corporation Way, Peabody, MA 01960, (508) 531-6666; FAX: (508) 531-6669. Micro Digital Introduces smx++ Micro Digital has announced smx++, an Application Program Interface (API) which allows C++ programmers to access smx (simple multitasking executive) multi-tasking features in object form. smx++ supports Borland C++ v3.1 and Microsoft C++ v7.0. smx++ is a C++ class library consisting of nine base classes and seven derived classes. Redundant and seldom-used smx functionality has been omitted and orthogonality has been improved, in order to create a simpler API. A shallow class hierarchy was designed to preserve performance. smx++ runs on top of smx and direct smx C function calls can still be performed, and smxProbe still works the same. The new design made all smx objects (e.g., tasks, messages, semaphores, etc.) fully dynamic--deletable as well as createable. C++ developers can derive classes from the smx++ classes. For example, device drivers can be derived from the Bucket and Pipe classes, (which have been derived from the Software I/O Bus (SIObus) class). smx++ is priced at $2995 (including a royalty-free license), with source available for $1000, and smxProbe for $500. Contact Micro Digital, Inc., 6402 Tulagi Street, Cypress, CA 90630-5630, (800) 366-2491 or (714) 373-6862; FAX: (714) 891-2363. Integrated Development Announces LibTools Integrated Development Corp. has announced LibTools, a set of programmer's tools for creating, managing, and exploring libraries of C, C++, Assembly, Xbase, and other Intel-, Microsoft-, and Borland-compatible object modules. LibTools provides extensive reporting and cross-referencing capabilities. LibTools can resolve public symbol conflicts, forecast overly-large executables, and verify module integrity. LibTools can show a complete list of the public and external references in a module, showing what will be linked in when you call a particular function. LibTools' LibComp utility can compare two libraries and produce a list of duplicate symbols, and LibTools can rename external and public symbols to resolve conflicts. LibTools also includes a Library Dump utility, which provides a detailed listing of the complete contents of a library. LibTools documentation includes tutorials, an introduction to library management, and tips on designing more granular libaries. Contact Integrated Development Corp., 190 Main Street, P.O. Box 592, Hampstead, NH 03841, (603) 329-5522 or (800) 333-3429; FAX: (603) 329-4842; CIS: 700441,2465. Archimedes Adds C Tools for Hitachi Microcontrollers Archimedes Software, Inc. has introduced C cross compilers and debuggers/simulators for the Hitachi H8/300 and H8/500 microcontroller families. The Archimedes C-Cross Compilers for the microcontrollers follow the ANSI C standard and support all the required libraries. The compilers provide several memory models (six of the H8/300 and 11 for the H8/500). Pre-defined in-line functions support interrupt handling. The compiler provides both single and double precision IEEE floating-point library functions. The compilers generate relocatable code and the Archimedes linker generates any of 32 different output formats for different emulators and PROM-programmers. The family of C-SPY High-Level Language Debuggers/Simulators for embedded applications support the Archimedes C-Cross compilers. C-SPY is avialble in simulator and emulator driver versions. The emulator versions support the MIME-700 in-circuit emulator from Pentica Systems, Inc. The H8/300 and H8/500 Cross Compilers are hosted on PCs, HP 9000, Sun SPARCs, and DEC MicroVax and Vax platforms. The compilers are priced starting at $1295 for H8/300 on a PC. C-SPY is supported on PC compatibles, and priced at $1195 for the H8/300 and $1995 for the H8/500. Contact Archimedes Software, Inc., 2159 Union Street, San Francisco, CA 94123, (415) 567-4010; FAX: (415) 567-1318. Nu-Mega Announces BOUNDS-CHECKER 2.0, for MS-DOS Memory Protection Nu-Mega Technologies, Inc., has announced BOUNDS-CHECKER 2.0, an MS-DOS memory protection tool that provides real-time memory and heap protection. BOUNDS-CHECKER 2.0 can detect problems in a program's heap, stack, or data segment; handles array over-run detection; finds illegal memory accesses outside a program; and finds code overwrites automatically. BOUNDS-CHECKER adds a Smart Mode feature, with built-in heuristics to determine the legitimacy of a memory access, and avoid unnecessarily flagging legitimate out-of-bounds accesses (e.g., video memory, BIOS variables, etc.). BOUNDS-CHECKER 2.0 doesn't require anything to be linked-in or compiled, but can work directly with both Microsoft C 7.0 and Borland 3.1 with the VROOM overlay, as well as support memory managers such as QEMM. Contact Nu-Mega Technologies, Inc., P.O. Box 7780, Nashua, NH 03060-7780, (603) 889-2386; FAX: (603) 889-1135. StratosWare Releases MemCheck for the Macintosh StratosWare has introduced versions of its error detection and prevention product, MemCheck for the Macintosh, for the Think C and MPW C environments. MemCheck requires no source code changes. MemCheck detects memory overwrites and underwrites, memory leaks, heap corruption, and other memory errors. MemCheck operates transparently, appearing only to report errors with source file and line information. MemCheck for the Macintosh detects failure of memory allocation routines, failure of many resource operations, invalid operations on unlocked or purged handles, and inappropriate use of non-resource handles. One include file per source module is required to configure projects, and an automated configuration tool is included, MemCheck can be switched on or off at runtime, linked out via the production library, or compiled out with no source code changes. Contact StratosWare Corporation, 1756 Plymouth Road, Suite 1500, Ann Arbor, MI48105, (313) 996-2944 or (800) 933-3284; FAX: (313) 747-8519. Scientific Endeavors Announces GraphiC/Win Windows Graphics Library Scientific Endeavors has announced GraphiC/Win, a version of its C graphics library for Windows. The features of the MS-DOS version of GraphiC have been provided under Windows, allowing scientists to create graphics for publication. GraphiC/Win creates and manages its own window and resources, and adds features to take advantage of the Windows environment. GraphiC/Win will create Windows metafiles and bitmaps and copy both to the Windows clipboard. GraphiC/Win uses Windows video and printer drivers. Graphics are stored using the high-resolution Tektronix 4105 format; graphics can be exported in Postscript, GEM, Lotus PIC, HPGL, HPGL/2, and TIFF formats. GraphiC's routines come as source code. GraphiC/Win is priced at $495. Contact Scientific Endeavors Corporation, 508 North Kentucky Street, Kingston, TN 37763, (615) 376-4146 or (800) 998-1571; FAX: (615) 376-1571. Instrumentation Software Adds Stand-Alone Libraries National Instruments has announced LabWindows for MS-DOS Version 2.2, an instrumentation software package (for instrument control, data acquisition, analysis, and presentation) which now includes stand-alone libraries for the Borland C++ and Turbo C++ compilers and the Microsoft Visual Basic for DOS (VBDOS) compiler. Users can access the Borland compiler and linker from within the LabWindows programming environment, or they can add the LabWindows libraries to Borland's development environment. The LabWindows instrument driver library includes over 260 instrument drivers. Version 2.2 includes a float data type DSP Analysis Library, new cursor functions, DPMI memory manager, and utilties for MS-DOS file and directory commands from within LabWindows. LabWindows can control GPIB, VXI, and RS-232 instruments, and plug-in data acquisition cards. Contact National Instruments, 6504 Bridge Point Parkway, Austin, TX 78730-5039, (512) 794-0100 or (800) 433-3488; FAX: (512) 794-8411. StatSci Introduces S+INTERFACE Application Building Toolkit Statistical Sciences, Inc. (StatSci), has introduced S+INTERFACE, a toolkit designed to assist users of the S-PLUS data analysis software on UNIX workstations. S+INTERFACE can create custom menu interfaces to S-PLUS, and provides access for separate C applications to the over 1000 data analysis functions of S-PLUS. Under X11-based systems, S+INTERFACE provides a macro language that can be used to create a Motif-style user interface. S+INTERFACE provides access to S-PLUS functions from another application as if S-PLUS were a subroutine library. S-PLUS is initiated as a separate process and C functions calls are used for communication. Contact Statistical Sciences, Inc., 1700 Westlake Ave. N, Suite 500, Seattle, WA 98109, (206) 283-8802 or (800) 569-0123; FAX: (206) 283-8691; E-mail: mktg@statsci.com. RTIS Announces Distributed Application Builder The Real-Time Intelligent Systems (RTIS) Corporation has announced their DAB Distributed Application Builder Software. DAB provides network-wide data exchange between computer programs running on PC compatibles connected by LANs and serial links. An introductory DAB Kit includes C libraries compatible with Microsoft or Borland compilers, a users manual, and identification keys for two computers. DAB supports NetBIOS compatible LANs (e.g., Novel and Lantastic). DAB creates a multiprocessing environment, with MS-DOS running in the foreground and communications software running in the background. Contact The Real-Time Intelligent Systems Corporation, 30 Sever Street, Worcester, MA 01609, (508) 752-5567; FAX: (508) 752-5491. WCSC Releases COMM-DRV Version 12.0 WCSC has released COMM-DRV Version 12.0, their serial communication development libraries and tools. COMM-DRV v12.0 supports Windows 3.x, MS-DOS, and DESQview. COMM-DRV supports dumb multiport cards, the ARNET SMARTPORT PLUS cards, and the Digiboard COMXi cards. COMM-DRV can be linked directly into MS-DOS or Windows applications, or it can be used as a DLL for Windows. COMM-DRV is priced at $189.95. Contact WCSC, 2470 S. Dairy Ashford, Suite 188, Houston, TX 77077, (800) 966-4832 or (713) 498-4832; FAX: (713) 568-3334. EMS Adds Products to Library of PD/Shareware C Utilities EMS Professional Software has added 70 new products to its Library of PD/Shareware C Utilities. The library includes 787 products for C/C++ programmers. A database accompanying the library indexes the products and includes descriptions. Searches by type, name, vendor, or free text are supported. The library is available on disk or CD-ROM. Contact EMS, 4505 Buckhurst Court, Olney, MD 20832, (301) 924-3594; FAX: (301) 963-2708. Shamus Software Announces Version 3.2 of its C Arithmetic Library Shamus Software Ltd. has announced version 3.2 of its MIRACL product, a Multi-precision Integer and Rational Arithmetic C library. New features include extensions conditionally-compiled in-line assembly. The libraries are of use primarily for implementing cryptography systems. Source is included. MIRACL is portable and supported platforms include: PC compatibles, Macintosh, Acorn, Sun, and VAX. Contact Shamus Software Ltd., 94 Shangan Road, Ballymun, Dublin 9, Ireland, Tel: 8425430. Liant Cuts Price and Boosts Execution Speed of LPI-C Compiler Liant Software has improved the execution speed of its LPI-C compiler, while cutting the price. LPI-C is bundled with CodeWatch, Liant's X/Motif source-level debugger. Liant reports that its LPI-C compiler v2.0 generates code that runs 50 percent faster than previous versions, achieving a 43,000 rating on the Dhrystone 2.1 benchmark on a 33MHz, i486 system. Liant also reduced the suggested list price of LPI-C from $895 to $595. Liant LPI-C v2.0 is a C compiler for UNIX applications on i386/i486 and Sun SPARC platforms. LPI-C is ANSI C compliant, and has passed the FIPS-160 ANSI/ISC C Validation-Suite (NIST-certified) and the Plum Hall ANSI C Validation Suite. Version 2 incorporates a new optimizer and an improved code generator. The optimizer provides global optimization across the entire compilation unit, loop unrolling, and function in-lining. Users can select either standard UNIX system header files or the LPI-C ANSI runtime library to ensure portability across operating systems and architectures for strictly conforming ANSI applications. Version 2.0 also compiles pre-ANSI sources including PC-based code. LPI-C also includes a windowed debugger, CodeWatch. Contact Liant SOftware Corporation, (508) 872-8700. ImageSoft Updates CommonView for OS/2 ImageSoft has introducted the OS/2 2.0 version of CommonView, their application framework of C++ classes for developing GUI-based applications. CommonView for OS/2 v2.0 supports 32-bit applications. ImageSoft has also announced an agreement with J Systems, Inc., to publish Object/Designer, an extensible C++, C, and Pascal application generator for Windows. Contact ImageSoft Incorporated, 2 Haven Avenue, Port Washington, NY 11050, (516) 767-2233; FAX: (516) 767-9067. Dyad Software Ships M++ Version 4.0 Dyad Software has begun shipping verion 4.0 of their M++ math library. The new version adds a set of spectral operations, a BitArray class, a PointerArray class, huge memory pointers for MS-DOS users, and a set of assembly language Basic Linear Algebra routines (BLAs). The spectral methods allow FFTs on vectors or arrays of vectors as well as multidimensional FFTs (up to four dimensions). M++ is available for MS-DOS C++ compilers (Borland, Microsoft, Zortech, and MetaWare) for $495, and for UNIX, WIndows NT, and OS/2 compilers for $695. Contact Dyad Software, Bellevue, WA, (206) 637-9426. Electronic Imagery Enhances Image Processing Software Electronic Imagery (EI) has announced enhancements to its image processing software. ImageScale Plus, ImageScale Plus for UNIX, and ImageScale Plus Developer's Toolkit for MS-DOS and UNIX applications, now include the JPEG compression/decompression algorithm. EI has released ImageManager, a file manager for pictorial images, documents, and associated text files in an SQL database, MSWindows environmennt. Another new release, ImageCount, supports image counting and object recognition that can define, count, number, and measure objects in imaging files and video frames. Contact Electronic Imagery, Inc., 1100 Park Central Boulevard South, Suite 3400, Pompano Beach, FL 33064, (305) 968-7100; FAX: (305) 968-7319. Liant Upgrades C-scape User Interface Management System Liant has announced a major upgrade (version 4.0) of its C-scape User Interface Management System, an object-oriented C development tool for creating portable text and GUI applications. Liant describes the most significant enhancement to C-scape as its greatly improved look and feel, with support for CUA (Common User Access) style boarders for both text and graphics mode, scroll bars, minimize/maximize buttons, menus, and other windowing functions. Contact Liant Software Corporation, Framingham, MA, (508) 872-8700. Data Entry Workshop Supports Interactive Design of Validated Entry Screens TurboPower Software has announced Data Entry Workshop, a collection of tools for writing validated data entry screens and other Windows controls. Data Entry Workshop builds controls in a three-step process: first, use Resource Workshop to place and edit the controls interactively; second, run the MAKESRC utility to generate the source code (C++ or Pascal); finally, use Borland's ObjectWindows Library to access the controls. Data Entry Workshop provides the following controls: Simple Entry Field, Numeric Entry Field, DEW Shade Control, Toolbox Control, Picture Entry Field, Spin Control, Meter Control, and Toolbar Control. Data Entry Workshop is designed for use with Borland C++, Borland Turbo Pascal for Windows, or Borland Turbo C++ for Windows. Data Entry Workshop includes full source code, comprehensive documentation, pop-up help, and example programs. Data Entry Workshop costs $189, and no royalty payments are required. Contact TurboPower Software, P.O. Box 49009, Colorado Springs, CO 80949-9009, (415) 322-3417. SET Laboratories Announces PC-METRIC 4.0 for C Set Laboratories, Inc., has announced version 4.0 of its software measurement and analysis package, PC-METRIC for C. Additions in version 4.0 include new measures of control flow complexity, iEEE standard size counting measures, and a completely revamped interactive query and analysis system for tracking metrics across releases. Contact SET Laboratories, Inc., P.O. Box 868, Mulino, OR 97042, (503) 829-7123; FAX: (503) 829-7220. Eighteen Eight Laboratories Introduces Three Interface Cards for PL2500 Eighteen Eight Laboratories has announced three new interface cards for the PL2500 family of AT-hosted Floating Point Array Processors. The new cards use the PL2500's SPAN32 bus to transfer data at up to 15 million bytes per second. The PL2500 on-board rountines are callable from C (also Fortran and Pascal) control programs. Contact Eighteen Eight Laboratories, 1247 Tamarisk Lane, Boulder City, NV 89005, (702) 294-5009 or (800) 888-1119; FAX: (702) 294-2611. Aggregate Releases GNU make Compatible NetMake Aggregate Computing, Inc., has released NetMake 1.2, a distributed, parallel version of the UNIX make utility. NetMake can use multiple systems across a network of Sun workstations and servers in parallel, to handle Sun, BSD, or GNU makefiles. Contact Aggregate Computing, Inc., 300 South Highway 169, Suite 400, Minneapolis, MN 55426, (612) 546-5579; FAX: (612) 546-9485. SunPro Announces Object Technology Agreement with Rouge Wave Software SunProg, the software development business of Sun Microsystems, Inc., has entered a technology development and licensing agreement with Rogue Wave Software, Inc., a supplier of C++ class library technology. The agreement centers on Rogue Wave's Tools.h++ class library, a toolbox of nearly 100 C++ classes. Contact SunPro, 2550 Garcia Avenue, Mountain View, CA 94043-1100, (415) 960-1300; FAX: (415) 969-9131. Genus Plans for GIF Toolkit and Printer Toolkit Genus Microprogramming has announced plans for a December release of GIF Toolkit and GX Printer, a printer toolkit. The GIF Toolkit provides over 100 routines for incorporating GIF images into applications, and conforms to the GIF89a specifications. The GX Printer toolkit supoprt grtaphics printing from the Genus PCX or GIF toolkits, the display, or any GX virtual buffer. Contact Genus Microprogramming, 1155 Dairy Ashford, Suite 200, Houston, TX 77079, (800) 227-0918 or (713) 870-0737. We Have Mail Mr. Plauger, I am a subscriber of The C Users Journal and I have been enjoying it from I started my subscription. In particular I want to ask you something. I am working in a project that requires sophisticated and unusual macros. Working on them I realized that the following code: { int i=1; (1, i)++; printf("Value of i is %d\n", i); } will print: Value of i is 2 This makes sense to me. But I have tried to compiled with four different compilers: Sun ANSI C Compiler complains saying that the result of 1 is not a LHS value. It is certainly not. GNU C Compiler compiles and executes correctly. Sun C++ Compiler compiles and runs correctly Because I am getting two different results with two compiler that are supposed to be ANSI compliant, which one is the correct one? Is the construction valid? Thank you very much for your help. And sorry to bother you with this kind of simple questions. Daniel M. German dmg@cs.wm.edu Believe it or not, all compilers are behaving properly. The C Standard says the result of a comma operator is an rvalue, and the ++ operator is defined only for modifiable lvalue operands. Applying ++ to an rvalue is thus undefined behavior, which leaves implementations free to do as they choose. The strictest approach is to issue a diagnostic, as the Sun ANSI compiler does. But a common extension is to find some lvalue behind the rvalue and apply ++ to the lvalue. That's what the other compilers seem to be doing. For what it's worth, I was the one who screamed loudest that the comma operator (and a few others) be rvalues. Hope this helps. -- pjp Dear Mr. Plauger, In the July 1991 issue of CUJ, Jonathan Walker III had a lovely article in which he presented an algorithm for positioning a generalized tree. Last spring and summer, a student and I wrote a couple of programs utilizing this algorithm. These programs take bracketted output from syntactic and morphological parsers and display visual representations (trees) in an X-window or a large curses pad (and use the vi keys, hjkl, to navigate through various parts of the tree). The X-window version has been tested on Sun3s, Sun4s, and MIPS machines, and the curses version on Suns, PCs running XENIX, and PCs running MS-DOS. There is also a primitive version using Borland's bgi for MS-DOS. We would like to make these programs available for use by linguists by putting them in an ftp site. Last summer I wrote to Mr. Walker at the address given in his article to ask for permission to do this (his code makes up about 25% of each program). I haven't received a reply from him, and I'm writing to you to ask first if CUJ has a more recent address for him), and secondly if, failing that, it would be possible for CUJ to give up permission to use the code in these small, public-domain applications. On another topic, I've been a subscriber of CUJ since I found out about it a few years ago, and have enjoyed it immensely as well as learned a great deal from it. I particularly enjoy your articles on Standard C. (At my age, 50, I give myself the luxury, however of sticking to K&R C (the first edition). I'm professionally a linguist, anyway, and not actively involved in training programmers. I've noticed that the computer science students I work with all code in Standard C. With many thanks for your help, Chet Creider Diane Thomas, CUJ Managing Editor responds: All CUJ code is now posted on USENET. Please check the notice about online source code located in the table of contents for details. Dear P.J. Plauger, My copy of the Journal arrived yesterday, and I was pleased to see the usual broad range of articles. I found your column of particular interest, as I have recently reviewed some of the coding errors that I make. I hope that the Journal will not drift too far from C to C++, as I consider C++ to be a very different language, requiring different design considerations and often used for very different projects. Perhaps you could consider a little coverage of Objective C, which appears to me to be a much more helpful framework for OOP. The article "Time Complexity" by Wilbon Davies (page 31) displays a misunderstanding of the bubble sort. The code shown is incomplete, exaggerating the time taken: swap = 1; while (swap == 1) { swap = 0; for (i = 0; i < n - 1; i++) { if (x[i] < x[i + 1]) { tmp = x[i]; x[i] = x[i + 1]; x[i + 1] = tmp; swap = 1; } } } A practical bubble sort expands this simply, collapsing the scope of the sort faster. Where the data is already sorted, only (n - 1) compares are performed, partially and randomly sorted data require progressively more work. The worst case is for reverse sorted input, where no improvement is made over the published form. Keeping the format above: last = n; while (last > 0) { limit = last - 1 last = 0; for (i = 0; i < limit; i++) { if (x[i] < x[i + 1]) { tmp = x[i]; x[i] = x[i + 1]; x[i + 1] = tmp; last = i; } } } A number of improvements can be made to this algorthim to give acceptable execution time, and reduce worst-case behaviour. In particular alternating between sorting forwards and backwards, this collapses the scope of the sort still faster. It also has the benefit of moving worst-case execution from a reverse sorted input, to an esoteric order to require the maximum exchanges. In several applications I have had the task of sorting large structures, rather than pointers, with the primary constraint on use of memory. I have found that an optimised version of the Bubble Sort has given very acceptable results. This incorrect version often features in articles comparing sort methods. The problem seems to be that the authors find it in several refernce books. In these books it is introduced briefly, only for a paragraph discussing its worst case operation. I would very much appreciate if articles in your Journal refering to "fastest bubble time," or similar, actually use the practical version, rather than propagate the brain-dead one on this occasion. I am happy to discuss this further, and you may edit the letter if you wish to publish it in the journal. Yours sincerely, Anthony Naggs Software/Electronics Consultant) Email: amn@vms.brighton.ac.uk P O Box 1080, or xa329@city.ac.uk Peacehaven, East Sussex BN10 8BT Phone: +44 273 589701 Great Britain When Kernighan and I wrote The Elements of Programming Style, we discovered that "improving" a bubble sort often made it run slower rather than faster. Naturally, you can favor certain patterns of input to advantage. But the extra baggage you add generally slows average behavior for random input. A "brain dead" bubble sort is quite fast enough for sorting small quantities of data. For sorting large quantities, you're better off switching to a better algorithm than gilding this particular lily. Optimizations can pay off in the middle region, if you can determine what that is. -- pjp Dear P.J.: Is there an e-mail address for the C Users Journal? I didn't receive the September '92 issue and, being a hard-up student, want to avoid the cost of an international phone call to sort it out. By the way, I would also like to commend you on the thoughtfulness and intelligence of your writing, I appreciate it very much. Thanks, Jane Anna Langley 105 Osborne Street South Yarra University of Melbourne Victoria 3141 AUSTRALIA (613) 03 820 3629 s342046@emu.insted.unimelb.edu.au Diane Thomas, CUJ Managing Editor, responds: All questions regarding subscriptions or any other customer relations topic can be sent to cujsub@rdpub.com Just to start off, I find the magazine that you publish is the best that I have come across. To better clarify this, I appreciated the points you brought up in the Editor's Forum about product reviews and how the magazine stands on the issue. After reading that, I examined the editorial box to the right of the article and I believe I have found a mistake. Under trademarks, OS/2 is listed under the competitor's company, Microsoft. In the article, "Debugging with Assertions" on page 42, there is an error in the code in listing 2: void CheckEmpty() { assert(StackPtr = 1) } It should be as follows: void CheckEmpty() { assert(StackPtr == 1) } Keep up the good work! Sincerely Eric V. Blood ericb@sierra.com Thanks.--pjp Dear Mr. Pugh, To gain control of the specific area of the screen scrolled when printf does a line feed from the last row, I installed my own video interrupt service routine (ISR) for int10h. My ISR chains to the original ISR after testing AH for function code 6 (scroll up). If (and only if) AH==6, my ISR replaces the value in CH with a row number passed from the calling program before chaining to the original ISR. This gives the calling program a method of preventing data above a given row from scrolling off the screen, as long as scrolling is done by int10h calls. (Some compilers implement cprintf, cputs, and putch of so they can scroll up without using int10h.) My test program reports the vector of the original video ISR and the vector of the my replacement ISR. A test version of my ISR displays a signature on row 14 of the screen whenever AH==6. The signature consists of the char equivalents of CH (0), CL (0), DH (24), DL (79), AH (6), AL (1), and the value to be put into CH (from the calling program. CH,CL is used by the original ISR as the upper-left row,col of the window to be scrolled, while DH,DL is used as its lower-right row,col. AL is the number of lines to be scrolled. On a PC Designs XT with Hercules video under DOS 3.1, the test program and my ISR work as expected. On an AST Premium with VGA under DOS 3.3, my ISR is not called at all, as indicated by the absence of the diagnostic signature, even though the vectors reported show that my ISR was properly installed. The same failure was noted on a Packard-Bell with EGA under DOS 3.3. Suspecting that EGA/VGA video subsystems bypass int10h for all scrolling functions, I studied PC & PS/2 Video Systems (Richard Wilton, Microsoft Press, 1987), Programmer's Guide to the IBM PC (Peter Norton, Microsoft Press, 1985), and back issues of C Users Journal and Dr. Dobb's Journal. I found nothing that either confirmed or dispelled that notion. Can you offer any suggestions that may lead to an answer to this puzzle? Thank you, Sid Sanders 5 Seneca Avenue Geneseo, NY 14454-9508 Sure looks like a bypass to me, but I'm hardly an expert in this area. Anybody out there got any ideas? I suggest you contact Mr. Sanders directly for speed. Send us a copy of your letter if you think the lore is worth sharing. -- pjp Re your editorial in the October 1992 C Users Journal: I bought a copy of the Shamus Software library MIRACL to fiddle with p. It has many mathematics routines and some top coding and decoding stuff. So I ported it to Coherent 310 and now to Coherent 401 which is the 32-bit system. I could compute p to 500 places in the 16-bit system and it does much more in the 32-bit version. I can get 50,000 places, but it takes 57 hours! It yields a 12 page printout and I am not sure of its accuracy at this time. I would like a copy of the "Spigot" algorithm originally due to Stanley Rabinowitz which really works. I got one in an article by Peter Morrison on comp.lang.c which produces rubbish only. Also the Coherent 401 produces LOTTO programs with one million games in them. Of course it takes up 39 Mbytes but I use grep to see how many winning games are in the file. I tried porting the MIX Multi-C library to Coherent but it is no good. There may be a bug in the way coherent uses the typedef enum. It has a definition of ECODE in the file mtc_defs.h. This reads typedef enum { 13 names of errors.. } ECODE. Then it has the line: typedef ECODE (*TRAVERSE_FUNC) (void*, void*, int, int, void*); This is not acceptable to the compiler, which produces this message: 58: mtc_defs.h: missing ')' 58: mtc_defs.h: missing semicolon 58: mtc_defs.h: declarator syntax 58: mtc_defs.h: external syntax Have you got any ideas on what I can do with this? Jonathan Kitchin Perth, Western Australia jon@dialix.oz.au Once a compiler produces a diagnostic, it often gets confused for awhile. Try omitting the earlier statement (or fixing it if you know how). There's a good chance the subsequent diagnostics will evaporate. -- pjp Dear Sir, Some time ago, I purchased your book, "The Standard C Library," together with the code disk. I just thought I would pass on that it has been one of the better investments I have made in computer science books. I find it very instructive and a good source of ideas and algorithms. (I recently had to implement malloc on an embedded system with no operating system! Studying how you did it gave me a good kick start.) I read your "Standard C" column on bugs (CUJ September 1992) with great interest. I wish there were more articles about people's bugs and problems. I think we could learn more from them than from some of the "how to do it" articles. No doubt you have received other letters on this, but there seems to be a bug in your bug fix. [Bug report omitted. Same as earlier reports -- pjp] I hope that this bug did not creep into the v1.1 code disk. Speaking of the v1.1 code disk. You mentioned in the column that it now available, but I couldn't see anything in CUJ about how to order it. Do you need some "proof of ownership" for the upgrade? I can't see a serial number on the disks. Or do you rely on your purchase records? [MasterCard number omitted, for obvious reasons. -- pjp] Yours faithfully, Ian Cargill 54 Windfield Leatherhead Surrey KT22 8UQ A Portable User Interface Using curses Matt Weisfeld Matt Weisfeld is currently employed by the Allen-Bradley Company in Highland Heights, Ohio. He responsible for the design and development of test software on VAX/VMS, UNIX, DOS, and other platforms. Matt is currently working on a book entitled Building and Testing Portable Libraries in C, to be published by QED in 1993. He can be reached on Compuserve at [71620,2171]. The topics of portability and user interfaces are usually not mentioned in the same conversation, because the hardware-dependent nature of terminal devices makes user interfaces notoriously difficult to port. However, you don't necessarily need to write a separate user interface for each platform, even when you need maximum portability. This article presents a library of routines that allows a programmer to create a portable, text-based user interface using curses. curses, the screen-handling package available as part of the C package on VMS and most flavors of UNIX or as shareware, allows you to update screens efficiently. curscr keeps an image of the current screen. You change this image by changing the standard screen, stdscr, or by creating a new screen. refresh or wrefresh change curscr to match stdscr. The User Interface The example user interface discussed here can be ported to VAX/VMS, MS-DOS/BORLAND C, Hewlett-Packard/HPUX, and SUN/GNU with revision. The display consists of a main window, a menubar, and a dialog box (see Figure 1). The main window occupies the entire screen and is the backdrop for all other constructs. At the top of the main window, a single-line menubar presents the user with the available program options. The user chooses an option either by entering the first letter of the option or by using the arrow and return keys. Choosing one of these options will activate a pulldown menu containing further options. The dialog box, used to print informational messages and accept additional user input, resides at the bottom of the main window. Library functions to handle these user interface constructs, and other specific tasks, simplify the process of building screen applications. The separate library files can be linked into specific user applications. There are three major reasons for using curses: portability, availability, and usability. Even if curses is inappropriate for a specific application, you can apply the methods presented here for creating a menubar and pulldown menus to any user interface. The libraries can be treated as shells, with the curses commands replaced by other user interface commands. The appropriate #ifdefs make the libraries portable to multiple platforms. Primary curses Screen Structures Listing 1 contains a simple curses application. Just as C defines stdout, stdin, and stderr for input and output, curses keeps a memory representation of the screen in stdscr. All window operations affect only this memory representation. To change the screen itself, kept in curscr, you execute refresh or wrefresh--even when deleting a window. WINDOW, a data structure in curses.h required by all curses applications, contains information such as window location and size. Each window created must correspond to a pointer of this structure type. All curses programs create stdscr by default. Other curses constructs must be explicitly created. In most cases, curses treats stdscr differently from other windows. For example, to clear a window, curses performs the wclear(win) command, but to clear stdscr, curses uses the clear command, with no parameters. Creating a Popup Window Since most operations for any user interface involve windows, I built a library function called popup to create a popup window. (Listing 3 contains popup and all other library code presented in this article.) To create a window that entirely covers stdscr, I call popup with MAX_ROWS and MAX_COLUMNS. WINDOW *mainwin; mainwin = popup (MAX_ROWS, MAX_COLUMNS, 0,0); The constants MAX_ROWS and MAX_COLUMNS represent the standard screen size. The header file menu.h (Listing 2) defines all the constants and structures for this user interface. There are three popup windows in this application: the menubar, the dialog box, and a window used for pulldown menus. These are global to all functions and thus are declared as extern in most of the files. Color presents a special problem when writing a portable routine for creating a popup window. Since PC curses has color capabilities, whereas VMS and UNIX do not, ifdefs are used to take advantage of this feature. The PC curses command wattrset controls color. The colors representing the foreground and background are ORed together with: wattrset(mainwin, F_RED B_BLACK); PC curs es also has many more box characters to choose from. Many different effects can be obtained by using colors and box characters on the PC. However, if you desire a simple border around a window similar to VMS and UNIX, simply set the background to black and execute the box command on all platforms. Special Keyboard Input Accepting Single Keystrokes The example application uses the arrow keys. This causes a portability problem when creating a windowed curses application. wgetch gets characters from a window. However, curses buffers this input and echoes it to the screen. To prevent line buffering and echo, making the program accept one keystroke at a time, you must call crmode to set the cbreak mode and noecho to unset the echo mode. Using the arrow keys (from the keypad) on MS-DOS is very straightforward. Either the getch or the wgetch commands will return the necessary key code. (For all codes see Listing 2.) Obtaining non-printable characters with VMS requires the use of low-level Screen Management (SMG) commands. (See the VMS SMG manual for a complete description.) The two SMG commands needed here are CREATE_VIRTUAL_KEYBOARD, which activates the program for keyboard input, and READ_KEYSTROKE, which returns a keystroke. (VMS returns a short.) Interpreting Escape Sequences On UNIX systems, use of the arrow keys, escape key, and return key presents another problem. Both the HP and SUN systems return keystrokes from the keypad with escape sequences. Some systems, such as the HP, include a function called keypad. This function activates the keypad and returns the proper key code directly, saving the programmer from having to interpret the escape sequences. When the keypad function is not available, the method for dealing with the escape sequences depends on the system. For example, on the SUN system, entering a keypad character will return an escape sequence in three parts. When the first getch is recognized as an escape, two more getch commands must be called in succession. The third character contains the code needed. The character codes used in these libraries are defined in the file menu.h as: UP_ARROW, DOWN_ARROW, LEFT_ARROW, RIGHT_ARROW, ESCAPE, and RETURN. Creating a Menubar In all environments, the same structure defines a menubar: typedef struct mbar { char string[80]; char letter; int pos; } MENUBAR; The first field holds the actual string that represents the particular option. For example, if one of the options across the top relates to printing, then the string print is a logical choice. The second field is the letter that invokes the option from the keyboard. The final field, called pos, holds the location of the string within the menubar. This example uses the code #define TCHOICES 3 MENUBAR menubar[TCHOICES] = { "file", 'f', O, "edit", 'e', O, "options", 'o', O, }; to create a menubar (see Listing 4). This declaration creates a menubar with three different options, file, edit, and options, invoked by entering an f, e, or o respectively. The positions are initially set to zero, and calculated, when needed, by the menubar routine. All the libraries use the constant TCHOICES to identify the number of options available. To add or delete options, adjust TCHOICES and add or delete the appropriate number of lines in the menubar structure. The routine topbar generates the menubar window (the window only, not its contents). In most cases, windows are created as separate entities. But since the menubar is a permanent part of stdscr, I made the menubar a subwindow of stdscr to improve efficiency. This technique allows you to refresh just stdscr, instead of refreshing both stdscr and the menubar. The code WINDOW *swin; if ((swin = subwin(win,3,(win->MAXX)-4,(win->BEGY)+1, (win->BEGX)+2)) == NULL) clean_up (); creates the menubar. The parameter list for the subwin command includes the window pointer of the parent. curses uses this pointer to calculate where to position the menubar (see curses.h). Beware not to use a pointer until you create the actual window. Until a newwin or subwin command is invoked, the pointer is null and passing a null pointer to a function may cause unexpected results. Even though the curses implementations for all the platforms is highly consistent, the variable names vary across different environments. (To position the menubar, you need the WINDOW structure coordinates of the parent window.) VMS uses _beg_x and beg_y while MS-DOS and UNIX use beg_x and beg_y. To make the code more portable, my code uses macros, such as BEGX. Displaying a Menu The function do_menubar performs all the tasks associated with selecting an option from a menubar. First, do_menubar prints the string in the window. The sample menubar includes three strings: file, edit, and options. You should space these three strings appropriately so that they fill the top of the screen evenly. This application uses a function called strmenu for spacing the strings. strmenu takes the menubar structure and the width of the parent window as parameters. strmenu proceeds in three stages. First, strmenu calculates the number of spaces allocated to each string by dividing the width of the parent window by the number of strings in the menubar (in this case three). Next, strmenu enters a loop that builds the menubar by copying each string and padding it with the proper number of spaces, highlighting the current choice (initially the one on the left) in upper case. Finally, strmenu returns the string pointer to the calling function, and prints the menubar using the mvwaddstr function. Detecting a Selection Once the menubar is in place, the user must be able to select one of the options. The user will type either the first character of the option or the return key to activate the highlighted option. By typing the left and right arrow keys, the user can highlight different options. After receiving a keystroke, a switch statement controls the action. If the program encounters an arrow, it moves the highlight either to the left or right (with allowances for wrap-around). An escape terminates the program, while a return breaks out of the loop and invokes the option currently highlighted. By default the program sends all other sequences back to the calling program as a character code. Creating a Pulldown Menu A pulldown menu is basically a popup window, except the pulldown routine must be able to adjust for windows of different sizes. To define the choices contained in each pulldown menu, I created the structure CHOICES: typedef struct choices { char string[20]; char letter; int (*funcptr)(); } CHOICES; As with the menubar structure, CHOICES contains the string that represents the option and the letter that invokes it. The third field, a function pointer, represents the function that will be executed when the option is chosen. The prototypes for these functions are in Listing 5. For example, suppose that invoking the menubar option file produces a pulldown menu with the options open, close, and exit. The application initializes the structure CHOICES choices1[3] = { "open ", 'o', c_open, "close", 'c', c_close, "exit ", 'e', c_exit, }; Thus, if the user chooses open, the application calls function c_open. (I used the c_ prefix to avoid possible function name conflicts.) In this example application, there are three options (see Listing 4). They are all tied together with typedef struct pmenu { int num; int maxlength; CHOICES *ptr; } PULLDOWN; The structure PULLDOWN contains three pieces of information. The first field represents the number of options in the pulldown. The second indicates the maximum string length. The option close has five letters, and thus 5 is the maximum length. When creating the pulldown, the menu should have equal proportions. Thus, the shorter strings are padded with spaces to match that of the longest. The third field is the pointer to the structure that hold the choice information for this particular menu. The initialization of the entire PULLDOWN structure is PULLDOWN pullmenu[3] = { 3, 5, choices1, 4, 6, choices2, 3, 7, choices3, }; To create a pulldown, the application calls the function do_pulldown. Choosing an option from the pulldown menu works like the menubar, except that do_pulldown uses UP_ARROW and DOWN_ARROW. Unlike the menubar, the pulldown menu must be erased after an option has been chosen, and whatever was underneath must be restored. The command touchwin performs this task. For this example, the pulldown window blocks both the menubar and stdscr, so both must be restored. Tying It All Together The program in Listing 4 demonstrates the advantages of building these libraries. The actual application requires only a dozen or so lines of code. The program consists of two basic parts: building the screen and creating the menus. The functions that the menu choices invoke are simply shells. (The programmer would substitute the appropriate functionality.) The only functions that perform any tasks are the version function, which displays the current program version, and the exit function, which terminates the application (Listing 6). To run the application, compile menu.c, uilibs.c, and funcs.c with the appropriate defines for the host platform. Then link them together with the curses library provided by the package. When the user starts the program, the menu screen will appear. Conclusion The curses interface has its limitations. A commercial package written in curses will most likely not show up on the store shelves. However, if you need a reasonably efficient and portable method of creating user interfaces, curses is an option. Figure 1 Sample menu screen using curses Listing 1 sample.c -- sample curses application #include #include /* define window pointer */ WINDOW *win; main() { /* must be called before any curses command */ initscr(); /* get space for window at coordinates /* (y,x,begy,begx) */ /* y is # of rows, x is # of columns */ /* begy & begx are the positions on the screen */ win = newwin(24,80,0,0); /* surround the window with characters provided */ box (win, '', '-'); /* place the string in win at position y,x */ mvwaddstr(win,2,2, "enter a character to erase window"); /* this must be done to view changes on screen */ /* otherwise changes are made only in memory */ wrefresh(win); /* get a single character from the keyboard */ wgetch(win); /* erase the window in memory */ werase(win); /* necessary to complete the erase on screen */ wrefresh(win); /* remove the window from memory */ delwin(win); /* must be called to end a curses program */ endwin(); } /* End of File */ Listing 2 menu.h /* screen dimensions*/ #define MAX_ROWS 24 #define MAX_COLUMNS 80 /* keystroke codes and color */ #ifdef HPUX #define UP_ARROW 3 #define DOWN_ARROW 2 #define LEFT_ARROW 4 #define RIGHT_ARROW 5 #define RETURN 10 #define ESCAPE 27 #endif #ifdef SUN #define UP_ARROW 65 #define DOWN_ARROW 66 #define LEFT_ARROW 68 #define RIGHT_ARROW 67 #define RETURN 10 #define ESCAPE 27 #endif #ifdef VMS #define UP_ARROW 274 #define DOWN_ARROW 275 #define LEFT_ARROW 276 #define RIGHT_ARROW 277 #define RETURN 13 #define ESCAPE 291 /* F11 */ #endif #ifdef BCC #define MAINCOLOR (F_RED B_BLACK) #define DIALOGCOLOR (F_CYAN B_BLACK) #define UP_ARROW 56 #define DOWN_ARROW 50 #define LEFT_ARROW 52 #define RIGHT_ARROW 54 #define RETURN 10 #define ESCAPE 27 #endif /* macros for portability */ #ifndef VMS #define BEGX _begx #define BEGY _begy #define MAXX _maxx #else #define BEGX _beg_x #define BEGY _beg_y #define MAXX _max_x #endif /* box characters */ #ifdef BCC #define SINGLE_SIDE -77 /* single bar */ #define SINGLE_ACROSS -60 #define DOUBLE_SIDE -70 /* double bar */ #define DOUBLE_ACROSS -51 #else #define SINGLE_SIDE '' #define SINGLE_ACROSS '-' #define DOUBLE_SIDE '"' #define DOUBLE_ACROSS '=' #endif #define TCHOICES 3 /* menubar structure */ typedef struct mbar { char string[80]; char letter; int pos; }MENUBAR; /* pulldown menu choices */ typedef struct choices { char string[20]; char letter; int (*funcptr)(); } CHOICES; /* pulldown menu structure */ typedef struct pmenu { int num; int maxlength; CHOICES *ptr; } PULLDOWN; /* prototypes */ WINDOW *topbar(WINDOW *); WINDOW *pulldown(int,int,int,int); WINDOW *popup(int,int,int,int); void move_window(WINDOW *win,int y, int x); void print_string(WINDOW *,int, int, char *); void erase_window(WINDOW *); void delete_window(WINDOW *); void refresh_window(WINDOW *); int to_dialogue(char *); void clear_dialogue(void); void touch_window(WINDOW *); char *strmenu(int, MENUBAR *, int); char menu_choice(char *); int clean_up(void); void repaint(void); char do_pulldown(int,PULLDOWN *, MENUBAR *); void set_stdscr(void); void execute_command(int, int, PULLDOWN *); char do_menubar(WINDOW *, MENUBAR *); void strtoupper(char *); void strtolower(char *); void set_stdscr(void); char get_keystroke(void); /* End of File */ Listing 3 uilibs.c -- library code #include #include #include #include #include #ifdef VMS #include #endif #ifdef BCC #include #include #include #endif #include "curses.h" #include "menu.h" extern WINDOW *dialogue; extern WINDOW *tbar; static bar_size; /* size of menubar */ static int.menu_pos; /* position in menubar */ /* display menubar */ WINDOW *topbar(WINDOW *win) { WINDOW *swin; int string_count, string_size; if((swin = subwin(win,3,(win->MAXX)-4,(win->BEGY)+1, (win->BEGX)+2)) == NULL) clean_up(); #ifdef BCC wattrset(swin, F_BLUE B_GRAY); #endif box (swin, SINGLE_SIDE,SINGLE_ACROSS); bar_size = (swin->MAXX)-2; menu_pos=0; return (swin); } /* print string to menubar */ char do_menubar(WINDOW *swin, MENUBAR *menubar) { char * menu; char buffer[80]; int status; #ifdef VMS int keyboard; short term_code; #else char term_code; #endif #ifdef VMS if ( (( status = SMG$CREATE_VIRTUAL_KEYBOARD(&keyboard))&1)!=1) clean_up(); #endif term_code = 0; while (term_code != RETURN) { /* get the new menubar string */ menu = strmenu(bar_size, menubar, menu_pos); mvwaddstr(swin, 1, 1, menu); wrefresh(swin); /* get a single keystroke */ #ifdef VMS if ( (( status = SMG$READ_KEYSTROKE (&keyboard,&term_code))&l)!=l) clean_up(); #endif #ifdef BCC term_code = wgetch(swin); #endif #ifdef HPUX term_code = getch(); #endif #ifdef SUN term_code = getch(); if (term_code == ESCAPE) { getch(); term_code = getch(); } #endif /* process keystroke */ switch (term_code) { /* arrows check for wrap-around */ case LEFT_ARROW: if (menu_pos == 0) menu_pos = TCHOICES-1; else menu_pos--; break; case RIGHT_ARROW: if (menu_pos == TCHOICES-1) menu_pos = 0; else menu_pos++; break; /* do nothing */ case RETURN: break; /* exit program */ case ESCAPE: clean_up(); break; /* return keyboard input */ default : return (term_code); break; } } /* return highlighted option */ return (menubar[menu_pos].letter); } WINDOW *popup(int rows,int columns,int sy,int sx) { WINDOW *win; win = newwin(rows, columns, sy, sx); if(win == NULL) { endwin(); clean_up(); } #ifdef BCC wattrset(win, F_BLACK B_GRAY); #endif box(win, SINGLE_SIDE, SINGLE_ACROSS); wrefresh(win); return (win); } /* erase windows and surrounding box */ void erase_window(WINDOW *win) { werase(win); box(win, ' ', ' '); wrefresh(win); } void delete_window(WINDOW *win) { delwin(win); } void refresh_window(WINDOW *win) { wrefresh(win); } void touch_window(WINDOW *win) { touchwin(win); wrefresh(win); } /* process pulldown menu options */ char do_pulldown (int i, PULLDOWN *pullmenu, MENUBAR *menubar) { WINDOW *subwin1; int j; int position, oldpos; char *ptr; int status; #ifdef VMS int keyboard; short term_code; #else char term_code; #endif #ifdef VMS if ( (( status = SMG$CREATE_VIRTUAL_KEYBOARD(&keyboard))&1)!=1) clean_up(); #endif subwin1 = popup( (pullmenu[i].num)+2, (pullmenu[i].maxlength)+2,stdscr->BEGY+3, (menubar[i].pos)+2); /* print pulldown options */ for (j=0;j #include #include #include #include #ifdef BCC #include #include #endif #include "curses.h" #include "menu.h" #include "internal.h" char choice; /* windows global to all routines */ WINDOW *dialogue; WINDOW *tbar; /* define menubar with three options */ MENUBAR menubar[TCHOICES] = { "file", 'f', 0, "edit", 'e', 0, "options", 'o', 0, }; /* define pulldown menu sub-choices */ CHOICES choices1[3] = { "open ", 'o', c_open, "close", 'c', c_close, "exit ", 'e', c_exit, }; CHOICES choices2[4] = { "copy ", 'c', c_copy, "paste ", 'p', c_paste, "delete", 'd', c_delete, "move ", 'm', c_move, }; CHOICES choices3[4] = { "version", 'v', c_version, "compile", 'c', c_compile, "link ", 'l', c_link, "run ", 'r', c_run, }; /* tie all choices into one struct */ PULLDOWN pullmenu[TCHOICES] = { 3, 5, choices1, 4, 6, choices2, 4, 7, choices3, }; main() { int i,j,k; initscr(); /* needed to return one keystroke at a time */ #ifndef VMS cbreak(); #else crmode(); #endif /* activate keypad code */ #ifdef HPUX keypad(stdscr, TRUE); #endif noecho(); #ifdef BCC cursoff(); #endif /* set up screen */ set_stdscr(); dialogue = popup(3, MAX_COLUMNS-4, MAX_ROWS-4, 2); clear_dialogue(); tbar = topbar(stdscr); /* enter loop to process options */ for (;;) { choice = do_menubar(tbar, menubar); for (i=0;i #include "curses.h" #ifdef BCC #include #endif #include "menu.h" #include "internal.h" /* all these functions are shells except for c_exit & c_version */ int c_open() { to_dialogue("open"); sleep(3); return; } int c_close() { to_dialogue("close"); sleep(3); return; } int c_exit() { to_dialogue("exit"); clean_up(); return; } int c_copy() { to_dialogue("copy"); sleep(3); return; } int c_paste() { to_dialogue("paste"); sleep(3); return; } int c_delete() { to_dialogue("delete"); sleep(3); return; } int c_move() { to_dialogue("move"); sleep(3); return; } int c_compile() { to_dialogue("compile"); sleep(3); return; } int c_link() { to_dialogue("link"); sleep(3); return; } int c_run() { to_dialogue("run"); sleep(3); return; } int c_version() { to_dialogue("Version 3.0"); sleep(3); return; } /* End of File */ A Prompting Function Dale A. Panattoni Mr. Panattoni received a Bachelor of Science degree in Computer Information Systems from DeVry Institute of Technology in Los Angeles, CA. He has worked the last four years as a Programmer/Analyst for DataStar Corporation, a company that specializes in business and fruit accounting software. You may contact him at DataStar Corporation, 6 South 55th Avenue, Yakima, WA 98908, (509) 453-2455, or at his home number, (509) 453-2455. Introduction In school, they tell you that when writing the user interface section of a program, you should write it as if a monkey will be sitting at the keyboard. Based on my experience, I sometimes think that it would be easier to write code for a monkey than a real user. A monkey may bang at the keyboard, but a user often times will try to outwit the program and unknowingly enter incorrect information. Several years ago I was shown a data input function, called prompt, that not only provided a good method for entering data, but also did basic validation of the data being entered. The function was printed in a book titled Variations in C by Steve Schustack (MicroSoft Press, 1985). After several modifications and enhancements, this function has become a cornerstone for all of my data input routines, both in MS-DOS and UNIX. In order to demonstrate prompt without requiring you to have a commercial windowing library, the version listed here has been stripped of all of its windowing library calls and replaced with Standard C I/O routines. To read a character from the keyboard, this version uses getchar. To display to the screen it uses printf. And to position the cursor, it uses a macro called MOVE_CUR. Since the MOVE_CUR macro uses ANSI codes, you need to include the ansi.sys driver in the file config.sys. Because of these changes, this function as it stands is written for MS-DOS, but it can easily be made to work with any MS-DOS or UNIX windowing library by changing just a few function calls. prompt accepts input from the user and returns to the calling program an integer value that represents a terminating key. The terminating key is the key that is pressed by the user signaling either that some action may need to be performed by the calling program or that the user is done entering data in the field. The calling program will determine what needs to be done based on that terminating key. For example, If the terminating key is an F1, you may want to display a help screen or a pick list pertaining to the current field. If the terminating key is what I call a "moving" terminating key--such as Up Arrow, Down Arrow, Home, End, Tab, or Enter--the calling program will probably want to move to the next or previous field. The keys.h header file included in prompt (see Listing 1) defines the values returned from getchar when the special keys, such as function and cursor keys, are pressed. This header file needs to be included not only with the file that defines prompt, but also with any program that calls it, if that calling program needs to know what terminating key caused prompt to return. When you implement a windowing library with prompt, this header file will probably need to be replaced by the windowing package's header file that defines its own return codes for the keyboard. Besides the definitions of the special keys, I have included definitions for four other terminating keys which can be returned by prompt. Because there are times when a data field requires specific validation, I wanted to know whether the user changed the data. If no changes occur, there is no need to do any validating. Therefore I have defined a NO_CHANGE value for Up Arrow, Down Arrow, Tab, and Enter (and only these four keys). They are the most common keys that a user would press when done entering information for a field. If you find that you use other terminating keys to end data input, you may need to add NO_CHANGE values for those keys as well. Calling the Prompting Function The function prompt expects nine parameters: data is a character pointer that points to a buffer holding the default value of the data field to be entered by the user. It may be initialized to NULL, or it may have a preset value. When prompt returns to the calling program, the buffer that data points to will be changed if the user has entered any data. match_char is a character code that represents what type of input will be accepted as being valid from the keyboard. These codes are setup and maintained in the match function. min_len is an integer value depicting a minimum number of characters to be entered by the user for this data field. The prompt function will not return a "moving" terminating key unless the user has met this requirement. max_len is an integer value depicting the maximum number of characters that can be entered for this data field. row and col are two integers which represent the starting row and column where the user will begin entering data. fyi is a character string that can be used either as the field's title, or as a "For Your Information" (FYI) line to give the user instructions about what is to be entered. If you decide not to use an FYI, you can pass a null string for this parameter. fyi_row and fyi_col are two integers which represent the starting row and column for the FYI message display. All of the row and column values refer to an entire screen of 12 rows and 80 columns. If you implement a windowing library with prompt, you will probably want to add another parameter for a Window pointer. Then all of the row and column numbers will be relative to the window being used. When calling prompt, you can be as simple as you want, or you can analyze the return code in detail. In the sample program (See Listing 2), I check to see what terminating key is returned by prompt to determine what field the user wants to go to next. If you have more than a couple of fields to call using prompt, I suggest making an array of "field" structures that will hold all of the values for prompt for each field. This has two immediate advantages. First, the parameters passed to prompt are more readable at a glance. And second, if the array of structures is used every time a field's data is displayed to the screen, when your client asks you to move screen fields around, you can move a field anywhere they want in one simple change. Using the Prompting Function prompt is a versatile function. It not only allows the user to enter in new data but gives the user the freedom to edit existing data without having to retype the entire field. When the program's cursor first arrives at a field, the user can overwrite its data by simply typing new information. prompt will wipe out the old data and replace it with the newly-entered information as long as the first key entered by the user is not Right Arrow. If Right Arrow is pressed as the first key, prompt will enter into an edit mode. Edit mode will allow the user to move through the field's data using Left Arrow, Right Arrow, Home, and End. The user also has the option of toggling between an Insert mode and a Typeover mode by pressing the Insert key. When in Typeover mode, the cursor appears as a block. When in Insert mode, the cursor appears as an underscore. Because this version of prompt has been stripped of a commercial windowing library in order to make it work with the Standard C library, I have added the change_cur function to change the cursor's appearance. If you implement a windowing library prompt, you will probably want to replace change_cur with a similar function from your windowing library. Implementation prompt is really quite basic when you look at it closely. (See Listing 3.) When first called, it displays the data field and the For Your Information line. After that, the function goes into a loop that gets characters from the user until a terminating key is pressed. For each character entered by the user, prompt will verify that it is valid based on the match code that was passed in as a parameter. If the character entered is found to be valid, it is displayed to the screen and the user is prompted to enter another character. All editing keys such as Backspace, Left Arrow, Right Arrow, Home, and End can be entered by the user. When a terminating key is entered, prompt will do one of two things depending on what kind of terminating key is pressed. If the terminating key is not a "moving" key, prompt simply returns to the calling program the terminating key that was pressed. If the terminating key is a "moving" key, prompt will make sure that the length of the field's data meets or exceeds the field's minimum length requirement. If the minimum length has been met, prompt will return to the calling program. But before returning, prompt checks to see if the user has entered anything for this field. If not, then a NO_CHANGE value for the "moving" key is returned. Otherwise, the "moving" terminating key is returned to the calling program. Real-Time Validation match is a function that you, the programmer, set up to define a series of one-character codes that will represent sets of valid data. (See bottom of Listing 3.) You can have as many sets as you like, and add them as often as is needed. This function is what I think makes prompt so great. Once the user has entered in the field's data, you can assume that the field is valid. prompt will not allow any characters to be entered that are not defined as being correct for a particular match code. There may be times however when you will still have to do some validating when prompt returns to the calling program. For example, while prompt can make sure that the user enters all of the correct characters that make up a date, it does not validate the date. But based on my experience with this function, prompt eliminates 50 to 60 percent of the user entry errors by keeping them from typing invalid keys from the beginning. Listing 1 keys.h -- defines the values returned from getchar when special keys are pressed /* */ /* Definitions Of Keys From The */ /* Keyboard */ /* */ #define C_UP 328 /* Up Arrow */ #define C_DOWN 336 /* Down Arrow */ #define C_PGUP 329 /* Page Up Key */ #define C_PGDN 337 /* Page Down Key */ #define C_CR 13 /* Enter Key */ #define C_BACK 8 /* Backspace Key */ #define C_LEFT 331 /* Left Arrow */ #define C_RIGHT 333 /* Right Arrow */ #define C_TAB 9 /* Tab Key */ #define C_HOME 327 /* Home Key */ #define C_END 335 /* End Key */ #define C_F1 315 /* F1 - F12 Keys */ #define C_F2 316 #define C_F3 317 #define C_F4 318 #define C_F5 319 #define C_F6 320 #define C_F7 321 #define C_F8 322 #define C_F9 323 #define C_F10 324 #define C_F11 325 #define C_F12 326 #define C_ESC 27 /* Escape Key */ #define C_INS 338 /* Insert Key */ #define C_DEL 339 /* Delete Key */ #define CR_NO_CHG -1 /* This Value Represents The ENTER Key Pressed By The User Without Making Any Changes To The Field's Data */ #define UP_NO_CHG -2 /* Up Arrow - No Changes */ #define TB_NO_CHG -3 /* Tab Key - No Changes */ #define DN_NO_CHG -4 /* Down Arrow - No Changes */ Listing 2 Sample Program using prompt () #include #include #include #include #include #include "./keys.h" /* Identify Field Indexes */ #define FNAME 0 #define LNAME 1 #define SEX 2 #define AGE 3 #define MOVE_CUR(row,col) printf("\x1B[%d;%df",row,col); /* Prototypes */ extern int prompt (char *, char, int, int, int, int, char *, int, int); struct fields { short row; /* Field Row */ short col; /* Field Column */ short fyi_row; /* FYI Row */ short fyi_col; /* FYI Column */ short min_len; /* Minimum Length */ short max_len; /* Maximum Length */ char match; /* Match Character Code */ char fyi[81] ; /* FYI Message */ }; struct fields field[] = { { 10,37,15,35,0,30,'A',"ENTER IN YOUR FIRST NAME."}, { 11,37,15,35,0,30,'A',"ENTER IN YOUR LAST NAME. "}, { 12,37,15,35,0, 1,'X',"ENTER IN YOUR SEX. (M/F) "}, { 13,37,15,35,0, 3,'#',"ENTER IN YOUR AGE. "} }; void main() { char xbuf[81]; /* Buffer */ char fname[31]; /* First Name */ char lname[31]; /* Last Name */ char sex; /* Male/Female */ int c; int age; int index; /* Initialize Variables And Draw Screen Titles */ fname[0] = lname[0] = sex = '\0'; MOVE_CUR(10,25); printf("FIRST NAME:"); MOVE_CUR(11,25); printf("LAST NAME:"); MOVE_CUR(12,25); printf("SEX M/F ..:"); MOVE_CUR(13,25); printf("AGE ......:"); age = 0; index = FNAME; while (1) { switch (index) { case FNAME: sprintf(xbuf,"%-s",fname); c = prompt(xbuf, field[FNAME].match, field[FNAME].min_len, field[FNAME].max_len, field[FNAME].row, field[FNAME].col, field[FNAME].fyi, field[FNAME].fyi_row, field[FNAME].fyi_col); switch (c) { case C_UP : case UP_NO_CHG: case C_END : /* Go To Last Screen Field */ index = AGE; break; case C_ESC : /* Exit Program */ exit(0); case C_CR : case C_DOWN: case C_TAB : strcpy(fname,xbuf); case CR_NO_CHG: case DN_NO_CHG: case TB_NO_CHG: /* Go To Next Field */ index = LNAME; break; } MOVE_CUR(10,37); printf("%-30.30s",fname); break; case LNAME: sprintf(xbuf,"%-s",lname); c = prompt(xbuf, field[LNAME].match, field[LNAME].min_len, field[LNAME].max_len, field[LNAME].row, field[LNAME].col, field[LNAME].fyi, field[LNAME].fyi_row, field[LNAME].fyi_col); switch (c) { case C_END : /* Go To Last Screen Field */ index = AGE; break; case C_ESC : /* Exit Program */ exit (0); case C_UP : case UP_NO_CHG: case C_HOME : /* Go To Previous Field */ index = FNAME; break; case C_CR : case C_DOWN: case C_TAB : strcpy(lname,xbuf); case CR_NO_CHG: case DN_NO_CHG: case TB_NO_CHG: /* Go To Next Field */ index = SEX; break; } MOVE_CUR(11,37); printf("%-30.30s",lname); break; case SEX: sprintf(xbuf,"%c",sex); c = prompt(xbuf, field[SEX].match, field[SEX].min_len, field[SEX].max_len, field[SEX].row, field[SEX].col, field[SEX].fyi, field[SEX].fyi_row, field[SEX].fyi_col); switch (c) { case C_UP : case UP_NO_CHG: /* Go To Previous Field */ index = LNAME; break; case C_ESC : /* Exit Program */ exit(0); case C_HOME: /* Go To First Screen Field */ index = FNAME; break; case C_CR : case C_DOWN: case C_TAB : sex = xbuf[0]; case C_END : case CR_NO_CHG: case DN_NO_CHG: case TB_NO_CHG: /* Go To Next Previous Field */ index = AGE; break; } MOVE_CUR(12,37); printf("%c",sex); break; case AGE: sprintf(xbuf,"%-d",age); c = prompt(xbuf, field[AGE].match, field[AGE].min_len, field[AGE].max_len, field[AGE].row, field[AGE].col, field[AGE].fyi, field[AGE].fyi_row, field[AGE].fyi_col); switch (c) { case C_UP : case UP_NO_CHG: /* Go To Previous Field */ index = SEX; break; case C_ESC : /* Exit Program */ exit(0); case C_HOME: case C_CR : case C_DOWN: case C_TAB : age = atoi(xbuf); case CR_NO_CHG: case DN_NO_CHG: case TB_NO_CHG: /* Go To Next Screen Field */ index = FNAME; break; } MOVE_CUR(13,37); printf("%- 3d",age); break; } } } /* End of File */ Listing 3 Prompt Function #include #include #include #include #include #include "./keys.h" #ifndef YES #define YES 1 #endif #ifndef NO #define NO 0 #endif #define M_INSERT 1 #define M_TYPEOVER 2 #define MOVE_CUR(row,col) printf("\x1B[%d;%df",row,col); #define NORMAL "\x1B[Om" /* ANSI Normal Video */ #define REVERSE "\x1B[7m" /* ANSI Reverse Video */ /* Prototypes */ extern int match (int, char); extern void change_cur (int, int); extern void main (void); int key; int prompt( data, match_ch, min_len, max_len, row, col, message, fyi_row, fyi_col) char data []; /* Return Message */ char match_ch; /* Code For Which Characters Are Allowed */ int min_len; /* Min Length Allowed */ int max_len; /* Max Length Allowed */ int row; /* Window Row */ int col; /* Window Column */ char *message; /* Prompt Message */ int fyi_row; /* FYI Window Row */ int fyi_col; /* FYI Window Column */ { char buf [80]; int terminate; /* Key That Caused Termination */ int index,i; /* General Index Variables */ int mode; /* Mode: INSERT or TYPEOVER */ int changed; /* Changed Flag: 1=YES, 0=NO */ int edit; /* Edit Flag: 1=YES, 0NO */ int frst_key; /* First Key Flag: 1=YES, 0=NO */ char more; /* YES/NO Flag . . . Used To Terminate Input */ /* Initializations */ edit = NO; frst_key = NO; changed = NO; mode = M_TYPEOVER; change_cur(0, 13); /* Set Cursor To Block Mode */ /* Display The FYI Message At The FYI Row, Colunn */ if (strlen(message)) { MOVE_CUR (fyi_row, fyi_col); printf(message); } printf(REVERSE); /* Change To Reverse Video */ strcpy(buf, data); /* Make A Copy Of The Default Return Value */ for (index = strlen(buf); index < max_len; ++index) { buf [index] = ' '; /* Fill Remainder Of Default Value With Blanks */ } buf [index] = '\0'; /* Display Default Value For Field */ MOVE_CUR (row, col); printf(buf); /* Position Cursor At Beginning Of Field */ MOVE_CUR (row, col); /* Get Input From User */ for (index = 0,more = YES; more; ) { key = getch(); if (key == 0) key = getch()+256; switch(key) ( case C_F1 : /* F1 Key */ case C_F2 : /* F2 Key */ case C_F3 : /* F3 Key */ case C_F4 : /* F4 Key */ case C_F5 : /* F4 Key */ case C_F6 : /* F6 Key */ case C_F7 : /* F7 Key */ case C_F8 : /* F8 Key */ case C_F9 : /* F9 Key */ case C_F10 : /* F0 Key */ case C_F11 : /* F11 Key */ case C_F12 : /* F12 Key */ case C_ESC : /* ESCAPE Key */ frst_key = NO; terminate = key; buf[index] = '\0'; more = NO; break; case C_INS : /* INSERT Key */ frst_key = NO; if (mode == M_INSERT) { /* Set The Mode To TYPEOVER and Change Cursor To Block Mode */ mode = M_TYPEOVER; change_cur(0, 13); } else { /* Set The Mode To INSERT And Change Cursor To Underline */ mode = M_INSERT; change_cur(12, 13); } putchar(bur[index]); MOVE_CUR(row, (col + index)); break; case C_BACK : /* BACKSPACE Key */ case C_DEL : /* DELETE Key */ frst_key = NO; changed = YES; if (index >= max_len) index = max_len - 1; for (i = index; i < max_len; i++) { if (buf[i+1] == '\0') buf[i] = ' '; else buf[i] = buf[i+1]; } /* Redisplay The Field */ MOVE_CUR(row, col); printf(buf); /* Reposition The Cursor. */ MOVE_CUR(row, (col + index)); putchar(buf[index]); MOVE_CUR(row, (col + index)); if (key == C_DEL) break; case C_LEFT : /* LEFT ARROW */ first_key = NO; if (index <= 0) break; if (index >= max_len) index = max_len -1; index--; MOVE_CUR(row, (col + index)); break; case C_RIGHT: /* RIGHT ARROW */ frst_key = NO; edit = YES; if (index >= (max_len - 1)) break; index++; MOVE_CUR(row, (col + index)); break; case C_HOME : /* HOME KEY */ case C_END : /* END KEY */ frst_key = NO; if (edit == YES) { if (key == C_END) index = end_of_fld(buf,(strlen(buf))); else index = 0; MOVE_CUR(row, (col + index)); break; } case C_DOWN : /* DOWN ARROW */ case C_UP : /* UP ARROW */ case C_PGDN : /* PAGE DOWN */ case C_PGUP : /* PAGE UP */ case C_CR : /* CARRIAGE RETURN */ case C_TAB : /* TAB Key */ frst_key = NO; /* Reset Mode and Edit Flag */ edit = NO; mode = M_INSERT; if (strlen(data) >= (unsigned)min_len && index == 0 && changed == NO) { /* If The User Has Not Changed The Value Of The Data Field, Return The Appropriate NO CHANGE Key For The terminating Key That Was Pressed. */ if (key == C_CR) { /* Return No Change For Carriage Return */ terminate = CR_NO_CHG; } if (key == C_UP) { /* Return No Change For Up Arrow */ terminate = UP_NO_CHG; } if (key == C_DOWN) { /* Return No Change For Down Arrow */ terminate = DN_NO_CHG; } if (key == C_TAB) { /* Return No Change For Tab Key */ terminate = TB_NO_CHG; } if (key != C_CR && key != C_UP && key != C_DOWN && key != C_TAB) terminate = key; strcpy(buf, data); more = NO; break; } /* If Minimum Length Requirement Has been Met, Quit */ if (index >= min_len) { terminate = key; more = NO; break; } /* Else Ignore */ break; default : if (index == max_len) /* At End Of Data Field - Do Nothing */ break; /* We Have Dealt With All Of The Keys That We Needed Above, So Now We Will Filter Out All Keys And Characters Above 122 (z) Here. */ if (key > 'z') /* Ignore Key */ break; if (match(index, match_ch) == YES) { /* Only Accept The User Entered Character If It Successfully Passed The match function. */ edit = YES; changed = YES; if (frst_key == YES) { /* If This Is The First Valid Key That The User Has Entered, Then Wipe Out The Field. */ for (i = 0; i < max_len; i++) buf[i] = ' '; frst_key = NO; } if (mode == M_INSERT) { /* Insert The User Entered Character Into The Field */ for (i = (max_len - 1); i > index; i--) buf[i] = buf[i-1]; } /* Overwrite The Buffer With The User Entered Character */ buf[index] = (char)key; /* Redisplay The Field */ MOVE_CUR(row, col); printf(buf); /* Reposition The Cursor. */ index++; if (index >= max_len) index --; MOVE_CUR(row, (col + index)); putchar(buf[index]); MOVE_CUR(row, (col + index)); } break; } } /* Redraw The Field In The Window's Normal Color Attribute */ printf(NORMAL); MOVE_CUR (row, col); printf(buf); strcpy(data, buf); data[(end_of_fld(data,(strlen(data))) + 1)] = '\0'; /* Change Cursor Back To Underline */ change_cur(12, 13); return (terminate); } static int match(index,match_ch) int index; char match_ch; { int matches = YES; switch (match_ch) { case '$' : /* 0-9 or ,$. */ if (!isdigit(key) && !strchr(".,$",key)) matches = NO; break; case '#' : /* 0 - 9 */ if (!isdigit(key)) matches = NO; break; case 'A' : /* A - Z */ if (!isalpha(key) && !strchr(" ",key)) matches = NO; break; case 'D' : /* Date: 0-9 or - or / or .*/ if (!isdigit(key) && !strchr(".-*/",key)) matches = NO; break; case 'l' : /* a-z, A-Z, 0-9, or ;:.,/?*-$#()'! or leading space */ if (!isalnum(key) && !strchr(" !@#$%^&*()-_=+[{]}';:.,/?",key)) matches = NO; break; case 'Q' : /* YyNn as Reply To Yes/No Question */ if (!strchr("YyNn",key)) matches = NO; key = toupper(key); break; case 'S' : /* 0-9 or + or - */ if (!isdigit(key) && !strchr("+-",key)) matches = NO; break; case 'T' : /* Time: 0-9 or . or : */ if (!isdigit(key) && !strchr(".:",key)) matches = NO; break; case 'X' : /* MmFf As Reply To Male/Female */ if (!strchr("MmFf",key)) matches = NO; key = toupper(key); break; default : matches = NO; break; } return (matches); } /* This Function Will Return A 0 Based Index Of The First Non-Blank Character At The End Of A String. */ int end_of_fld(string, length) char *string; int length; { int i; for (i = length - 1; i >= 0; i--) { if (string[i] != ' ' && string[i] != '\0') return(i); } return(0); } /* This Function Will Change The Appearance Of The Cursor To A Block Or An Underscore */ void change_cur(start, end) int start; int end; { union REGS r; r.h.ah = 1; r.h.ch = start; r.h.cl = end; int86(0x10, &r, &r); } /* End of File */ Mapping Functions for Repetitive Structures Steven K. Graham Steven K. Graham has worked for Hewlett-Packard, served on the Faculty at UMKC, and is currently a Senior Engineer with CSI. Higher-order functions take other functions as arguments. They are usually used to apply a function to some repetitive data object, such as a list or a tree. Such use of higher-order functions is referred to as mapping, and the higher-order functions are called mapping functions. The key benefit of higher-order functions is the extra layer of abstraction they introduce. In essence, mapping functions allow you to create custom control structures that can be applied to data structures routinely used in a particular environment. Mapping functions do not simplify code intended for a single use or a single purpose. Only when a data object is processed repeatedly in similar ways--where the control flow is the same, but the functions performed vary--is a mapping function valuable. As an example, consider the general case of processing records in a particular kind of data file. This can be regarded as mapping the processing function onto the records of the file. A mapping function could be built that takes any given record processing function and applies it to all the records of a file. Uses for Mapping Functions Despite sparse use in traditional programming languages, mapping functions are used in many familiar contexts. In mathematics, a higher-order function is used to express iterated sums: Click Here for Equation The summation applies the function F to each integer from 1 to n, and accumulates the sum. If n were 5 and F(x) = x x, then sum = 11 + 22 + 33 + 44 + 55. It can be argued that a large part of the success of mathematics is due to developing effective language and notations to express complex ideas. In UNIX, the notion of higher-order commands has appeared in commands that take other commands as arguments and control their application. sed is a good example of this. Some languages provide convenient methods for expressing higher-order functions. Lisp and its dialects such as Scheme, provide functions as first class objects--functions can be passed as arguments, returned as values, assigned as array elements and so on. So Lisp programmers commonly use mapping functions and are accustomed to exploiting their power. Other languages restrict the use of functions to varying degrees, but higher-order functions are possible in languages readily available, such as C. Because of C's (relatively) strong typing, mapping functions force the processing functions they apply to return a particular type, and it is necessary to coerce types to pass a function argument to a mapping function. Example Programs Listing 1 illustrates a C version of summation. It defines two mapping functions, sum and dsum, and three processing functions which are mapped onto a range of integers, pi, id, and sq. Note that sum and dsum are basically for loops that process the integers from a to b, at each step accumulating the result of applying the function parameter term to the current value of i. Because of C's typing, you need two versions to accommodate different return types despite the identical control structures, so dsum is used to accumulate a floating-point sum. One of the processing functions, id, is particularly noteworthy, because it is only in the context of higher-order functions that a programmer ever needs an identity function that does nothing but return its argument value. However, id is needed for sum to simply add the integers from a to b. The call to dsum, approximates p using a partial sum of an infinite series. This is drawn from the identity that Click Here for Equation which can be written as Click Here for Equation with the function pi (in Listing 1) providing the value for each term, given the value for i. Higher-order functions become mapping functions when applied to repeating structures, such as lists or trees. A list is comprised of an item and a pointer to the rest of the list (which is also a list). A tree is comprised of a node, with subtrees as children. This mapping approach can be applied to the directory tree of a UNIX file system (see Listing 2). The function mapdir is a recursive function that maps its argument filefn onto all the files in the hierarchy below a commandline directory argument. This example applies the function suidfile that prints out the name of files that have the setuid bit set. The setuid bit indicates that when the file is executed, it will change its effective user ID to the owner of the file when it is run. Programs that set the user ID can lead to gaping holes in system security. (Sun's Programmer's Guide to Security Features comments on its use, "1. Don't do it unless absolutely necessary.") Using mapdir, it is simple to search an entire directory structure for programs that set the user ID. This example could be part of a suite of programs designed to explore the file system for potential security problems. A more elaborate system could be devised to check other file attributes, such as comparing file permissions to the permissions of the directory that contains them. The real power of this approach is that the mapdir function is independent of any particular use. Simply by calling mapdir with the function printfile (in Listing 3) as an argument, you have a program to display directories recursively. By creating routines that check for unusually large files, for unusually small files, or for large numbers of files within a single directory, you can build tools to help manage disk usage. The functions and structures used in mapdir are drawn from the standard include files, dirent.h, sys/dirent.h and sys/stat.h. These examples were developed on a Sun SPARCstation using SunOS 4.1. dirent.h provides the DIR struct, along with the functions opendir, closedir, and readdir to manipulate it. sys/dirent.h provides the struct dirent, for directory entries: struct dirent { off_t d_off; /* offset of next disk dir entry */ unsigned long d_fileno; /* file number of entry */ unsigned short d_reclen; /* length of this record */ unsigned short d_namlen; /* length of string in d_name */ char d_name[255+1]; /* name (up to MAXNAMLEN + 1) */ }; For the example programs, the field d_name, a string that specifies the file name for a particular directory entry, was the only field used. stat.h provides the following structure to store information about files: struct stat { dev_t st_dev; /* device file resides on */ ino_t st_ino; /* file serial number */ mode_t st_mode; /* file mode */ short st_nlink; /* # of hard links to file */ uid_t st_uid; /* owner's user ID */ gid_t st_gid; /* owner's group ID */ dev_t st_rdev; /* dev identifier for * /* dev identifier for * * special files */ off_t st_size; /* file size in bytes * * - (off_t is a long) */ time_t st_atime; /* last access time */ int st_spare1; time_t st_mtime; /* last modify time */ int st_spare2; time_t st_ctime; /* last status change time */ int st_spare3; long st_blksize; /* preferred block size */ long st_blocks; /* # of blocks allocated */ long st_spare4[2]; }; The example programs use the st_mode field, as well as some of the macros and constants defined in stat.h. In particular, the macros S_ISDIR, S_ISLNK, and the bit mask S_ISUID are used to examine the st_mode field, which records file types and permissions. stat.h also declares the stat function that reads the file information. stat has the declaration int stat(path, buf) char *path; struct stat *buf; The mapdir program can be made more general. In Listing 3, mapif.c, the mapdir function has been modified to accept three functions as arguments: a predicate function, a then function and an else function. The purpose is to not only map these functions across the file system, but also provide alternative actions depending on the results of the predicate function. The example uses a function that tests for the setuid bit, a printfile function, and a "do nothing" function, to reproduce the behavior of the earlier example from Listing 2. Variations to Mapping Functions In these example, the functions that are mapped rely on a single filename argument. This can be varied. A file descriptor could be passed. The complete path could be passed in addition to the filename. Other parameters might specify the level in the file hierarchy or information about the directory containing the file. Global variables or additional parameters could be set using command-line arguments and used to provide thresholds for searching for unusually large or small files, or perhaps a search string for building a grep-like function out of mapdir and a routine that searches a single file. Besides operating on files within the file system, a mapping function could be written that processes the directories rather than (or in addition to) processing the files, for example, counting the number of files in each directory. The file system isn't the only regularly structured data in a UNIX system. A simple procedure based on the mapping notion could be written that applies a function to each line within a file. Using such a routine, functions could operate on the passwd file, checking for users with no password, or checking directory protections on the home directories of each user. A function could be written to process every user's .cshrc file, perhaps to incorporate changes uniformly. UNIX, perhaps more than any other system, is characterized by an abundance of tools. As I mentioned at the outset, many of these tools incorporate the notion of function or command arguments. The find command, with an - exec parameter, processes the file system in much the way that mapdir does. So why bother with mapdir? First, the key point is not the particular example, but the general idea: whenever possible, abstract general behaviors over regular structures into mapping functions. Second, tools built in C rather than as shell scripts or command invocations are more flexible. Their input and output can be varied; alternatives can be handled; and programs can be tailored to particular uses. Finally, the performance of different approaches will vary. In some cases, a program may be more efficient than a command with respect to some essential resources. (C) 1993 by Steven K. Graham Listing 1 Mapping and processing functions int sum(a,b, term) int a,b, (*term) (); { int i, sum = 0; for (i=a; i #include #include #include #define LASTCHR(s) s[strlen(s)-1] int suidfile(name) char *name; { struct stat stbuf; stat(name, &stbuf); if (stbuf.st_mode & S_ISUID) printf("%s\n", name); return 0; } int mapdir(name, filefn) char *name; int (*filefn) (); { DIR *dirp; struct dirent *entry; struct stat stbuf; char olddir[MAXPATHLEN]; char cname[MAXPATHLEN]; getwd(olddir); strcpy(cname, name); if (chdir(name)) { printf("Couldn't change to directory %s\n", name); return -1; } if ((dirp = opendir(name)) == NULL){ printf("Unable tto open DIR %s\n", name); chdir(olddir); return -1; } for (entry=readdir(dirp); entry != NULL; entry=readdir(dirp)) { if (0 != stat(entry->d_name, &stbuf)) printf("Can't read file information %s\n", entry->d_name); else if (strcmp(entry->d_name, ".") == 0 strcmp(entry->d_name, "..") == 0) /* don't pursue these entries */ ; else if (S_ISLNK(stbuf.st_mode)) /* don't pursue links */ ; else if (S_ISDIR(stbuf.st_mode)) { if (LASTCHR(cname) != '/') strcat(cname, "/"); strcat(cname, entry->d_name); mapdir(cname, filefn); strcpy(cname, name); } else (*filefn) (entry->d_name); } closedir(dirp); chdir(olddir); return 0; } char *setupdir(buf, name) char *name; char *buf; { char curdir[MAXPATHLEN]; getwd(curdir); if (name[0] == '/') /* absolute pathname */ strcpy(buf, name); else { /* relative pathname*/ strcpy(buf, curdir); if (buf[strlen(buf)-1] != '/') strcat(buf, "/"); /* may be at root */ strcat(buf, name); } return buf; } int main(argc, argv) int argc; char **argv; { char *filefn; char name[MAXPATHLEN]; if (argc < 2) { printf("Usage: %s ",argv[0]); return -1; } setupdir(name, argv[1]); filefn = (char *) suidfile; return mapdir(name, filefn); } /* End of File */ Listing 3 mapdir -- maps its argument onto all the files in the hierarchy below the command-line directory argument. #include #include #include #include #define LASTCHR(s) s[strlen(s)-1] int do_nothing(name) char *name; { return 0; } int printfile(name) char *name; { printf("%s\n", name); return 0; } int suidfile(name) char *name; { struct stat stbuf; stat(name, &stbuf); return (stbuf.st_mode & S_ISUID); } int mapdir(name, predfn, thenfn, elsefn) char *name; int (*predfn)(), (*thenfn)(), (*elsefn)(); { DIR *dirp; struct dirent *entry; struct stat stbuf; char olddir[MAXPATHLEN]; char cname[MAXPATHLEN]; getwd(olddir); strcpy(cname, name); if (chdir(name)) { printf("Couldn't change to directory %s\n", name); return -1; } if ((dirp = opendir(name)) == NULL){ printf("Unable tto open DIR %s\n", name); chdir(olddir); return -1; } for (entry=readdir(dirp); entry != NULL; entry=readdir(dirp)) { if (0 != stat(entry->d_name, &stbuf)) printf("Can't read file information %s\n", entry->d_name); else if (strcmp(entry->d_name, ".") == 0 strcmp(entry->d_name, "..") == 0) /* don't pursue these entries */ ; else if (S_ISLNK(stbuf.st_mode)) /* don't pursue links */ ; else if (S_ISDIR(stbuf.st_mode)) { if (LASTCHR(cname) != '/') strcat(cname, "/"); strcat(cname, entry->d_name); mapdir(cname, predfn, thenfn, elsefn); strcpy(cname,name); } else if ((*predfn) (entry->d_name)) (*thenfn) (entry->d_name); else (*elsefn) (entry->d_name); } closedir(dirp); chdir(olddir); return 0; } char *setupdir(buf, name) char *name; char *buf; { char curdir[MAXPATHLEN]; getwd(curdir); if (name[0] == '/') strcpy(buf, name); else { strcpy(buf, curdir); if (buf[strlen(buf)-1] != '/') strcat(buf, "/"); /* may be at root */ strcat(buf, name); } return buf; } int main(argc, argv) int argc; char **argv; { char *predfn, *thenfn, *elsefn; char name[MAXPATHLEN]; if (argc < 2) { printf("Usage: %s ",argv[0]); return -1; } setupdir(name, argv[1]); predfn = (char *) suidfile; thenfn = (char *) printfile; elsefn = (char *) do_nothing; return mapdir(name, predfn, thenfn, elsefn); } /* End of File */ A Natural Language Processor Russell Suereth Russell Suereth has been consulting for over 12 years in the New York City and Boston areas. He started coding and designing systems on IBM mainframes and now also builds PC software systems. You can write to Russell at 84 Old Denville Rd, Boonton, NJ 07005, or call him at (201) 334-0051. Hardware and software are sold with CD-ROMs, sound boards, and speech synthesizers. These components allow the user and the computer to interact in a human manner. This stage of user and computer interaction can be extended further with human language. The natural language processor presented in this article is a practical application of the use of human language. Processing Human Language A natural language processor takes a human statement, analyzes it, and responds in a way that appears human. But this appearance isn't a mystery. A natural language processor performs certain processes based on the words in the input sentence. In many ways, processing human language is just another data-processing function. The input sentence is the input transaction record. The words in the sentence are fields in the input record. The dictionary is a master file of word information. The meaning is derived from a combination of information in the input sentence, the dictionary, and program code. The generated response to the sentence is the output. Some simple natural language processors process one-word commands that don't require much analysis, such as find, up-date, and delete. This type of processor uses a small dictionary to identify the commands. When used with a database, commands to such a processor could execute functions to find, update, and delete records. A similar processor can be connected to a remote-controlled car and used with the commands forward, reverse, and stop. Other natural language processors are more sophisticated and can process multiple input sentences. A larger dictionary contains information about the word such as whether it is a noun, a preposition, or a verb. This complex processor can analyze a sentence, identify parts of the sentence, derive meaning from the sentence, and generate an appropriate response or action. Theoretically, a complex processor may parse through a book and derive meanings and themes from the sentences. It may then generate a response based on these meanings and themes. Building a Processor The natural language processor presented here is a miniature version of the complex processor just described. It can process several English input sentences such as Jim is running in the house, Bill is walking on the street, and Sue is going to a store. It can also process sentences such as Where was Bill walking? and generate a response that states Bill was walking on the street. An actual session with the processor is shown in Figure 1. This miniature processor, however, has a limited capability. It can only be used with the kinds of input sentences shown in the figure. But this capability can be increased by expanding existing routines and adding new ones. Some areas of possible expansion will be indicated throughout this article. A natural language processor must perform some amount of analysis on the sentence and the words in it. The analysis may be simple and only identify that the words are valid by matching them with the dictionary. Or, it may be complex and identify structures, phrases, meanings, and themes in the sentence. The analysis this particular processor performs identifies phrase structures and meanings. This processor uses transformational grammar to identify underlying phrase structures in the input sentence. Each word in the sentence has a type such as noun or verb. Specific combinations of these types make a certain phrase structure. For example, the words a, the, and these are determiners and are located right before a noun. A determiner-noun combination such as the house is a kind of noun phrase. Phrase structures must be identified in order to identify sentence elements such as subject, action, and place. Figure 2 shows some abbreviated phrase structures used in transformational grammar. There are a limited number of underlying phrase structures in transformational grammar. But there are an unlimited number of possible sentences. The relationship of structures to possible sentences is shown with the two sentences Jim is running in the house and Sue is going to a store. The underlying structure (name-auxiliary-verb-preposition-determiner-noun) is the same in these two examples, although the words make the two sentences different. The limited number of structures can be coded in the program without coding the many combinations of words that may occur. Transformational grammar also defines how words are used together or used in a certain context. For example, the woman drove the car is correct but the table drove the car is not. Words have restrictions to determine how they can be used together properly. The inanimate object table cannot be used with the action verb drove. Only humans can drive so a HUMAN restriction should be assigned to the verb drove. Additional restrictions of MAN, WOMAN, PERSON, and TEENAGER should also be assigned. The natural language processor in this article uses only one restriction labeled ING for words such as going, running, and walking. An expanded natural language processor may use many additional restrictions. One of these is verb tense so that Jim was runs in the house is recognized as an incorrect sentence. Processing the Sentence Listing 1 contains the natural language processor. The main routine first calls the initialize routine. This routine initializes some variables that are used throughout the program. Each entry in the subjects, actions, and places arrays is initialized. The first entry contains the subject, action, and place of the first input sentence. The second entry contains this information for the second input sentence. The arrays have 20 entries and can contain information from 20 input sentences. Next, the main routine opens the dictionary file named diction. If the dictionary opened successfully, the main control flow of the program is entered. The main control flow of the program is a while statement that loops once for each input sentence. Several processes occur within the while statement for each sentence. From a broad view, the program extracts the sentence words and matches them with the dictionary, which contains information about the word. The program loads this word information into arrays that it analyzes to determine the underlying structure of the sentence. The program identifies the subject, action, and place words in the sentence and copies these words to their arrays, which contain an entry for each input sentence. The program determines the appropriate response and then generates that response. When there are no more input sentences the program closes the dictionary and then ends. From a detailed view, the while loop executes until the input sentence is null, that is, when only Enter is pressed on the keyboard. First, the program calls the reset_sentence routine. This routine initializes variables that are used for each input sentence. reset_sentence initializes the word_ct variable, which will contain the number of words in the sentence. reset_sentence also initializes array entries. These arrays contain an entry for each word in the sentence, allowing up to 10 words in an input sentence. The type_array contains five additional entries for each word, allowing up to five possible types for the word. Examples of types are noun, preposition, and verb. The main routine then parses through the input sentence to extract each sentence word. For each word in the sentence main calls the get_record routine and increments word_ct. The get_record routine reads each record in the dictionary and calls the match_record routine to determine whether the dictionary word matches the sentence word. If they match, then the types variable is incremented to accommodate another type for the same word. The dictionary can contain multiple types for the same word. The word jump, for instance, can be a noun or a verb and would have a dictionary record for each type. When the end of the dictionary file has been reached, the types variable is checked to see whether the word was found. If the word wasn't found and the first character of the word is uppercase, then the word is a name. Names such as Jim and Sue aren't kept in the dictionary. The word is then copied to word_array for later processing. The routine can be modified to find the word faster by changing the dictionary to an indexed file. The dictionary appears in Listing 2. It is the same dictionary used in the session shown in Figure 1. get_record calls the match_record routine for each record in the dictionary. match_record compares the passed sentence word with the word in the current dictionary record. match_record extracts the word from the dictionary with the extract_word routine, then it matches the extracted dictionary word with the passed word. If the match is successful, then the type is extracted from the dictionary record and copied to type_array. If the type is a verb, then the root is extracted from the dictionary with the extract_root routine and copied to root_array. In a group of similar words such as run, runs, ran, and running, the word run is the root. Each verb in the dictionary has a root. The root will later identify a group of similar words that may be used in a generated response sentence. An expanded natural language processor that generates many different responses would find the root invaluable. For example, given the input sentence Jim is running in the house, a generated response may be Why does Jim run in the house? or It appears that Jim runs often. The response words run and runs can be identified in the dictionary through the common root run. The Underlying Structure The check_underlying routine identifies underlying phrase structures in the input sentence. The code shows two specific underlying structures that can be identified. The first underlying structure is a question sentence that starts with a WH word. A WH word is a word such as where or what that starts with the letters wh. The next word in the input sentence must be an auxiliary which is labeled AUX. The next word must be a name, and the last word must be a verb. This underlying structure has the types: WH-AUX-NAME-VERB. It can be used for many similar sentences such as Where was Bill walking and Where was Sue going. The check_underlying routine calls check_type which compares the passed type with the possible types in type_array. The type_array variable holds the possible types for each input sentence word. If the first input sentence word has a WH type, then it matches the structure for the first word. Each word in the input sentence is checked to see if it matches the type in the structure. If all the input sentence words match, then the sentence has the underlying structure WH-AUX-NAME-VERB. Once the underlying structure is matched, the correct type is copied to prime_types. The prime_types array identifies that the word jump, for example, is a noun rather than a verb. Verbs refer to actions and nouns refer to places. This type identification will be used later to identify words in the sentence that refer to an action or a place. Next, the kind of phrase is assigned. Auxiliaries and verbs are assigned to verb phrases, determiners and nouns are assigned to noun phrases, and prepositions are assigned to prepositional phrases. Phrases are combinations of specific, adjacent word types. For instance, a noun phrase has a DET-NOUN combination. Phrase identification will be needed later to locate words in the sentence that identify a place. For example, a place can be identified by the prepositional phrase in the house. In an expanded natural language processor, phrase identification would be increased to several processes. What initially looks like a noun phrase such as the house may be a prepositional phrase such as in the house after additional sentence analysis. The second underlying structure that can be identified has the types NAME-AUX-VERB-PREP-DET-NOUN. This structure can be used for sentences such as Bill is walking in the street and Sue is going to a store. The two coded underlying structures will only accept input sentences such as Sue is going to a store and Where was Sue going? Other kinds of sentences can be processed when other underlying structures are coded. Additional underlying structures that can be coded are shown in Figure 2. Notice that the two coded structures aren't shown. These two structures were created only for explanatory purposes instead of the more lengthy code required for all the underlying structures. In an expanded natural language processor, the transformational grammar structures would be coded. For example, one coded structure would be DET-NOUN to identify a kind of noun phrase. Identifying Sentence Elements The elements of subject, action, and place help convey the meaning in the input sentence. Subject identifies who or what the sentence is about, action identifies the activity that is performed, and place identifies where the activity occurred. In the input sentence Sue is going to a store, the subject is Sue, the action is going, and the place is to a store. Without the identification of these elements, the sentence would be merely composed of meaningless words, types, and phrases. Three routines in the processor identify the sentence elements. The check_subject routine looks at each word in the input sentence. If the word is a name, then check_subject copies the word to the subjects array, which contains a subject entry for each input sentence. The check_action routine also looks at each word in the input sentence. If the word is a verb, then check_action copies the root of the word to the actions array, which contains an action entry for each input sentence. The root will be useful when expanding the processor. It will allow the processor to determine that Jim ran on the street and Jim runs on the track are similar actions. It will also allow an appropriate form of run to be used in a response statement. For example, Jim ran should be used to describe past tense and Jim runs to describe present tense. The check_place routine looks at each word in the input sentence too. If the word is in a prepositional phrase, then check_place concatenates the word to the places array, which contains a place entry for each input sentence. Each word in the prepositional phrase refers to a place and will be concatenated to the places array. With this information, a simulated understanding of the sentence can be derived. The processor does this by matching the subject and action words in the current input sentence with information in previous sentences. For example, one input sentence can be Jim is running in the house. The processor will place Jim in the subjects array, run in the actions array, and in the house in the places array. A later input sentence can be a question that asks, Where was Jim running? The processor will identify Jim as the subject and run as the action. Since this is a question, the processor will also search the subjects array and actions array for the words Jim and running. When a match is found, the corresponding places array will be used to create a response that states, Jim was running in the house. If a person saw only the input sentences and responses as shown in Figure 1, then the processor would appear to have some degree of understanding. But this is only an appearance. The processor generates a canned response of words that are based on a combination of input sentence words and information in the arrays. An expanded natural language processor can contain code for a number of responses. It can also contain the routines check_manner and check_time to identify how and when something occurred. These two routines should allow prepositional phrases to identify elements of place such as in my house as well as elements of manner and time such as in my joy and in the morning. Generating a Response The response is the output statement from the natural language processor. After reading and processing an input sentence, the natural language processor must generate an appropriate response in acknowledgement. When a person speaks, the listener responds to let the speaker know the words were heard and understood. The response may be a simple nod of the head or several sentences that explain the listener's understanding. Two considerations that determine the kind of response are the listener's knowledge of the spoken information, and whether the spoken words were a question or a statement. The make_response routine uses these two considerations to generate responses to the input sentence. First, it checks to see whether the first word in the input sentence is Where. If it is, then the input sentence is a question. The second consideration is whether the processor has knowledge of information in the input sentence. The processor has this knowledge when the information exists in sentence elements from previous sentences. In the question Where was Jim running, the subject is Jim, and the action is running. Since it's a where-question, it's asking for a location associated with the words Jim, and run which is the root of running. The processor keeps information from previous sentences in the subjects, actions, and places arrays. The places array contains locations. The make_response routine searches the subjects and actions arrays for a match with Jim and run. When it finds a match the associated entry in the places array will contain the information to generate a response that states where Jim is running. When the input sentence is a where-question, and make_response does not find Jim and run, the processor doesn't have enough information to indicate Jim's location. The routine then moves the statement I don't know to the response variable. When the input sentence is not a question, it is simply a statement of fact. The routine then moves Ok to the response variable. Other kinds of responses can be coded and generated. For example, an array of You don't say, Please go on, and Tell me more statements can be coded and used as responses. When the input sentence is a where-question, and make_response finds Jim and run, the make_response routine calls the make_answer routine. make_answer creates an answer by placing the associated subject, action, and place words together in the response variable. The make_answer routine is passed an array index that relates the appropriate entries in the subjects and places arrays. First, the routine copies the appropriate subject to the response variable giving Jim. It then concatenates was to the variable to give Jim was. Next, it calls the get_verb_ing routine to retrieve the ING version of the action word. The ING version is a word such as running or walking. The ING verb must be used in the response because other selections of Jim was runs and Jim was ran are incorrect. The get_verb_ing routine reads each record in the dictionary file. It calls the match_verb_ing routine to determine whether the record contains the correct ING verb. The correct ING verb has an ING restriction. The correct ING verb also has a root that matches the action in the input sentence. If match_verb_ing finds the correct ING verb, the get_verb_ing routine concatenates it to the response giving Jim was running. Finally, the make_answer routine concatenates the appropriate place words to the response resulting in Jim was running in the house. Expanding the Processor There are several ways this natural language processor can be expanded. The simplest would be to add words to the dictionary. Adding words would enable more words to be used in the coded, underlying structures. The number of generated responses may increase when words are added to the dictionary. More words cause more possible word combinations which allow more possible generated responses. The additional words, though, will require word restrictions that define how the words can go together properly. When expanding the processor in this way, expect to add restrictions to the dictionary and to enhance the program code that processes the restrictions. Another way to expand the processor is to consider the sentence tense. Basically, a sentence refers to something in the past, present, or future. Sentence tense affects the words used in the sentence as well as its context and meaning. In this miniature processor, the auxiliary that helps define the tense is ignored. The generated response is even hard-coded with the auxiliary was. The first step in this expansion would be to add a tense restriction in the dictionary for auxiliary and verb words. For example, the word is would have a present-tense restriction and will a future-tense restriction. Next, the code would have to be expanded to accommodate the several kinds of auxiliary structures. For example, auxiliaries can occur as have, could have, or could have been. An overall auxiliary tense would have to be derived from these individual auxiliary words. A verb with the appropriate tense can be retrieved from the dictionary after the overall auxiliary tense has been determined. Expanding this natural language processor will provide additional human language capabilities. But not all processors require the same capabilities. The capabilities that are needed for a given processor depend, in part, on the kinds of input sentences and words that are expected. This natural language processor, presented in its current form, doesn't read a book or answer questions about the book. But expanding this processor will make that capability possible. (C) 1993 by Russ Suereth. Figure 1 A processor session Sentence: Jim is running in the house Response: Ok Sentence: Sue is going to a store Response: Ok Sentence: Bill is walking on the street Response: Ok Sentence: Where was Jim running Response: Jim was running in the house Sentence: Where was Bill going Response: I don't know Sentence: Where was Sue going Response: Sue was going to a store Figure 2 Phrase structures Sentence = Noun Phrase + Verb Phrase Noun Phrase = Pronoun, or Name, or Noun, or Determiner + Noun Verb Phrase = Verb, or Verb + Place, or Verb + Manner, or Verb + Time, or Verb + Reason, or Auxiliary + Verb, or Auxiliary + Verb + Place, or Auxiliary + Verb + Manner, or Auxiliary + Verb + Time, or Auxiliary + Verb + Reason The following examples are words that may be found in the phrase structures: Pronoun = he, she, they, it Name = Jim, Florida, Moby Dick Common Noun = house, store, street Determiner = a, the, that, those Verb = run, walk, go, read Auxiliary = is, was, have, did, could Place = in the house, to a store Manner = with ease, quickly Time = in the morning, at noon Reason = due to the snow, since it rained Listing 1 NATURAL.C - A natural language processor /* copyright 1993 by Russ Suereth */ #include #include #include #include #define ING 73 /* Restriction for ING word */ void initialize(void); void reset_sentence(void); void get_record(char *); char *extract_word(void); int match_record(char *, int); char *extract_root(void); void check_underlying(void); int check_type(char *,int); void check_subject(void); void check_action(void); void check_place(void); void make_response(void); void make_answer(int); void get_verb_ing(void); int match_verb_ing(void); FILE *infile; char dic_record[80]; int sentence; int word_ct; char word_array[10] [15]; char root_array[10][15]; char prime_types [10] [11]; char phrases [10] [11]; char type_array[10][5][11]; char subjects [20] [15]; char actions[20][15]; char places[20][31]; char response[80]; void main() { char *cur_word; char in_sentence[80]; initialize(); if ((infile = fopen("diction", "r+")) == NULL) { printf ("\nError opening dictionary\n"); exit(0); } printf("\nSentence: "); while(gets(in_sentence)) { if (in_sentence[O] == '\0') break; reset_sentence(); cur_word = strtok(in_sentence, " "); while(cur_word != NULL) { get_record(cur_word); cur_word = strtok(NULL," "); if (++word_ct > 9) break; } check_underlying(); check_subject(); check_action(); check_place(); make_response(); printf("Response: %s\n\nSentence: ", response); if (++sentence > 19) break; } /* end while */ fclose(infile); return; } /*****************************************************/ /* Initialize variables (subjects, actions and */ /* places arrays contain entries for 20 sentences). */ /*****************************************************/ void initialize() { int i; for (i=0; i<20; i++) { subjects[i][0] = '\0'; actions [i][0] = '\0'; places[i][0] = '\0'; } sentence = 0; return; } /****************************************************/ /* These variables are initialized for each new */ /* input sentence (each of the 10 word entries for */ /* the input sentence has 5 type_array entries). */ /****************************************************/ void reset_sentence() { int i,j; word_ct = 0; for (i=0; i<10; i++) { word_array[i] [0] = '\0'; root_array[i][0] = '\0'; prime_types[i] [0] = '\0'; phrases[i][0] = '\0'; for (j=0; j<5; j++) type_array[i][j][0] = '\0'; } return; } /****************************************************/ /* Get all the records from the dictionary. If the */ /* passed word is not in the dictionary, then the */ /* word could be a name. */ /****************************************************/ void get_record(char *pass_word) { int types = 0; rewind (infile); fgets(dic_record, 80, infile); while (! feof(infile)) { if (match_record(pass_word, types) == 0) types++; fgets(dic_record, 80, infile); } if (types == 0) { if (isupper( (int) pass_word[0])) strcpy(type_array[word_ct][types], "NAME"); else strcpy(type_array[word_ct][types], "NOTFOUND"); } strcpy(word_array[word_ct], pass_word); return; } /*******************************************************/ /* Compare the passed word with the word in the */ /* current dictionary record. If they are the same, */ /* then extract the type (NOUN, VERB, etc.). If the */ /* type is a VERB, then also extract the root and */ /* and copy it to the root array. */ /*******************************************************/ int match_record(char *pass_word, int types) { int i, j; char *root; char *dic_word; dic_word = extract_word(); /* Check if passed word equals dictionary word */ if (strcmpi(pass_word, dic_word) != 0) return(1); /* Word found, get the type */ for (i=14,j=0; i<20; i++) { if (isspace(dic_record[i])) break; type_array[word_ct][types][j++] = dic_record[i]; } /* Trim the type */ type_array [word_ct][types][j] = '\0'; if (strcmp(type_array[word_ct][types], "VERB") == 0) { root = extract_root(); strcpy(root_array[word_ct], root); } return(0); } /******************************************************/ /* Extract the word from the dictionary. The word is */ /* 14 characters in length and starts in column 1. */ /******************************************************/ char *extract_word() { int i, j; char dic_word[15]; for (i=0,j=0; i<14; i++) { if (isspace(dic_record[i])) break; dic_word[j++] = dic_record[i]; } /* Trim the dictionary word */ dic_word[j] = '\0'; return(dic_word); } /******************************************************/ /* Extract the root from the dictionary. It */ /* identifies a group of similar words (the root for */ /* run, ran, runs and running is run). It is 14 */ /* characters in length and starts in column 35. */ /******************************************************/ char *extract_root() { int i, j; char root[15]; for (i=34,j=0; i<48; i++) { if (isspace(dic_record[i])) break; root[j++] = dic_record[i]; } /* Trim the root */ root[j] = '\0'; return(root); } /******************************************************/ /* Determine if the input sentence contains a known, */ /* underlying structure. If it does, then assign the */ /* correct types and phrases for the words. */ /******************************************************/ void check_underlying() { int i; /* Structure WH-AUX-NAME-VERB */ i = 0; if ( (check_type("WH", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("NAME", i+2) == 0) && (check_type("VERB", i+3) == 0) ) { strcpy(prime_types[i], "WH"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "NAME"); strcpy(prime_types[i+3], "VERB"); strcpy(phrases[i], "WHQUESTION"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "NOUNPHRASE"); strcpy(phrases[i+3], "VERBPHRASE"); return; } /* Structure NAME-AUX-VERB-PREP-DET-NOUN */ if ( (check_type("NAME", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("VERB", i+2) == 0) && (check_type("PREP", i+3) == 0) && (check_type("DET", i+4) == 0) && (check_type("NOUN", i+5) == 0) ) { strcpy(prime_types[i], "NAME"); strcpy(prime_types [i+1], "AUX"); strcpy(prime_types [i+2], "VERB"); strcpy(prime_types[i+3], "PREP"); strcpy(prime_types[i+4], "DET"); strcpy(prime_types[i+5], "NOUN"); strcpy(phrases[i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "VERBPHRASE"); strcpy(phrases[i+3], "PREPPHRASE"); strcpy(phrases[i+4], "PREPPHRASE"); strcpy(phrases[i+5], "PREPPHRASE"); return; } return; } /****************************************************** /* Compare the passed type with all the types for */ /* this word in the type_array. If the type is */ /* found, then return 0. The pass_number parameter */ /* identifies the word in the input sentence. */ /*****************************************************/ int check_type(char *pass_type, int pass_number) { int i; for (i=0; type_array[pass_number][i][0]; i++) { if (strcmp(type_array[pass_number][i], pass_type) == 0) /* Passed type is found in array */ return(0); } /* Passed type is not found in array */ return(1); } /*****************************************************/ /* If the correct type is "NAME", then the word */ /* refers to a subject so copy the word to the */ /* subjects array. */ /*****************************************************/ void check_subject() { int i; for (i=0; i= 0; i--]) { if ( (strcmp(subjects[i], subjects[sentence]) == 0) && (strcmp(actions[i], actions[sentence]) == 0) && (strlen(places[i]) != 0) ) { make_answer(i); return; } } /* Not enough information in actions and */ /* subjects arrays. */ strcpy(response, "I don't know"); return; } /****************************************************/ /* Generate a response that states the location of */ /* where the subject and action occured. */ /****************************************************/ void make_answer(int prev_sentence) { strcpy(response, subjects[prev_sentence]); strcat(response, " "); strcat(response, "was "}; get_verb_ing(); strcat(response, places[prev_sentence]); return; } /****************************************************/ /* Retrieve the ING version of the word from the */ /* dictionary (the ING version of run is running). */ /****************************************************/ void get_verb_ing() { rewind (infile); fgets(dic_record, 80, infile); while (! feof(infile)) { if (match_verb_ing() == 0) break; fgets(dic_record, 80, infile); } return; } /******************************************************/ /* If the root in the current dictionary record */ /* matches the root in the actions array, and the */ /* current dictionary record has an ING restriction, */ /* then extract the dictionary word and return O. */ /******************************************************/ int match_verb_ing() { int i; char *root; char *dic_word; root = extract_root(); if (strcmp(actions[sentence],root) == 0) { /* Root found, look for ING restriction */ for (i=24; i<33; i++) { if (isspace(dic_record[i])) break; if (dic_record[i] == ING) { dic_word = extract_word(); strcat(response, dic_word); return(0); } } } return(1); } /* End of File */ Listing 2 DICTION -- the dictionary file /* copyright 1993 by Russ Suereth */ a DET the DET house NOUN street NOUN store NOUN jump NOUN go VERB go goes VERB go going VERB I go went VERB go run VERB run runs VERB run running VERB I run ran VERB run walk VERB walk walks VERB walk walking VERB I walk walked VERB walk jump VERB jump jumps VERB jump jumping VERB I jump jumped VERB jump is AUX was AUX to PREP in PREP on PREP where WH Mixed Numbers in C P.J. LaBrocca P.J. LaBrocca is the author of ReCalc(TM), a set of rational expression calculators that never give answers (well, almost never), and run identically on PCs, Macintoshes, and Apples. He has a BS and MA in Chemistry and teaches computer science at Peter Rouget Middle School 88 in Brooklyn, NY 13782, (607) 746-7175. Introduction This article describes a method of representing and manipulating mixed numbers in C. A simple mixed number calculator, MixCalc, exercises the functions. You can use MixCalc to help you do your kids' fractions homework. The first part presents the package for working with mixed numbers in a C program. It describes mixed_t, the data structure used to represent mixed numbers, and the functions that do calculations with them. The discussion then turns to MixCalc, a rational number calculator (Listing 1 and Listing 2). MixCalc accepts infix expressions involving any combination of whole numbers, fractions and mixed numbers and returns an answer as either a whole number, a fraction, or a mixed number. For example, given this input 3 23 / ( 12 + 2 ) MixCalc returns 1 715 A vertical bar, , is used as the fraction bar to differentiate it from the division sign. I based the code for MixCalc's parser (Listing 3) on the recursive-descent parser from Bjarne Stroustrup's The C++ Programming Language. I modified it to work with mixed_ts and added some error recovery. A rational number is any number that can be expressed as the quotient of two whole numbers. Integers are rational numbers. A proper fraction's value is less than one. An improper fraction's value is equal to or greater than one. A mixed number is the sum of an integer and a proper fraction. Since a mixed number can be expressed as an improper fraction it is a rational number. I use the term mixed number loosely to refer to any rational number. Most programming languages do not provide built-in support for fractions or mixed numbers. You approximate mixed numbers as decimal numbers using floating-point types. For most applications this is fine. However, mixed numbers have one big advantage over decimal numbers--they are always exact. Using decimal numbers, 1 divided by 3 results in the approximation 0.33333...; the mixed number result is exactly 13. Of course, there is a down side. In certain calculations the integers used for the numerator and denominator can overflow. However, in an application using, say, the English measuring system, five-eighths looks a lot better as 58 than as 0.625. Mixed Number Package The data structures and functions for mixed numbers are in Listing 4 - Listing 8, fraction.c, mixed.h, error.c, mix_num2.c, and primes.c. Since I wanted the intermediate results of calculations available for display (for an application not discussed here) I provided storage for each possible component of a mixed number. Integer types are used for the whole number part, the numerator part, the denominator part and the sign. Fractions must be factored into primes in order to reduce them. The factors of the numerator and denominator are stored in a two-dimensional array of integers. Including the arrays rather than pointers to dynamically-allocated storage simplifies the code at the expense of stack size. At compile time, be sure to provide a generous stack. Include mixed.h in each file that uses the functions, and call init_primes once before doing any calculations. init_primes sets up a table of prime numbers using the sieve of Eratosthenes. Declare mixed numbers using mixed_t. It's a good idea to initialize mixed_ts as soon as possible. You assign values by calling mix_init or mix_clear. mix_init accepts a pointer to a mixed_t, and three integer values for the whole number, numerator, and denominator parts of the mixed number. The sign of the mixed_t is determined by the first integer; the others should be positive. mix_clear sets the mixed number to zero. Both functions truncate the arrays holding the prime factors of the numerators and denominators. The typedef Integer controls the size of the int used to store the parts of mixed_ts. Use any size as long as it's signed. Naturally, larger ints make overflow less likely, but take more room and time. The output function mix_print, and any input functions you supply, need adjustment if you change the size of Integer. The sign of a mixed_t is stored in an int as +1 or -1. It is used in intermediate calculations. Notice that a mixed_t with value zero has its whole number and numerator parts set to zero, but its denominator part is set to one. A denominator is also a divisor, and division by zero is meaningless. Also note that a fraction that has not yet been factored has the first elements of its factors array set to one. The functions for manipulating mixed_ts are in fraction.c. They are used by the functions that perform arithmetic operations on mixed numbers, and for converting mixed numbers to equivalent forms. To reduce fractions to lowest terms you need to know the prime factors of the numerator and denominator. These are provided by a call to mix_factor. mix_factor takes the address of a mixed number and puts its prime factors into the factors array. The whole number part is not considered. mix_factor is called by mix_reduce, which returns a pointer to a mixed number with the fraction part in lowest terms. mix_reduce loops through the two arrays of factors comparing them element by element, and either ignoring them or multiplying them into temporary variables. For example, consider reducing the fraction 4290. After the call to mix_factor the arrays look like this: Numerator 2 3 7 1 Denominator 2 3 3 5 1 The pointers top and bottom point to the prime factors under consideration, one from the numerator, the other from the denominator. If the elements are equal, the factors are common. Advancing the pointers cancels the common factors. In the example, the two's, and then the three's are ignored. When the factors are unequal the smaller is multiplied into its temporary variable and its pointer is advanced. This continues until the sentinel, i.e., an element equal to one, is encountered in one or both arrays. Any remaining factors are multiplied into the corresponding temporary variable. The rest of the function adjusts the whole number part, if needed, and makes sure the mixed number is in a consistent state if it started out equal to zero. An improper fraction is one in which the numerator is greater than or equal to the denominator. In the case of the mixed numbers developed here this means that the whole number part must be zero. mix_make_improper converts a mixed_t into an improper fraction. This function is called by mix_add and mix_sub. You can it call before mix_print to display a mixed_t as an improper fraction. mix_print displays a mixed number on standard output in a reasonable form within the limits of a text-based environment. Some details are presented later. mix_num2.c contains the functions for doing arithmetic with mixed numbers. The four basic operations are provided, plus negation and calculating the lowest common denominator. Addition and subtraction are accomplished by converting the mixed numbers to improper fractions, bringing them to a common denominator and adding the numerators. For multiplication the numerators are multiplied, then the denominators. The division function takes the reciprocal of the second mixed number and calls the multiplication function. The possibility of taking the reciprocal of zero presents a complication. As it stands, mix_recip complains if the denominator will become zero. Otherwise it returns the reciprocal. mix_recip does not alter its argument. Your program should check if a mixed number is zero before dividing by it. The arithmetic functions return unreduced results. mix_neg reverses the sign of a mixed_t. mix_lcd returns the lowest common denominator of two mixed_ts. If during calculations the numerator becomes zero, the denominator gets set to one. If the value of a mixed number becomes zero the sign is set to positive. These actions ensure a consistent starting point for further calculations. A greatest_common_divisor function could be used to "pre-reduce" fractions during calculations. However, taking out gcds does not guarantee lowest terms. To ensure lowest terms mix_reduce still has to be called. There might be some practical advantage, say, when adding up a long list of fractions if you avoid overflow. However, 61% of the time random number pairs have a gcd equal to one. Most of the time calling a gcd function adds overhead but little else. Note that mix_add and mix_sub alter the representation, but not the value, of their arguments. To preserve the original representation copy it to another mixed_t. Limits Storing numbers in computer memory has limitations. Range and domain errors are reported for many calculations involving floating-point types. For integer types errors may or may not be reported. If you are lucky bizarre results let you know something is amiss. When working with mixed_ts here are some limits to keep in mind. Whole numbers, numerators and denominators are stored in longs. If the size of a calculation's result doesn't fit in a long the result is wrong. You might get a run-time error or garbage. The functions do not take any precautions. The largest prime number in the prime number table is 15991. Numbers with prime factors bigger than that produce all kinds of annoying errors. Some examples are given later. The factors array holds 40 prime factors, a generous amount. If more need to be stored other memory gets over-written with the usual unpredictable results. Actually you might consider reducing the size. The mixed_ts in this version are big so stack overflow is likely. Increase the amount of stack space allocated at compile and link time as needed. Scanning a Mixed Number MixCalc's parser is based on the one presented in Stroustrup's The C++ Programming Language. I discuss only those parts I modified for MixCalc. The main alteration was to the function than scans the input. For MixCalc the scanning function, gettok (Listing 9), has to recognize mixed numbers in their varying guises. gettok reads the standard input, breaks it up into tokens, returns the tokens to the parser and constructs a mixed_t. If it detects an unrecognized token it issues a diagnostic to standard error and resets MixCalc via a call to longjmp. The setjmp/longjmp combination provides a good method of error recovery for a calculator program where an error in the input stream makes the rest of the expression meaningless. A variable of type jmp_buf is defined and a call is made to setjmp before entering the main loop. If longjmp is called later on the environment is reset to the way it was when setjmp was called. An error in an expression causes the rest of the current line to be discarded. MixCalc resumes on the next line. Scanning a mixed number presents a problem similar to scanning a floating-point number. Valid input can come in several forms. An acceptable mixed number may consist of a whole number, a fraction, or a whole number followed by a fraction. An optional sign may precede any of these. An irksome part of scanning a mixed number is that it is natural to have space embedded in it. gettok tackles these problems. Positive and negative signs are treated as operators, not as part of the mixed number. They are passed to the parser. When gettok encounters a digit it pulls in an integer using scanf. Then white space is skipped. The next character can be a digit, a fraction bar, or some other character (an operator or an unknown character). If the next character is a digit then the scanner expects an integer, optional white space, a fraction bar, optional white space, and another integer. Otherwise an error message is displayed and MixCalc resets itself. If the next character is a fraction bar then an integer preceded by optional white space must follow. Otherwise an error as above is generated. For each of the three possible successful scans gettok constructs a mixed_t and returns the token NUMBER. Note that the fraction bar is not an operator in MixCalc, rather it is a delimiter. Compiling MixCalc MixCalc compiles as presented here with Microsoft C 6.00 or C/C++ 7.0, and Zortech C/C++ 3.0. A makefile (Listing 10) is provided for building MixCalc using Microsoft and Zortech. It's set to use the Microsoft compilers as shipped. To switch to Zortech move the # on the lines referring to the CC macro and the linker. The file works with Microsoft's NMAKE and Zortech's MAKE utilities. I also use this makefile to backup source files and update hardcopy. For example, make backup copies those files that changed since the last backup to a floppy on drive a:. To use backup and hcopy with Zortech's MAKE add exclamation points before the commands. For Microsoft's PWB create a project called mixcalc. Add the seven C files to the project. Then choose build from the project menu. The PWB does not use the makefile provided with MixCalc. I like to compile everything with the large model, but any model should work. For more information on using MixCalc see the sidebar "How to Use MixCalc." How to Use MixCalc MixCalc is a rational number calculator. It accepts expressions consisting of any combination of whole numbers, fractions, and mixed numbers and returns a result in the appropriate reduced form. +,-, * and / perform addition, subtraction, multiplication, and division, respectively. Parentheses change the order of operations and nest arbitrarily deep. Several expressions, separated by semicolons, may appear on one line. A carriage return ends a line. MixCalc accepts input from the command line or from a redirected file. An error in the input flushes the current line and resets MixCalc. Start MixCalc by typing mixcalc at the DOS prompt. To do a series of calculations from a text file called examples, type mixcalc < examples To save the results of your calculations to a file called results, type mixcalc < examples > results An error causes a message to be displayed and halts the redirected session. Several lines of a MixCalc session with some comments follow. White space is optional except to separate the whole number part from the fraction part of a mixed number. > 12*12 14 >12 * 12 14 > 2 23+1 56 4 12 Parentheses work just as expected. > 5 79 + 11 23 * 3 12 46 1118 > (5 79 + 11 23) * 3 2 61 118 The fraction bar is not an operator; it has meaning only within a fraction. > (1/2)3 Unknown token: > 1/23 1 12 This is acceptable to MixCalc, but might not be what was intended. The above example is "one divided by two-thirds," NOT "one divided by two, divided by 3." One-half over three equals one-sixth: > (1/2)/3 16 End a MixCalc session by hitting control-Z (or control-C). All of the limits of the mixed number package apply to MixCalc. Numbers near the largest prime available to MixCalc are used in some examples to force errors. Here's how two compilers dealt with them. Microsoft C/C++ 7.0 > 115991 + 115991 215991 > 116000 + 115991 - integer divide by 0 > 116001 + 115991 - integer divide by 0 Zortech C/C++ 3.0 > 115991 + 115991 215991 > 116000 + 115991 1 > 116001 + 115991 721426102 1879503890-2113923584 In the first example everything stays within bounds. The numerator becomes 2x15,991 and the denominator 1,5991x15,991. Both fit in longs and have prime factors less than or equal to 15,991. The second example produces the numerator 31,991, which is a prime bigger than the biggest available to MixCalc. Note the denominator, 255,856,000, fits in a long and factors nicely. Listing 1 mixcalc.h - header for MixCalc /* mixcalc.h */ /* Adapted from The C++ Programming Language by Bjarne Stroustrup Modified by P.J. LaBrocca */ #ifndef MIXCALC_H #define MIXCALC_H #include "mixed.h" #include enum token_value { NAME = 1, NUMBER, END, ENDFILE = END, PLUS = '+', MINUS = '-',MUL = '*', DIV = '/', PRINT = ';',ASSIGN = '=', LP = '(', RP = ')' }; extern enum token_value curr_tok; extern mixed_t *M; //extern mixed_t number_value; //parser.c extern jmp_buf startup; //function prototypes mixed_t expr(void); mixed_t term(void); mixed_t prim(void); enum token_value get_token(void); #endif /* End of File */ Listing 2 mixcalc.c - a rational number calculator /* mixcalc.c */ /* Copyright 1992 by P.J. LaBrocca */ #include "mixed.h" #include "mixcalc.h" #include #ifdef __ZTC__ /* For Zortech Compiler */ _stack = 15000; #endif enum token_value curr_tok; void main() { mixed_t ans; mixed_t work; init_primes(); mix_clear( &ans ); mix_init( &work, 3, 1, 7 ); setjmp( startup ); M = &work; while(1) { printf("> "); curr_tok = gettok(); if(curr_tok == END) { break; } if(curr_tok == PRINT) continue; ans = expr( ); mix_print( mix_reduce( &ans) ); } } /* End of File */ Listing 3 parser.c - parser For mixcalc.c /* parser.c */ /* Adapted from The C++ Programming Language by Bjarne Stroustrup Modified by P.J. LaBrocca */ #include "mixed.h" #include "mixcalc.h" #include mixed_t number_value; mixed_t expr(void) { mixed_t left = term(); mixed_t tmp; for(;;) switch(curr_tok) { case PLUS: curr_tok = gettok(); tmp = term(); left = mix_add( &left, &tmp ); break; case MINUS: curr_tok = gettok(); tmp = term(); left = mix_sub( &left, &tmp ); break; default: return left; } } mixed_t term() { mixed_t d; mixed_t left = prim(); for(;;) switch(curr_tok) { case MUL: curr_tok = gettok(); d = prim(); left = mix_mul( &left, &d ); break; case DIV: curr_tok = gettok(); d = prim(); if( (d.whole == 0) && (d.num == 0) ) { fprintf( stderr, "Division by 0\n"); fflush( stdin ); longjmp( startup, 1 ); } left = mix_divide( &left, &d ); break; default: return left; } } mixed_t prim() { struct name *n; mixed_t e; switch(curr_tok) { case NUMBER: number_value = *M; curr_tok = gettok(); return number_value; case MINUS: curr_tok = gettok(); e = prim(); mix_neg( &e ); return e; case PLUS: curr_tok = gettok(); return prim(); case LP: curr_tok = gettok(); e = expr(); if(curr_tok != RP) { fprintf( stderr, "')' expected\n"); fflush( stdin ); longjmp( startup, 1 ); } curr_tok = gettok(); return e; case END: return * mix_init( &e, 1, 0, 1 ); default: fprintf( stderr, "primary expected, found %c\n", curr_tok); fflush( stdin ); longjmp( startup, 1 ); } } /* End of File */ Listing 4 fraction.c - functions for manipulating mixed_ts /* fraction.c */ /* Copyright 1992 by P.J. LaBrocca */ #include #include "mixed.h" mixed_t *mix_init( mixed_t *m, Integer w, Integer n, Integer d ) { m->sign = POSITIVE; m->whole = w; if( w < 0) { m->sign = NEGATIVE; m->whole *= -1; } m->num = n; m->den = d; m->factors[ NUMER ][ 0 ] = 1; m->factors[ DENOM ][ 0 ] = 1; return m; } mixed_t *mix_clear( mixed_t *m ) { m->whole = 0; m->num = 0; m->den = 1; m->factors [ NUMER ][ 0 ] = 1; m->factors [ DENOM ][ 0 ] = 1; m->sign = POSITIVE; return m; } mixed_t *mix_factor( mixed_t *m ) { Integer n; int i; Integer *pi; Integer *pp; for( i = 0; i < 2; ++i) { pp = Primes; /* point to global array of primes */ (i != 0) ? (n = m->den) : (n = m->num); pi = &m->factors[i][0]; while(n > 1) { if( !(n % *pp) ) { /* if there is no remainder */ n = (Integer) (n / *pp); /* factor the prime out of number */ *pi = *pp; /* save the prime */ ++pi; continue; /* try the prime again */ } ++pp; /* next prime */ } *pi = 1; pp = Primes; } return m; } mixed_t *mix_reduce( mixed_t *m ) { Integer tnum = 1, tden = 1; Integer *top = &m->factors[NUMER][0]; Integer *bot = &m->factors[DENOM][0]; if( m->num == 0) { return m; } if( m->den == 1 ) { m->whole += m->num; m->num = 0; return m; } mix_factor( m ); /* got to factor to reduce */ /*accumulators for reduced numerator & denominator*/ while(*top != 1 && *bot != 1) { /* neither factor is sentinel */ if(*top == *bot) { /* if the current factors are equal..*/ ++top; /* ..cancel them & continue */ ++bot; continue; } /* otherwise accumulate the smaller*/ (*top < *bot) ? (tnum *= *top++) : (tden *= *bot++); } while(*top != 1) /* any remaining factors are */ tnum *= *top++; /* multiplied in */ while(*bot != 1) tden *= *bot++; if(tnum == tden) { /*ie, n/d == 1*/ ++m->whole; /*add 1 to whole*/ m->num = 0; m->den = 1; } else if(tnum > tden) { /*improper fraction*/ m->whole += (Integer) (tnum / tden); m->num = tnum % tden; m->den = tden; } else { /*proper fraction*/ m->num == tnum; m->den = tden; } if(m->num == 0) { /* keep zero-valued fractions*/ m->den = 1; /* in consistent state*/ if(m->whole == 0) mix_clear( m ); } return m; } void mix_make_improper( mixed_t *m ) /* converts invoking instance*/ { /* into an improper fraction*/ m->num += m->whole * m->den; /* if possible*/ m->whole = 0; } /* If sizeof( Integer ) changes change %ld */ void mix_print( mixed_t *m ) { printf("\t"); if( m->sign == -1 ) printf("-"); if( m->whole != 0 ) printf("%ld", m->whole); if( m->num != 0 ) printf(" %ld%ld",m->num, m->den); if( (m->whole == 0) && (m->num == 0) ) printf("0"); printf("\n"); } /* End of File */ Listing 5 mixed.h /* mixed.h */ /* Copyright 1992 by P.J. LaBrocca */ #ifndef MIXED_H #define MIXED_H typedef long Integer; #define MAXFACTOR 40 /* maximum number of factors */ #define NUMER 0 /* index for numerator */ #define DENOM 1 /* index for denominator */ #define POSITIVE 1 #define NEGATIVE -1 typedef struct { Integer whole; Integer num; Integer den; int sign; Integer factors[2][MAXFACTOR]; } mixed_t; extern Integer Primes[]; /* space for prime numbers */ mixed_t mix_error(char *s); void init_primes( void ); mixed_t *mix_init( mixed_t *m, Integer w, Integer n, Integer d ); mixed_t *mix_clear( mixed_t *m ); mixed_t *mix_factor( mixed_t *m ); mixed_t *mix_reduce( mixed_t *m ); void mix_make_improper( mixed_t *m ); void mix_print( mixed_t *m ); mixed_t mix_sub(mixed_t *x, mixed_t *y); mixed_t mix_add(mixed_t *x, mixed_t *y); mixed_t mix_mul(mixed_t *x, mixed_t *y); mixed_t mix_recip(mixed_t *f); - mixed_t mix_divide(mixed_t *f, mixed_t *g); Integer lcd(mixed_t *f, mixed_t *g); void mix_neg(mixed_t *f); #endif /* End of File */ Listing 6 error.c /* error.c */ /* Copyright 1992 by P.J. LaBrocca */ #include "mixed.h" #include mixed_t mix_error(char *s) { mixed_t t; fprintf(stderr, "Error: %s\n", s); return *mix_init( &t, 1, 0, 1 ); } /* End of File */ Listing 7 mix_num2.c -- functions for doing arithmetic with mixed numbers /* mix_num2.c */ /* Copyright 1992 by P.J. LaBrocca */ #include "mixed.h" #include mixed_t mix_sub(mixed_t *x, mixed_t *y) { mixed_t sum, xt, yt; mix_clear( &sum ); mix_make_improper(x); mix_make_improper(y); xt.num = x->num * y->den; xt.den = x->den * y->den; yt.num = y->num * x->den; yt.den = y->den * x->den; sum.num = x->sign * xt.num - y->sign * yt.num; if(sum.num < 0) { sum.num = labs(sum.num); sum.sign = NEGATIVE; } sum.den = xt.den; /*xt.den == yt.den at this point*/ if(sum.num == 0) { sum.den = 1; if(sum.whole == 0) sum.sign = POSITIVE; } return sum; } mixed_t mix_add(mixed_t *x, mixed_t *y) { mixed_t sum, t, xt, yt; mix_clear( &sum ); mix_make_improper(x); mix_make_improper(y); xt.num = x->num * y->den; xt.den = x->den * y->den; yt.num = y->num * x->den; yt.den = y->den * x->den; sum.num = x->sign * xt.num + y->sign * yt.num; if(sum.num < 0) { sum.num = labs(sum.num); sum.sign = NEGATIVE; } sum.den = xt.den; /*xt.den == yt.den at this point*/ if(sum.num == 0) { sum.den = 1; if(sum.whole == 0) sum.sign = POSITIVE; } return sum; } mixed_t mix_mul(mixed_t *x, mixed_t *y) { mixed_t product; Integer xn, yn; mix_clear( &product ); xn = x->sign * (x->whole * x->den + x->num); yn = y->sign * (y->whole * y->den + y->num); product.num = xn * yn; product.den = x->den * y->den; if(product.num < 0) { product.num = labs(product.num); product.sign = NEGATIVE; } if(product.num == 0) { product.den = 1; if(product.whole == 0) product.sign = POSITIVE; } return product; } mixed_t mix_recip(mixed_t f) /*reciprocal*/ { /* does not alter f*/ Integer tmp; mix_make_improper( &f ); if(f.num == 0) { mix_error("denominator will become zero"); return f; } tmp= f.num; f.num = f.den; f.den = tmp; return f; } mixed_t mix_divide(mixed_t *f, mixed_t *g) { mixed_t rec = mix_recip( *g ); return mix_mul( f, &rec ); } Integer mix_lcd(mixed_t *f, mixed_t *g) { int i = 0, j = 0; Integer low[30]; Integer *l = low; Integer t; mix_factor(g); mix_factor(f); white(1) { if(f->factors[1][i] == 1) { while(g->factors[1][j] != 1) *l++ = g->factors[1][j]++]; break; } else if(g->factors[1][j] == 1) { while(f->factors[1][i] != 1) *l++ = f->factors[1][i++]; break; } else if(f->factors[1][i] == g->factors[1][j]) { *l++ = f->factors[1][i]; ++i; ++j; } else if(f->factors[1][i] > g->factors[1][j]) { *l++ = g->factors[1][j]; ++j; } else if(f->factors[1][i] < g->factors[1][j]) { *1++ = f->factors[1][i]; ++i; } } *l = 1; t = 1; i = 0; while(low[i] !=1) t *= low[i++]; Return t; } void mix_neg(mixed_t *f) { if(f->sign == NEGATIVE) f->sign = POSITIVE; else f->sign = NEGATIVE; } /* End of File */ Listing 8 primes.c -- function to set up table of prime numbers /* primes.c */ /* Copyright 1992 by P.J. LaBrocca */ #include #include "mixed.h" #define NUM 16000 /* highest number to check for being prime */ Integer Primes[ 2000 ]; void init_primes( void ) { char *mark = malloc(NUM * sizeof(char) ); Integer *pr = Primes; /* point to global array */ Integer j, k; for(j = 0; j < NUM; ++j) mark[j] = 1; /* mark everything prime */ for(j = 4; j < NUM; j += 2) mark[j] = 0; /* scratch off all the even numbers ... */ *pr++ = 2; /* ... except for 2; put it in primes array */ for(j = 3; j < NUM; j += 2) /* check each odd number: */ if(mark[j]) { /* if it's marked... */ *pr++ = j; /* ..record it in array.. */ for(k = j + j; k < NUM; k += j) mark[k] = 0; /* ..and scratch off all its multiples */ } free( mark ); } #undef NUM /* End of File */ Listing 9 gettok.c - scanning function that must recognize mixed numbers in various forms /* gettok.c */ /* Copyright 1992 by P.J. LaBrocca */ #include #include #include "mixed.h" #include "mixcalc.h" #define LookAhead(x) while( (x = getchar()) == '' x == '\t') jmp_buf startup; mixed_t *M; enum token_value CurrentToken; enum token_value gettok( void ) { int c; Integer tmp1, tmp2, tmp3; LookAhead( c ); if( c == EOF ) return CurrentToken = ENDFILE; if( isdigit(c) ) { ungetc( c, stdin ); scanf("%ld", &tmp1); LookAhead(c); if( isdigit(c) ) { ungetc( c, stdin ); scanf("%ld", &tmp2); LookAhead(c); if(c != '') { fprintf( stderr, "Expected ''\n"); fflush( stdin ); longjmp( startup, 1 ); } LookAhead(c); if( !isdigit(c) ) { fprintf( stderr, "Expected a digit\n"); fflush( stdin ); longjmp( startup, 1 ); } ungetc( c, stdin ); scanf("%ld", &tmp3 ); if( tmp3 == 0 ) { fprintf( stderr, "Denominator cannot be zero\n"); fflush( stdin ); longjmp( startup, 1 ); } mix_init( M, tmp1, tmp2, tmp3 ); return CurrentToken = NUMBER; } if( c == ''){ LookAhead(c); if( !isdigit(c) ) { fprintf( stderr, "Expected a digit\n"); fflush( stdin ); longjmp( startup, 1 ); } ungetc( c, stdin ); scanf("%ld", &tmp2 ); if( tmp2 == 0 ) { fprintf( stderr, "Denominator cannot be zero\n"); fflush( stdin ); longjmp( startup, 1 ); } mix_init( M, 0, tmp1, tmp2 ); return CurrentToken = NUMBER; } ungetc( c, stdin ); mix_init( M, tmp1, 0, 1, ); return CurrentToken = NUMBER; } if( c == '\n' c == ';') return CurrentToken = PRINT; switch ( c ) { case '*': case '+': case '-': case '/': case '(': case ')': return CurrentToken = c; default: fprintf( stderr, "Unknown token: %c\n", c); fflush( stdin); longjmp( startup, 1 ); } } /* End of File */ Listing 10 A makefile #makefile for mixcalc.exe #CC =ztc -c -ml CC = cl /c /AL .c.obj: $(CC) $*.c all :mixcal.exe mixcalc.exe :mixcalc.obj fraction.obj parser.obj mix_num2.obj primes.obj gettok.obj error.obj # blink mixcalc+fraction+parser+mix_num2+primes+gettok+error,,, /NOI link mixcalc+fraction+parser+mix_num2+primes+gettok+error,,, /NOI /ST:15000 ; mixcalc.obj : mixed.h mixcalc.h mixcalc.c fraction.obj : mixed.h fraction.c mix_num2.obj : mixed.h mix.num2.c primes.obj : mixed.h primes.c error.obj : mixed.h error.c gettok.obj : mixed.h mixcalc.h gettok.c parser.obj : mixed.h mixcalc.h parser.c backup: backup1 backup2 backup3 backup4 backup1: mixcalc.c fraction.c gettok.c parser.c cp -m $? a:\mixb touch backup1 backup2: mix_num2.c primes.c error.c cp -m $? a:\mixb touch backup2 backup3: makefile mixed.h mixcalc.h #mixctxt cp -m $? a:/mixb touch backup3 backup4: cp -m mixcalc.exe a:\mixb touch backup4 hcopy: hcopy1 hcopy2 hcopy1: mixcalc.c fraction.c gettok.c parser.c mixcalc.h pr -W -e4 -o5 $? > prn touch hcopy1 hcopy2: mix_num2.c primes.c error.c makefile mixed.h #mixctxt pr -W -e4 -o5 $? > prn touch hcopy2 Standard C Formal Changes to C P.J. Plauger P.J. Plauger is senior editor of The C Users Journal. He is convenor of the ISO C standards committee, WG14, and active on the C++ committee, WG21. His latest books are The Standard C Library, published by Prentice-Hall, and ANSI and ISO Standard C (with Jim Brodie), published by Microsoft Press. You can reach him at pjp@plauger.com. Current Status The last meeting of the C standards committees, ISO JTC1/SC22/WG14 and ANSI-authorized X3J11, occurred jointly last December near Dulles Airport. I left that meeting with a warm sense of accomplishment and a humongous amount of homework. Both were a direct result of having the meeting go the way I'd hoped, for a change. In soap operas, senators and board chairpeople always finagle their political goals against all odds. In the real world, we lowly Convenors of standards committees mostly go with the flow. I summarized the administrative highlights in my "Editor's Forum" last issue. (That was in CUJ March 1993, but see also the "Editor's Forum" for January 1993 and for November 1992.) Here again is a brief synopsis of what happened: WG14 finally voted out an amendment to the C Standard. We were charged several years ago by SC22, our parent committee, to produce such a "normative addendum" to correct several perceived flaws in the coverage and expression of the C Standard. With a bit of eleventh-hour compromising, we finally got agreement within WG14. Now the amendment must survive at least two rounds of balloting within SC22 before it becomes formal. SC22 finally established sensible procedures for interpreting and correcting ISO programming language standards. Various authorized agencies issue Defect Reports to the Convenor (me again). She/he (I) must log them, acknowledge them, and submit them to the Working Group to develop a response. A Technical Corrigendum patches the standard, while a Record of Response simply explains the standard. (I helped develop these procedures at the SC22 plenary in Finland last August.) ANSI has adopted the ISO C Standard verbatim as its own, replacing the original C Standard, with slightly different formatting. That was a prelude to having X3J11 start an I-Project to track development of the normative addendum. The English of all this is that responsibility for maintaining the C Standard has passed from ANSI to ISO. X3J11 recognizes that its principal current business is to apply its expertise in interpreting the C Standard. WG14 has requested that X3J11 develop initial interpretations and X3J11 has graciously agreed to do so. WG14 retains ultimate responsibility for publishing the responses, as I described above. For these and other reasons, I have resigned as Secretary and member of X3J11. I get all the glory I need as Convenor of WG14, thank you. And I seem to have more than enough work as well. That homework I mentioned earlier has occupied me for over a month, off and on, since the last meeting. The responsibility lies with me to prepare the normative addendum for SC22 balloting as a Committee Draft (CD). I also inherit several years of X3J11 interpretations as a huge batch of Defect Reports to log and organize. My aim in writing this report is not to win your symphathy. (But I'll take any I can get by the way.) Rather, it's to spell out the current formal activities in the C standard arena. Note that this report does not cover the work of X3J11.1 (a.k.a. the Numerical C Extension Group, or NCEG). That's because the charter of that subcommittee is to produce only a Technical Report (TR). In X3 land, a TR does not have the force of a standard. It is simply advisory. You'll hear more about X3J11.1 in future installments of this column. I need to cover a lot of administrivia first, so please bear with me. I promise to give you a few technical details of what's happening to Standard C before I'm done. The Normative Addendum The normative addendum has, until recently, consisted of three contributions, each put forth by a separate member body: The UK contribution endeavors to clarify several dozen areas where people found the C Standard unclear. Some issues arose from queries within the UK. Others arose from early Requests for Interpretation (RFIs) submitted to X3J11. All took the form of examples to be added to the C Standard. (Examples are part of the C Standard, but don't affect the definition of the C language. Hence, they are a good vehicle for adding clarification without running the risk of inadvertently changing the language.) The Danish contribution adds macros and several alternate ways to spell some of the punctuators and operators in C. The idea is to provide a way to write C source code more readably in character sets that commandeer things like braces and the tilde character for other graphics. The C Standard includes trigraphs for this purpose, but nobody pretends that using them makes for readable code. Even if you can't replace all trigraphs, proponents argue, any improvement in readability is worth supporting. The Japanese contribution adds extensive support for manipulating large character sets in C. The C Standard provides only the bare minimum of the functionality you need to play with Kanji or other large character sets. The Japanese delegation has developed a much more ambitious extension to C for this purpose. The biggest change we agreed to last December was to delete the UK contribution from the normative addendum. Don't think we considered it unimportant--quite the contrary. Rather, we observed that the new machinery for handling Defect Reports offered more apropos vehicles for publishing the work of the UK delegation. So we threw this piece over the wall, as it were. The other two pieces got final approval at the meeting. Both, however, suffered from a serious shortcoming. They needed to be translated into better "standardese." The Danish contribution evolved as a one- or two-page statement of intent. The Japanese contribution was remarkably refined, given the difficulty that English presents to the Japanese. But still there were places where the wording was a bit rough, or where more formal jargon was called for. Lucky for me, Dave Prosser took it upon himself to correct these problems. As the final Redactor (editor) of the C Standard, Dave speaks standardese like an ISO bureaucrat. He also understands C better than practically anybody else I know. By the time he completed a pass over the normative addendum, I had little left to do except carp at details, then make a stack of review copies. As of this writing, a review committee is checking our work. I will then submit the document to SC22 for CD balloting, once we get everyone's approval. By the time you read this, the balloting should be under way. My goal is to have the balloting period close shortly after the next X3J11 meeting (New York City in May), and before the next WG14 meeting London in July). That's all part of a little game of brinksmanship that we Convenors play all the time. Defect Reports Meanwhile, back at the ranch, I have this great stack of interpretations from X3J11. Four dozen Requests for Interpretation have percolated through ANSI official channels since the C Standard was approved in 1989. Over the years, X3J11 has patiently addressed and debated every one. The result has been two Technical Information Bulletins (TIBs) summarizing the RFIs and committee responses. I described some of the earliest RFIs and responses in these pages way back when. (See "Standard C: A Matter of Interpretation," CUJ June 1990, and "Standard C: Interpreting the Nasties," CUJ July 1990.) Other people have also discussed some of the interpretations here and in other publications. Sadly, however, the TIBs have yet to be officially published by ANSI. Now it looks like they never will be. An administrative foulup or two delayed the publication of TIB #1. Then ANSI switched over to the ISO C Standard and the situation changed. No longer was ANSI obliged to interpret the C Standard, since it was now an ISO document. Worse, it wasn't clear whether ANSI was even permitted to issue interpretations, under the agreement with ISO. TIB #2 sailed straight into the same swamp. Now both are mired in bureaucratic uncertainty. We didn't want to lose all those probing questions to public view. And we certainly didn't want to waste the carefully crafted responses. So I accepted the obligation to treat each of the ANSI RFIs as a separate Defect Report. I've ensured that Defect Reports #001 through #048 correspond to ANSI RFIs #01 through #48. (And I've already been handed Defect Report #049 through a separate channel, even before the dust has settled on the changeover.) I've built this 100-page (typeset) Defect Report Log. It contains the original ANSI RFIs, each accompanied by a "suggested response"--the response crafted by X3J11 for publication in a TIB. And remember all those examples from the UK contribution of the normative addendum? Well, I dealt them out as appropriate among the RFIs. Each example is labeled as a "suggested correction" to the C Standard. That's not the end of it, of course. X3J11 developed most of its responses under a severe constraint. We were originally told that we could not change a single word of the C Standard. Even if a slight change of wording, or an added sentence, could clarify our intent without changing the language definition, we couldn't make the change. Thus, we put a lot of energy into rationalizing that you could read the C Standard the way we intended. That's not the best way to respond to a serious complaint from a confused questioner. Now WG14 has machinery for making such clarifications, as Technical Corrigenda. The sentiment among many members of both WG14 and X3J11 is that we should not waste this opportunity. We could simply publish the two ANSI TIBs as an ISO Record of Response. That would get the interpretations out to the public (at last) fairly quickly. But it would leave us in the position of rationalizing bad standards language instead of fixing it. So my task instead is to circulate this Defect Report Log among the membership of both committees. I hope that X3J11 can give us prompt guidance about the best way to respond to each Defect Report. Either we accept the explanation from the ANSI TIB, we include the example from the UK contribution, or we develop amended wording to clarify the C Standard. (I like to think that only one of these three options will suffice in each case.) I hope for prompt guidance because this process has already dragged on for too long. The sooner we can clarify the gray areas of Standard C for the world at large, the happier I'll be. The Danish Contribution Now for a few technical details. The Danish contribution requires that all implementations of Standard C add a header called . (The name honors the ISO standard which corresponds to ASCII, except that it permits certain graphics to be substituted for those we Americans know and love.) Listing 1 shows the contents of this header. Note that you can use this file as is with any variant of ISO 646. It just prints funny on some national variant of that character set. The idea, in fact, is to confine most of the funny printing to just this header (which you should seldom feel moved to print.) You can then write: if (x != 0 x != XMAX) ..... as if (x ne 0 or x ne XMAX) ..... and the code should be readable with any national variant. If that is not important to you, don't include the new header. Then none of the new macros conflict with any names you choose. Besides this header, all implementations of Standard C must also recognize alternate spellings for six tokens: <: :> <% %> %: %:%: /* are the same as */ [ ] { } # ## /* respectively */ Because they are just alternate ways to spell the same token, you can balance <: with }, if you want to be perverse. And if you "stringize" one of these alternate forms, you get a different result than when using the older token (or its trigraph form). Thus: #define STR(X) #X printf(STR( <: ) STR( { ) STR( ??< )); prints <:{{. Before you start writing letters, let me make a few observations: Not all of these alternate forms are needed to solve problems with ISO 646, despite the name. Some help with EBCDIC as well. None of these alternate forms help much with using certain changeable characters inside character constants and string literals. You still need to use trigraphs sometimes. bitand is hardly a great name to use for & as the address-of operator. You can improve on these names all sorts of ways. In fact, many people have. In further fact, suggesting alternative lists has been a popular indoor sport at C standards meetings for several years now. The point is that this addition is not perfect. I'm pretty convinced after years of trying, though, that perfection is unattainable here. This particular approach is good enough. It can also be argued that the problem this addition solves is small and rapidly getting smaller. Others are pretty convinced, though, that it is still a problem worth solving. I believe the clutter is small enough that the rest of us should be tolerant. The Japanese Contribution By far the largest part of the normative addendum is the Japanese component. I count one new macro, three new type definitions, and 60 (!) new functions. The basic idea is to provide a complete set of parallels between functions that act on one-byte characters and functions that act on the newer wide characters. It's too bad we have to invent a whole set of variant names for these new functions. (That's one of the ways that C++ has improved code hygiene over C.) But I believe the time is ripe to introduce better wide-character support. Windows NT traffics consistently in (16-bit) wide characters. It exemplifies the new trend toward supporting large and varied character sets. Multiple sets of 256 characters just don't cut it for systems and applications with an international market. I plan to devote next month's column to a detailed look at the Japanese contribution. It's too big to due justice to in the space remaining here. For now, I'll simply summarize what it contains: A set of functions analogous to those in lets you classify wide characters much the way you do conventional ones. You can also define your own categories of characters and test for them. A set of functions analogous to those in lets you manipulate wide-character strings much the way you do conventional ones. A set of functions analogous to those in lets you convert numeric wide-character strings much the way you do conventional ones. Additions to the wide-character conversion functions in give you much tighter control over the conversion process. A function analogous to strftime in lets you encode time information as a wide-character string much the way you do conventional ones. Additional conversion specifiers for the existing print and scan functions let you convert between occasional multibyte sequences in files and wide characters internal to the program. A set of functions analogous to those in lets you manipulate "wide-character streams". These are files of multibyte characters that appear internally as sequences of wide characters. Some of this stuff sounds redundant, and it is. Still, there are good reasons for each of the additions. I'll do my best to convince you of that next month. Listing 1 The header #define and && #define and_eq&= #define bitand& #define bitor #define compl ~ #define ne != #define not ! #define or #define or_eq = #define xor ^ #define xor_eq^= // End of File On the Networks It's Back? Sydney S. Weinstein Sydney S. Weinstein, CDP, CCP is a consultant, columnist, lecturer, author, professor, and President of Datacomp Systems, Inc., a consulting and contract programming firm specializing in databases, data presentation and windowing, transaction processing, networking, testing and test suites, and device management for UNIX and MS-DOS. He can be contacted care of Datacomp Systems, Inc.,3837 Byron Road, Huntingdon Valley, PA 19006-2320 or via electronic mail on the Internet/USENET mailbox syd@DSI.COM (dsinc!syd for those that cannot do Internet addressing). Once again, and with no explanation, comp.sources.unix. seems to have gotten restarted again, just in time for the new year. However, comp.sources.x is still silent. Well, while it is still active, I guess it's time to make the best of comp.sources.unix. Welcome back Paul. (The moderator is Paul Vixie.) Let's see if we can keep it coming and empty out the backlog, as many of these sources are over a year old. First on the new list is mytinfo from Ross Ridge . mytinfo is a single library that combines the functionality of the standard UNIX termcap and terminfo libraries. It has the special ability of being able to fetch terminal descriptions from both termcap and terminfo databases regardless of which set of functions, termcap or terminfo, are used. It can even read terminal descriptions from terminfo source files. Also included is a tool to convert from/to termcap and terminfo source and binary formats. It was posted as Volume 26, Issues 77-79. If you need restrict privileged access (super-user access) to a small set of commands, then op from David Koblas is just what you need. Posted as Volume 26, Issue 80, it provides restrictions on who can execute the commands, what commands they can execute, and even what arguments are to be allowed to the commands. Just perfect for restricting mounting of CD-ROM drives or floppies as UNIX file systems (a task, alas, that requires super user-access). A variant on John Walker's settime for setting the clock on a Sun Sparcstation from a dial up access service was posted by John Rushford as nisttime for Volume 26, Issue 81. This version is for System V flavors of UNIX and supports accessing both the NIST master clock at the National Institute of Standards and Technology and the USNO master clock at the Naval Observatory. The script will dial the respective clock, receive the correct time, and then update the systems clock. Copying boot tapes, or other multi-volume tapes under UNIX can be troublesome. You often have to spool them to disk first, and there isn't enough room. Thad Floryan has provided tprobe-1.0 which can reveal a tapes-existing saveset layout, copy the tape using drives located anywhere on the network, and perform media conversions. Published as Volume 26, Issue 84, it even includes documentation. If your C is missing the library routine strftime then the version submitted by Arnold Robbins will fit the bill. strftime is a format-string-controlled internal time representation to ASCII conversion filter. It supports locale for internationalization. strftime, and a date command wrapper, are Volume 26, Issue 85. Utilities Galore Utilities abound this time in comp. sources. misc. If you are tired of nohup leaving nohup.out files lying around, consider nhup from Gnanasekaran Swaminathan . It works like nohup, as it runs the command in the background, immune from hangup and quit signals, but unlike nohup, if the input and output are not redirected, they are set to /dev/null and not a nohup. out file. nhup is Volume 33, Issue 80. pdcurses is a public domain Curses C library that is compatible with the UNIX System V 3.2 curses library. It is written to support most of the popular DOS and OS/2 C Compilers. This new version, 2.0, is almost a total rewrite of the prior 1.4 version, by a new maintainer, Mark Hessling . Posted as Volume 33, Issues 81-91, a dmake 3.8 flavor makefile is included for both DOS and OS/2. New in 2.0 include X/Open routines, Color support in System V format, OS/2 port, and many functions supported as both functions and macros. Farrell McKay released a bug-fixed version of his xcb-2.1 program for Volume 33, Issue 92. xcb provides easy access to the cut/paste buffers in the X server. New is WM_DELETE_WINDOW protocol support, and fixing of two major bugs. MetalBase, a simple database engine, portable between platforms (UNIX, MS-DOS, etc.) has been posted as mbase by Richard Parvin Jernigan for Volume 33, Issues 119-126. This latest release, 5.0, has more field types, three-phase locking to ensure multi-user access without any system-based IPC, and internal caching to provide twice the speed and up for rebalancing indices. A conversion utility for MetalBase 4.0 and 4.1a relations is provided. Other features include runtime encryption on all platforms, creation of relations on-the-fly, and a flexible report writer. Lee Hounshell has submitted his C++ object library var for Volume 33, Issues 127 and 128. var provides a "super-string" class with associate array capabilities. The var class does a pretty good job of offering a data object that assumes its "type" at runtime, based on context of use. Rarely will you need to declare ints, or longs, or doubles or char[] or even string objects! var does it all (or at least tries to). An alternative user interface for ftp was created by Mike Gleason . ncftp was posted as Volume 34, Issues 14-16. It supports autologin as anonymous, access to sites in your .netrc file using a substring of their name, use of a pager to view remote directory listings, a transfer progress meter, activity logging, and much more. It's not as impressive as ftptool, but it doesn't require an X play either. A bug-fix patch (patch 1) was posted in Volume 34, Issue 20. Two C++ classes for outputting formatted numbers were contributed by Scott Kirkwood . The first, fformat for Volume 34, Issue 23, supports scaling and editing, truncation to fit using SI notation (K, M, etc.) for floating-point numbers. The second, iformat, in Issue 24 is for integer numbers and offers similar capabilities. A collection of X11 image processing and display utilities was contributed by John Cristy for Volume 34, Issues 28-54 with patches in Issues 85-87, 88, 89, 98 and 118. The imagemagick utilities read and write MIFF images and has utilities to access GIF, JPEG, PostScript, PPM, RLE, SUN Raster, TIFF, Utah Raster, VICAR, X Bitmap, X Window Dump formats. It can display images, support animation, perform processing on images (scaling, rotation, color reduction, and more), and much more. Support for the JPEG4 release is also included. (See the next item.) In addition, the Independent JPEG Group has updated its JPEG software to version 4 in Volume 34, Issues 55-72 jpeg in this new version provides significant speedups, improved image quality, error recovery, and better portability. A matrix package for C++ was submitted for Volume 34, Issues 103-110 by Robert Davies . newmat07 supports the matrix types: Matrix, UpperTriangularMatrix, LowerTriangularMatrix, DiagonalMatrix, SymmetricMatrix, RowVector, ColumnVector, BandMatrix, UpperBandMatrix, LowerBandMatrix, and SymmetricBandMatrix. Only one element type (float or double) is supsuported. The package includes the operations *, +, -, (defined as operators) inverse, transpose, conversion between types, submatrix, determinant, Cholesky decomposition, Householder triangularisation, singular value decomposition, eigenvalues of a symmetric matrix, sorting, fast fourier transform, printing, an interface with Numerical Recipes in C, and an emulation of exceptions. A faster version to freeze was released as an update by Leonid A. Broukhis for Volume 34, Issues 125-127 with a patch in Issue 128. Version 2.4 of this file compression program is at least 20% faster than before and memory requirements have been reduced such that it now fits in small model for segmented 16-bit architectures. On the patch front, ecu had patch 4 issued against version 3.20 by Warren Tucker . This patch, to the extended cu utility, fixes some problems with SVR4 locking logic, problems with FAS 2.10, and adds console xon/xoff support. The patch is Volume 33, Issue 79. cproto was updated to patch 5 in Volume 34, Issue 3 and patch 6 in Issue 5 by Chin Huang . This patch fixes some bugs and adds options to rewrite function declarations to include both old and new definitions controlled by an ifdef flag and support of more type qualifiers. Patch 6 just fixed a problem with compilation on pre-ANSI C compilers. Oraperl, the oracle user subroutine extensions to Perl were patched to level 3 in Volume 34, Issue 31. The principal change is that the bind and do functions now return a row count. Remind version 3, received patch 1 to fix some portability bugs and add a new PostScript output format for its calendar which is much more attractive than the old ASCII one. The patch is Volume 34, Issues 79 and 80. The Info-ZIP group submitted patch 1 to unzip50 for Volume 34, Issue 124. This patch fixes some compatibility problems with PKZIP 2.04c and one bug in password checking. Games or Alt? There seems to be a war between comp.sources.games and alt.source for posting improvements to the tetris game, it seems to have been everywhere this past month. In comp.sources.games the version was submitted by Qiang Alex Zhao for Volume 15, Issue 44-45, with patchs in Issue 46, 47 and 55. This version is more portable, and uses no special toolkits. Plus the blocks are bigger and easier on the eyes. In addition, three flavors of the minesweeper game for X were posted. xmines, an Xview variant from Manolis Lourakis in Volume 15, Issue 28, xbomb using Xlib from John Heidemann in Volume 15, Issues 48-51, and xdemineur also using Xlib from Marc Baudoin in Volume 15, Issues 52-54. From down under comes dinkum2, the Australian adventure game. In Dinkum you'll search for treasure in the Australian Outback. You give the commands and it'll do the dangerous work. It's a text-based adventure-type game submitted by Gary Allen for Volume 15, Issues 36-43. A version of accordian solitaire was submitted by Eric Lechner for Volume 15, Issue 27. In accordian a deck of cards is dealt to the playing table (either a card at a time, or the whole deck at once), and the goal is to move stacks of cards around until a single stack of cards is left on the table. Stacks of cards can only move 1 or 3 spaces up the line of cards, onto other stacks whose top card is of the same suit or rank as the current stack's top card. If you end up with a single stack, you win the game! A multi-player X11 tank game and server that supports network-wide access and a rather large display (at least 1024 x 800 is recommended, but 1080 wide is even better) was submitted by Bernard Hatt for Volume 15, Issues 29 and 30 with Patch 1 in Issue 31. xtb uses UDP sockets for communications and robot tanks for adding difficulty. Another multi-player network wide game is robot_hunt from Steve Boswell . This curses-based game uses sockets for communication. The object is to hunt down other players in a maze and kill them before they, or the robot, kill you. Robot hunt is Volume 14 Issues 32-35. Previews from alt.sources As usual, plenty in alt.sources so here are just a few highlights of what's to come in the main stream groups. Steven Grimm is just about ready to post his Workman X-based CD player to the main stream groups. In the interim, several beta copies have appeared, with the latest being on December 19, 1992 in nine parts. It supports an XView interface and support most of the standard audio CD player features using the systems CD player. It also supports a database of CDs and provides information from the database about the CD in the player. An X11/Motif file manager was posted by Jan Newmarch in 17 parts on November 19, 1992. xmfm is a file manager that shows files as icons in panes. It divides the display of a directory into three areas--executable files, directories, and ordinary files--to distinguish between them. When a file is selected a range of actions can be performed on it. This is configurable on a per-user basis. xmfm also allows arbitrary programs to be run from within it. A patch was posted on November 24, 1992. Psroff was updated with patch 15 issued on January 13, 1993. This patch appears here due to the long delay in the comp.sources.unix queue. This large patch provides many small changes including paper and orientation changes for both LJ and PS modes. xcoral is a multiwindows mouse-based text editor, for X Window system, with a built-in browser to navigate through C functions and C++ classes hierarchies. xcoral provides variables width fonts, menus, scrollbars, buttons, search, regions, kill-buffers and a 3-D look. Commands are accessible from menus or standard key bindings. xcoral is a direct Xlib client and was posted on January 15, 1993 by Lionel F. in 14 parts. Questions & Answers Message Catalogs Kenneth Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member on the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. Q I was reading your column entitled, "Linked Lists, Strings, and Internationalization," in the January 1993 issue of The C Users Journal, and was particularly interested in the section about the X/Open message catalog. Yours is the first description of the layout of the file I've seen. We have Sun workstations here and are planning on using a message file scheme, but Sun has no documentation about the format of a message file, despite supporting gencat, catopen, etc. Where can I find information about the format of message files (the text file that we'd create, not the .cat file), preferably with examples? I enjoyed the article--it was very interesting and well written. I also agree with your comments about Gimpel's PC-lint/Flexelint--I swear by it. Many thanks. Alec Sharp Louisville, CO A The information I have on message catalogs comes from my teaching OSF UNIX. The DEC documentation for this system explains precisely the organization of these message catalogs. I think similar information should be available from X/Open, since that is where the interface originated. I think it would be ironic if you had to purchase the documentation for a Sun UNIX feature from OSF. Strings Again Q I am writing in reference to your comments on how to handle strings in the August 1992 issue of The C Users Journal (page 132). The presented approach is quite acceptable, but there is something I do not like. Whenever the programmer introduces a new string, he or she must do three things--add the string to the array (char *string[]=..); define the corresponding constant (#define INIT_STR 1); and use it (printf(get_string(INIT_STR));). I think the programmer should only do two things: define the string and use it. For example: defstr ( InitStr, "Init String" ) ... printf (InitStr); or printf (getstr(InitStr)); Of course, having multiple identical strings in code is not acceptable. One possible solution (tested with Borland C++ compiler) is shown in Listing 1. The file STRDEF.H contains the string definitions. File STRDEMO.C illustrates how the string definitions are used. There are two additional files: STRHNDL.H and STRHNDL.C, but being written once, they do not need to be changed any more. All the strings appear as global data in file STRHNDL.C. The file STRHNDL.H provides the necessary extern declarations making the strings accessible. The same approach could be easily adapted to handle multilingual messages. This is illustrated in Listing 2. This solution is far from being perfect--at least, because of using a lot of global data. I am sure there are better solutions. Perhaps you could suggest one. Also, I am interested in your opinion about using an object-oriented approach to the problem. Stefan Ganev Bourgas, Bulgaria A I agree with you that having to create a #define INIT_STR and to put the string into an array seems like double work. There is always the potential for a mismatch between the value of INIT_STR and its position in the array. Your macro approach helps eliminate this problem. I would suggest a minor change in your second example. If you concatenate the name of the array in your #define defstr(x,y,z) macro, the name space of the character arrays would be practically guaranteed to not conflict with any other external names. The replacement might look something like: char *STR__##strx[] = { y, z }; The same change would be made in the other places that you refer to the array. I explained the X/Open alternative in the January issue. In that system, you can use identifying names with the strings and it will produce a header file with those names #defined. That eliminates the mismatch and extra coding. You asked about an object-oriented approach to this problem. A goal of my design would be to make the calling program not aware of where or how the strings are actually stored. That is, the calling module should not have to be recompiled if just the strings are changed. Preferably it should not have to be relinked, as only resource values would have changed. Your approach meets most of these criteria, except for the necessity to relink. I would define a StringResource object as: class StringResource { public: StringResource(Language_Identifier lid); String get_string(String_Identifier sid); //... } A header file would contain the values for your particular application. This could be generated automatically from a text file that looked like: INITIAL_STRING "This is the first string" ANOTHER_STRING "This is another string" As an example, the header file my_prog.h might look like: enum Language_Identifier {ENGLISH}; enum String_Identifier { INITIAL_STRING, ANOTHER_STRING, //... }; A calling program would look something like Listing 3. I used the stream class for input and output and assumed that the operator << was overloaded for Strings. If your String class had a conversion to char *, you could use the Standard C library functions, such as puts(sr.get_string(INITIAL_STRING)). I would not define such a conversion for the StringResource class. Whether you decide to compile the string file into the code or keep it in some external file is completely transparent to the calling program. The get_string function returns a String object that contains the requested characters in either case. The StringResource example here assumes that you only require one set of strings per program. That is the typical case. Going to multiple sets complicates the picture. Readers might send in their suggestions for what a multi-set design would look like. Truncating a File In the January 1993 issue of The C Users Journal in the question titled "Truncating a File in Place and Portability," on page 106, you say "For the MS-DOS version, the functions would perform the operations you indicate... while the MS-DOS version may fail if there is insufficient disk space." The operations indicated were copying the file and deleting and renaming. I thought you should know that both the Microsoft C compiler and the Borland compiler offer a file truncate function, chsize (or _chsize) which can truncate a file in place, just as the UNIX approach may. Further, this is a single DOS call. It's quick and has no disk free-space problem. For those without an equivalent function in their runtime libraries, the trick in MS-DOS is simply to write to an open file handle with a count of zero. Anyway, your main point (about writing a single function that is portable, whose implementation uses OS dependent code) is right on. I just wanted to point out something many long-time DOS programmers don't realize; a file may be easily truncated or extended, without copying. Regards, and keep up the good work. Dave Angel A Thanks for your comments. Truncating files is not something I do often. In fact, the only time I can ever remember doing it is demonstrating how to use the function on UNIX. I just like to expand a bit on the main area of discussion. It all boils down to what you consider your standard interface to the environment in which you program. This standard interface consists of the set of functions which you can expect to have on any system. The Standard C library is a good starting place for such a set of functions. Actually when using C++, I hide even the standard C functions from use in my programs. I also hide the iostream class, even though its interface is becoming more standard. I basically have created a set of classes and methods for each class which are as primitive as possible. The member functions are easily written in terms of Standard C and operating system functions. This permits me to easily port my code to any environment which may not support either the Standard C library or the C++ streams library. The functions are simple enough to implement for whatever the system supports. The class for File looks like Listing 4. I have not reproduced all the classes and their implementation in their entirety for space and copyright reasons. The File::open, File::close, File::read, and File::write member functions call their respective operating system functions. On UNIX and MS-DOS, these are the open, close, read, and write functions with the appropriate parameters supplied. The Standard does support unbuffered reading and writing using fopen and setvbuf calls. To run on a system that only supports those calls one needs only to change the way Internal_File::open works and to alter the appropriate calls in each of the other member functions. This class can also eliminate the problems of the mode (text or binary) of the file opened by open. One vendor's implementation of using a global variable for the type of file has caused me grief in the past. The testing program for this class has code that looks like Listing 5. You may wonder where the number of bytes read or written is being kept. It is part of Internal_ByteArray. Now stout-hearted C programmers will probably be upset at my use of a separate function to return the error code for an operation. It actually gets returned from Internal_File members. The Internal_File::error is an inline function that simply returns the value of the last error set. I could have set up a variable as: Internal_File::Error error; and programmed as: if ( (error = new_file.write(buffer) ) != Internal_File::No_error) { // Do something about it } But I didn't. Listing 1 Files for string handling /* file STRDEF.H */ defstr ( InitStr, "Init String") defstr ( OtherStr, "Other String") /* file STRDEMO. C */ #include "strhndl.h" #include int main () { puts(InitStr); puts(OtherStr); return 0; } /* file STRHNDL.H */ #define defstr (x,y) extern char x[]; #include "strdef.h" /* file STRHNDL.C */ #define defstr(x,y) char x[] = y; #include "strdef.h" /* End of File */ Listing 2 Files for handling multilingual messages /* file STRDEFB.H */ defstr ( InitStr , "init String Lang 1", "Init String Lang 2" ) defstr ( OtherStr , "Other String Lang 1", "Other String Lang 2" ) /* file STRDEMOB.C */ #include "strhndlb.h" #include int main () { set_language(LANG_1); puts(get_str(InitStr)); puts(get_str(OtherStr)); set_language(LANG_2); puts(get_str(InitStr)); puts(get_str(OtherStr)); return 0; } /* file STRHNDLB.H */ enum language { LANG_1, LANG_2 }; extern int Language; #define defstr(x,y,z) extern char *x[]; #include "strdefb.h" #define set_language(x) Language = x #define get_str(x) x[Language] /* file STRHNDLB.C */ #include "strdefb.h" int Language; /* End of File */ Listing 3 A calling program #include "my_prog.h" #include "stringre.hpp" #include int main() { StringResource sr(ENGLISH); cout << sr.get_string(INITIAL_STRING); } // End of File Listing 4 The File class class FileName; // String containing legal filenames class Internal_ByteArray; // Array of bytes (can include nuls) // Includes length class Internal_ByteOffset; // a long class Internal_ByteSize; // a long class Internal_File { public: enum Mode {Read, Write, ReadWrite}; enum Error {No_error, Does_not_exist, Read_only, End_of_file, ... } Internal_File(); ~Internal_File(); // calls close() if not closed Error open(FileName name, Mode mode = Read); Error create(FileName name); Error close(); Internal_ByteSize read(Internal_ByteArray buffer); Internal_ByteSize write(Internal_ByteArray buffer); Error seek(Internal_ByteOffset offset); Error error(); //... }; // End of File Listing 5 A test program #include #include "int_file.hpp" #include "boolean.hpp" #include "program.hpp" main() { Internal_File old_file; Internal_File new_file; Internal_ByteArray buffer(100); Internal_ByteSize bytes; Boolean done; old_file.open(FileName("abc.dat")); if (old_file.error() != Internal_File::No_error) { cout << "Unable to open file abc.dat"; program_exit(Failure); } new_file.create(FileName("def.dat")); if (new_file.error() != Internal_File::No_error) { cout << "Unable to open for writing def.dat"; program_exit(Failure); } // Copy the files do { old_file.read(buffer); switch(old_file.error()) { case Internal_File::No_error: done = False; break; case Internal_File::End_of_file: done = True; break; default: cout << "Error in reading"; done = True; break; } new_file.write(buffer); if (new_file.error() != Internal_File::No_error) { cout << "Error in writing"; done = True; break; } } while (!done); old_file.close(); new_file.close(); } // End of File Code Capsules Sorting with qsort Chuck Allison Chuck Allison is a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. If you've been programming for very long you've probably noticed that many of your programs spend a noticeable amount of time ordering data elements. In his classic work "Sorting and Searching" (The Art of Computer Programming, Vol. III, Addison-Wesley, 1973), Don Knuth reports that the average program spends over 25% of its time sorting. Sorting is useful for grouping related items, eliminating duplicates, comparing or merging related sets, and helping searches. You'll certainly want to use an efficient algorithm for such a frequent activity. This article describes the use of qsort, the sort function supplied with the standard C library. In most compilers, qsort is an implementation of the Quicksoft algorithm, which is, on average, the most efficient of all general-purpose sorts. (However, an implementation is not required to use the Quicksort algorithm--it just needs to sort arrays according to the specifications illustrated below) .qsort is general purpose because you can use it to sort an array of elements of any type, with simple or compound sort keys. It only requires that any two elements can be compared and that the array is in memory .qsort modifies the array by swapping elements so that they come out in ascending order. The program in Listing 1 shows how to use qsort to sort an array of integers. qsort (declared in ) takes four arguments: the array to sort, the number of elements in the array, the size of each element in bytes, and a user-supplied compare function. A compare function, f, must have the prototype int f(const void *p1, const void *p2). (Using void *allows pointers to any type). qsort repeatedly calls the compare function, passing it pointers to two array elements. The compare function must behave like strcmp, i.e., it must return a negative number if the value that p1 represents precedes that of p2, 0 if the values are equal, and a positive number otherwise. In this example icomp does this by returning the difference of the two integers being compared. Note that the incoming pointers must be cast to the appropriate type (int * in this case) in order to reference the values to compare. Listing 2 shows how to sort an array of strings (i.e., an array of pointers to char) in descending order. You simply supply qsort with a compare function that reverses the order of the sort by negating its return value. Since some_strings[] is an array of char *, qsort passes pointers to char * (i.e., char **) to the compare function scomp. scomp dereferences once and passes the pointers to char to strcmp. Don't overlook the fact that it is the pointers that qsort reorders, and not the strings themselves. The program in Listing 3 sorts an array of structures in two different ways: first by name (last, then first), and then by age. Most of the time it is faster to swap pointers than structures. (Structures tend to be bigger than pointers.) When this is the case, it will be more efficient to pass qsort an array of pointers to structures instead of an array of actual structures. This of course requires another level of indirection in the compare functions (see Listing 4). How Generic Sorting Works qsort is able to sort any array because you supply it with all it needs to know to do the job: where the elements are, how many, how big each one is, and how to compare them. To illustrate the technique, I'll take the simplest of comparison-sort algorithms, selection sort, and make it generic. Selection sort makes n--1 passes on an array of n elements. In the first pass, it compares the first element to the rest and swaps it with any that precede it in value (so after the first pass, the smallest element is now the first). It repeats the process with the second element, and so on until the array is in ascending order (see Listing 5). To make this process generic, I'll parameterize the four items mentioned above, just like qsort (I'll call it ssort--see Listing 6). The statements char *p1 = (char *) array + i*size; char *p2 = (char *) array + j*size; determine pointers to the ith and jth array elements. These addresses are then passed to the user-supplied compare function. If the elements are out of order, ssort puts them in order by swapping their associated blocks of storage. Listing 7 shows that usage of ssort is identical to qsort. Indexes Sometimes you want to access data in a certain order without actually disturbing its contents. This is especially important if you need multiple orderings repeatedly in the same program--you don't want to have to sort more than once for each ordering. This is the motivation for keeping indexes. If you don't mind having the data array global, indexes are easy with qsort. Listing 8 shows how to construct an index for an array of integers. idx is the index array (sometimes called a permutation vector) for some_ints. After initializing it to the "identity" permutation (i.e., idx[i] = i), call qsort to sort idx, not some_ints. You use idx as an indirection device to access some_ints in ascending order, although some_ints itself hasn't changed. Note that some_ints must be global because the compare function needs access to it. Indexes have many uses. Listing 9 shows how to extract only unique elements from an array non-destructively. This program can easily be modified to print a listing of the unique tokens in a text file. The program in Listing 10 populates an array with the alphanumeric tokens read from standard input. It then prints only unique tokens in ascending order, while the original array still contains all the tokens in their original order (see Figure 1). This is quite a feat for such a short program. Although this lazy technique is not worthy of a serious parser (since it picks up tokens that start with a numeral, or that appear in strings or comments) you may find a use for it. I also cheated a little by using ranges in the scansets. The format descriptor %*[^A-Za-z0-9] instructs scanf to ignore any contiguous sequence of non-alphanumeric characters, while %[A-Za-z0-9] matches any such sequence (for an explanation of scansets, see "Code Capsules: Text Processing I", CUJ, October 1992). But scanset ranges are not guaranteed to be portable in ANSI C, although many popular compilers support them. If yours does not, replace A-Za-z0-9 with all 62 alphanumeric characters spelled out. There is of course a lot more to sorting. I have only considered internal sorting (i.e., when you are sorting an array confined to memory). Some other sorting techniques perform more efficiently than qsort by using the characteristics of the data to optimize the process (e.g., radix sort, which only works on integral types). Like Quicksort, qsort is not a stable sort. This means that elements with equal keys may lose their relative ordering after sorting (e.g., if you sort the data in Listing 3 by last name only, Edsel Ford may precede or follow Henry Ford--their relative ordering is indeterminate). If you are interested in external sorting or want to explore the innards of many different sorting algorithms, see Knuth's book cited in the beginning of this article. (Next month's column will also discuss external sorting.) In this column I have tried to illustrate how much mileage you can squeeze out of a single, well-designed sort function. Figure 1 Outout from index3.c (input is the text of sort1.c) 0 integers 12 ints 15 main 37 n 4 p1 40 p2 NELEMS printf Sort qsort a Sreturn b size c sizeof const some d sort1 define static for stdio h stdlib i t icomp void include with int Listing 1 Uses qsort to sort integers /* sort1.c: Sort integers with qsort() */ #include #include #define NELEMS 4 static int icomp(const void *, const void *); main() { size_t i; int some_ints[NELEMS] = {40, 12, 37, 15}; qsort(some_ints,NELEMS,sizeof some_ints[0], icomp); for (i = 0; i < NELEMS; ++i) printf("%d\n",some_ints[i]); return 0; } static int icomp(const void *p1, const void *p2) { int a = * (int *) p1; int b = * (int *) p2; return a - b; } /* Output: 12 15 37 40 */ /* End of File */ Listing 2 Sorts strings in descending order /* sort2.c: Sort strings in descending order */ #include #include #include #define NELEMS 4 static int scomp(const void *, const void *); main() { size_t i; char *some_strings[NELEMS] = {"well","hello","you","rascal"}; qsort(some_strings, NELEMS, sizeof some_strings[0], scomp); for (i = 0; i < NELEMS; ++i) puts(some_strings[i]); return 0; } static int scomp(const void *p1, const void *p2) { char *a = * (char **) p1; char *b = * (char **) p2; /* Negate for descending order */ return -strcmp(a,b); } /* Output: you well rascal hello */ /* End of File */ Listing 3 Sorts structures first by name, then age /* sort3.c: Sort structures */ #include #include #include #define NELEMS 4 struct person { char last[16]; char first[11]; char phone[13]; int age; }; static int name_comp(const void *, const void *); static int age_comp(const void *, const void *); main() { size_t i; static struct person some_people[NELEMS] = {{"Lincoln","Abraham","555-1865",161}, {"Ford","Henry",#555-1903",98}, {"Ford","Edsel","555-1965",53}, {"Trump","Donald","555-1988",49}}; qsort(some_people, NELEMS, sizeof some_people[0], name_comp); puts("By name:"); for (i = 0; i < NELEMS; ++i) printf("%s, %s, %s %d\n", some_people[i].last, some_people[i].first, some_people[i].phone, some_people[i].age); qsort(some_people, NELEMS, sizeof some_people[0], age_comp); puts("\nBy age:"); for (i = 0; i < NELEMS; ++i) printf("%s, %s, %s %d\n", some_people[i].last, some_people[i].first, some_people[i].phone, some_people[i].age); return 0; } static int name_comp(const void *p1, const void *p2) { struct person *sp1 = (struct person *) p1; struct person *sp2 = (struct person *) p2; int order = strcmp(sp1->last,sp2->last); if (order == 0) order= strcmp(sp1->first,sp2->first); return order; } static int age_comp(const void *p1, const void *p2) { struct person *sp1 = (struct person *) p1; struct person *sp2 = (struct person *) p2; return sp1->age - sp2->age; } /* Output: By name: Ford, Edsel, 555-1965 53 Ford, Henry, 555-1903 98 Lincoln, Abraham, 555-1865 161 Trump, Donald, 555-1988 49 By age: Trump, Donald, 555-1988 49 Ford, Edsel, 555-1965 53 Ford, Henry 555-1903 98 Lincoln, Abraham, 555-1865 161 */ /* End of File */ Listing 4 Sorts structures via a pointer array /* sort4.c: Sort structures via pointers */ #include #include #include #define NELEMS 4 struct person { char last[16]; char first[11]; char phone[13]; int age; }; static int name_comp(const void *, const void *); static int age_comp(const void *, const void *); main() { size_t i; static struct person some_people[NELEMS] = {{"Lincoln","Abraham","555-1865",161}, {"Ford","Henry","555-1903",98}, {"Ford","Edsel","555-1965",53}, {"Trump","Donald","555-1988",49}}; struct person *some_ptrs[NELEMS] = {some_people,some_people+l,some_people+2, some_people+3}; qsort(some_ptrs, NELEMS, sizeof some_ptrs[0], name_comp); puts ("By name:"); for (i = 0; i < NELEMS; ++i) printf("%s, %s, %s %d\n", some_ptrs[i]->last, some_ptrs[i]->first, some_ptrs[i]->phone, some_ptrs[i]->age); qsort(some_ptrs, NELEMS, sizeof some_ptrs[0], age_comp); puts("\nBy age:"); for (i = 0; i < NELEMS; ++i) printf("%s, %s, %s %d\n", some_ptrs[i]->last, some_ptrs[i]->first, some_ptrs[i]->phone, some_ptrs[i]->age); return 0; } static int name_comp(const void *pl, const void *p2) { struct person *sp1 = * (struct person **) p1; struct person *sp2 = * (struct person **) p2; int order = strcmp(sp1->last,sp2->last); if (order == 0) order = strcmp(sp1->first,sp2->first); return order; } static int age_comp(const void *pl, const void *p2) { struct person *sp1 = * (struct person **) p1; struct person *sp2 = * (struct person **) p2; return sp1->age - sp2->age; } /* End of File */ Listing 5 Illustrates the Selection Sort Algorithm /* select1.c: Illustrate selection sort */ #include #define NELEMS 4 main() { int a[NELEMS] = {40, 12, 37, 15); size_t i, j; for (i = 0; i < NELEMS-1; ++i) for (j = i+1; j < NELEMS; ++j) if (a[i] > a[j]) { int t = a[i]; a[i] = a[j]; a[j] = t; } for (i = 0; i < NELEMS; ++i) printf ("%d\n", a[i]); return 0; } /* Output: 12 15 37 40 */ /* End of File */ Listing 6 A generic selection sort function /* ssort.c: Generic selection sort */ #include void ssort(void *array, size_t nelems, size_t size, int (*comp)(const void *, const void *)) { size_t i, j, n; for (i = 0; i < nelems-1; ++i) for (j = i+1; j < nelems; ++j) { char *p1 = (char *) array + i*size; char *p2 = (char *) array + j*size; if (comp(p1,p2) > 0) for (n = 0; n < size; ++n) { /* Swap elements byte-by-byte */ char t = p1[n]; p1[n] = p2[n]; p2[n] = t; } } } /* End of File */ Listing 7 Illustrates a generic Selection Sort /* select2.c: Use the selection sort function */ #include #define NELEMS 4 extern void ssort(void *, size_t, size_t, int (*)(const void *,const void *)); static int comp(const void *, const void *); main() { size_t i; int a[NELEMS] = {40, 12, 37, 15); ssort(a,NELEMS,sizeof a[0],comp); for (i = 0; i < NELEMS; ++i) printf("%d\n",a[i]); return 0; } static int comp(const void *p1, const void *p2) { int a = * (int *) pl; int b = * (int *) p2; return a - b; } /* Output: 12 15 37 40 */ /* End of File */ Listing 8 Illustrates how to construct an index for an array of integers /* index1.c: Sort indirectly via an index */ #include #include #include #define NELEMS 4 static int comp(const void *, const void *); static int some_ints[NELEMS] = {40,12,37,15}; main() { size_t i; size_t idx[NELEMS] = {0,1,2,3}; qsort(idx,NELEMS,sizeof idx[0],comp); printf("Sorted array:\n"); for (i = 0; i < NELEMS; ++i) printf("%d\n",some_ints[idx[i]]); printf("\nOriginal array:\n"); for (i = 0; i < NELEMS; ++i) printf("%d\n",some_ints[i]); return 0; } static int comp(const void *pl, const void *p2) { size_t i = * (size_t *) pl; size_t j = * (size_t *) p2; return some_ints[i] - some_ints[j]; } /* Output: Sorted array: 12 15 37 40 Original array: 40 12 37 15 */ /* End of File */ Listing 9 Extracts unique elements using an index /* index2.c: Filter out duplicate integers */ #include #include #define NELEMS 7 static int comp(const void *, const void *); static int some_ints[NELEMS] = {40,12,37,12,40,15,15}; main() { int last; size_t i; size_t idx[NELEMS] = {0,1,2,3,4,5,6}; qsort(idx,NELEMS,sizeof idx[0],comp); /* Output only unique items */ printf("%d\n",(last = some_ints[idx[0]])); for (i = 1; i < NELEMS; ++i) if (some_ints[idx[i]] != last) printf("%d\n",(last = some_ints[idx[i]])); return 0; } static int comp(const void *p1, const void *p2) { size_t i = * (size_t *) p1; size_t j = * (size_t *) p2; return some_ints[i] - some_ints[j]; } /* Output: 12 15 37 40 */ /* End of File */ Listing 10 Extracts unique alphanumeric tokens from standard input /* index3.c: Print unique tokens from standard input */ #include #include #include #define MAXWIDTH 32 #define MAXTOKENS 1024 static int comp(const void *, const void *); static char tokens [MAXTOKENS] [MAXWIDTH]; main() { size_t i, n; size_t idx[MAXTOKENS]; char *last; char *format = "%*[^A-Za-z0-9]%[A-Za-z0-9]"; /* Read tokens & initialize index array */ for (n = 0; n is included, it goes from ANSI C to vendor C. Many of the packages available on UNIX (curses, the Berkeley openedir routines, etc.) are available on DOS. On DOS, machines constantly hang up during development. At least on UNIX they core dump and allow post mortem analysis. (You can work backwards. On DOS you generally have to work forwards from main.) In A. Levinson's letter in June, he talks about a find command on UNIX. Most UNIX commands are available on MS-DOS. (Most are freeware. The MKS toolkit is of high quality and runs on DOS.) In addition, the Berkeley opendir routines work extremely well on DOS, giving portability to directory searches. Using multiple systems, it is highly desirable to have them work as similarly as possible, at both an application and a source code level. This makes portability of user skills and writing source code. Source code is useful not only for changing the behavior of programs but to understand how programs work. It is possible to write portable DOS/UNIX source code for almost everything which fits on DOS. In the June issue, 20,000 commercial packages are talked about in DOS's favor. Many of these packages are quite bad. As an experienced programmer, I find the lack of make-able source code is a hinderance. What I've seen on both DOS and UNIX is the available of freely copyable source improves the quality of programs. (I've recently bought a PC at home and have not spent a cent on software. I'm typing this in Elvis, a vi clone GNU puts out (which runs on DOS and UNIX)). I also use less (instead of more). Both run on MS-DOS and UNIX. Sincerely, Marty Leisner leisner.henr801c@xerox.com leisner@eso.mc.xerox.com (716) 654-7931 As one of the original users of UNIX, I have to agree with much of what you say. If you have the time and the inclination, you can do a lot and learn a lot as a UNIX programmer. But if you just want to buy software that works, it's hard to beat DOS for price and availability. -- pjp Dear Mr. Plauger, First and foremost, I apologize for misspelling your name in a previous letter! You've mentioned in recent editorials and comments that you're working with the ANSI C++ committee. That makes you a perfect target for my suggestions. I like the language a lot (just as a I like C), and think that it's well-designed and well-thought-out. I do have a problem, though. A number of my functions--especially those critical to speed and/or size--have been written in assembler. (More to the point, many of them were written some time ago, and I really don't want to re-write them.) There's no easy way to marry external functions to classes at present, and I wish the committee would address this. I can't be the only one with this problem. I could use short "dispatcher" functions as class members, and let them call my library, but that adds to code size and slows execution. Why not instead add class prototyping to the language, with the ability to write member functions in other source modules? Say, much in the same way that ANSI function prototyping has been implemented for C? (You can put the prototypes into a common header and define the actual functions where you choose.) (Actually, it'd be nice if someone finally acknowledged that no one writes a 64K source file, and provided ways to implement encapsulation across several source modules. Why not use something like a master project file with individual protected sub-project files? Better yet, why not create a pointer-to-member concept that will let me prototype a class, then put the members--Y member--where I choose?) Just a thought. Thanks, Stephen M. Poole, CET 122 N. Main Street Raeford, NC 28376 The C++ committee is just starting to get more formal about handling requests for extensions. They have a considerable backlog to work off at present, and growing resistance to new ideas. A recent issue of SIGPLAN Notices (sorry, I forget which) lays the ground rules for proposing extensions to the committee. -- pjp Dear Sir, I read with interest in the November 1992 issue of CUJ, the Q & A discussion of "Check Digits for Error Detection." I was recently involved in a project requiring the entry of credit card numbers into a database for billing purposes. During this project I ran across the check-digit algorithm used for credit card numbers, which may be of interest to Mr. O'Haire. Like the Social Security tradition alluded to by Mr. Pugh, I received this one through word of mouth. However, this one has been verified in daily use with thousands of card numbers and it works for MC/VISA, AMEX, and Discover. It does correct for transposition of adjacent digits, which is a common typing mistake. Starting with the last digit of the credit card number and proceeding in reverse order, multiply each digit of the card number by an alternating sequence of 1s and 2s to form individual sums. In those cases where the multiplication exceeds 10, subtract 9 from the sum. Now add all the sums together and the result must be divisible by 10. That is, the last digit of the card number is fixed to make the total sum divisible by 10. Consider the following example: Card # 4011-7231-9528-4803 Compute 3 x 1 + O x 2 + 8 x 1, etc. to get a total sum of 70, which is divisible by 10. I hope you find this algorithm of interest and of possible use. Sincerely, Christopher R. Skonicki 7304 Jonathan Way Louisville, KY 40228 Thanks. I'm personally always curious about these little algorithms that permeate our lives. -- pjp Editor: I am writing in reference to the "Ross Data Compressoion" article (CUJ, Oct. 1992, p. 113). I have tried the program on several text and binary files and it works. However, there appears to be something not right in the code. In comprs.c (p. 114), the variable ctrl_bits is declared to be of type unsigned integer. And, a couple dozen lines later, it is used in an assignment statement: *ctrl_idx = ctrl_bits; However, ctrl bits is not initialized prior to the assignment. I would like to use this code as a starting point of some work. I cannot, however, without a priori reason, trust an uninitialized variable. I would really appreciate it if you would clear up this little matter. Respectfully, August Grammas 4376 Cove Island Dr. Marietta, GA 30067 Ed Ross replies: The variable ctrl_bits is not initialized for two reasons. First, there is no meaningful value to which it can be initialized. Second, it is not necessary because it will be filled with information, one bit at a time, before it is referenced. Mr. Grammas sees that there is no assignment to ctrl_bits in the listing before it is referenced and wrongly assumes that the program will reference an uninitialized value. He does not see that at runtime ctrl_bits is referenced only when the variable ctrl_cnt indicates that ctrl_bits contains a full 16 bits of information. Sir: Since CUJ is the only computer magazine to which I subscribe, to you I write to vent my frustrations. Earlier this year (June) I ordered software from one of your advertisers, Strategic Software Designs Inc. (SSDI). Although SSDI was happy to accept my money (via VISA), SSDI was unable to deliver the software, and unwilling to answer my phone calls and faxes. Finally, on October 12 a phone call was answered and I was told a VISA credit was being processed. Either I was misled, or the American banking system is in bad shape. To run an application development shop, I cannot afford to waste time or money. Yet I have wasted both on one order. So, why am I writing? To ask for help ... To suggest a "Bravo and Beefs" department--a forum for developers to air their feelings. A place I can read the testimonials of satisfied and dissatisfied customers. A place where next month I will read that 300 people wrote saying they are happy with SSDI products and services, and one person who wrote with the same problem. Or, the opposite. A place for the smaller software houses such as SSDI to be critically reviewed by their customers and peers. The "Letters to the Editor department should be the forum for comments on CUJ quality, content, and style. And since this is a letter to the editor, I will address a comment to the editor: remember, a datum is, but data are. Except for the minor financial loss, and reading too many "the data is," I feel better already. As they say, no pain, no gain. And the fewer CUJs, the fewer informed decisions. Yours truly, Peter Eberhardt Eberhardt Associates, Inc. 288 Laird Dr. Toronto, Ontario Canada M4G 2X3 (416) 429-5705 You make a cogent case for a customer's gripe center, but I doubt we'll ever get into that business wholesale. An occasional plaintive letter such as yours is more than enough to keep me alert to rip-offs. As for the great "data" controversy, I favor its use as a collective noun rather than a plural. The data is read in much the same way the coffee is brewed, and unlike the way the peanuts are shelled. Of course, we could go the home-boy route and pioneer the use of "the data be read in." Somehow, I suspect that would lead to a clash of cultures, however. -- pjp Dear Sirs; I can understand how genetic engineering can create the seedless apple-orange, (see cover of CUJ January 1993) but I am puzzled about where you get the seeds to grow them. Wayne Beard 1825 E. Third Tempe, AZ 85281-2901 You get them the same way you deal with impossible software requirements -- you subcontract. -- pjp Dear Mr. Plauger: re: new unit for angular measurement I was encouraged by Mr. Bertrand's article, which brought the CORDIC technique, involving a different method of dividing up the circle, to my attention. However, I had several complaints about the article, and considered proposing a follow-up article. Looking at the original article by Jack E. Volder reduced most of my objections to mere quibbles. I believe that Mr. Bertrand's very first paragraph should have been presented more emphatically, to stress that the CORDIC technique is not highly accurate and thus is only appropriate for monitor displays (and similar uses). Members of the general public, such as myself, may overlook this point, while trying to grasp the subsequent exposition and code. Much of the appeal of Mr. Volder's technique is in its simplicity. I suspect that this was necessary, in the late 1950s, to achieve adequate speed. This aspect is not obvious from Mr. Bertrand's presentation using C code. However, I do accept the obvious: that modern computers, running C, can still outpace the original CORDIC hardware. On the other hand, I also suspect that we could now afford to improve the CORDIC technique by the judicious use of conditionals. I find it very annoying that 45 degrees can never be exactly represented, even though this is the very first rotation that we encounter. Having landed on it, initially, it seems ludicrous to flop around it for thirteen more rotations, but never get back to it. With only a two-credit C course as preparation, even I could improve things here. In this case, it might even save time. I do not see the expansion of the vector presenting a serious problem, when a three-way choice is possible for the angular rotation at each step (i.e. clockwise, counter-clockwise, or zero rotation). It will be expanded at each step in all three cases. The article also reawakened an interest of mine: devising a new unit of angular measure. While high-level mathematicians may prefer radian measure, most applications are presented in terms of degrees. I believe that the degree is no more appropriate to the computer than is decimal notation. (I have come a long way since I was first offended by computers that did not return degree measurements in minutes and seconds.) Even in the absence of binary-based computers, I would still be troubled by the degree. Ideally, our alternative to the radian would be based on classically constructible angles. These in turn lead us to consider the five known Fermat primes: 3, 5, 17, 257, and 65537. We are also free to bisect any angle repeatedly. Dividing one revolution by three times five times the third power of two only gets us down to three degrees. Obviously dividing by the next three Fermat primes, or further powers of two will never allow us to produce an angle of exactly one or two degrees. We could produce as small an angular unit as we might need by using only a large enough power of two as a divisor, but this would not allow for exact expressions for angles of 60 or 36 degrees. Since these angles, their multiplies, and their repeated halves are required for regular polygons and classic polyhedra, we must include three and five as factors in the divisor. This is a problem with the "CORDIC angle units" (i.e. CAU). The CAU also fail to divide up the circle as finely as do seconds. We need to divide the circle by a number larger than two to the twentieth power just to equal the second. This should not be insurmountable, as even home computers are moving to a 32-bit standard, with 64-bit systems becoming available for engineers and scientists. If you accept three and five as factors of the divisor, the question then becomes: should any of the other Fermat primes also be factors, or should we be satisfied to only use angle bisecting for any further refinement? I am not so sure myself, and this is an important question for a new unit -- if it is to ever gain acceptance. (Note that the degree has sufficed for about three thousand years, prior to the presence of the electronic binary computer. A new unit must do all that the old unit did, and have a clear advantage for use, internally, by a computer, as does the hexadecimal vis-a-vis the decimal number system.) I think that I am borrowing somewhat from the idea that "form should follow function." (I believe that it was D'Arcy Wentworth Thompson that discussed this from a biological standpoint.) In the case of angular measurement, the unit should allow for exact expressions for classically occurring angles. The degree does this. It should also not unduly emphasize insignificant angles. The degree fails here. Consider an angle of 50 degrees. It does not justify special mention, except that it is multiple of the degree. Finally, the new unit should work well in the computer environment; since the computer is able to save so much human effort. Here, the degree is acceptable, but not the radian. (The radian, being essential at the theoretical level, is not threatened in the least.) In general, radical expressions are unavoidable in the values of the trigonometric functions. This implies some inexactness even with the most powerful computers that we can imagine. I believe that this problem is exacerbated by the level of nesting of radicals in expressions (after all denominators are rationalized). Each application of the half-angle formulas add one more to the level of nesting. Constructions based on the Fermat primes have a similar problem, but it does not appear to me to be as bad. For ease of discussion, I will name angles in degrees, and start with 90 degrees; since none of its trigonometric functions involve radicals. The half-angle formulas lead us to 45 degrees with one radical (for all but the tangent). Dividing ninety degrees by three, the first Fermat prime, gives thirty degrees; which also requires one radical (for all but the sine). At the next step, 22.5 degrees requires one radical nested within another. Using the next Fermat prime, five, gives an angle of 18 degrees. Its functions also require one nested radical. Next, 11.25 degrees requires nesting three radicals. This is also the case for the angle produced when 90 degrees is divided by seventeen. The functional expressions are much more complicated than those for 11.25 degrees, but what is important for the sake of accuracy, I believe, on a computer, is that the maximum level of nesting is three. Note that this angle is already less than half of 11.25 degrees. I assert that 5.625 degrees will require nesting of four radicals for its functional values, as will 90/257 degrees (or 0.35019... degrees). The expressions for the former are much easier to obtain, but I believe that the work was done for the latter by 1832; although I have not seen Richelot's article. The point is that the work need only be done once, and then fed into the computer. Even if we only use the first three Fermat primes as factors, we get down to an angle of about 0.35294... degrees (i.e. 90/255), with only the nesting of six radicals. (I believe that this is conservatively stated, as I prefer to have monomial expressions in the numerator, rather than allow polynomials, which might avoid some nesting. This way, subsequent steps, if required, are easier to obtain.) We can come very close to the angle just mentioned, through the use of angle-bisecting only, with 256 as the denominator; but this should require the nesting of eight radicals. It is the nature of these Fermat primes that their continued product is only one less than a power of two. Thus, 3 times 5 times 17 times 257 equals 65,535. (At some point, infinite series will be considered as an alternative to angle bisecting, or constructible angles. I do not yet have any compelling reasons for using 16, 257, or 65537 as factors, and perhaps powers of two would be easier to work with. Definitely, both three and five must be factors.) A denominator (for one revolution) in the order of two to the twenty-fourth power should be more than adequate for any applications within the next few decades, if not centuries. This might be obtained as the product of: 3, 5, 17, 257, and 256. An alternative is offered by the factors: 3, 5, 17, and 65536. Angles are troublesome at best. In general, if we can state the angle exactly, we can not give a expression (in a finite number of terms) for the functional values (e.g. 40 degrees); and if we can express the functional values exactly, we can't give an exact expression for the angle (e.g. acute angles of the 3-4-5, right triangle). This is what makes constructible angles so important: they allow an angle, as well as the trigonometric functions to be stated exactly with a finite number of terms (theoretically, if not practicably). The question may be, whether a new unit is justified. Theoretical work will continue to use the radian, and appplications will continue to only need an adequate approximation. Thank you for your consideration. Sincerely yours, Lem Chastain 8210 4th Avenue, 3-J Brooklyn, New York 11209-4431 I think you lost me more than once there. I, for one, favor the Eurpoean approach of expressing angles in quadrants. (A quadrant equals 90 degrees or p/2 radians.) It sure simplifies computing most of the trignometric functions. -- pjp Dear Sir, After reading "The CORDIC Method for Faster sin and cos Calculations" by Michael Bertrand, I felt that you and Mr. Bertrand might be interested in the fact that the firm I used to work for employed the CORDIC method in its circular interpolation algorithm for its line of Computer (based) Numerical Controls (CNC) since the mid 1970s. The method was chosen for precisely the reasons Mr. Bertrand states: speed and accuracy. In essence, our controls during circular interpolation, in which the control was required to move the slides of a machine tool (usually a milling machine or a lathe), had to compute a new point on a circle of known radius and origin on every real-time clock interrupt. Depending on the type of control, the real-time clock interrupt occurred once every 8 or 10 milliseconds. Our controls were closed-loop systems, which meant that, during the realtime clock interrupt the current positions of the slides were read from A/D converters and correction factors for the error in the previous commanded position also had to be computed and factored into the newly computed position. All of this took time, of course, and the system was a foreground/background arrangement, where all other control functions were performed in the background. We had no floating-point hardware, so finding a method for rapidly (and accurately) computing sines and cosines in software was critical to the success of our circular interpolation scheme. The CORDIC technique fit the bill. We used it first on a control with a CPU based on the instruction set of the Data General Nova 10 mini, and then on a control that used the ubiquitous 8086 and the (not so ubiquitous?) 80186. The interpolation computations were done in fixed-point arithmetic and we held computational accuracy at 32 bits. By the mid 1980s, however the line controls had suffered from the malady of "creeping featuritis" and it was necessary to equip more and more of the controls with with the 8087 math co-processor. Sincerely Yours, Edward Kotlarczyk HC 31 Box 5254 B-1 Wasilia, AK 99654 Dear Mr. Plauger Rodney M. Bates' article ("Debugging with Assertions," The C Users Journal, October 1992) offers two suggestions that may make sense for his applications, but are not universally reasonable. Bates writes that he "leave[s] assertions active in released programs." In Bates' specialty (compilers) there is little harm done by a failed assertion screeching to a halt. But compilers are unusual in this regard. What about transaction-processing, process control, and real-time data acquisition applications? I, for one, don't want an assertion failure to lose track of my ATM deposit. I especially don't want one to turn off my car or shut down the 727's engines while I'm flying over Chicago. Bates also writes that "If a function of yours is called by code written by someone else . . . parameter validity checks you write should be error checks rather than assertions." That's reasonable if you're designing code for public consumption, but it adds complexity, inefficiency and defect risks if you're designing it as part of a larger system. When project team members work together on software they should carefully specify the interfaces between modules and use assertions to test compliance with these interface specifications. This second point brings me to a semantics issue. Bates suggests the terms ensurer and relyer to describe the relationship between the calling and called routines. These terms suggest defensive programming doctrine. Many of us who once followed this doctrine have found it impairs software performance and increases complexity (read: risk of defects.) I suggest compliant and reliant are better terms. A compliant routine complies with, but doesn't necessarily ensure that all data meet the reliant routine's needs. When it doesn't actively ensure data validity, a compliant routine must also be reliant on a compliant calling routine. My previous criticisms aside, this article was sound. It was a much needed overview of assertion. I hope it inspires more programmers to use this simple, powerful tool. Sincerely, William J. Hoyt, Jr. President The Softcraft Labatory 15 Columbus Avenue Middletown, CT 06457 (203) 346-9219 You can use assertions to advantage even in an embedded application. Record the failed assertion, preferably with a time stamp, in a permanent log; then reboot the system. If you're really sophisticated, you can provide several levels of fallback. Pick the appropriate level of panic for each assertion failure. Heavy handed as this approach is, I often find it better than letting the code just stumble onward. -- pjp Dear CUJ: In response to a letter from Bill Casey (CUJ, Vol. 11 No. 01, p. 136) regarding the purchase of an optional source-code companion disk for the book entitled The C Toolbox by William James Hunt, Mr. Casey stated that, "... the programs (listed in the book) referred to other functions which were not available anywhere in the book...". Having read through code in various chapters of the book I must respond to Mr. Casey's observation by asking, "Where? What functions? Please point them out". I've used code fragments from Chapter 5, "Tools for Sorting" and Chapter 6, "BTREE: An Indexed File Module," either having modified blocks of Hunt's code with my own enhancements or just reading Hunt's code to understand his approach to particular problems. Following Hunt's examples has revealed no missing function source that I can see except for Standard C or compiler-specific library calls where you wouldn't expect to see such source or function declarations unless you cruise your compiler's C library header files or license the compiler's library source. It should be noted that on page xvii of the Introduction chapter Hunt states, "Sample solutions for many of the enhancements discussed in the book are included on the source disks." Hunt's source disk is excellent (helpful batch and project files including many enhancement functions/code complete with discussions). The disk also points out some typo errors, one in particular being the correction of a parameter in a function declaration presented in the book. It'd be nice if P.J. Plauger would provide a "corrections only" disk for his The Standard C Library book. (C'mon Plauger, break down, man. Drop it into the public domain). If it appears I feel somewhat stongly about Mr. Casey's accusations of Hunt's book it is only because I have found, The C Toolbox, Second Edition to be one of the few books employing the C language that is not a rehash of K&R and addresses tough problems over a wide host of fronts. One of the few "rides worth the fare." Phil Pistone Chicago II. Dear Editor: Just about the most common programming task required for each application I write for Windows is located at the beginning of WinMain, namely code to ensure that at most one instance of the application can exist. All this code has to do is return to the previous instance, if any. Sounds simple, right? Well, there's no Windows function to do it! Not only that, but the existing published examples I've seen fail in a number of simple situations. Here is some generalized code that you can plug right into your applications to accomplish this common task. (The code is available on the montly code disk.) This code has been tested in several applications. Sincerely yours, David Spector President Springtime Software 81 Amherst Avenue Waltham, MA 02154 617-894-9455 [In correspondence, please reference DS061 to avoid confusion.] Sounds like one of those nuisancy little operations that the designers of Windows forgot to make easy. Thanks for sharing the code. -- pjp Dear Mr Plauger, I work for a small belgian company as Technical Support Manager and am only an occasional C programmers. That is probably why I am faced to my current problem. Some times ago, I have been asked by one of our clients whether I could provide him a tool that would warn him when less than 25%, say, of his disk space were remaining free. As I was not aware of such an existing utility, I decided to write it myself. For that purpose, I used a Borland C++ function called getdfree that is returning disk information in a structure of type dfree declared in dos.h as struct dfree { unsigned df_avail; /* nb of available clusters */ unsigned df_total; /* total nb of clusters */ unsigned df_bsec; /* bytes per sector */ unsigned df_sclus; /* sectors per cluster */ } All the information I needed is there and I calculated the percentage of free space with the formula (unsigned long) dfree.df_avail * 100 / dfree_df_total Then I added a test condition and a warning message. And that was it. All was going well until I gave the program to other customers. Some of them complained that they were getting the warning message although there remained a lot of free space on their machine and so asked me for a more accurate program. I checked their configuration and found that those complaining clients are using very large hard disks (several hundreds of Mb). I tested the program at the office on our very large Novell server volumes and also got similar problems. A more precise debugging showed that the problem is coming from the fact that if the disk has more than 64k clusters (maximum value for an unsigned integer variable), the calculated percentage is wrong. So my question is: do you know a way to have this program works correctly with any disk? Note that there should be one as various utilities like Norton Utilities from Symantec and PCShell from Central Point Software are giving complete and accurate information for any disk. Many thanks in advance, Frederic Naisse Technical Support Manager 30, Clos des Pinsons B-1342 Limelette BELGIUM Dear Dr. Plauger, I thought I'd be the last person to write to any programming magazine about an error, but I have found what appears to be an oversight in your String Library article. Being a processor of strings, I typed in the first listing and found that the str_nmid function returns the wrong value for the returned sub-string. Here, again, is the old zero offset thing. I include my solution to the problem. char *sr_nmid(char *str, size_t pos, size_t num) { char *mid; size_t len = strlen(str); pos--; if(pos >>= len) { /*outside str*/ *str= \0 ; return(str); } if(pos + num >> len) { num= len - pos; } mid = &str[pos]; memmove ((void *)str, (void *)mid, num); str[num] = \0 ; return (str); } My change is the pos-- directive which makes it return the correct value. In passing, I find CUJ simply the best for getting the job done. The others are too full of ads and themselves. Thanks William H. Logan Global Weather Dynamics, Inc. 2400 Garden Road Monterey, CA 93940 The C Users Journal Perhaps an error in the code from article "An Essential String Function Library" by William Smith, January 1993. I believe the routine str_vcat will always duplicate the second argument. Simply deleting all references to Strl will correct the problem. char * str_vcat (char * Dest, char * Strl, ...) char * str_vcat (char * Dest, ...) <=== above line changed to { va_list VarArgList; char * Str; va_start (VarArgList, Dest); Str = va_arg(VarArgList, char *); strcat (Dest, Strl); <====delete this line while (Str != nil) { strcat(Dest, Str); Str = va_arg(VarArgList, char *); } va_end(VarArgList); return (Dest); } M. Thomas Groszko Steelcase Inc. CD-4E-22 6100 East Paris Avenue Caledonia, Michigan 49316-9139 616-698-4580 Testing Your Objects Norman Wilde Norman Wilde has been writing and testing programs since the Fortran II days of the early sixties. Since 1987 he has taught in the Masters of Software Engineering program at the University of West Florida and has done research on software maintenance and on the impact of object orientation on maintenance. With coresearchers from Bellcore, he published an article "Maintaining Object Oriented Software" in the January issue of IEEE Software. Norman can be reached on Internet as wilde@cs.uwf.edu. Testing units individually before integrating them into a software system is critical to efficient system development. In traditional systems, the units are subroutines (for example, C functions), which can be first tested alone, then integrated into larger and larger collections until the system is complete. Detecting and correcting errors is much easier at the unit level than at later stages of integration because the amount of code to be examined is much smaller. For object-oriented programs, the important units are not subroutines but classes of objects (the C++ member functions that act as subroutines are typically too short to warrant individual testing). Unit testing, in this case, means exercising thoroughly the life-cycle behavior of an object: its creation, its state changes, and its destruction. Unit testing entails writing test driver programs--i.e., small programs that set up the data needed to test the unit, call it, and then print out the results of each test. Building test drivers for objects can get complicated, because the object to be tested will often have objects of other classes as members or as member function parameters. Consequently, a good object test driver must construct quite a variety of objects. The example in Listing 1 shows a header for several simplified classes that might appear in an invoicing application. An invoice contains a client and a list of items. The total discount for the invoice depends on the type of client and on the product codes of the items. To test the discount calculation, you need to construct Client and Item objects that are used to create the Invoice object and to exercise the Invoice member functions. A good object test driver must also step through a variety of inputs, trying a range of values for each object (errors often appear only at extreme values, as for example, when the list of items is empty or when it contains just one item--for a discussion of appropriate test data, see the box entitled, "Choosing Test Data"). Finding subtle errors requires checking combinations of circumstances, such as a special product code that appears in a list with just one item. Enumerating the different combinations that should be checked and writing the driver code to generate them can be very tedious. Using a Little Language for Testing To simplify the process of building object test drivers, you can define a "little language" that lets you specify and generate test drivers for the objects in your program. Figure 1 shows the components of the testing system. The driver generator language syntax file DGEN.SYN defines the little language. A parser generator (AnaGram in this case) converts the syntax file into a C program called DGEN.C, which compiles into the driver generator DGEN.EXE. The driver generator reads test specifications written in the little language and produces a C++ test driver, which is compiled and linked with the code for the objects being tested. Listing 2 shows a sample test specification for tests of the Invoice object class from Listing 1. The little language allows C++ style comments, as shown in the first two lines. The block within braces that follows will be simply copied over into the test driver program so that the program will include the headers it needs. There are really only two important kinds of statements in the little language: set variable declarations and runtest statements. A set variable declaration names a set of objects that the test driver will choose from in performing the tests. For example, the declaration Client someClient { [Client(RETAIL)] [Client(WHOLESALE)] [Client(FOREIGN)] } (from Listing 2) describes a set named someClient with three Client objects of different client types, and gives initializers for each object. The declarations that follow this in Listing 2 describe a set of integers called numItems and two identical sets of Item objects called someItem and otherItem. The final set, called theInvoice, has just one member. Note that its initializer uses the someClient set. During each actual test run, the test driver will choose just one member from the someClient set to construct the invoice object. Thus the test driver will do the work of creating objects which contain other objects in different combinations. Each runtest statement specifies a series of actual test runs. For example, the statement runtest "r1" combining someClient theInvoice numItems someItem { ... } tells the driver to select all possible combinations of values from the set variables someClient, theInvoice, numItems and someItem. For each such combination, the code in the block will be executed to test the totalDiscount calculation. As Listing 2 shows, this code writes out information about the objects being used and sends a series of addItem messages to the invoice, followed by a final totalDiscount message. Once you have defined the little language, you can generate flexible test driver programs quite easily. One important feature of this method is that the test specification file provides simple and compact documentation of each test. When the system changes in the future and tests need to be rerun, the test driver can be quickly recreated. It is good practice to make sure that key output from each run is also saved for comparison with future test results. The Little Language for Driver Generation Listing 3 shows the most important part of the AnaGram grammar of the driver generator. (The complete syntax, alone with other files for this technique, is available on the C Users Journal's monthly code disk and via Internet and CompuServe). A test specification simply consists of a series of statements followed by end of file. A statement is either a block of embedded C++ code, a runtest statement, or a declaration. There is one kind of declaration for each class of objects to be used in the test; the example includes Item, Client, and Invoice objects as well as integers. The driver generator parses the test file and identifies each kind of statement. When it encounters a declaration, it calls the addTVars function to add the declared set variables to a symbol table. Strings stored with each variable give its type and its list of initializers. When a runtest statement is parsed, its list of variables is passed up to the makeComb function along with its C++ block. makeComb emits C++ code to construct every combination of the variable initializers and send them to the C++ block. The Generated Test Driver Figure 2 shows a structure chart for the test driver generated from Listing 2. A hierarchy of functions is generated for each runtest statement in the test specification, with each function responsible for setting the value of one of the set variables in the statement. Each function contains a local array of objects for its variable, declared using the initializers in the corresponding variable declaration. The main function generates an integer vector, vals, that represents a combination to be tested. Each other function in the chain uses the value of one element of vals to select a single object out of its local array and pass it to the function below. Thus each function has available a single value for each of the variables above it and can use those values to initialize its own objects. The bottom function contains the block of C++ code from the runtest statement that actually does the test. It has available values for all of the variables. Thus the main function simply needs to generate all possible vectors vals and call the top function in the chain for each one. There are a couple of variations to be considered in using this technique. First, the objects can be passed down the chain either by value or by reference. The C++ default is to pass function arguments by value, which means that a copy of each argument will be made using a copy constructor; if you don't code a copy constructor, then the C++ compiler creates one for you, and it may sometimes have side effects that surprise you! You control the selection of pass-by-reference or pass-by-value in the driver generator syntax file. The addTVars function takes two string arguments that are used in declaring the types of objects. The first is used in declaring the array of objects; the second, in declaring the arguments passed between functions. Thus, for example, in Listing 3 the line addTVars (vl, "Client","Client&", il) says that for Client set variables the array will be declared using a type of "Client" while the function arguments will be declared as "reference to Client". The decision can thus be taken differently for each class of objects. If you pass arguments by reference, then you will have more control over object creation and destruction during tests. However, I have uncovered several memory management bugs in my objects using pass-by-value, since the object constructors and destructors were thoroughly exercised. Another question to consider is whether you really want to try all combinations of values from the sets you have declared. The number of test cases can be very large, and although the running of the test cases is automatic, you still must do the hard work of carefully examining the output for errors. Sometimes it may be sufficient to vary the different objects one at a time, holding the variables not being varied at some standard value. The complete driver generator I have been using allows this option. Conclusions You can't completely eliminate the hard work required for unit-testing your objects: you must think carefully about the values you want to use and the sequences of messages you will send to each object; most important, you must review the output of each test very carefully to ensure that there is no concealed error. By using a little language and a parser generator, however, you can automate a considerable part of the work of setting up and running tests. You still have a lot of work, but you'll be able to devote more effort to inventing good test data and less to the mechanics of the testing process. Further Reading The classic book on software testing is The Art of Software Testing, by Glenford Myers (New York: John Wiley and Sons, 1979). While somewhat old, it is probably still the best source of practical hints on testing software. Although several sources describe the useful technique of developing "little languages" to solve problems, the method still is not used as often as it might be. Jon Bentley gives several examples in Chapter 9 of More Programming Pearls: Confessions of a Coder (Reading, MA: Addison-Wesley, 1988). Parser generators usually are discussed at a level of theory that makes it hard to see how useful these tools can be for tasks other than just compiler writing. John Levine, Tony Mason, and Doug Brown describe the yacc-based tools in lex & yacc, (Sebastopol, CA: O'Reilly & Associates, 1992). Another readable source is chapter 4, "Introduction to Syntax Directed Parsing," of the AnaGram User's Guide (Jerome T. Holland, 22 Forty Acres Dr., Wayland, MA 01778). Choosing Test Data As implied by the title of Glenford Myers' classic book on the subject, The Art of Software Testing, choosing good test data is a combination of science and art. Most methods fall into one of two categories: "black box" or "white box." Black-box methods of finding test data ignore the code and work from its specification, that is, from a knowledge of the task the software must perform. Myers suggests that testers should try to write one test for each equivalence class of inputs--that is, one for each range of inputs for which the program should perform the same way. For example, if a program is specified to accept up to five transactions in each batch, then it makes little sense to run three separate tests for batches of two, three, and four transactions; one test with just one of these values is probably sufficient. Another good black-box rule is to check just inside and outside boundary values, where off-by-one errors may be revealed. Thus in this same example, a good test set would probably include tests with an empty batch, with a one-transaction batch, with a five-transaction batch, and with a six-transaction batch. It is important to include tests with incorrect data, such as the empty batch and the six-transaction batch in this example, since a well-engineered program should behave predictably even when its input is wrong. By contrast, white-box testing involves using the code itself to help find test data. The idea is to make sure that all the code is "covered" in some way, meaning that it all has been tested. Of the several possible criteria for coverage, the simplest is "statement coverage," the goal of which is to ensure that the test set causes every executable statement in the program to be executed at least once. Stronger criteria involve making sure that each side of each branch is taken or guaranteeing that each subcondition within every compound-branch condition is evaluated to both true and false. In doing white-box testing, the tester normally uses a test coverage tool. The tester runs a series of tests, then uses the tool to identify statements, branches, or conditions that have not yet been exercised. The tester then identifies further tests to try to get coverage as close as possible to 100 percent. Both black-box and white-box methods have their advantages. White-box testing is easier to automate and evaluate, but may not find certain kinds of bugs, such as the omission of some required program function. Black-box testing needs more creativity, but may detect more subtle errors. Thorough testing, in OOP or otherwise, should probably involve a combination of both approaches. Parser Generators A parser generator produces programs that read input that is described by a grammar and perform specified actions based on what they find in that input. The earliest application of parser generators was in compiler writing and the best-known parser generator is the early UNIX tool yacc ("yet another compiler compiler"). PC versions of yacc are available from MKS (519-884-2251) and Abraxas (503-244-5353). The examples in this article use the AnaGram grammar-based programming system from Jerome T. Holland (508-358-7968), which provides an integrated PC environment for developing, testing, and debugging grammars, as well as some very useful extensions to conventional parsing techniques. Figure 3 shows how a parser generator works. The syntax file contains "productions," which describe the input that the generated program will handle, and "reduction procedures," which say what actions will be taken or output produced when each production is matched in the input. Listing 3 is an example of part of a syntax file. The productions for a runtest statement give a good example of how the generated parser works. runtest statements look like the following: runtest "r1" combining someClient theInvoice numItems someItem { ... } that is, they consist of the keyword runtest, a run name in quotes, the key word combining, a list of the variables to be used in the run, and finally a C++ block that will be executed for each combination of the variables. The syntax file of Listing 3 declares a runStatement consisting of a runHeader, one or more spaces, and a testSpec. Then runHeader and testSpec are described on the following lines. The production for runHeader shows that it starts with the runtest keyword, followed by spaces and the string with the run name (string is declared elsewhere in the syntax file). This production shows how a user-supplied reduction procedure called setRunName can manipulate the data collected during parsing. A local variable called n is declared to hold the semantic value of the string, in this case a character pointer to the string characters. When a complete runHeader has been identified, the parser will call setRunName (which is coded like any C or C++ function) and pass it this pointer. In this particular case, setRunName simply stores the string into a global variable for later use. The production for testSpec is similar but illustrates more generally how reduction procedures can get the data they need. When a complete testSpec is identified, the makeComb reduction procedure is called (in this case the procedure writes all the code needed for one set of tests). The semantic values it uses are a pointer to the block and a pointer to the variable list; it also makes use of the global run-name string previously saved by setRunName. The production for variableList illustrates recursion, which is very useful for lists of things, and also shows how typed semantic values are created. The variableList production has two rules, saying that a variableList can be either a single variable or a variableList followed by a variable. This is just a recursive way of saying that a variableList is one or more variables. The single variable rule will be the first to be matched during parsing. When it is matched, the next call is to the listNew reduction procedure, which creates a new STRLIST (string list) and returns a pointer to it. The AnaGram-generated parser makes this pointer the semantic value of the variableList; note that the type of the pointer is declared in the first line of the variableList production so that normal C type checking can be enforced. When a second variable is encountered, the second rule will match, since there is now a variableList followed by a variable. Both are passed to the listAdd reduction procedure, which appends the new variable to the list and returns a pointer to the new list. When all variables have been encountered and the complete variableList has been built, the last pointer generated will be passed up to the testSpec production as variable vl. Parser generators can thus be used for a wide range of problems in which a program's main purpose is to operate on structured input. Their big advantage is that the structure of the input is explicit in the syntax file and not distributed through the program, so that changes in input requirements tend to be far easier to accommodate. Figure 1 Components of the testing system Figure 2 Structure chart for the test driver Figure 3 How a parser generator works Listing 1 Header for classes in an invoicing system // File: invoice.h // Copyright Norman Wilde 1993 // Header for classes used in an Invoice #include enum ClientType {RETAIL, WHOLESALE, FOREIGN}; class Item { long prodCode; long quant; float basePrice; friend ostream& operator << (ostream& s, Item& i); public: Item(long aCode, long aQuant, float aPrice ); float value(); long code(); }; class ItemList { Item * theItem; ItemList *nextItem; public: ItemList(Item *anItem ); void addItem(Item *anItem ); Item * currentItem(); ItemList * next(); ~ItemList(); }; class Client { ClientType cType; friend ostream& operator << (ostream& s, Client& c); public: Client(ClientType aClientType); ClientType type(); }; class Invoice { Client *iClient; ItemList *iList; friend ostream& operator << (ostream&s, Invoice& i); public: Invoice(Client *aClient); void addItem(Item *anItem); float totalDiscount(); }; /* End of File */ Listing 2 Sample test specification // File: t001.tst // Tests for Invoice::totalDiscount() { #include "testgen.h" #include "invoice.h" } // set declarations Client someClient { [Client(RETAIL)] [Client(WHOLESALE)] [Client(FOREIGN)] } int numItems {[1] [0] [4] } Item someItem otherItem { [Item(100, 10, 1.00)] [Item(1100, 5, 5.00)] } Invoice theInvoice { [Invoice(&someClient)] } // test run declarations runtest "r1" combining someClient theInvoice numItems someItem { cout << "TESTING CLIENTS & NO. OF ITEMS\n"; cout << "client: " << someClient << "\n"; cout << "items are:\n"; for (int i = 0; i < numItems; i++) { theInvoice.addItem(&someItem); cout <<" "<< someItem << "\n"; } cout << "total discount:" << theInvoice.totalDiscount() << "\n"; } runtest "r2" combining someClient theInvoice someItem otherItem { cout << "TESTING MIXING ITEMS\n"; cout << "client: " << someClient << "\n"; cout << "items are:\n"; theInvoice.addItem(&someItem); cout << " " << someItem << "\n"; theInvoice.addItem(&otherItem); cout << " " << otherItem << "\n"; cout << "total discount:" << theInvoice.totalDiscount() << "\n"; } // End of File Listing 3 Partial grammar for a test specification file // File: DGEN.SYN // Copyright Norman Wilde 1993 // AnaGram grammar for a test specification file // (partial) grammar -> statement..., eof =writeDriver(); statement -> embeddedBlock, space?... -> runStatement, space?... -> declaration, space?... embeddedBlock -> block:eb =addBlock(eb); runStatement -> runHeader, space..., testSpec runHeader -> "runtest", space..., string:n =setRunName(n); testSpec -> "combining", space..., variableList:vl, block:b =makeComb(vl,b); (struct STRLIST *) variableList -> variable:v, space... =listNew(v,""); -> variableList:lst, variable:v, space... =listAdd(lst,v,""); declaration -> intDeclaration -> ItemDeclaration -> ClientDeclaration -> InvoiceDeclaration // Declaration of a set of integers intDeclaration -> "int", space..., variableList:vl, "{", space?..., initializorList:il, "}" =addTVars(vl,"int","int",il); // Declaration of a set of Item objects ItemDeclaration -> "Item", space..., variableList:vl, "{", space?..., initializorList:il, "}" =addTVars(vl,"Item","Item&",il); // Declaration of a set of Client objects ClientDeclaration -> "Client", space..., variableList:vl, "{", space?..., initializorList:il, "}" =addTVars(vl,"Client","Client&",il); // Declaration of a set of Invoice objects InvoiceDeclaration -> "Invoice", space..., variableList:vl, "{", space?..., initializorList:il, "}" =addTVars(vl,"Invoice","Invoice&",il); // End of File Templates in C++ Nicholas Wilt Nicholas Wilt is a software engineer currently residing in eastern Massachusetts. His interests include photorealistic computer graphics, 80x86 assembler and C++ programming. Mr. Wilt welcomes constructive feedback about his work; he can be reached via CompuServe [75210, 2455] or in care of the editorial offices of CUJ. Templates are a recent addition to the C++ programming language. By providing a way to define parameterized data types, they complement the mechanisms for reusability provided by inheritance. Templates come in two related flavors: function templates and class templates. Function Templates The function template in Listing 1 implements Insertion Sort as described by Bentley (1986). Listing 2 contains a program to test the Insertion Sort template. A few points are worth mentioning about the InsertionSort function template. First, it does not declare a function called InsertionSort; rather, it describes a family of over-loaded functions named InsertionSort. If you call InsertionSort with an int *, the compiler generates an instance of the InsertionSort function template that sorts integers. If you call InsertionSort with another pointer type, the compiler generates another instance of the function that can sort that type. This approach to a generic implementation of Insertion Sort compares favorably to the qsort library function's approach, which requires a function call for every comparison. Templates let you implement a sorting routine that is both generic and efficient. InsertionSort can be called for any user-defined class, as well as the primitive data types. If you call it with a user-defined class, you must be sure the copy constructor and assignment operator work properly, and you must overload operator< for the class. Function templates can resolve many issues of code reusability. Insertion Sort is not a very involved algorithm (hence its use as an example), but template implementations of more complicated algorithms such as Quicksort pay off handsomely in reduced maintenance costs. Templates are useful in less generalized contexts as well. Template-based numerical algorithms allow the fast implementation that uses float and the stable implementation that uses double to share the same source code. All the function template does is save you typing and maintenance. Unfortunately, every time you call a function template with a different argument for the parameter, the compiler generates another instance of the function template. As far as the resulting executable is concerned, you might as well have implemented separate InsertionSort functions. For applications where you apply a large variety of types to a function template, this can be an overwhelming use of space. Class Templates Class templates offer even greater potential for code reuse than function templates. They let you describe a family of related classes generically, then provide specific parameters later, as needed. This approach to generality is called a parameterized data type. The template defines a family of related classes, all with members and member functions of the same name that behave similarly. Here is a partial definition of a linked list class template to introduce you to the syntax: template class LinkedList { void Insert(const T&); T *Query(const T&) const; }; Like function templates, the template definition itself does not define any classes. Classes result from instantiating the class template. The above template would be instantiated using the same angle brackets that were used to define it; that is, LinkedList is a linked list of integers class, Linked-List is a linked list of UserDefinedTypes class. Barring their origin, classes that result from instantiating a class template are just like other classes. Templates are especially well-suited to implementing container classes. As an example, I have implemented a balanced binary tree class template. Balanced binary trees are tremendously useful data structures for keeping track of dynamic collections of ordered objects. Almost all operations on a balanced binary tree require only O(logN) time where N is the number of nodes in the tree. These operations include insertion, deletion, query, minimum, and maximum. In addition, if you have a pointer to a node in the binary tree, you can find the predecessor or successor of that node in O(logN) time. The balanced binary tree in this article is a red-black tree as described by Cormen et al. (1990). The binary tree implementation makes use of three class templates: BinaryNode, Binarytree, and BinaryTreeIter. BinaryNode implements the node of a binary tree. Since a binary tree node is really the root of a subtree, BinaryNode implements most of the functionality required for binary trees. It contains some member functions that are protected, including LeftRotate, RightRotate, and DeleteFixup. These functions are used to balance the tree after an insertion or deletion, so they are usable only by classes that inherit from instances of BinaryNode. BinaryTree encapsulates a single, balanced binary tree. It contains a pointer to a BinaryNode (the root of the tree), and makes available member functions such as Insert that are implemented in terms of BinaryNode. BinaryTreeIter, an iterator class for BinaryTree, makes it unnecessary for users of BinaryTree to deal directly with BinaryNodes. Its constructors take a tree as input; the iterator then has a pointer to the root node of the binary tree. Subsequent operations operate on the subtree pointed to by this root node; the Min function, for example, finds the minimum node in the subtree and points the iterator at it. The Contents member function returns a pointer to the contents of the root node in the subtree, or 0 if the iterator is not pointing at anything. BinaryTree, like the InsertionSort function template given earlier, has been designed to work with any class as long as operator< is defined. Listing 3 contains btree.h, the definitions for the three class templates BinaryNode, BinaryTree and BinaryTreeIter. (The code to maintain a red-black tree is extensive, so the source code listing has not been provided in its entirety. Rather, Listing 3 contains the btree.h header, where the class templates are defined.) Listing 4 contains the program that tests the three class templates that implement binary trees. It thoroughly exercises the BinaryTree class by inserting, querying and deleting integers, and checking the invariants for the binary tree between operations. It also uses the BinaryTreeIter class to traverse the nodes and ensure that they appear in the right order. As with function templates, class templates save you typing and maintenance. But every time a class template is instantiated with different arguments, the compiler generates another whole instance of the class. Again, in situations where the compiler instantiates the template many times, this can be an overwhelming use of space. Class Templates And Inheritance Class templates work best when combined with inheritance. In the case of the BinaryTree template, you can extend its capabilities by having another class template inherit from it. This declares a family of classes who all inherit from related members of another family of classes. As an example, I'll develop a template implementation of the order statistics tree, an augmentation of the balanced binary tree as described by Cormen et al. (1990). An order statistics tree supports all the operations that a binary tree supports, plus a very efficient Selection operation. The Selection operation chooses the ith element in the sorted order of an N-element array. For example, numbering from zero, the median of an array for odd N is the (N--1)/2th element. For a single selection operation, optimal O(N) algorithms have been derived. The order statistics tree supports selection in O(logN) time, where N is the number of elements in the tree. Since N elements must be inserted into the order statistics tree before performing selection, the order statistics tree takes O(NlogN) time for a single selection operation. This asymptotic runtime is not efficient compared to the linear algorithms; however, when the number of selection operations grows, the advantage offered by the O(logN) time for a single selection gets more noticeable. The order statistics tree works well when the data sets from which it selects do not change very much between selections. A good application is the median neighborhood operation in image processing. The median operation performs noise reduction by replacing each pixel with the median of its neighbors (including itself, so a total of nine pixels are considered). You might implement this algorithm by reading the pixel values for each pixel of its neighbors and itself, computing the median, and replacing the pixel value. But the data set for the next pixel is similar to the data set for the pixel just computed. (They share three elements.) If you maintain an order statistics tree to compute the median values, pseudocode for a single pixel would look like: Compute median using O(lgN) selection of the OS tree Delete leftmost 3 pixel values from OS tree Insert rightmost 3 pixel values from OS tree The 3x3 median problem has been studied extensively, and the above technique is probably not the most efficient. See Paeth (1990) for an implementation that requires at most 19 comparisons per pixel. For larger neighborhoods, however, more of each pixel's neighborhood is shared with those of its neighbors. At some point the order statistics tree becomes a good way to solve large-kernel median problems. Implementing the Order Statistics Tree To implement the order statistics tree, I derive OSNode and OSTree class templates from the BinaryNode and BinaryTree templates. The syntax is: template class OSNode : public BinaryNode { // class definition }; This declares a template for a family of classes called OSNode. All of the classes in the family inherit publicly from BinaryNode. OSNode is a binary tree node that incorporates the necessary modifications to make it a node in an order statistics tree. Whenever possible, it calls the BinaryNode implementations of its member functions to maintain the order statistics tree. This implementation is inconvenient because the left, right, and parent pointers are declared as BinaryNode *s in the BinaryNode class template. C++ will not automatically promote a pointer to an inherited class to a pointer to its base class. So, because the OSNode class template considers these pointers to point to OSNodes, the code for OSNode member functions contains many typecasts from BinaryNode * to OSNode *. OSTree inherits from BinaryTree just like OSNode inherits from BinaryNode. OSTree does not need to overload as many functions, however, since the node templates do most of the work. For example, the BinaryTree version of DeleteItem works just fine for OSTree. Listing 5 gives os.h, the header for the order statistics tree implementation. Listing 6 gives os.cpp, the C++ program that tests OSTree. Conclusion Function and class templates offer a clean way to write reusable, highly maintainable code, while retaining the efficiency of reimplementing the functions or classes as the need arises. Templates are by no means a panacea, especially when a large number of instances are needed, but they can be a viable option among the innumerable options presented by C++. (C) 1993 Nicholas Wilt. Bibliography Arvo, J. and D. Kirk. 1989. In An Introduction to Ray Tracing. Boston: Academic Press. Bentley, J. 1986. Programming Pearls. Reading, MA: Addison-Wesley. Cormen, T., C. Leiserson and R. Rivest. 1990. Introduction to Algorithms. Cambridge, MA: MIT Press. Paeth, A. 1990. "Median finding on a 3x3 grid,". Graphics Gems. Boston: Academic Press. Preparata, F. P. and M. I. Shamon. 1985. Computational Geometry. New York: Springer-Verlag. Listing 1 Insertion Sort function template // Copyright (C) 1992 by Nicholas Wilt. All rights reserved. template void InsertionSort(T *base, int n) { for (int i = 1; i < n; i++) { T temp = base[i]; for (int j = i; j && temp < base[j - 1]; j--) base[j] = base[j - 1]; base[j] = temp; } } /* End of File */ Listing 2 insort.cpp -- C++ program to test Insertion Sort function template // Copyright (C) 1992 by Nicholas Wilt. All rights reserved. #include #include #include "insort.h" #define NUMINTS 10 int main() { int i; int x[NUMINTS]; randomize(); for (i = 0; i < NUMINTS; i++) { x[i] = rand(); cout << x[i] << ' '; } cout << '\n'; InsertionSort(x, NUMINTS); for (i = 0; i < NUMINTS; i++) cout << x[i] << ' '; cout << '\n'; return 0; } /* End of File */ Listing 3 Definitions for class templates to implement a binary tree // ================================================================= // btree.h // Template-based balanced binary tree class. // Copyright (C) 1992 by Nicholas Wilt. All rights reserved. // ================================================================= #ifndef __BTREE______LINEEND____ #define __BTREE______LINEEND____ // BinaryNode is the class template that does all the work. // All the binary tree primitives are implemented in here. template class BinaryNode { protected: // For node colors enum RedBlack { Black, Red }; public: T x; // Node contents enum RedBlack clr; // Color of node (red or black) BinaryNode *l, *r, *p; // Left, right and parent pointers protected: // Tree manipulations used during insertion and deletion virtual BinaryNode *LeftRotate(BinaryNode *root); virtual BinaryNode *RightRotate(BinaryNode *root); virtual BinaryNode *DeleteFixup(BinaryNode *x, BinaryNode *p); public: // Constructors. Node always starts out red. BinaryNode(); BinaryNode(const T& X, BinaryNode *P = 0, BinaryNode *L = 0, BinaryNode *R = 0); virtual ~BinaryNode() { } static void PostOrderDeletion(BinaryNode *r); virtual BinaryNode *Dup(BinaryNode *P) const; // Operations defined on binary trees. All run in 0(lgN) time. virtual BinaryNode *Min(); virtual BinaryNode *Max(); virtual BinaryNode *Pred(); virtual BinaryNode *Succ(); virtual BinaryNode *Query(const T& q); virtual BinaryNode *InsertNode(BinaryNode *root); virtual BinaryNode *Insert(const T& AddMe); virtual BinaryNode *DeleteNode(BinaryNode *z); virtual BinaryNode *DeleteItem(const T& q); virtual BinaryNode *DeletePassbk(T q, T *passbk); // Returns 0 if the red-black invariant holds. virtual int CheckInvariant(int i, int num); // Returns number of black nodes from root to leftmost node. int BlackToMin(); }; template class BinaryTreeIter { public: // Create iterator for tree, initially pointing at root. BinaryTreeIter(BinaryTree& tree); // Create iterator for tree, initially pointing at the node // queried by q. BinaryTreeIter(BinaryTree& tree, const T& q); // Reset iterator to point to root of given tree. void Reset(BinaryTree& tree); // Returns pointer to the contents of the current node, or // 0 if the current node is 0. T *Contents() const; // Sets iterator to point to minimum node in the subtree. void Min(); // Sets iterator to point to maximum node in the subtree. void Max(); // Sets iterator to point to the current node's predecessor. void Pred(); // Sets iterator to point to the current node's successor. void Succ(); // Queries subtree for the given key. int Query(const T&); protected: BinaryTree *tree; // Pointer to the tree being scanned BinaryNode *subtree; // Subtree currently being considered }; // BinaryTree class template. template class BinaryTree { protected: BinaryNode *root; public: // Default constructor. BinaryTree(); // Copy constructor. BinaryTree(const BinaryTree& x); // Assignment operator. BinaryTree& operator= (const BinaryTree& x); // Destructor. ~BinaryTree(); virtual T *Min() const; virtual T *Max() const; virtual T *Pred(const T& q) const; virtual T *Succ(const T& q) const; virtual T *Query(const T& q) const; virtual void Insert(const T& addme); virtual void DeleteItem(const T& q); virtual void DeletePassbk(const T& q, T *passbk); virtual int IsEmpty() const; virtual int CheckInvariant(); // The following are accessible only to classes that inherit // from BinaryTree, since they deal directly with BinaryNodes. protected: virtual BinaryNode *InsertPassbk(const T& addme); virtual BinaryNode *QueryNode(const T& q) const; virtual void DeleteNode(BinaryNode *delme); friend BinaryTreeIter; }; template BinaryTreeIter::BinaryTreeIter(BinaryTree& Tree) { tree = &Tree; subtree = tree->root; } template BinaryTreeIter::BinaryTreeIter(BinaryTree&Tree, const T& q) { tree = &Tree; subtree = tree->root; if (subtree) subtree = subtree->Query(q); } template void BinaryTreeIter::Reset(BinaryTree& Tree) { tree = &Tree; subtree = tree->root; } template T* BinaryTreeIter::Contents() const { return (subtree) ? &subtree->x : 0; } template void BinaryTreeIter::Min() { if (subtree) subtree= subtree->Min(); } template void BinaryTreeIter::Max() { if (subtree) subtree = subtree->Max(); } template void BinaryTreeIter::Pred() { if (subtree) subtree = subtree->Pred(); } template void BinaryTreeIter::Succ() { if (subtree) subtree = subtree->Succ(); } template int BinaryTreeIter::Query(const T& x) { subtree = (subtree) ? subtree->Query(x) : 0; return subtree != 0; } // Private enum to implement the red-black tree. // enum RedBlack { Black, Red }; template BinaryTree::BinaryTree() { root = 0; } template BinaryTree::BinaryTree(const BinaryTree& x) { root = x.root->Dup(0); } template BinaryTree& BinaryTree::operator=(const BinaryTree& x) { BinaryNode::PostOrderDeletion(root); root = x.root->Dup(0); return *this; } template BinaryTree::~BinaryTree() { BinaryNode::PostOrderDeletion(root); } template T* BinaryTree::Min() const { return (root) ? &root->Min()->x : 0; } template T* BinaryTree::Max() const { return (root) ? &root->Max()->x : 0; } template T* BinaryTree::Pred(const T& q) const { BinaryNode *p = (root) ? root->Query(q) : 0; if (p) { BinaryNode *r = p->Pred(); return (r) ? &r->x : 0; } else return 0; } template T* BinaryTree::Succ(const T& q) const { BinaryNode *p = (root) ? root->Query(q) : 0; if (p) { BinaryNode *r = p->Succ(); return (r) ? &r->x : 0; } else return 0; } template T* BinaryTree::Query(const T& q) const { BinaryNode *p = (root) ? root->Query(q) : 0; return (p) ? &p->x : 0; } template void BinaryTree::Insert(const T& addme) { if (root) root = root->Insert(addme); else root = new BinaryNode(addme); } template BinaryNode * BinaryTree::InsertPassbk(const T& addme) { if (root) root = root->Insert(addme); else root = new BinaryNode(addme); return root->Query(addme); } template void BinaryTree::DeleteItem(const T& q) { if (root) root = root->DeleteItem(q); } template void BinaryTree::DeletePassbk(const T& q, T *passbk) { if (root) root = root->DeletePassbk(q, passbk); } template int BinaryTree::IsEmpty() const { return root == 0; } template int BinaryTree::CheckInvariant() { return (root) ? root->CheckInvariant(0, root->BlackToMin()) : 0; } template BinaryNode * BinaryTree::QueryNode(const T& q) const { return (root) ? root->Query(q) : 0; } template void BinaryTree::DeleteNode(BinaryNode *delme) { if (root) root = root->DeleteNode(delme); } template BinaryNode::BinaryNode() { clr = Red; l = r = p = 0; } template BinaryNode::BinaryNode(const T& X. BinaryNode *p = 0, BinaryNode *L = O, BinaryNode *R = 0): x(X) { clr = Red; p = P; l = L; r = R; } template void BinaryNode::PostOrderDeletion(BinaryNode *r) { if (r) { PostOrderDeletion(r->l); PostOrderDeletion(r->r); delete r; } } template BinaryNode * BinaryNode::Dup(BinaryNode *P) const { BinaryNode *ret = new BinaryNode(x); ret->clr = clr; ret->l = l->Dup(ret); ret->r = r->Dup(ret); ret->p = P; return ret; } template BinaryNode * BinaryNode::Min() { BinaryNode *x = this; while (x && x->l) x = x->l; return x; } template BinaryNode * BinaryNode::Max() { BinaryNode *x = this; while (x && x->r) X = x->r; return x; } template BinaryNode * BinaryNode::Min() { BinaryNode *x = this; while (x && x->l) x = x->l; return x; } template BinaryNode * BinaryNode::Max() { BinaryNode *x = this; while (x && x->r) x = x->r; return x; } template BinaryNode * BinaryNode::Pred() { BinaryNode *trav = this; if (trav->l) return trav->l->Max(); BinaryNode *y = trav->p; while (y && trav == y->l) { trav = y; y = y->p; } return y; } template BinaryNode * BinaryNode::Succ() { BinaryNode *trav = this; if (trav->r) return trav->r->Min(); BinaryNode *y = trav->p; while (y && tray == y->r) { trav = y; y = y->p; } return y; } template BinaryNode * BinaryNode::Query(const T& q) { BinaryNode *trav = this; while (trav) { if (q < trav->x) trav = trav->l; else if (trav->x < q) trav = trav->r; else return tray; } return 0; } template BinaryNode * BinaryNode::LeftRotate(BinaryNode *root) { BinaryNode *ret = root; BinaryNode *y = r; r = y->l; if (r) r->p = this; y->p = p; if (p) { if (this == p->1) p->l = y; else p->r = y; } else ret = y; y->l = this; p = y; return ret; } template BinaryNode * BinaryNode::RightRotate(BinaryNode *root) { BinaryNode *ret = root; BinaryNode *x = l; l = x->r; if (l) l->p = this; x->p = p; if (p) { if (this == p->l) p->l = x; else p->r = x; } else ret = x; x->r= this; p = x; return ret; } template BinaryNode * BinaryNode::InsertNode(BinaryNode *addme) { BinaryNode *root = this; if (! addme->p) root = addme; else { if (addme->x < addme->p->x) addme->p->l = addme; else addme->p->r = addme; } clr = Red; while (addme != root && addme->p->clr == Red) { BinaryNode *y; if (! addme->p->p) break; if (addme->p == addme->p->p->l) { y = addme->p->p->r; if (y && y->clr == Red) { addme->p->clr = Black; y->clr = Black; addme->p->p->clr = Red; addme = addme->p->p; } else { if (addme == addme->p->r) { addme = addme->p; root = addme->LeftRotate(root); } addme->p->clr = Black; if (addme->p->p) { addme->p->p->clr = Red; root = addme->p->p->RightRotate(root); } } } else { y = addme->p->p->l; if (y && y->clr == Red) { addme->p->clr = Black; y->clr = Black; addme->p->p->clr = Red; addme = addme->p->p; } else { if (addme == addme->p->l) { addme = addme->p; root = addme->RightRotate(root); } addme->p->clr = Black; if (addme->p->p) { addme->p->p->clr = Red; root = addme->p->p->LeftRotate(root); } } } } root->clr = Black; return root; } template BinaryNode * BinaryNode::Insert(const T& AddMe) { BinaryNode *x = this; BinaryNode *y = 0; while (x) { y = x; x = (AddMe < x->x) ? x->l : x->r; } BinaryNode *addme = new BinaryNode(AddMe, y); return InsertNode(addme); } template BinaryNode * BinaryNode::DeleteFixup(BinaryNode *x, BinaryNode *p) { BinaryNode *root = this; while (x != root && (! x x->clr == Black)) { BinaryNode *w; if (x == p->1) { if (! p) return root; w = p->r; if (! w) return root; if (w->clr == Red) { w->clr = Black; p->clr = Red; root = p->LeftRotate(root); w = p->r; if (! p ! w) return root; } if ( ((! w->l) w->l->clr == Black) && ((! w->r) w->r->clr == Black)) { w->clr = Red; x = p; p = p->p; continue; } else if ((! w->r) w->r->clr == Black) { w->l->clr = Black; w->clr = Red; root = w->RightRotate(root); w = p->r; if (! p ! w) return root; } w->clr = p->clr; if (p) p->clr = Black; w->r->clr = Black; if (p) root = p->LeftRotate(root); x = root; } else { if (! p) return root; w = p->l; if(! p ! w) return root; if (w->clr == Red){ w->clr = Black; p->clr = Red; root = p->RightRotate(root); w = p->1; if(! p ! w) return root; } if ( ((! w->r) w->r->clr = Black)&& ((! w->l) w->l->clr == Black)){ w->clr = Red; x = p; p = p->p; continue; } else if ((! w->l) w->l->clr == Black){ w->r->clr = Black; w->clr = Red; root=w->LeftRotate(root); w = p->l; if (! p ! w) return root; } w->clr = p->clr; if (p) p->clr = Black; w->l->clr = Black; if (p) root = p->RightRotate(root); x = root; } } if (x) x->clr = Black; return root; } template BinaryNode* BinaryNode::DeleteNode(BinaryNode*z) { BinaryNode*root = this; BinaryNode*x,*y; if (! z) return root; y = (! z->l ! z->r) ? z : z->Succ(); x = (y->l) ?y->l : y->r; if(x) x->p = y->p; if(y->p) { if(y == y->p->l) y->p->l = x; else y->p->r = x; } else root = x; if(y ! = z) z->x = y->x; if(y->clr == Black){ if (root) root = root->DeleteFixup(x,y->p); } delete y; return root; } template BinaryNode* BinaryNode::DeleteItem(const T& q) { return DeleteNode(Query(q)); } template BinaryNode* BinaryNode::DeletePassbk(T q, T *passbk) { BinaryNode*z = Query(q); if(z) *passbk = z->x; return DeleteNode(z); } template int BinaryNode::CheckInvariant(int i, int num) { int ret; if(! l && (i ! = num + clr == Black)) return -1; if(! r && ( i ! = num + clr == Black)) return -1; ret = (l) ? l->CheckInvariant(i + (clr == Black),num) : 0; if (ret) return ret; return r && r->CheckInvariant(i + (clr == Black), num); } template int BinaryNode::BlackToMin() { BinaryNode*trav = this; int ret = 0; while (trav) { ret += (trav->clr == Black); trav = trav->l; } return ret; } #endif /*End of File*/ Listing 4 Program to test class templates in Listing 3 #include "btree.h" #undef VERBOSE #ifdef __TURBOC______LINEEND____ extern unsigned int_stklen = 24576; #endif const int NumInts = 1000; const int NumReps = 1000; int compare_ints(const void *a,const void *b) { return *(const int*) a - *(const int *) b; } int main() { int*arr = new int[NumInts]; int seed; BinaryTree *inttree; randomize(); for (int i = 0; i < NumReps; i++) { int j; int min = INT_MAX; int max = INT_MIN; inttree = new BinaryTree; cout << i << '\n'; for(j = 0; j < NumInts; j++) { int x = rand(); if (x < min) min = x; if (x > max) max = x; arr[j] = x; inttree->Insert(x); if (inttree->CheckInvariant()) cout << "Problems\n"; } if (min ! = *inttree->Min() max ! = *inttree->Max()) cout << "Min/max problems\n"; for (j = 0; j < NumInts; j++){ BinaryTreeIter q(*inttree, arr[j]); if (*q.Contents() ! = arr[j]) cout << "Serious problem\n"; } BinaryTreeIter *inorder = new BinaryTreeIter(*inttree); inorder->Min(); if (min != *inorder->Contents()) cout << "Problems in inorder initialization\n"; j = 0; qsort(arr, NumInts, sizeof(int), compare_ints); while(inorder->Contents()) { inorder->Succ(); int *next = inorder->Contents(); if (next&&(*next < min *next ! = arr[++j])) cout << "Problems in inorder traversal\n"; if (next) min = *next; } for(j = 0; j < NumInts; j++){ inttree->DeleteItem(arr[j]); if(inttree->CheckInvariant()) cout << "Problems deleting\n"; BinaryTreeIter q(*inttree, arr[j]); if (q.Contents() && arr [j+1] ! = arr[j]) cout <<"Problems deleting II\n"; } delete inttree; } return 0; } /*End of File*/ Listing 5 Definitions for order Statistics tree //======================================================== // os.h // Header for order statistics tree (OSTree) class. //======================================================== #include "btree.h" template class OSNode : public BinaryNode { public: int size; public: OSNode(const T& x, int Size); OSNode(const T& x, OSNode *p = 0, OSNode *l = 0, OSNode *r = 0); BinaryNode*LeftRotate(BinaryNode *root); BinaryNode*RightRotate(BinaryNode *root); BinaryNode*DeleteNode(BinaryNode *z); BinaryNode*Insert(const T& AddMe); T *Select(int i); void PrintNodes(); int NumNodes(); int Rank(); friend void CheckNumNodes(BinaryNode*x); }; template OSNode::OSNode(const T& X, int Size): BinaryNode(X) { size = Size; } template OSNode::OSNode(const T& x,OSNode *p = 0, OSNode *l = 0, OSNode *r = 0): BinaryNode(x, p, l, r) { size = 0; } template BinaryNode* OSNode::LeftRotate(BinaryNode *root) { OSNode*ret = (OSNode *) BinaryNode:: LeftRotate(root); ((OSNode *)p)->size = size; size = (l) ? ((OSNode*)1)->size + 1 : 0; size += (r) ? ((OSNode*)r)->size + 1 : 0; return ret; } template BinaryNode* OSNode::RightRotate(BinaryNode *root) { OSNode *ret = (OSNode*)BinaryNode:: RightRotate(root); ((OSNode *)p)->size = size; size = (l) ? ((OSNode*)l)->size + 1 :0; size += (r) ? ((OSNode*)r)->size + 1 :0; return ret; } template T* OSNode::Select(int i) { OSNode *trav = this; while(trav){ int rank = (trav->l) ? ((OSNode *) trav->l)->size+ 1 : 0; if(i == rank) return &trav->x; if (i < rank) trav = (OSNode*) trav->l; else { trav = (OSNode*) trav->r; i -= rank + 1; } } return 0; } template int OSNode::NumNodes() { int ret = (l)?((OSNode*)l)->NumNodes(): 0; ret += (r)?((OSNode*) r)->NumNodes() : 0; return ret + 1; } template int OSNode::Rank() { int ret = (l) ? ((OSNode*)l)->size : 0; return ret + 1; } templatevoid CheckNumNodes(BinaryNode *x) { if(((OSNode *) x)->NumNodes() ! = ((OSNode *) x)->size) cerr <<"Problems\n"; } template BinaryNode* OSNode::DeleteNode(BinaryNode*z) { OSNode *trav; if (z && z->l && z->r) trav = (OSNode *) z->Succ(); else trav = (OSNode*) z; while (trav) { trav->size--; trav = (OSNode *)trav->p; } return BinaryNode::DeleteNode(z); } templateBinaryNode* OSNode::Insert(const T& AddMe) { OSNode *x = this; OSNode *y = 0; while (x) { x->size++; y = x; x = (AddMe < x->x) ? (OSNode *) x->l: (OSNode *) x->r; } OSNode *addme = new OSNode(AddMe,y); return BinaryNode::InsertNode(addme); } templatevoid OSNode::PrintNodes() { OSNode *trav = (OSNode*) Min(); while (trav) { cout <x << "(" <size <<") "; trav = (OSNode *) trav->Succ(); } } templateclass OSTree : public BinaryTree { public: OSTree(); T *Select(inti); void Insert(const T& AddMe); void PrintNodes(); void CheckNodes(); }; template OSTree::OSTree() { root = 0; } template T* OSTree::Select(int i) { return (root) ? ((OSNode *)root)->Select(i) : 0; } template void OSTree::Insert(const T& AddMe) { if (root) root = ((OSNode *)root)->Insert(AddMe); else root = new OSNode(AddMe); } template void OSTree::PrintNodes() { if(root) ((OSNode*)root)->PrintNodes(); } template void OSTree::CheckNodes() { if (root){ BinaryNode *trav = root->Min(); while (trav){ if (((OSNode *)trav)->NumNodes() !=( (OSNode*) trav)->size + 1) cerr << "Problems\n"; trav = trav->Succ(); } } } /*End of File*/ Listing 6 C++ program to test the OStree class template #include #include #include "os.h" #undef VERBOSE extern unsigned int_stklen = 24576; int compare_ints(const void *a, const void *b) { return *(int*)a - *(int *)b; } const int NumInts = 1000; const int NumReps = 1000; int main(int argc, char *argv[]) { int i; int *arr = new int [NumInts]; int seed; OSTree *tree; for (int j = 0; j ; cout << "Repetition" << j <<'\n'; switch (argc){ case 1: if (NumReps == 1) randomize(); else srand(seed = j); break; case 2: srand(atoi(argv[1])); break; case 3: cerr << "Usage:" <Insert(arr[i]); tree->CheckNodes(); #ifdef VERBOSE tree-> PrintNodes(); cout <<'\n'; #endif } #ifdef VERBOSE cout <<'\n'; #endif qsort(arr, NumInts, sizeof(int), compare_ints); for (i = 0; i < NumInts; i++){ int inx; int j, k; for ( j = 0; j < NumInts - i; j++) if (*tree -> Select(j) != arr[j]) { cout << "Problems, seed =" <DeleteItem(arr[inx]); tree->CheckNodes(); #ifdef VERBOSE tree->PrintNodes(); #endif for(j = inx; j < NumInts - 1; j++) arr[j] = arr[j+1]; #ifdef VERBOSE cout << *tree->Select(i) << ' '; cout << '\n'; #endif } #ifdef VERBOSE cout <<'\n'; #endif delete tree; } return 0; } /* End of File */ An Object-Oriented Frog Pond Charles Havener Charlie Havener is a software engineer at Epsilon Data Management in Burlington, MA, where he is developing a custom query language compiler in C++ for the CM5 Connection machine super computer. He holds two masters degrees in EE and CS and teaches an advanced C++ lab course in the Northeastern University State-of-the-Art program. He may be reached on the Internet, cdh@world.std.com or cdh@epsilon.com Introduction In this very special frog pond you will encounter objects that change. A Tadpole will change into a Bullfrog even as an external pointer to it on a linked list does not change. The implementation technique that permits this bit of magic to happen is the powerful C++ "envelope and letter class" idiom. You can find it presented by James Coplien in his book Advanced C++ (Coplien 1992). To construct the various kinds of creatures that are the letter classes, you need a virtual constructor in the Creature base class. You can implement this special kind of constructor by making use of an enumeration type, as in: enum type{ BULLFROG, TADPOLE, ...}; But that is rather unsatisfying. The job can be better accomplished using a variation on the clone technology presented by Tom Cargill in his article "Virtual Constructors Revisited" (Cargill, Sept. 1992). Here there be magic. Watch carefully and see if you can keep your wits, sanity, and sense of humor intact as we all go for a swim in the object-oriented frog pond. A Boring Little Frog Pond Figure 1 shows a first attempt at a class hierarchy to represent various creatures in a pond. I will concentrate on frogs, but you can imagine extending it to different kinds of fish, bugs, and other slimy things. Listing 1 shows how a Pond can be instantiated and populated with creatures. The Pond class consists of an embedded singly-linked list of pointers to creatures as the only data object (see Figure 2), and two member functions. The Pond::insert function sticks freshly created creatures into the linked list by calling the list's insert function. I have chosen to use the generic singly-linked list from the Rogue Wave tools.h++ library, but anything you have available will do. This list just contains pointers to the objects it controls. It does not "own" the objects themselves just because you inserted them. You still own them. The Pond::activate function is a bit trickier. You can imagine the creatures having several member functions, like move and soundoff, that have the same signature. They take no arguments and return nothing. Then, using the weird but useful C++ pointer-to-member syntax, you can bind a pointer to a virtual move or soundoff function in the Creature base class. You pass this pointer as an argument to the activate member function of Pond. The activate function iterates over the list, using the generic singly-linked list iterator in the case of Rogue Wave. That's how you broadcast the desired function activation request to all creatures in the pond. Simple huh? It is easy to imagine storing lots of additional information about the frogs in the Frog base class. Some examples are gender, color, and age. But what about age? Remember, as tadpoles age they turn into frogs. Changing an Object in Situ Say the Creature base class contained a pointer to a creature as a data member and Creature::move forwarded the move operation through this pointer. Then you could alter the pointer somehow and thus effectively change the body of the creature at runtime. This interesting technique is called the "envelope and letter class" idiom (depicted in Figure 3), as I mentioned earlier. The "letter" is one of a set of classes publicly derived from the base or "envelope" class. The outside world accesses the letter only through the envelope. Listing 2 shows how the Creature base class has been fleshed out with a data member that points to a creature. The virtual move member function merely passes the work along to the letter object through the pointer. Note that the constructor now takes an enumerated type as one of its arguments. Using an enumerated type is not considered good style. It means the base class must know about all of its derived children and must be modified any time a new derived class is invented. To add different kinds of fish, you have to alter the base class enumerated type. You also have to alter the so-called virtual constructor for class Creature. It has an ugly case statement, the goto of object-oriented programming. Another interesting new aspect of Listing 2 is the protected constructor in class Creature. This or some similar mechanism is needed to initialize the Creature class when it is part of the letter class. That is, when a Frog is instantiated, some constructor for the base class Creature must be invoked. In this example you don't need a default constructor for Creature. It is not meant to be instantiated by itself. Bruce Eckel shows a good approach in his article "Virtual Constructors" (Eckel Mar/Apr 1992). A protected default constructor does the job nicely. Each Creature now has an envelope with the object pointer set to the letter, and a letter with its object pointer set to zero. This approach wastes some space. Attempts to reuse data that is common to the envelope and letter by placing it in a union requires the utmost care. If we needed a default constructor in the base class, then we just make a constructor in the base Creature class that takes a funny argument such as a class Noop{}. (See Coplien's polymorphic Number example in Advanced C++.) The constructors, especially the copy constructor, of the derived classes must be sure to invoke the special base constructor or there can be infinite recursion. Clearly Listing 2 is less than perfect. It shares the defect with Listing 1 of using plain old pointer to char instead of a const pointer to char. Furthermore, copy constructors are needed everywhere or utter failure at runtime is guaranteed. And we have not yet done any object changing in place. So let's move on. A Complete Solution Listing 3 combines everything I have discussed, together with an extension to the clone technology presented by Cargill in "Virtual Constructors Revisited" to eliminate the need for an enumerated type and a case statement. Cargill optimizes away the very need for an envelope and letter class, but here I want to keep it. So the clone techniques have to be carefully applied. Careful attention has been paid to getting all the copy constructors properly implemented, an essential part of success. Not to beat a dead horse, but copy constructors are frequently left out when the class has either no data or only simple predefined types such as integers. However, when inheritance is involved, as here, explicit copy constructors are needed to invoke the base-class copy constructors. Even Coplien accidently left out the copy constructors from an "envelope and letter" example in the first edition of Advanced C++. Let's consider how things change in place before looking closer at the clone technique. The first thing to notice is that the Creature base class has sprouted quite a few additional functions and a static data member, altered, which is also a pointer to Creature. This pointer is needed so the move member function in the envelope Creature base class can test it after the letter class move is called. If the letter class has decided to change itself into another object, it must communicate this desire to the envelope in some way. It can't do so itself. That is like pushing on a string. The letter class has absolutely no way of communicating to the envelope except through a return value from a function, such as move, or through a global variable. The static Creature variable altered is a restricted global variable. It serves the purpose so long as the class doesn't have to be re-entrant. It must be static or the letter class would just be setting the variable called altered in its own base and not in the envelope base. Remember, there are two copies of all the data in the Creature base class. Rather than do anything complicated for this example in terms of time and ageing, the Tadpole::move function just increments the age each time it is called. It makes a new Bullfrog when the time comes and sets the static class variable altered for use by the envelope class. The move member function in the envelope class tests altered. If the variable is non-zero, the function deletes the existing pointer to Creature, Tadpole for example, and replaces it with altered, which in this case is a pointer to Bullfrog. Finally the function resets altered to zero. The clone technique uses a static class member newCreature(const Creature&) of class Creature to make new creatures. Every class now has a clone member function which returns a pointer to Creature. These clone functions return pointers to objects that will be letter objects. The static newCreature(Creature &) function requires a Creature constructor that takes a pointer to a creature as its argument. This is absolutely essential because the newCreature function must return a pointer to an envelope Creature, not a letter Creature. If you think you've been having a bad day, just try debugging an envelope/letter program when you accidently pass around pointers to letter objects instead of envelope objects! Recall that the Creature pointer in the letter object is set to zero. Trying to pass work off through a null pointer does bad things. C++ does an outstanding job of finding bugs at compile time by enforcing type checks. However, the envelope/letter idiom moves a program into the realm of runtime type modification. It is much easier to get in trouble. The main program creates new creatures of any kind by invoking the static Creature::newCreature function and passing it a reference to some creature to be cloned. If the kind of creature you want is not already extant, make an anonymous one, as shown in Listing 3, which the compiler will arrange to delete at its convenience. Presumably, it would be desirable to modify the attributes of the new clone using various member functions in a straightforward fashion. This example merely maintains the name. Conclusion You have seen a cute but usable example of the "envelope and letter class" idiom. You have also seen how it can be combined with clone technology to provide an extendable framework for situations that require objects that change in place at runtime. It's not an easy idiom to understand or to get right. But once you master it, you will find it to be a powerful addition to your kit of C++ tools. (C) 1993 Charles Havener References Cargill, Tom. September 1992. "Virtual Constructors Revisited," C++ Report. Coplien, James O. 1992. Advanced C++. Reading, MA: Addison-Wesley. Eckel, Bruce. March/April 1992. "Virtual Constructors," C++ Report. Figure 1 A simple class hierarchy Figure 2 An embedded, singly-linked list of pointers to creatures Figure 3 Creature envelope and letter Listing 1 Program showing how a pond can be instantiated and populated with creatures // main.cc - Pond main program Listing 1 #include main() { PMF_VV pf = &Creature::move; // bound to a virtual Tadpole wiggly("Wiggly"); Bullfrog croaker("Croaker"); Pond walden; walden.insert(&wiggly); walden.insert(&croaker); walden.activate( pf ); // Tell all residents to move return 0; } ------------ output ----------------- Wiggly Swimming jerkily Croaker - I'm jumping ----------------------------------- // Frog.h #ifndef FROG_H #define FROG_H #include #include // Rogue Wave tools.h++ class Creature { public: virtual void move() {} virtual ~Creature() {} }; declare(GSlist, Creature) class Frog : public Creature { protected: char *name; public: Frog(char *p="Noname") { name = p; } char *Name() { return name; } virtual void move() {} virtual ~Frog() {} }; class Tadpole : public Frog { public: Tadpole(char *name) : Frog(name) {}; void move(){cout << name << " Swimming jerkily\n";} }; class Bullfrog : public Frog { public: Bullfrog(char *name) : Frog(name) {}; void move(){cout << name << " - I'm jumping\n"; } }; class Pond { GSlist(Creature) list; // singly linked creature list public: // activate takes a ptr to any member func of Frog // that takes a void arg and returns a void // It turns out move() is the only one so far // It iterates over all objects derived from // Creature base class in the Pond void activate( void (Creature::*pf) () ); void insert(Creature *); }; // this typedef must be after the class declarations // it may be useful to simplify declarations in the users // program. useful for pointing to move(void) typedef void (Creature::*PMF_VV)(); #endif -------------------------------------------------------- // Frog.cpp #include #include void Pond::insert(Creature *f) { list.insert(f); } void Pond::activate( PMF_VV pmf_vv ) { GSlistIterator(Creature) it(list); Creature *resident; while ( resident = (Creature )it() ) { (resident->*pmf_vv)(); } } // End of File Listing 2 Creature base class with a data member that points to a creature // main.cpp - Frog Pond Listing 2 #include main() { PMF_VV pf = &Creature::move; // bound to a virtual Pond walden; Creature *p1 = new Creature(Creature::TADPOLE,"Wiggly"); walden.insert(p1); Creature *p2 = new Creature(Creature::BULLFROG,"Croaker"); walden.insert(p2); walden.activate( pf ); // Tell all creatures to move return 0; } /-------------------------------------------------------- // Frog.h #ifndef FROG_H #define FROG_H #include #include class Creature { public: enum type{BULLFROG,TADPOLE}; Creature(type,char *); virtual void move() {object->move(); } protected: Creature() : object(0) {}// default invoked by derived private: Creature *object; }; declare (GSlist, Creature) class Frog : public Creature { protected: char *name; public: Frog(char *p="Noname Frog") : name(p) {} ~Frog (); char *Name() { return name; } virtual void move() (} }; class Tadpole : public Frog { public: Tadpole(char *name) : Frog(name) {}; void move(){cout << name << " Swimming jerkily\n"; } }; class Bullfrog : public Frog { public: Bullfrog(char *name) : Frog(name) {}; void move() { cout << name << " - I'M JUMPING\n"; } }; class Pond { GSlist(Creature) list; // singly linked creature list public: void activate( void (Creature::*pf) () ); void insert(Creature*); }; // this typedef must be after the class declarations // it may be useful to simplify declarations in the users // program. useful for pointing to move(void) typedef void (Creature::*PMF_VV)(); #endif //-------------------------------------------------------- // Frog.cpp #include #include void Pond::insert(Creature *f) { list.insert(f); } void Pond::activate( PMF_VV pmf_vv ) { GSlistIterator(Creature) it(list); Creature *resident; while ( resident = it() ) { (resident->*pmf_vv)(); } } Creature::Creature(Creature::type t,char *name) { if ( t == TADPOLE ) object = new Tadpole(name); else if( t == BULLFROG) object = new Bullfrog(name); else { cerr << "Invalid type in Creature ctor/n"; exit(2); } } Frog::~Frog(){} // End of File Listing 3 The complete frog pond program // main.cpp - Frog Pond Listing 3 #include main() { PMF_VV pf = &Creature::move; // bound to a virtual Pond walden; Creature *p1; Creature *p2; p1 = Creature::newCreature(Tadpole("Wiggly") ); walden.insert(p1); p2 = Creature::newCreature(Bullfrog("Croaker") ); walden.insert(p2); cout << "First time ------------------\n"; walden.activate( pf ); // Tell all creatures to move cout << "second time -----------------\n"; walden.activate( pf ); // Tell all creatures to move cout << "third time ------------------\n"; walden.activate( pf ); // Tell all creatures to move return 0; } First time -------------------- WigglyAge 1 Swimming jerkily Croaker - I'm jumping second time ------------------- Wiggly Age 2 Swimming jerkily still, but changing into Bullfrog Croaker - I'm jumping third time -------------------- Wiggly - I'm jumping Croaker - I'm jumping // Frog.h #ifndef FROG_H #define FROG_H #include #finclude class Creature { public: Creature(Creature *ptr); Creature(const Creature &c) : object(c.object) {} virtual void move(); virtual Creature *clone() const; static Creature *newCreature(const Creature&); virtual const char *getname() const {return object->getname(); } virtual ~Creature(); // void change(Creature *c); protected: Creature() { object = 0; } // invoked by derived Creature *object; static Creature *altered; }; declare(GSlist,Creature) class Frog : public Creature { protected: const char *name; public: Frog(const char *p); Frog(const Frog &f); ~Frog(); virtual void move() = 0; virtual const char *getname() const { return name; } Creature * clone() const; }; class Tadpole : public Frog { int age; public: Tadpole(const char *name = "Noname Tadpole"); void move(); Tadpole(const Tadpole& t) : Frog(t), age(t.age) { } // invoke Frog copy ctor Creature * clone() const; }; class Bullfrog : public Frog { public: Bullfrog(const char *name="Noname frog"); Bullfrog(const Bullfrog &b) : Frog(b) {} void move() {cout << name <<" - I'm jumping\n"; } Creature * clone() const { return new Bullfrog(*this); } class Pond { GSlist(Creature) list; // singly linked creature list public: void activate( void (Creature::*pf) () ); void insert(Creature*); }; typedef void (Creature::*PMF_VV)(); #endif // Frog.cpp #include #include #include Creature *Creature::altered = 0; // private static data void Pond::insert(Creature *f) { list.insert(f); } void Pond::activate( PMF_VV pmf_vv ) { GSlistIterator(Creature) it(list); Creature *resident; while ( resident = (Creature *)it() ) { (resident->*pmf_vv)(); } } Creature *Creature::newCreature(const Creature& prototype) { Creature *C = new Creature(prototype.clone()); // we must return an envelope, clone just makes a letter return C; } Creature * Tadpole::clone() const { Tadpole *t = new Tadpole(*this); // Tadpole copy ctor return t; } Creature::Creature(Creature *objptr) { object = objptr; } Creature::~Creature() { } Creature *Creature::clone() const { Creature *p = object->clone(); Creature *envelope = new Creature(p); return envelope; } void Creature::move() { object->move(); if ( altered ) { delete object; object = altered; altered = 0; } } Creature * Frog::clone() const // never get here { return 0; } Frog::Frog(const char *n) : name(n) { } Frog::Frog(const Frog &f) : Creature(f), name (f.getname()) { } Tadpole::Tadpole(const char *n) : Frog(n),age(0) { } Bullfrog::Bullfrog(const char *n) : Frog(n) { } void Tadpole::move() { age++; if ( age < 2 ) cout << name << "Age " << age << " Swimming jerkily\n"; else { cout << name <<" Age " << age << " Swimming jerkily still, but changing into Bullfrog\n"; Creature *f=Creature::newCreature(Bullfrog(name)); Creature::altered = f; } } Frog::~Frog() { } // End of File Performance Tuning a Complex FFT Tim Prince Tim Prince has a B.A. in physics from Harvard and a Ph.D in mechanical engineering from the University of Cincinnati. He has 25 years of experience in aerodynamic design and computational analysis. He can be reached at Box 3224, Rancho Santa Fe CA 92075. Introduction Much of the current lore on how to program in C and other languages for best execution speed is based on older architectures such as PDP, VAX, and 8088, and early compilers. Modern compilers for RISC architectures must do a good job of automatic register allocation and instruction scheduling or lose the benchmark wars. While much excellent literature on algorithms is available, published code often falls short on performance. In this article, I show step-by-step performance enhancements for a commonly-used algorithm, the complex fast Fourier transform. This algorithm is taken as a typical example of resolution of performance bottlenecks, as much as for its practical interest. Basic Code The starting point is the complex FFT code given by Press et al. (1988), slightly modified for compatibility with their earlier FORTRAN version. (The original is credited to N. Brenner.) Using single precision (float) data, the overhead of trig function evaluations is minimized by calculating sin only in the outer loop, using double arithmetic to prevent error accumulation from degrading the results. The modifications consist of passing arguments by reference, using zero-based arrays, and not using shift operators when multiplication is meant, as modern compilers make these conversions internally. Each version has been tested using the test drivers published by the same authors. Timing tests are performed on batches of 1,000 transforms using 1,024 data points, using clock. The published version took 4.6 milliseconds for a 1,024 transform on an HP730. Precision Conversions Outside Inner Loop The code was originally written for K&R compilers. Some of these compilers did not round the tempr and tempi results down to float precision, as they tried to promote all floating-point arithmetic to double. Declaring all local variables double produces slightly better accuracy than the original code, at least when Standard C compilers are used. If more than seven significant digits are required, everything including the data[] arrays must be typed double. Placing (float) casts on the double variables in the inner loop permits the compiler to move these data conversions to the pre-loop setup code block, helping architectures that do not perform operations of mixed precision efficiently. This change does not reduce the accuracy significantly. The 1,024 transform now takes 3.85 milliseconds. Help Compiler Optimize Data Fetches Sufficient information is contained in the code to prove that j is always greater than i+1 where the data arrays are accessed, enabling a hypothetical optimizing compiler to determine that the j subscripts cannot point to the same data as the i subscripts. Existing compilers do not perform such an analysis. In the original version, the generated code allowed for the possibility that modifying data[j] and data[j+1] (which were stored first) might change data[i] and data[i+1]. This prevents the use of register values of data[i] and data[i+1]. Most compilers do take advantage of the non-overlap of pointers that differ by a known constant. The cure is simple--copy the data into a local register-eligible variable that is not affected by assignments to array elements. With this change, the compiler is free to generate code that fetches the data sooner. Earlier data fetches reduce the impact of cache misses, although this should not be a factor in the present application. Now the same FFT is performed in 3.6 milliseconds. Simplify Use of Trig Functions As the published code calculates cos(theta) from sin(theta/2), and sin(theta/2) is a value that will be used in the next outer loop iteration, rearrangement allows just one computation of sin per outer loop iteration. The published code, rather than adding 1 in this calculation of cos, delays the addition to the end of the recursion formula in the middle loop. This is a standard technique for preservation of accuracy, but it is unnecessary since the recursion formulae are already coded in higher precision than the inner loop. With cos calculated in the outer loop, the benchmark timing is 3.5 milliseconds. Numerical results were not affected by this substitution. By tinkering with the assembly code, it is possible to get this down to 3.3 seconds. Since the cache hit rate is higher than normal, certain load instructions may be delayed to places where an instruction requires the result of the previous instruction, so that all instructions are given two cycles to execute. The final source code is shown in Listing 1. Pipeline Priming (Partial Unrolling) All of the modifications discussed previously should be helpful or at least neutral on any modern computer. We have gained 30% in speed on the HP computer, without doing anything specific for that architecture. As most instructions on the HP take more than one instruction clock cycle to execute, the compiler attempts to schedule instructions so that operations can overlap. In several places, the generated code does not have the optimum number of instructions between an instruction that changes the contents of a register and one that uses the new contents of the register. Because of the sequential nature of the code, only two of the four multiplication operations are executed in parallel with addition operations. Two of the last four add and subtract operations are capable of being written as two-operand operations, without introducing an extra register copy. These are eligible to be executed in parallel with a multiply operation on the HP. If you want to get more parallelism and allow more cycles for completion of each instruction, you need loop unrolling. Loop unrolling techniques performed by many current compilers increase the time required to start up the loop. This is unsatisfactory, as the average inner loop count for an FFT of order 1024 is only 5. You can prime the pump, so to speak, by rearranging the code to permit pipelines to fill sooner. As the code normally is written, the only operations that are ready to be performed at the top of the loop are data fetches from memory to register. You need enough data in registers to permit the floating-point arithmetic pipelines to start immediately. On the HP architecture, you need to perform the first two multiplications also at the end of the previous iteration of the loop, so that they may be performed in parallel with addition or subtraction operations. This keeps the pipelines full closer to the end of the loop body. One copy of the first part of the loop body is placed ahead and the other at the end of the iterated part of the loop. The remainder of the original loop body is copied after the iterated part of the loop. All of the data used in the new loop body are copied into registers before any results are stored in the data arrays, in order to get around the lack of compiler analysis for data overlap. Changing the way j is calculated for the next loop iteration saves us from having to calculate it in the post-loop body, but makes no difference otherwise. Arranging the conditions to run one less iteration of the loop, you end up performing exactly the same job, except that there are two copies of all of the loop body code and, in effect, one iteration of the loop is always performed. The unrolled inner loop is shown in Listing 2. If it were necessary to allow for a zero iteration case, the whole unrolled inner loop could be placed within an if block. In this application, the original inner loop could have been written as a do-while, to emphasize the fact that at least one iteration is always taken. Although this saves a couple of instructions, there is no measurable change in performance. Current compilers transform a while(){} loop into if() do{} while() when that will improve performance. The code will perform nearly as well if the inner loop is allowed to iterate once more, so that the duplicate code for the latter part of the original loop is not needed. The data array must be made large enough to avoid picking up data outside the array. Zero values will do in the extended part of the array. Values that generate underflows or exceptions will take a long time even though they do not affect the results. For the FFT algorithm, arrangements similar to Listing 2 are useful on a range of architectures. The benchmark now executes in 3.1 milliseconds, a 45% improvement over the original code. The effect of unrolling will be greater on an architecture that takes more advantage of parallelism than the HP 7x0 series. While the code generated by the HP compiler does not appear to be optimum, no significant additional performance has been achieved by tinkering with it. Conclusion A published implementation of the complex FFT can be speeded up 30% by simple changes in the source code that eliminate unnecessary data conversions and redundant memory access. Execution speed can be increased another 12% by appropriate source code unrolling techniques on a typical pipelined RISC workstation. These performance increases are larger when cache hit rate is reduced. The unrolling technique shown performs well even with a small number of loop iterations. References Press, Flannery, Teukolsky, Vettering. 1988. Numerical Recipes in C. Cambridge University Press. Vettering, Teukolsky, Press, Flannery. 1985. Numerical Recipes Example Book (FORTRAN). Cambridge University Press. Listing 1 Tuning an FFT by simplifying the use of trig functions #include #define SWAP(a,b) tempr=a;a=b;b=tempr void four1(float *data, int *nn, int *isign) { /* altered for consistency * with original FORTRAN */ /* Press, Flannery, Teukolsky, Vettering "Numerical * Recipes in C" tuned up ; Code works only when *nn is * a power of 2 */ int n, mmax, m, j, i; double wtemp, wr, wpr, wpi, wi, theta, wpin; float tempr, tempi, datar, datai; n = *nn * 2; j = 0; for (i = 0; i < n; i += 2) { if (j > i) { /* could use j>i+1 to help * compiler analysis */ SWAP(data[j], data[i]); SWAP(data[j + 1], data[i + 1]); } m = *nn; while (m >= 2 && j >= m) { j -= m; m >>= 1; } j+=m; } theta = 3.141592653589795 * .5; if (*isign < 0) theta = -theta; wpin = 0; /* sin(+-PI) */ for (mmax = 2; n > mmax; mmax *= 2) { wpi = wpin; wpin = sin(theta); wpr = 1 - wpin * wpin - wpin * wpin; /* cos(theta*2) */ theta *= .5; wr = 1; wi = 0; for (m = 0; m < mmax; m += 2) { for (i = m; i < n; i += mmax * 2) { j = i + mmax; /* mixed precision not significantly more * accurate here; if removing float casts, * tempr and tempi should be double */ tempr = (float) wr*data[j] - (float) wi *data[j + 1]; tempi = (float) wr *data[j + 1] + (float) wi *data[j]; /* don't expect compiler to analyze j > * i+1; original code stored data[j..] * first and avoided using data[ri] */ data[i] = (datar= data[i]) + tempr; data[i + 1] = (datai = data[i + 1]) + tempi; data[j] = datar - tempr; data[j + 1] = datai - tempi; } wr= (wtemp = wr) * wpr - wi * wpi; wi = wtemp * wpi + wi * wpr; } } } /* End of File */ Listing 2 An unrolled inner loop for (m = 0; m < mmax; m += 2) { j = m + mmax; tempr = (float) wr *(data1r = data[j]); tempi = (float) wi *(data1i = data[j + 1]); for (i = m; i < n - mmax * 2; i += mmax * 2) { /* mixed precision not significantly more * accurate here; if removing float casts, * tempr and temp1 should be double */ tempr -= tempi; tempi = (float) wr*data1i + (float) wi *data1r; /* don't expect compiler to analyze j > i+1 */ data1r = data[j + mmax * 2]; data1i = data[j + mmax * 2 + 1]; data[i] = (datar = data[i]) + tempr; data[i + 1] = (datai = data[i + 1]) + tempi; data[j] = datar - tempr; data[j + 1] = datai - tempi; tempr = (float) wr *data1r; tempi = (float) wi *data1i; j += mmax * 2; } tempr -= tempi; tempi = (float) wr *data1i + (float) wi *data1r; data[i] = (datar = data[i]) + tempr; data[i + 1] * (datai = data[i + 1]) + tempi; data[j] = datar - tempr; data[j + 1] = datai - tempi; wr= (wtemp = wr) * wpr - wi * wpi; wi = wtemp * wpi + wi * wpr; } /* End of File */ The Windows Messaging System Mike Klein This article is not available in electronic form. Standard C Large Character Set Support P.J. Plauger P.J. Plauger is senior editor of The C Users Journal. He is convenor of the ISO C standards committee, WG14, and active on the C++ committee, WG21. His latest books are The Standard C Library, published by Prentice-Hall, and ANSI and ISO Standard C(with Jim Brodie), published by Microsoft Press. You can reach him at pjp@plauger.com. Introduction Last month, I described the new machinery for processing Defect Reports--to interpret or correct the C Standard. (See "State of the Art: Formal Changes to C," C Users Journal, April 1993.) I also outlined the "normative addendum" that will add a number of features to Standard C. Those changes are now being balloted for approval. Some are designed to help programmers write C source code that is more readable. I described them last month. But most of the changes are designed to ease writing programs that manipulate large character sets, a topic of growing interest. My goal this month is to start showing you the other, more extensive group of changes in the works for Standard C. Understand, they are still subject to modification. Member nations of ISO, and other interested parties, get several rounds of comments before we in WG14 freeze the normative addendum. I like to think, however, that this particular material is reasonably stable. It should give you a portent of how Standard C will look in another year or so. Equally important, my goal is to suggest some ways you might actually use these new capabilities. To some extent, the conversion to using a large character set is simple. Just look for all the places where you manipulate text as type char. Replace them with objects and values of type wchar_t. Whatever the implementation chooses for a large character set, its codes must be representable as that type. But all sorts of subtle changes must occur as well. I'll endeavor to show you some of them. The Lightweight Approach I begin by emphasizing that many programs can already speak Japanese, as it were. Programs often just read and write text without interpretation. You can feed them strings containing single-character and multibyte sequences intermixed, and they don't mess them up. (If they do, you can often fix the programs with little effort. Typically, you just stop presuming that you can break a string of text arbitrarily between any two characters.) A display or printer that does the right thing with these multibyte sequences may well show Kanji characters, just the way the user desires. Your program neither knows nor cares what happens to them outside its boundaries. Multibyte is usually the representation of choice outside programs. If you plan to do little or no manipulation of such text, it can also be the representation of choice inside a program as well. But lets say you have occasion to manipulate small amounts of text from a large character set within your program. You need to distinguish character boundaries so you can rearrange text or insert other characters. In that case, wide characters are the representation of choice inside the program. You want to deal, at least sometimes, in elements and arrays of type wchar_t. Standard C already provides a minimal set of functions for converting back and forth between multibyte and wide character. You can also write wide-character constants, as L'x', and wide-character strings, as L"abc". Several people have demonstrated that you can do quite a bit of work with just what's already standardized. (See, for example, my book, The Standard C Library, Prentice Hall, 1992.) One thing we chose not to add to the original C Standard, however, was the ability to read and write wide characters directly. A format string can contain multibyte characters as part of its literal text, but you cannot easily read and write data of type wchar_t. You have to convert wide characters to a multibyte string in a buffer before you print it out. Or you have to read a multibyte string into a buffer before you convert it to wide characters. That can be a nuisance. Print and Scan Additions So one of the changes in the normative addendum is to add capabilities to the existing print and scan functions. These are the functions declared in with print or scan as part of their names. Each family of functions now recognizes two new conversion specifiers, %C and %S. For the print functions: %C writes to the output the multibyte sequence corresponding to the wchar_t argument. A shift-dependent encoding begins and ends in the initial shift sequence. That can result in redundant shift codes in the output, but the print functions make no attempt to remove them. %S writes to the output the multibyte sequence corresponding to the null-terminated wide-character string pointed to by the pointer to wchar_t argument. The same rules apply for shift codes as for %C. You can use a precision, as in %.5S, to limit the number of array elements converted. In no event will either of these conversion specifiers produce a partial multibyte sequence. They may, however, report an error. A wide-character encoding may consider certain values of type wchar_t invalid. Try to convert one and the print functions will store the value of the macro EILSEQ in errno. That macro is now added to the header . As usual, the behavior of the scan functions for the same conversion specifiers is similar, but different from that for the print functions: %C reads from the input a multibyte sequence and converts it to a sequence of wide characters. It stores these characters in the array designated by the pointer to wchar_t argument. You can use the field width to specify a character count other than one, as in %5C. The input sequence must begin and end in the initial shift state (if states matter). %S does the same job, but with two critical differences. The multibyte sequence to be converted is determined by first skipping leading whitespace, then consuming all input up to but not including the next whitespace. Here, whitespace has its old definition of being any single character for which isspace returns true in the current locale. (Yes, that can cause trouble with some multibyte encodings.) The resultant sequence must begin and end in the initial shift state. The other difference is that the scan functions store a terminating null after the sequence of converted wide characters. A multibyte encoding may consider certain character sequences invalid. Try to convert one and the scan functions will store the value of the macro EILSEQ in errno. The print functions may generate no incomplete multibyte sequences, but the scan functions aren't nearly as tidy. They can leave the input partway through a multibyte character, or in an uncertain shift state, for all sorts of reasons. Remember, both families of functions are still essentially byte-at-a-time processors. Staying Light These additions to the print and scan functions were discussed before the C Standard was frozen, as I indicated earlier. We rejected them at the time because they offered only partial solutions to several problems: eliminating redundant shift states when generating a stream of multibyte characters keeping track of shift states across function calls when scanning a stream of multibyte characters producing and consuming whole multibyte characters even in the absence of shift codes. The new "wide-character streams" solve such problems, but at a higher cost in machinery. I'll discuss them in a later installment. So why did WG14 agree to add machinery which is known to be inadequate? Because, for many applications, it is good enough, thank you. People writing "internationalized" applications have discovered a whole spectrum of needs. Earlier, I described a class of programs that need essensially no special support for large character sets. Others can use what's already in the C Standard. The biggest nuisance that many practicing programmers keep reporting is this omitted ability to read and write wide characters directly. For a small cost in code added to the print and scan functions, you get enough code to help many programmers. The Header But let's say you need to get more serious about large character sets. In that case, you need more machinery. The normative addendum provides lots more machinery when you include the new header . Listing 1 shows a representative version of this header. The actual types used in the type definitions may vary among implementations, as can the value of the macro WEOF. I have simply made common choices. As you can see, this header declares quite a few new functions. Most of the new names are weird, but they can conceivably clash with those in existing C programs. That bodes ill for backward compatibility, a prime virtue in any update to a programming language standard. WG14 ameliorated the problem by deviating slightly from past practice. The name of every current library function is reserved in the external name space whether or not you include the header that declares it. That lets an optimizing compiler look for function names such as sqrt with fewer fears that it's guessing wrong in treating it special. But the names of functions declared in are not similarly reserved. If you include the new header in any translation unit of a program, you have to look for conflicts. Otherwise, you can ignore the new stuff without fear. You'll notice a handful of type definitions in the new header. The first three are old friends. size_t and wchar_t are, in fact, already declared in multiple headers. struct tm, up to now, has been declared only in the header . It is needed here to declare the function wcsftime. Note, however, that in it is declared as an incomplete structure type. You still have to include if you want to complete the type declaration, so you can poke at its innards. Wide Meta-Characters Books on C seldom emphasize the point, but C has long supported a "meta-character" data type. It is implemented as type int with additional semantic constraints. Look, for example, at what the function fgetc returns. We're promised that the int return value is one of two things: a character code that can be represented as type unsigned char, or the value of the macro EOF, declared in Further, we are assured that EOF is distinguishable from all valid character codes. The typical implementation, in fact, has eight-bit characters and sets EOF to --1. So valid values for a meta-character are in the closed interval [--1, 255]. We use meta-characters all the time. We read them with fgetc or one of its sisters, getc or getchar. We test them with the functions declared in , such as isdigit or isalnum. And, provided we filter out stray EOFs, we can even write them with fputc or one of its sisters, putc or putchar. Pretty handy. It is only natural that the world of wide characters should demand analogous machinery. Thus defines a macro and a type that support programming with wide meta-characters: The type wint_t can represent any value representable as type wchar_t. The macro WEOF can be represented as type wint_t and is distinguishable from all valid wide-character codes. Now you should be able to understand why so many of the functions are declared the way they are in . They simply carry on the old, and demonstrably useful, C tradition of trafficking in meta-characters. Only now they're wide meta-characters. There's one small subtlety I hate to let slide by without notice. The type wint_t is often represented with more bits than the type wchar_t. That gives the implementor lots of choices for the value of WEOF. But that need not be the case. It's perfectly valid for wint_t to be the same size as wchar_t. Then, the value of WEOF must be chosen from among the invalid wide-character codes. (And there better be at least one to choose from.) So don't fall into the habit of thinking of wchar_t as some unsigned integer type. And don't assume that WEOF is --1, or any other negative value. You may one day be surprised. (If you want to write highly portable code, in fact, prepare for the possibility that types char and int are the same size. But that's a whole 'nother sermon.) State Memories One of the notorious shortcomings of the Standard C library is its dependence on private memory. A number of functions maintain static storage to remember various values between calls. Some storage is even shared among different functions. The drawbacks of private memory in library functions are well known: What a function does each time you call it depends on more than just the value of its arguments. That makes its interface harder to learn and to use properly. strtok, for example, is notoriously hard to master. You can't share the use of a function with memory among different tasks at the same time. Or you have to be very careful to save and restore context, as with setlocale and localeconv. A stored pointer from one call can be sabotaged by a later call. tmpnam(NULL), for example, returns a pointer to a filename that changes with each call. Even worse, a call to gmtime changes the data pointed at by an earlier return from localtime. We knew these shortcomings when we developed the C Standard. Much as we wanted to fix many of the offending functions, we decided it was too troublesome to change them. Too much existing code would have to be rewritten. Worse, we perpetuated this practice when we added several functions that manipulate large character sets. As they parse or generate multibyte strings, they may have to keep track of the current shift state. So the functions mblen, mbtowc, and wctomb have private memory. You initialize the memory to the initial shift state with one kind of call. You then progress through a multibyte string with other kinds of calls, and the memory tracks the shift state. Our major reason for using private memory was to avoid adding yet another argument to each of these functions. That argument also needs a special type definition as well. We didn't want all that semantic detail if people weren't going to parse multibyte strings all that much. After all, we've gotten by with strtok all these years, haven't we? Well, it turns out we were wrong. Seems lots of people are manipulating multibyte strings these days. And they chafe at the shortcomings I outlined above. Thus the new type defintion mbstate_t, and the new functions that use it. I'll describe them all next month. Conclusion You'll find one more new type definition in . The type wctype_t serves a limited role. It describes the "handle" returned from the function wctype. You use the handle only in the wide-character testing function iswctype. The handle corresponds to one of an open-ended set of character classifications. What are the classifications and how can you make new ones? The answer is sufficiently complex that I must also defer it to next month. It has taken me a whole installment just to lay the groundwork for understanding the large character set support being added to Standard C. A glance at Listing 1 shows that declares a mess of functions. By now, you can probably guess what many of them do. Your guesses are probably even right in most cases. My next task is to reinforce your prejudices where you're right and show you the surprises elsewhere. Tune in next month. Listing 1 The header typedef unsigned int size_t; struct tm; typedef unsigned short wchar_t; typedef int wint_t; typedef char mbstate_t; typedef unsigned int wctype_t; #define WEOF ((wint_t)-1) int iswalnum(wint_t wc); int iswalpha(wint_t wc); int iswcntrl(wint_t wc); int iswdigit(wint_t wc); int iswgraph(wint_t wc); int iswlower(wint_t wc); int iswprint(wint_t wc); int iswpunct(wint_t wc); int iswspace(wint_t wc); int iswupper(wint_t wc); int iswxdigit(wint_t wc); wint_t towlower(wint_t wc); wint_t towupper(wint_t wc); wctype_t wctype(const char *property); int iswctype(wint_t wc, wctype_t desc); double wcstod(const wchar_t *nptr, wchar_t **endptr); long int wcstol(const wchar_t *nptr, wchar_t **endptr, int base); unsigned long int wcstoul(const wchar_t *nptr, wchar_t **endptr, int base); wchar_t *wcscpy(wchar_t *s1, const wchar_t *s2); wchar_t *wcsncpy(wchar_t *s1, const wchar_t *s2, size_t n); wchar_t *wcscat(wchar_t *s1, const wchar_t *s2); wchar_t *wcsncat(wchar_t *s1, const wchar_t *s2, size_t n); int wcscmp(const wchar_t *s1, const wchar_t *s2); int wcscoll(const wchar_t *s1, const wchar_t *s2); int wcsncmp(const wchar_t *s1, const wchar_t *s2, size_t n); size_t wcsxfrm(wchar_t ;s1, const wchar_t *s2, size_t n); wchar_t *wcschr(const wchar_t *s, wint_t c); size_t wcscspn(const wchar_t *s1, const wchar_t *s2); wchar_t *wcspbrk(const wchar_t *s1, const wchar_t *s2); wchar_t *wcsrchr(const wchar_t *s, wint_t c); size_t wcsspn(const wchar_t *s1, const wchar_t *s2); wchar_t *wcsstr(const wchar_t *s1, const wchar_t *s2); wchar_t *wcstok(wchar_t *s1, const wchar_t *s2, wchar_t **ptr); size_t wcslen(const wchar_t *s); size_t wcsftime(wchar_t *s, size_t maxsize, const wchar_t *format, const struct tm *timeptr); int wctob(wint_t c); int sisinit(const mbstate_t *ps); int mbrlen(const char *s, size_t n, mbstate_t *ps); int mbrtowc(wchar_t *pwc, const char *s, size_t n, mbstate_t *ps); int wcrtomb(char *s, wchar_t wc, mbstate_t *ps); size_t mbsrtowcs(wchar_t *dst, const chat **src, size_t len, mbstate_t *ps); size_t wcsrtombs(char *dst, const wchar_t **src, size_t len, mbstate_t *ps); int fwprintf(FILE *stream, const wchar_t *format; ...); int fwscanf(FILE *stream, const wchar_t *format; ...); int wprintf(const wchar_t *format, ...); int wscanf(const wchar_t *format, ...); int swprintf(wchar_t *s, size_t n, const wchar_t *format, ...); int swscanf(const wchar_t *s, const wchar_t *format; ...); int vfwprintf(FILE *stream, const wchar_t *format, va_list arg); int vwprintf(const wchar_t *format, va_list arg); int vswprintf(wchar_t *s, size_t n, const wchar_t *format, va_list arg); wint_t fgetwc(FILE *stream); wchar_t *fgetws(wchar_t *s, int n, FILE *stream); wint_t fputwc(wint_t c, FILE *stream); int fputws(const wchar_t *s, FILE *stream); wint_t getwc(FILE *stream); wint_t getwchar(void); wint_t putwc(wint_t c, FILE *stream); wint_t putwchar(wint_t c); wint_t ungetwc(wint_t c, FILE *stream); /* End of File */ Stepping Up To C++ Inheritance, Part 2 Dan Saks Dan Saks is the founder and principal of Saks & Associates, which offers consulting and training in C++ and C. He is secretary of the ANSI and ISO C++ committees Dan is coauthor of C++ Programming Guidelines, and codeveloper of the Plum Hall Validation Suite for C++ (both with Thomas Plum). You can reach him at 393 Leander Dr., Springfield OH, 45504-4906, by phone at (513)324- 3601, or electronically at dsaks@wittenberg.edu. In my last column, I introduced inheritance as a technique for creating new (derived) classes from existing (base) classes. (See "Inheritance, Part 1", CUJ, March 1993). I considered only the simplest and most common form of inheritance--public inheritance. In this article, I'll explore other forms of inheritance and offer some guidance that may help you decide when to use, and when not to use, inheritance. In part one, I derived a float_vector class from a float_array class. The float_array class is a dynamic array of float elements. That is, it is an array whose number of elements can vary at runtime. The lower subscript bound for all float_arrays is always zero. The float_array class definition appears in Listing 1. The float_vector class is a dynamic array of float whose lower subscript can be an integer value other than zero. The float_vector class definition appears in Listing 2. (When I presented this listing last time, I accidentally omitted the int return type from the inline function definitions for low and high.) The non-inline float_vector member function definitions appear in Listing 3. My test program for float_vectors and float_arrays appears as Listing 4. I concluded last time by questioning the merits of deriving float_vector from float_array. Although public inheritance offers a compact notation for creating float_vector from float_array, it's not the only way to do it. Let's look at an alternative. Aggregation If all you want to do is simply create float_vector from float_array without rewriting much code, you can do that very straightforwardly using aggregation. Aggregation (also known as composition) is simply embedding an object of one type as a member of another. That is, rather than derive float_vector from float_array: class float_vector : public float_array { public: // public member functions ... private: int_low; }; you implement float_vector as a class with a float_array member: class float_vector { public: // public member functions ... private: float_array fa; int _low; }; Then, you implement each float_vector member function by calling the corresponding float_array member. For example: size_t float_vector::length() const { return fa.length(); } Here, the length of a float_vector is simply the length of its float_array member. Listing 5 shows a definition for class float_vector implemented with a float_array member. It differs from the class definition in Listing 2 in only two ways: It has a named float_array member rather than an inherited float_array subobject. It explicitly declares length as a member function because it no longer inherits any members. The inline member functions in Listing 5 also reflect these subtle changes: In the float_vector constructor inline float_vector::float_vector(int lo, int hi) : _low(lo), fa(hi - lo + 1) {} the constructor initializer for the float_array subobject refers to the float_array by its name, fa. In Listing 2, the float_array subobject has no name, so the initializer refers to the subobject by its type. The high member function computes the float_array's length using fa. length, not just length. Listing 6 shows the non-inline member functions for the floor_vector class defined (by aggregation) in Listing 5. These functions are identical to those in Listing 3, except for the calls to float_array::operator[]. In Listing 3, each float_vector::operator[] calls the operator[] for the inherited float_array subobject using the explicitly-qualified function name, as in return float_array::operator[] (i - low()); In Listing 6, each float_vector::operator[] calls operator[] for the float_array member object using the usual member function call notation. I could have written the calls as return fa.operator[](i - low()); but I took advantage of the overloaded operator notation and wrote the calls as simply return fa[i - low()]; In retrospect, I could have used the operator notation in the derived float_vector: :operator[] as well, as shown in Listing 7. Notice that the local variable fa is a float_array * in the non-const float_vector::operator[] function, and const float_array *in the const function. Aggregation vs. Inheritance Using aggregation, class float_vector has nearly all the functionality as it did when implemented by inheritance. But there are some critical differences. When you compile the test program in Listing 4, the compiler balks at the first call to display inside main, namely display("fa", fa); When you derive float_vector from float_array, this call works fine. Even though display expects its second argument to be a float_array, it accepts float_vector fa because a float_vector IS A float_array. But when using aggregation, a float_vector is not a float_array. Rather, a float_vector HAS A float_array, or equivalently, a float_vector IS IMPLEMENTED BY A float_array, so a float_vector does not convert implicitly to float_array. If you try writing an additional display function for float_vectors as void display(const char *s, const float_vector &fv) { cout << S << " = " << fv << endl; } then the compiler complains that there's no operator<< for float_vectors. Once again, even though there's an operator<< for float_arrays (declared in Listing 1), it doesn't apply to float_vectors built by aggregation. You can cure that problem by defining inline ostream &operator<< (ostream &os, const float_vector &fv) { return os << fv.fa; } (Note that this is not a member function, but it must reference private data in fv, so class float_vector must declare it as a friend function.) This clears up all the problems in calling display, but the program still doesn't compile, because the compiler rejects the later declaration float_array fc = fa; Disregarding the possibility that the compiler might introduce temporary objects, this declaration compiles as if it were float_array fc (fa); The compiler wants to call a float_array constructor that accepts a float_vector argument. (Remember, fa is a float_vector here.) When a float_vector IS A float_array, the compiler readily binds a float_vector to a float_array &, so the previous declaration calls the float_array copy constructor float_array(const float_array &); But, when a float_vector HAS A float_array, the compiler refuses to do the conversion. Of course, you could go back to the float_array class and write a constructor that accepts a float_vector argument, but you shouldn't. Adding this constructor forces all float_array users to also accept this float_vector class. It introduces an implicit conversion into an existing class that could break code using that class. Furthermore, you cannot assume that all users wishing to extend a class have access and can alter the source for that class. Thus, I've been looking at ways to reuse an existing class without rewriting it. Adding a conversion operator to class float_vector is a better approach: float_vector::operator float_array() { return fa; } This operator converts a float_vector to a float_array by returning a copy of the float_vector's float_array member. With this addition, you can compile the test program (Listing 4) using the float_vector class implemented by aggregation. Once you add a conversion operator, you no longer need versions of the display function and operator<< that accept a float_vector as their second argument. If the only declaration for display is void display(const char *s, const float_array &fa); but you have declared a conversion operator float_vector::operator float_array(); then the call display("fa", fa); (where fa is a float_vector), translates into display("fa", float_array(fa)); That is, the compiler obtains a float_array by applying the conversion operator to float_vector fa. This conversion operator doesn't really let you treat float_vectors as if they were float_arrays. The conversion operator returns a float_array by value. That is, it returns a temporary copy of the float_array object inside a float_vector, not a reference to the float_array subobject. For example, suppose you write float_vector fv; ... const float_array &r = fv; If you had derived float_vector from float_array, then the reference declaration binds r directly to the float_array subobject of fv. But, if you had declared float_vector with a float_array member, then the reference declaration binds cr to a temporary float_array object created by copying the float_array member of fv. Now, suppose you write float_array &r = fv; If you had derived float_vector from float_array, then this also binds r directly to the float_array subobject of fv. But, if you had declared float_vector with a float_array member, then the compiler will reject the reference declaration because you cannot bind a non-const reference to a temporary object. The error should go away if you declare an overloaded pair of conversion operators float_vector::operator float_array &(); float_vector::operator const float_array &() const; I say it should go away because I believe this is valid C++. Unfortunately, many compilers complain that this pair of conversions causes ambiguities. Listing 5 shows the class and inline function definitions for class float_vector implemented with a float_array member. Listing 6 contains the corresponding non-inline member function definitions. If the overloaded pair of conversions to (const and non-const) float_array & gives your compiler fits, replace them both with float_vector::operator float_array() { return fa; } and your compiler should accept the test program. Choosing the Right Design If inheritance and aggregation are so similar, how do you choose which one to use? It's largely an matter of judgment based on experience. But by and large, you will find that these general rules apply: Use public inheritance for IS-A. If you want the new class you're creating to have all the properties of the existing class by default, then derive it. Use aggregation for HAS-A or IS-IMPLEMENTED-BY- A. If you do not intend objects of the new class to be objects of the existing class, but you want to use the capabilities of the existing class to simplify implementing the new class, then embed an object of the existing class as a member of the new class. Both Cargill (1992) and Meyers (1992) offer additional rationale for and examples of these guidelines. Meyers calls aggregation layering. Sometimes it's easy to tell IS-A from HAS-A. For example, a circle is a shape and a triangle is a shape, but shapes have attributes like color. Therefore, you should publicly derive classes circle and triangle from class shape, and include a member of type color in class shape. Other times, the distinctions are very subtle. For instance, is a float_vector a float_array? It all depends on how you define the concepts. If you define float_array as a dynamic array of N elements and float_vector as a dynamic array of N elements with a user-definable lower subscript bound, then you could argue that a float_vector is a float_array. But, if you define float_array as a dynamic array of elements subscripted 0 through N-l, you'll have a harder time arguing that a float_vector is a float_array. If you have trouble designing a particular class relationship, I suggest you write down the reasoning behind your design in clear and precise prose, using complete sentences. If possible, have your peers review it too. If you can't make a convincing case for an IS-A relationship, then don't use public inheritance. Protected Members In addition to public and private, C++ offers a third access specifier--protected. Protected access is similar to private access, except that members and friends of derived classes can access protected base class members. As always, derived class members and friends cannot access private base class members. For example, Listing 8 shows a base class B with protected member j and private member k. Member function foo of class D derived from B can access j but not k. In the first part of this article, I said I was forced to make a simplifying assumption in my float_vector class, namely, that you only extend a vector's high bound. The word "forced" was a bit misleading. You could write the non-const float_vector:: operator[] to extend the vector's low bound as well. But, because the float_vector class can't access its inherited array member (the pointer to the dynamically-allocated elements), extending the low bound would have been more complicated and much slower than extending the high bound. Plum and Saks (1992) recommend that if the original float_array class had declared its private data members (array and len) as protected, then the derived float_vector class could access those members directly. A float_vector::operator[] could then extend a vector's low bound more efficiently. But this efficiency comes at a price. You should resist the urge to hedge against inflexible class designs by declaring members that should be private as protected. Protected members increase coupling between a base and its derived classes. The protected members are part of the base's interface. Any application program can gain access to protected base class members by simply deriving a class. If you must use protected members, use protected member functions rather than data. Protected data members don't protect themselves. The author of a derived class can easily corrupt its base class subobject by misusing inherited protected data. Protected member functions supplied by the base class can do a better job of keeping inherited private data in a consistent state. Non-Public Inheritance In addition to public inheritance, C+ + also supports private and protected inheritance. For example, class D : private B { ... }; In fact, private is the default access specifier for base classes, which is why you must explicitly specify public if that is what you want. Here's what the different base class access specifiers mean: public--if B is a public base class of D, then public members of B are accessible as public members of D, and protected members of B are accessible as protected members of D. protected--if B is a protected base class of D, then public and protected members of B are accessible as protected members of D. private--if B is a private base class of D, then public and protected members of B are accessible as private members of D. Last time, I said that whenever B is a public base class of D, then a D is a B. Specifically, you can: convert a D *to a B * convert a D & to a B & initialize a B & with a D In fact, these conversions apply whenever B is an accessible base class of D. The ARM (Ellis and Stroustrup, 1990) and C++ draft (Shopiro 1992) both state that "a base class is accessible if its public members are accessible." I believe this really means that a base class B is accessible with respect to a derived class D if the members of D inherited from public members of B are accessible as members of D. A public base class is always accessible with respect to its derived class, so those IS-A conversions, above, always apply. But, for protected and private inheritance, a base class is accessible only in certain contexts, as illustrated by Listing 9. Here, B is a protected base class of T. Inside a member (or friend) of T, like T::foo, B is accessible with respect to T, so the IS-A conversions apply. But, outside T's members and friends, as in main, B is not an accessible base of T, so the IS-A conversions do not apply. In many ways, private inheritance is like aggregation; it provides an alternative technique for defining IS-IMPLEMENTED-BY-A. In fact, the choice between aggregation and private inheritance is often a matter of personal preference. Both Cargill (1992) and Meyers (1992) suggest favoring aggregation over private inheritance. I agree. Meyers also provides an example to illustrate a exceptional case where you might prefer private inheritance. I have yet to see a use for protected inheritance. I would certainly be interested to see one. References Cargill, Tom. 1992. C++ Programming Style. Reading, MA: Addison-Wesley. Meyers, Scott. 1992. Effective C++. Reading, MA: Addison-Wesley. Plum, Thomas and Dan Saks. 1992. C++ Programming Guidelines, Cardiff, NJ: Plum Hall Books. Listing 1 float_array class definition // fa1.h - a dynamic array of float using a subscripting // object #include class fa_index { friend class float_array; public: fa_index &operator=(float f); operator float(); private: fa_index(float_array *f, size_t i); float_array *fa; size_t ix; }; class float_array { friend class fa_index; public: float_array(size_t n = 0); float_array(const float_array &fa); ~float_array(); float_array &operator=(const float_array &fa); float operator[](size_t i) const; fa_index operator[](size_t i); size_t length() const; private: void extend(size_t i); float *array; size_t len; }; ostream &operator<<(ostream &os, const float_array &fa); inline size_t float_array::length() const { return len; } /* End of File */ Listing 2 float_vector class definition // fv1.h - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object // implemented by inheritance from float_array #include "fa1.h" class float_vector : public float_array { public: float_vector(int lo: 0, int hi = 0); float operator[](int i) const; fa_index operator[](int i); int low() const; int high() const; private: int_low; }; inline float_vector::float_vector(int lo, int hi) :_low(lo), float_array(hi - lo + 1) {} inline int float_vector::low() const { return_low; } inline int float_vector::high() const { return_low + length() - 1; } /* End of File */ Listing 3 Non-inline float_vector member function definitions // fv1.cpp - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object // implemented by inheritance from float_array #include "fv1.h" #include float float_vector::operator[](int i) const { assert(i >= low()); return float_array::operator[](i - low()); } fa_index float_vector::operator[](int i) { assert(i >= low()); return float_array::operator[](i - low()); } /* End of File */ Listing 4 Test program for float_vectors and float_arrays // tv1.cpp - a test program for float_vectors and // float_arrays #include #include "fv1.h" void display(const char *s, const float_array &fa) { cout << s <<" =" << fa << endl; } int main() { int i, low, high; cout << "low? "; cin >> low; cout << "high? "; cin >> high; float_vector fa(low, high); for (i = fa.low(); i <= fa.high(); ++i) fa[i] = i; display("fa", fa); float_vector fb = fa; display("fb", fb); for (i = low; i < low + 2 * fa.length(); ++i) { fb[i] = i * i; display("fb", fb); } cout << "fb.low() = "<< fb.low() << '\n'; cout << "fb.high() = "<< fb.high() << '\n'; float_array fc = fa; display("fc", fa); i = fc.length(); fc[i - 1] = 123; display("fc", fc); cout << "fa[" << low - 1 << "] = "; cout << fa[low - 1] << '\n'; return 0; } /* End of File */ Listing 5 Definition for class float_vector implemented with a float_array member // fv2.h - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object // implemented by aggregation with a float_array member #include "fa1.h" class float_vector { public: float_vector(int lo = 0, int hi = 0); float operator[](int i) const; fa_index operator[](int i); size_t length() const; int low() const; int high() const; operator float_array &(); operator const float_array &() const; private: float_array fa; int_low; }; inline float_vector::float_vector(int lo, int hi) : _low(lo), fa(hi - lo + 1) { } inline size_t float_vector::length() const { return fa.length(); } inline int float_vector::low() const { return _low; } inline int float_vector::high() const { return_low + fa.length() - 1; } inline float_vector::operator float_array &() { return fa; } inline float_vector::operator const float_array &() const { return fa; } /* End of File */ Listing 6 Non-inline member functions for the float_vector class // fv2.cpp - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object // implemented by aggregation with a float_array member #include "fv2.h" #include float float_vector::operator[] (int i) const { assert(i >= low()); return fa[i - low()]; } fa_index float_vector:: operator [] (int i ) { assert{i >= low()); return fa[i - low()]; } /* End of File */ Listing 7 A dynamic vector of float // fv1.cpp - a dynamic vector of float (with a possibly // non-zero low-bound) using a subscripting object // implemented by inheritance from float_array //include "fv1.h" #include float float_vector::operator[] (int i) const { assert(i >= low()); const float_array *fa = this; return (*fa)[i - low()]; } fa_index float_vector::operator[] (int i) { assert(i >= low()); float_array *fa = this; return (*fa)[i - low()]; } /* End of File */ Listing 8 A base class (B) with a protected member (j) and a private member (k) class B { protected: int j; private: int k; }; class D : public B { public: int foo(); }; int D::foo() { ++j; // ok return k; // error: k not accessible } /* End of File */ Listing 9 Code illustrating a base class accessible only in certain contexts // test conversions from derived to base using // non-public base classes class B { public: int i; }; class T : protected B { public: void foo(); }; void T::foo() { T t; B *p = &t; // ok, B accessible } class V : private B { public: void bar(); }; void V::bar() { V v; B &r = v; // ok, B accessible } int main() { T t; V v; B *p = &t; // error, B not accessible // base of T B &r = v; // error, B not accessible // vase of V return 0; } /* End of File */ Questions & Answers Identifiers Kenneth Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member on the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. Q I share your oft-repeated conviction that longer, more suggestive identifiers are a good thing in source code, and I would like to see more people adopting them. However, what you recommend in the February 1993 issue carries it too far. There is a real risk that what you advise would result in verbose, long-winded text. A good writer of natural language, like a good writer of source code, has a sense of what an ordinary reader can be presumed to know, either from context or otherwise, and from that can write more concisely. In natural language it is an effective practice to describe some specifics on introduction of a term or concept and from then on to use a short, concise identifier. For example, at the start of a report one may speak of "the size of the engine implied by the consumer rating index" and thereafter say "size" to mean the same thing. The traditional source code equivalent is to use a concise identifier and comment on it when first introduced or whenever necessary, and we have long been told that this identifier should be suggestive and pronounceable. In contrast, you wish to put a lot more of the specifics into the identifier name. For example you recommend the more descriptive names when given a choice between virtual int get_engine_size_by_model_name(...); virtual int get_engine_size_by_consumer_rating(...); versus virtual int getsizeM(...); virtual int getsizeCr(...); No doubt you have come across source code with loads of comments in the right margin that just state the obvious and do not enhance understanding, but the reader still has to look at all the comments for fear of missing something important. It seems to me that what you are recommending carries a risk of making this same mistake, but with the comments now contained in the identifiers. The resulting source code could be like a sort of legalese, where the reader must digest large quantities of high-chaff verbiage. It would not be an appetizing meal. Humans are good at remembering the specifics when the identifier is suggestive. When you were reading David Copperfield, how often did you have to look up the same word twice? Not too often, I'll bet. Nowadays when you do have to look something up, it takes just a few keystrokes. And you will get more reliable information than you could get from a lengthy name. You say that having to look something up distracts from the flow of the plot. But you don't mention that it is also distracting to be presented with extraneous information, for example, to be told by the member function name that the engine size is from the consumer rating in a subroutine that just needs an engine size. More troubling, you would have the comment repeated again wherever that member function name was invoked, including when the parameter name was making it redundant. That would drive the reader to distraction. Since class member functions are always backward references and may be invoked frequently, a concise but suggestive name accompanied by a true comment in the class definition is more readable, and closer to how it's done in natural language. Lastly, overloaded operators are an effective means of concise expression in natural language (look at metaphors) and they hold out the promise of doing the same thing in source code. You won't see metaphors in legalese, but you will see a lot of redundancies, because lawyers prefer no unwritten inferences. What you propose is more like legalese than the natural language. You gave your correspondent David A. Dennerline a bad rap. Sean Furlong New York, NY A We agree that reasonable identifiers are a good thing. I may go to the long extreme for emphasis. I agree with your statement that a program that is too wordy is almost as bad as one that is not descriptive enough. To paraphrase Mark Twain, we've identified the problem, we're just trying to agree on the details. Let me respond to some of your points. I might suggest that the length of names could be inversely proportional to their frequency of appearance. To use a short name for a function that may be seldom used simply requires the reader to look up its meaning. The association between name and meaning may be easily forgotten. I also feel my associative memory is starting to fill up, so associations should only be used if there is distinct performance advantage to reading speed. On the other hand, a name as get_engine_size_of_car_by_model_- name_specified_by_a_character_- string (my_model_name); is definitely too long and repetitive. We generally assume that a name is going to be either a char * or a String type. And the of_car part is redundant, since the function is a member of class Car. That would be like having to describe Mark Twain in the first paragraph as a humorist/writer. (However, this may need to be stated for readers unfamiliar with American literature). If the name of the arguments that were passed to a function identified the function sufficiently, then additional description on the name is redundant, as you suggest. For example, engine_size = car.get_engine_size_by_model_- name(my_model_name); could be shortened to something like: engine_size : car.get_engine_size_ by - mn(my_model_name); Unfortunately the creator of the function has no control over its user. So if the user codes: ensz = car.get_engine_size_by_mn(md); then it is a bit more difficult for the reader to understand its meaning from the context. As I suggested, if you use a standard abbreviation list for all function and member names, then the name could be shortened even more. For example, if you had a file call abbrev.doc with es engine_size mn model_name cr consumer_rating then the function might be called with: es = car.get_es_by_mn(mn); My guideline for this usage is that these abbreviations should be used everywhere. For example, in all references to the same type of object, such as engine size, es should be used; not eng_size in one place or engsz in another. A comment for the function name is not even needed in this case, since the true meaning of the function simply requires knowing a standard set of abbreviations. The user of the function should use the exact same abbreviations in the call. If it were called with: ensz = car.get_es_by_mn(mdnm); then the reader would not only have to recall the meaning of the standard abbreviation, but also remember the meaning of the caller's abbreviations. The only danger in using short abbreviations is that the code space between them decreases and therefore the possibility of misreading increases. What I mean by code space is the number of characters that need to be changed to transform one abbreviation into another. Say el and et were used for engine_length and engine_torque. The abbreviations have only a one letter difference, while the long names have a six letter difference. I agree with you that the seemingly redundant language that is seen in complex legal documents seems a bit overbearing. The wording of "consumer readable" contracts is closer to what I would suggest as a reasonable model for program specification. Having a standard place for definitions of terms (e.g. the meanings of abbreviations) and using those terms throughout the program helps make it understandable, as you state. The opportunity for conciseness of overloaded operators is definitely an advantage. However, conciseness can be a disadvantage. You probably have heard of one-line APL programs that are not maintained, but rewritten, as the original coder's intent was hidden in its conciseness. I do not know any people who would write i = multiply(add(1,2),add(3,4)); instead of i = 1+ 2* 3 + 4; When I give my "That's an Object, I Object" presentation to C+ + students, only one person has ever objected to being able to code: String a_string b_string, a_string; = b_string + c_string; However, almost all (except for one group) have objected to overloading the - (hyphen) operator. The meaning of the following could not be agreed upon: b_string = a_string - c_string; Does it delete the value of c_string from a_string if it finds it at the end of a_string or anywhere within a_string? If the latter, does it take out the first instance, the last instance, or all instances? The one group that did agree had a common basis for agreement. The VMS operating system uses - (hyphen) to designate removing c_string from a_string if it occurs at the end of a_string. If an entire group of programmers has a common basis for abbreviating or overloading, then this collective knowledge can be used for shorter programs. I don't think I gave David Dennerline a bad rap. He was arguing for the same points that I made--emphasize readability over short code. As for David Copperfield, I can't remember how often I looked up the same words. I think my associative memory had quite a bit of spare storage then. It took me some time to read the book (a whole summer). I'm sure I often looked up the same word in separate sittings. Modems Strings Taking CUJ at its word, I am faxing this question to you in hopes that you'll be able to answer it or point me in the right direction. I am a developer working on modem communication programs. The company I work for has a large installed base of users with an extremely wide variety of modems. I have a very difficult time trying to support my programs when the user knows absolutely nothing about the modem and worse yet does not even have the manual that came with the modem to help me solve their problem(s). Each time I hear, "Well Procomm has modem installation" it makes me more desperate to find what I have been looking for, namely, a comprehensive, single source of modem commands and S-register settings for most if not all of the so called Hayes-compatible modems out there. Any help you could give me would be greatly appreciated (along with all the other help you have given me in your column.) Andrew W. Ackard I wish what you request was available. I have not seen such an item, but perhaps one of our readers has. I just went through the pains of a similar project. A central computer was used to collect data files from client computers. We had no control over the type of modems used for the client computers. The client programs had to be set up so that they were completely automatic--start them up and leave them alone till the files were transferred. A set-up program was used to customize the initialization string for each client's modem. Of the items on the string, about two-thirds were the same for all modems, the rest were different. Some were necessary simply because the default settings were different on each modem. Unfortunately, once you get past the simple commands, to the ones that were introduced with speeds higher than 2,400 baud, differences emerge. The S-registers appear to be different among just the three modems whose manuals I have handy. For example, S13 is used as a UART-status, code-guard time by one, but is reserved or undefined by the other two. Maybe by combing through ten or so manuals and doing similar comparisons, you may come up with a reasonable list. On a side note, you might have heard that Hayes is sueing other modem manufacturers for patent infringement. It is concentrating on the use of the escape sequence (silence followed by a string +++ followed by silence) to return to command mode. Although it wants to sue on duplication of the remainder of the commands, it supposedly has less of a leg to stand on. Standardization would be a boon to programmers and users, but such suits tend to discourage vendors from exact replication. File truncation I noted in your response to Jeff Dragovic in "Q?/A!", (CUJ, January 1993, p. 107) an implication that the only way to truncate a file under MS-DOS was to copy the file up to the desired point, close the copy, and discard the original. Under MS-DOS, writing zero bytes will truncate the file being written at the current file position. This is documented under MS-DOS function 40H, which is usually the system call underlying most MS-DOS fwrite and write functions. Not only that, if you seek beyond EOF and then write zero bytes, the file will be extended to the new file position, but without any actual data being written to the file. It's a great way to create really big files really fast when preallocating for any reason. While there is, of course, no guarantee that an ANSI implementation will actually honor a request to write zero bytes, I venture to suggest that under MS-DOS it will always be so, simply because of this functionality. Hope this helps! Karl Auer Canberra ACT Australia Thank you for your letter. I also received a similar response from Lloyd at Lloyd@debug.cuc.ab.ca. I don't have a last name (or is it the first name?), since the trailer was lost in electronic garbage land. He mentioned that Borland C has a chsize function, that calls function 40H. Borland also has a set of _open, _read, _write, and _close calls that call 40H directly. Listing 1 shows their use in truncating a file. The corresponding program using open, write, and close does not truncate the file. That would have made the functions disagree with their generally accepted operation. Seeking beyond the end of a file and writing extends the file in both MS-DOS and UNIX. Under MS-DOS the additional disk blocks in the gap are physically allocated at this time, as you mentioned. With UNIX, the blocks are not allocated. If you seek and write to them at a later time, they will then be allocated. This poses an interesting twist if one is porting between the two systems. Under MS-DOS, you would be sure you had disk space for the file if the write worked after the seek. Under UNIX, you will not be sure until all the blocks in the intervening gap are written. This difference might be hidden by another glue routine which might look like: enum e_file_extend {FROM_BEGINNING, FROM_CURRENT, FROM_END}; int file_extend(int file_descriptor, long size_in_bytes, enum e_file_extend); With MS-DOS, it would perform the seek as you described. Under UNIX, it would actually need to do the writes. (KP) Listing 1 Borland functions used in truncating a file #include #include #include #include #include void main() { int file; int count_written; file =_open ("TEMP", O_RDWR); if (file < 0) { printf("Error %d in file open\n", errno); perror(""); ) count_written = _write(file,&file,0); printf("Count written %d", count_written); _close(file); } /* End of File */ Code Capsules File Processing Chuck Allison Chuck Allison is a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. File systems differ greatly from one environment to another. There is no universal approach to issues such as directory structure, components of, length of, and acceptable characters for filenames, file modes (e.g., block vs. stream, text vs. binary), file versioning, or file locking. This sometimes makes it difficult to write a portable program that uses files. This capsule will illustrate some of the most commonly-used file I/O functions from the Standard C library. The examples below should work on most operating systems. (You may have to substitute certain keystrokes for the ones mentioned here.) Text Filters The following short program copies a line at a time from standard input to standard output: /* copy1.c */ #include main() { char s[BUFSIZ]; while (gets (s)) puts(s); return 0; } Unless you instruct otherwise, the standard I/O functions perform buffered I/O, that is, they collect data and then transfer it a buffer at a time for efficiency. BUFSlZ, defined in stdio.h (as 512 by my compiler), is the size of the internal buffers used by the standard I/O functions. BUFSIZ is a good choice for the size of your buffers if you don't have a good reason to choose differently. On operating systems like MS-DOS and UNIX, this program is more interesting than it appears. In conjunction with redirection, it can be used to actually create a file: C:> copy1 >file1 After entering the lines of text, enter Ctrl-Z (in MS-DOS, Ctrl-D in UNIX) on a line by itself to signal the end of input. To make a copy of an existing file, enter: C:> copy1 file2 Such a program that reads only from standard input and writes only to standard output is called a filter. It is also possible to redirect I/O from within the program itself via the freopen library function, which disconnects an open file pointer from its file and connects it to a new one. The program in Listing 1 disconnects stdin and/or stdout from the console and connects them to the files entered on the command line. You can invoke this program without explicit redirection C:> copy2 file1 file2 The names of the files become arguments to main (see the box called "Command-Line Arguments"). Using freopen is convenient because it avoids explicit opening and closing of files, but it disallows any interaction with the user (since standard I/O has been redirected to files). Most interactive applications, however, require both file and console I/O. The version in Listing 2 shows how to open files explicitly--no redirection is performed, and both filenames are required. The function fgets needs to know the size of your buffer, and it places the newline character into the buffer if there is room for it. (gets discards the newline.) fputs, therefore, doesn't append a newline to the string it writes (puts does). Using fgets with puts produces double-spaced output, while mising gets with fputs prints everything on one line. Both fgets and gets return NULL upon end-of-file or error; any additional error-checking is not usually required. You should check for output errors, however, especially on a PC system where running out of disk space is not uncommon. You do this with a call to ferror. For example, you should replace the while loop in Listing 2 with the following: /* copy4.c */ . . . while (fgets (buf,BUFSIZ,inf)) { fputs (buf,outf); fflush (outf); if (ferror(outf)) return EXIT_FAILURE; } . . . Since file I/O is buffered, you should flush the output buffer before checking in case there is a disk overflow error. Once the error state of a file is set, it remains unchanged until you reset it by calling clearerr or rewind. Binary Files The examples so far work only with text files (i.e., files of lines delimited by \n). In order to be able to copy any file (e.g., an executable program) on most non-UNIX systems, it is necessary to open the file in binary mode. In text mode under MS-DOS (the default file mode there), each newline character in memory is replaced with a \r\n pair (CR/LF) on the output device, while the process is reversed during input. (This is a carry-over from CP/M days.) In addition, a Ctrl-Z is interpreted as end-of-file, so it is impossible to read past a Ctrl-Z in text mode. Binary mode means that no such translations are made--the data in memory and on disk are the same. (NOTE: MS-DOS text and binary modes are not analogous to the cooked and raw modes of UNIX; binary mode has no effect in UNIX). The program in Listing 3 can copy any type of file. A b appended to the normal open mode indicates binary mode. The functions fread and fwrite read and write blocks of data. They both return the number of blocks (not bytes) successfully processed. In the example in Listing 3, the items just happen to be bytes, i.e., blocks of length 1. When fwrite returns a number less than the number of items requested, you know that a write error has occurred (so an explicit call to ferror is not necessary). fwrite stores numeric data in binary on the output device--so you cannot read it with normal text utilities. Record Processing The functions fread and fwrite are suitable for processing files of fixed-length records. The program in Listing 4 populates a file from keyboard input (terminated by Ctrl-z) and then randomly accesses certain records. I use stderr for printing prompts since it is always attached to the console and is unbuffered on most systems. Figure 1 contains the results of a sample execution. A + (plus sign) in the open mode request indicates update mode, which means that both input and output are allowed on the file. You must separate input and output operations, however, with a call to fflush or to some file positioning command, such as fseek or rewind. fseek positions the read/write cursor a given number of bytes from the beginning of the file (SEEK_SET), the end of the file (SEEK_END), or from the current position (SEEK_CUR). rewind(f) is equivalent to fseek(f,OL,SEEK_SEF). Arbitrary byte positions passed to fseek only make sense in binary mode, since in text mode there may be embedded characters you know nothing about. The function ftell returns the current position in a file, which value can be passed to fseek to return to that position (this synchronized use of fseek and ftell works even in text mode). Since fseek and ftell take a long integer argument for the file position, they are limited in the size of file they can correctly traverse. If your system supports larger file position values, then use fgetpos and fsetpos instead. fsetpos is only guaranteed to work correctly for values returned by fgetpos, not for arbitrary integral values. The program in Listing 5 puts fgetpos and fsetpos to good use in a simple 4-way scrolling browser for large files. It only keeps one screenful of text in memory. If you want to scroll up or down through the file, it reads (or re-reads) the adjacent text and displays it. When scrolling down (i.e., forward) through the file, the file position of the data on the screen is pushed on a stack, and the program reads the next screenful from the current file position. To scroll up, it retrieves the file position of the previous screen from the stack (see Listing 6). Although this is the crudest of algorithms for viewing text, it can view a file of any size (if you make the stack large enough), and performance is acceptable on systems that cache disk operations (really!). The display mechanism is also crude but fairly portable--it uses ANSI terminal escape sequences for clearing the screen and positioning the cursor (see Listing 7--if you're using MS-DOS, you must load ANSI.SYS from your CONFIG. SYS file). You could customize this program into an efficient tool by adding buffering and fast screen writes. Temporary Files When your program requires a scratch file for temporary processing, you need a unique name for that file. If the name isn't important to you, let tmpnam do the work of creating the filename for you: char fname[L_tmpnam]; tmpnam(fname); f = fopen(fname .... tmpnam will supply at least TMP_MAX unique names before it starts repeating. The macros L_tmpnam and TMP_MAX are defined in stdio.h. Don't forget to delete the file before the program terminates: remove (fname); If you don't need to know the name of the file, but just want access to it, a better approach most of the time is to let tmpfile give you a file pointer to a temporary file. It returns a pointer to a file opened with mode wb+ (this is usually adequate for scratch files). The best part is that the file is deleted automatically when the program terminates normally (i.e., abort isn't called). Listing 8 contains the program esort.c, an external sort program (i.e., a program that can sort files larger than can fit in memory). When a file larger than 1000 lines or that can fit in available memory is read, esort breaks it into subfiles. Each subfile is sorted internally with qsort, and then the sorted subfiles are merged to standard output (see last month's capsule for a discussion of qsort). The subfiles disappear automatically when the program halts. If for some reason the program aborts (i.e., one of the assertions failed--a write error, for example), the subfiles are not deleted and you can examine them for clues to what went wrong. Although file systems vary widely, many common operations can be done portably. The examples in this article (keyboard signals excepted) should work unchanged in any ANSI-C environment. Next month I'll examine useful features found in UNIX-compatible environments. Command-Line Arguments One of the reasons for C's success in building software tools is the ease with which you can access command-line arguments. All tokens on the command line (even the program name) are arguments to main. The following program prints each command-line argument on a line by itself. /* ecko: print command-line arguments */ #include main(int argc, char * argv[]) { int i; for(i=0; i ecko one two three then the output is C: \ECKO. EXE one two three The first argument (argc) is the number of arguments entered (4 in this case). The second argument (argv) is an array of pointers to character as shown in Figure 2. The first argument (argv[O]) is always the full path name of the program's executable file (except on versions of MS-DOS prior to 3.0, where it is an empty string). Wildcards Most operating systems allow you to use wildcard characters on the command line (e.g., * matches 0 or more characters, ? matches 0 or 1 in MS-DOS). These are usually associated with filenames. For example, the command C> ecko *.c should print the names of all .c files in the current directory, one-per-line. Not all operating environments do wildcard processing by default, however (UNIX does, MS-DOS and VAX/VMS do not). To activate it using Borland C under MS-DOS, you will need to link with the file \BORLANDC\LIB\WILDARGS.OBJ: C> bcc ecko. c \borlandc\lib\wildargs.obj The file for Microsoft C is \C700\LIB\SETARGV. OBJ. Command-Line Options Many programs accept options (sometimes called switches) on the command line to alter execution. You specify options by a special prefix character, usually a dash or a forward slash. For example, Microsoft C accepts, as two among many, the options -c (do not link), and -Od (turn off optimization): C> cl -c -Od ecko.c These options must precede the filenames to be processed. Listing 9, a sample program for printing a directory listing, shows how to process options before other arguments. Valid options are: -l-give detailed directory information -r-sort in reverse alphabetical order The output of the command C> list -lr *.c *.h is Dir: *.c (Order = -1, Full = 1) Dir: *.h (Order = -1, Full = 1) This technique allows you to combine options in the same argument (as shown above), or to separate them (-l -r). The code that actually reads the directory (function dir) would likely be non-portable, since file systems are highly platform-dependent. With another technique you can intersperse options among the other arguments, so that each file is processed with its own set of options. The file printer program in Listing 10 can process a command such as C> pr -n -c10 file1.c -nc1qL file2.c file3.c This command prints 10 copies of file1.c with line numbers on the default printer, and prints one copy of the other files without line numbers formatted for a laser printer (the flag n is a toggle). The options are: -c#-print # copies -n-print with line numbers -qC-print on queue C (S=std, L=laser,etc.) Figure 1 Execution of records. c from Listing 4 Last: Lincoln First: Abraham Age: 188 Last: Bach First: Johann Age: 267 Last: Tze First: Lao Age: 3120 last: Tze, first: Lao, age: 3120 last: Lincoln, first: Abraham, age: 188 Figure 2 An array of pointers to character Listing 1 A copy filter that allows redirection /* copy2.c */ #include #include main(int argc, char *argv[]) { char s[BUFSIZ]; /* Open optional input file */ if (argc> 1) if (freopen(argv[1],"r",stdin) == NULL) return EXIT_FAILURE; /* Open optional output file */ if (argc > 2) if (freopen(argv[2],"w",stdout) == NULL) return EXIT_FAILURE; while (gets(s)) puts(s); return EXIT_SUCCESS; } /* End of File */ Listing 2 Copies files via explicit file pointers /* copy3.c */ #include #include main(int argc, char *argv[]) { if (argc == 3) { char s[BUFSIZ]; FILE *inf, *outf; if ((inf = fopen(argv[1],"r")) == NULL) return EXIT_FAILURE; if ((outf = fopen(argv[2],"w")) == NULL) return EXIT_FAILURE; while (fgets(s,BUFSIZ,inf)) fputs(s,outf); fclose(inf); fclose(outf); return EXIT_SUCCESS; } else return EXIT_FAILURE } /* End of File */ Listing 3 Copies binary files /* copy5.c */ #include #include main(int argc, char *argv[]) { if (argc == 3) { char buf[BUFSIZ]; FILE *inf, *outf; if ((inf = fopen(argv[1],"rb")) == NULL) return EXIT_FAILURE; if ((outf = fopen(argv[2],"wb")) == NULL) return EXIT_FAILURE; while (!feof(inf)) { int nitems = fread(buf,1,BUFSIZ,inf); if (fwrite(buf,1,nitems,outf) != nitems) return EXIT_FAILURE; } fclose(inf); fclose(outf); return EXIT_SUCCESS; } else return EXIT_FAILURE; } /* End of File */ Listing 4 Processes fixed-length record files /* records.c: Illustrates file positioning */ #include #include #include #define MAXRECS 10 struct record { char last[16]; char first[11]; int age; }; static char *get_field(char *, char *); main() { int nrecs; char s[81]; struct record recs[MAXRECS], recbuf; FILE *f; /* Carefully store records */ for ( nrecs = 0; nrecs < MAXRECS && get_field("Last",s); ++nrecs ) { strncpy(recs[nrecs].last,s,15)[15] = '\0'; get_field("First",s); strncpy(recs[nrecs].first,s,10)[10] = '\0'; get_field("Age",s); recs[nrecs].age = atoi(s); } /* Write records to file */ if ((f = fopen("recs.dat","w+b")) == NULL) return EXIT_FAILURE; if (fwrite(recs,sizeof recs[0],nrecs,f) != nrecs) return EXIT_FAILURE; /* Position at last record */ fseek(f,(nrecs-1)*sizeof(struct record),SEEK_SET); fread(&recbuf,1,sizeof(struct record),f); printf("last: %s, first: %s, age: %d\n", recbuf.last,recbuf.first,recbuf.age); /* Position at first record */ rewind(f); fread(&recbuf,1,sizeof(struct record),f); printf("last: %s, first: %s, age: %d\n", recbuf.last,recbuf.first,recbuf.age); return EXIT_SUCCESS; } static char *get_field(char *prompt, char *buf) { /* Prompt for input field */ fprintf(stderr,"%s: ",prompt); return gets(buf); } /* End of File */ Listing 5 A simple four-way scrolling file browser /* view.c * * VALID COMMANDS: * D scroll Down 1 screen * U scroll Up 1 screen * B go to Bottom of file * T go to Top of file * L scroll Left a partial screen * R scroll Right a partial screen * Q or X quit */ #include #include #include #include #include "stack.h" #include "ansi.h" #define NROWS 24 /* Height of screen - 1*/ #define NCOLS 79 /* Width of screen - 1 */ #define HORIZ 32 /* Horiz. scroll increment */ #define MAXSTACK 500 /* Max # of screens */ /* Buffer for current screen */ static char Screen[NROWS] [BUFSIZ]; static size_t Nlines, /* Number of lines to display */ Offset = 0; /* Horizontal display offset */ static void read_a_screen(FILE *); static void display(void); static void display_banner(char *, size_t); main(int argc, char *argv[]) { FILE *f; char *file; fpos_t top_pos, stk_[MAXSTACK]; size_t stkptr_ = 0; int c; /* Open input file */ if (argc == i (f = fopen(argv[1],"r")) == NULL) { fputs("No such file.\n",stderr); return EXIT_FAILURE; } file = argv[1]; top: /* Display initial screen */ rewind(f); fgetpos(f,&top_pos); read_a_screen(f); display(); display_banner(file,stkptr_+1); for (;;) { switch(c = toupper(getchar())) { case 'D': /* Display the next screen */ if (!feof(f)) { PUSH(top_pos); fgetpos(f,&top_pos); read_a_screen(f); display(); } break; case 'U': /* Display the previous screen */ if (stkptr_ > O) { top_pos = POP(); fsetpos(f,&top_pos); read_a_screen(f); display(); } break; case 'T': /* Display first screen */ stkptr_ = O; goto top; case 'B': /* Display last screen */ while (!feof(f)) { PUSH(top_pos); fgetpos(f,&top_pos); read_a_screen(f); } display(); break; case 'L': /* Scroll left */ if (Offset > 0) { Offset -= HORIZ; display(); } break; case 'R': /* Scroll right */ if (Offset < BUFSIZ-HORIZ) { Offset += HORIZ; display(); } break; case 'Q': /* Quit */ case 'X': cls(); return EXIT_SUCCESS; } if (c != '\n') getchar(); /* Eat '\n' */ display_banner(file,stkptr_+1); } } static void read_a_screen(FILE *f) { int i; clearerr(f); for ( i = O; i < NROWS && fgets(Screen[i],BUFSIZ,f); ++i ) Screen[i] [strlen(Screen[i])-1] = '\0'; Nlines = i; } static void display(void) { int i; /* Display a screenful of text */ cls(); for (i = O; i < Nlines; ++i) if (Offset < strlen(Screen[i])) { setcur(i+2,1); fprintf(stderr,"%.*s",NCOLS,Screen[i]+Offset); } } static void display_banner(char *fname, size_t screen_num) { char banner[NCOLS+1]; /* Display banner line in reverse video */ setcur(1,1); rvs(); sprintf(banner, "File: %s Screen: %d Columns: %d-%d", fname,screen_num,Offset+1,0ffset+NCOLS); strcat(banner," Command? "); fprintf(stderr,"%-*.*s",NCOLS,NCOLS,banner); reset(); setcur(1,strlen(banner)+1); } /* End of File */ Listing 6 A simple stack /* stack.h: Macros for a stack - * * A simple-minded stack mechanism. The user's * program must define the following in the scope * where they are used: * * MAXSTACK The dimension of the stack * size_t stkptr_ = 0; The stack pointer * T stk_[MAXSTACK]; The stack: an array of type T * * Execution aborts if the stack overflows or * underflows. */ # include /* Stack operations: */ #define PUSH(x) \ (assert(stkptr_ < MAXSTACK), stk_[stkptr_++] = (x)) # define POP() \ (assert(stkptr_ > 0), stk_[--stkptr_]) /* End of File */ Listing 7 ANSI terminal escape sequences /* ansi.h: ANSI terminal escape sequences */ #include #define setcur(row, col) \ fprintf(stderr,"\O33[%d;%dH",(row),(col)) #define save_cur() fputs("\O33[s",stderr) #define restore_cur() fputs("\O33[u",stderr) #define cls() fputs("\O33[2J\O33[H",stderr) #define beep() putc('\OO7',stderr) #define home() fputs("\O33[H",stderr) #define clear_line() fputs("\033[2K",stderr) #define clear_eol() fputs("\O33[K",stderr) #define up(n) fprintf(stderr,"\O33[%dA",(n)) #define down(n) fprintf(stderr,"\O33[%dB",(n)) #define right(n) fprintf(stderr,"\O33[%dC",(n)) #define left(n) fprintf(stderr,"\O33[%dD",(n)) #define reset() fputs("\033[0m",stderr) #define bold() fputs("\O33[1m",stderr) #define underl() fputs("\O33[4m",stderr) #define blink() fputs("\O33[5m",stderr) #define rvs() fputs("\O33[7m",stderr) /* End of File */ Listing 8 An external sort program /* esort: Sort large files by merging subfiles */ #include #include #include #include #define MAXLINES 1000 #define MAXSUBFILES 15 struct subfile { FILE *f; char line[BUFSIZ]; }; int comp(const void *, const void *); void make_subfile(char *[], sizet, struct subfile *, size_t); main() { int i, merge_flag = 0; size_t nlines, nfiles = 0, min_idx; static char s[BUFSIZ], *lines[MAXLINES]; static struct subfile subfiles[MAXSUBFILES]; /* Read file - form subfiles if needed */ for (nlines = 0; fgets(s,BUFSIZ,stdin); ++nlines) { if (nlines == MAXLINES (lines[nlines] = malloc(strlen(s)+1)) == NULL) { /* Sort lines to a temporary merge file */ merge_flag = 1; make_subfile(lines,nlines,subfiles,nfiles++); line[nlines = O] = malloc(strlen(s)+1); assert(lines[O] != NULL); } strcpy(lines[nlines],s); } if (merge_flag) { /* Form last merge file from remaining lines */ make_subfile(lines,nlines,subfiles,nfiles++}; /* Prepare to read temporary files */ for (i = 0; i < nfiles; ++i) { FILE *f = subfiles[i].f; rewind(f); fgets(subfiles[i].line,BUFSIZ,f); } /* Do the merge */ while (nfiles) { struct subfile *sfp; /* Find next output line */ for (min_idx = 0, i = 1; i < nfiles; ++i) if (strcmp(subfiles[i].line, subfiles[min_idx].line) < 0) min_idx = i; sfp = &subfiles[min_idx]; /* Output the line */ fputs(sfp->line,stdout); fflush(stdout); assert(!ferror(stdout)); /* Get the next line from this file */ if (fgets(sfp->line,BUFSIZ,sfp->f) == NULL) subfiles[min_idx] = subfiles[--nfiles]; } } else { /* Sort singleton file */ qsort(lines,nlines,sizeof lines[0],comp); for (i = 0; i < nlines; ++i) { fputs (lines[i],stdout); fflush(stdout); assert(!ferror(stdout)); } } return 0; } int comp(const void *p1, const void *p2) { return strcmp(* (char **) pl, * (char **) p2); } void make_subfile(char *lines[],size_t nl, struct subfile subf[],size_t nf} { int i; FILE *f = tmpfile(); assert(f); /* Write sorted subfile to temporary file */ qsort(lines,nl,sizeof lines[O],comp); for (i = 0; i < nl; ++i) { fputs(lines[i],f); fflush(f); assert(!ferror(f)); free (lines[i]); } subf[nf].f = f; } /* End of File */ Listing 9 Template for a program to print a directory listing /* list.c: List directory*/ #include #include #include int Order = 1, Full = 0; static void dir(char *); main(int argc, char **argv) { char *p; /* Process options */ while (--argc && **++argv == '-') { for (p = *argv+1; *p; ++p) switch(toupper(*p)) { case 'L': Full = !Full; break; case 'R': Order = -1; break; default: fprintf(stderr, "list: Invalid flag: %c\n",*p); return EXIT_FAILURE; } } /* List all files by default */ if (argc == 0) dir("*.*"); else while (argc--) dir(*argv++); } static void dir(char *template) { /* Replace this stub with working code */ fprintf(stderr,"Dir: %s (Order = %d, Full = %d)\n", template,Order,Full); } /* End of File */ Listing 10 Template for a file printer program /* pr.c: Skeleton of a file printer program */ # include #include #include static int Copies = 1, /* Default to one copy */ Number = O; /* Don't print with line numbers */ static char Queue = 'S'; /* Default to standard printer */ static void process(char *); main(int argc, char *argv[]) { int i; char *s; /* Process each argument immediately */ for (i = 1; i < argc; ++i) { if (argv[i][O] == '-') for (s = argv[i]+l; *s; ++s) switch (toupper (*s)) { case 'C': /* Build number for copies */ if (isdigit (s[1])) for (Copies = O; isdigit(s[1]); ++s) Copies = Copies*10 + (s[1] - '0'); break; case 'N': /* Toggle line numbering */ Number = !Number; break; case 'Q': /* Select print queue */ ++s; Queue = toupper(*s); break; default: fprintf(stderr,"pr: Bad option: -%c\n",*s); return EXIT_FAILURE; } else process (argv [i] ); } return EXIT_SUCCESS; } static void process(char *s) { printf ("Processing %s... \n" ,s); printf("\tCopies: %d, Number: %d, Queue: %c\n", Copies,Number,Queue); } /* The output after executing the command line above is Processing file1.c... Copies: 10, Number: 1, Queue: S Processing file2.c... Copies: 1, Number: O, Queue: L Processing file3.c... Copies: 1, Number: O, Queue: L */ /* End of File */ CUG Library Editors Victor R. Volkman Victor R. Volkman received a BS in Computer Science from Michigan Technological University. He has been a frequent contributor to The C User Journal since 1987. He is currently employed as Senior Analyst at H.C.I.A of Ann Arbor, Michigan. He can be reached directly at the HAL 9000 BBS (313) 663-4173 or as sysop@hal9k.ann-arbor.mi.us on Usenet. The C Users Group (CUG) Library offers several complete text editors with full source code supplied, including STEVIE (CUG313), RED (CUG318), and SE (CUG331). Each of these editors has been released into the public domain without any licensing or usage requirements. As such, you can use them in their original condition, modify them as the basis of your own text editor, or embed them into your existing applications where you need text-editing capabilities. I'll discuss the benefits and limitations of each editor and how you can take advantage of them. For a quick overview of their basic statistics and advanced features, see Table 1 and Table 2. STEVIE (CUG313) The STEVIE editor by Tony Andrews (Boulder, CO) is a highly-portable clone of the popular UNIX vi editor. STEVIE first appeared on the Atari ST platform and its name stands for "ST Editor for VI Enthusiasts." However, the current version of STEVIE can be compiled for the Atari ST running TOS or Minix, UNIX System V or BSD, and PCs running MS-DOS or OS/2. Others have reported success in porting STEVIE to the Amiga and Data General machines, although platform-specific code for those ports is not included in this release. My own testing covered STEVIE implementations in MS-DOS v5.0 and OS/2 v2.0. I received STEVIE v3.71 (released 06/08/89) from CUG on a single 5-1/4 inch 360K diskette for the PC. CUG can also supply releases on a 3-1/2 inch diskette. In any case, CUG distribution does not include any binary executables so you must compile it yourself before you can run it. STEVIE displays text with ANSI terminal emulation escape sequences. These escape sequences were popularized by the DEC VT-100 series of terminals. Ideally, ANSI terminal support provides maximum display portability with minimum programming effort. However, the MS-DOS implementation (ANSI.SYS) is notoriously slow and impedes the performance of STEVIE. I found STEVIE to be incompatible with both ANSI.SYS in MS-DOS v5.0 and DVANSI.COM in DESQview/386. Specifically, scroll-down (CTRL-D) and scroll-up (CTRL-U) (keystrokes fail to refresh about half the screen. Although the STEVIE documentation does not mention this problem, it does suggest NANSI.SYS by Daniel Kegel (Altadena, CA) for ANSI emulation. Accordingly, I tested NANSI.SYS v3.3 which worked without problems and clocked screen refreshes 40 percent faster than ANSI.SYS. STEVIE does not check for the presence of ANSI emulation, and will render an illegible set of escape strings if it is missing. If STEVIE had used the ANSI Device Status Report (ESC[6n), then it could have reported the absence of ANSI support and recommended corrective action. STEVIE indirectly supports the EGA/VGA 43- and 50-line display mode because of its configurability. However, you will need both an ANSI.SYS that supports 43-line mode (like NANSI.SYS) and a utility to switch your graphics card into 43-line mode. After setting up the display mode while still in DOS, you must start STEVIE and then issue the command :set lines=43. Scrolling and refreshing worked smoothly on my ATI VGA/Wonder+ (1024K RAM) using L43.COM and NANSI.SYS. I also tried the proprietary ATI 132-column x 44-line mode and found it usable although STEVIE still enforced line wrapping at 80 columns. Because STEVIE v3.71 was released in 1989, the documentation only claims compatability with Microsoft C v5.1 in the MS-DOS environment. However, I compiled the STEVIE source code without errors using the Microsoft C v6.00A. The quality of the source code could be improved with the addition of ANSI function prototypes. The source code ships configured for UNIX and must be modified to build under other platforms. The documentation omits the exact procedure for building on MS-DOS, so I'll describe it here. Specifically, you must comment out both the #define UNIX and #define TERMCAP and uncomment #define DOS in the file ENV.H. Once built, the Microsoft EXEHDR utility reported that STEVIE.EXE was 105K in length and required 129K to run. Building STEVIE for OS/2 v2.0 required more modification. Around the time STEVIE was released, OS/2 v1.1 was current, so some incompatiblities with v2.0 were to be expected. Kevin Nickerson (Ann Arbor, MI) lent me his expertise in porting STEVIE to OS/2 v2.0 using the C/2 compiler. Specificially, he reported the following changes required to correctly build STEVIE: Uncomment #define OS2 in ENV.H Remove declarations like extern char *malloc; which collided with the ANSI headers in all files. Add STDLIB.H to ANSI headers included in STEVIE.H Change inchar to call KbdCharln instead of getch in OS2.C Add handling for EOF case to inchar Change the write and read calls to fwrite and fread in OS2.C Once built, STEVIE for OS/2 v2.0 behaved well in an OS/2 command window.STEVIE was friendly in its CPU usage and had no noticeable impact when idle. The main STEVIE documentation consists of the 12-page User Reference in both plain ASCII and nroff text formats. The ASCII version of the manual is formatted to exactly 66 lines per page with no formfeeds. The documentation clearly states that it is intended for users already familiar with the vi editor. Since vi is already well-documented both in print and online tutorials, this seems reasonable. (See bibliography.) For an idea of which advanced features STEVIE supports, see Table 2. The User Reference does provide brief one-line descriptions of all :set commands, and single and double-letter commands (e.g. yy). Additionally, an online listing of these commands appears when you issue the :help command in STEVIE. The STEVIE documentation also provides a few additional pages of information in Porting Notes and Source Notes. The Porting Notes provide general tips on how to move STEVIE to an unsupported environment. However, specific instructions for compiling in each individual supported environment are missing. The Source Notes cover the most important data structure in STEVIE--the doubly-linked list of text line structures. STEVIE allocates to each line buffer the exact number of characters needed to hold it. Since STEVIE is freeware, without any remuneration to the author, one normally does not expect extensive support. In the documentation, Andrews concedes that since STEVIE is stable, new releases may not be forthcoming. He goes on to say that he will accept bug reports and enhancements to the source code from time to time. Nevertheless, since three years have elapsed since the last release, it may be unrealistic to expect active support at this time. RED (CUG318) The RED editor by Edward Ream (Madison, WI) is an extremely compact editor that uses virtual memory techniques to handle files bigger than memory alone would allow. RED is also notable for its interrupt-driven displays and pattern-matching search and replace. RED first appeared in an article in the January 1983 issue of Dr. Dobb's Journal. CUG318 contains RED v7.0 as released on January 26th, 1990. This version claims compatability with both Microsoft C and Borland Turbo C. RED can edit very large files, yet only needs 56K of memory to run. The secret behind this is the work file that RED creates as it loads your file. The work file is an indexed set of doubly-linked 1K blocks. Each block contains the actual text associated with lines in the edit file. The resulting temporary file is at least as big as the original file. RED keeps at least 10 of these 1K blocks in memory at any time. When the user scrolls out of the region of the file that is in memory, RED swaps out the least recently used blocks. In an attempt to push RED to its limits, I asked it to load a 691K file containing 32,767 lines of text. Although it required several seconds to load up, RED was then able to rapidly traverse the file. Next, I increased the file up to 34,000 lines of text and tried RED again. RED reported several internal errors in building the temporary file and refused to load the file. This problem is due to RED using 16-bit signed integers rather than 32-bit longs to represent line numbers. Although RED will handle very long text files well, none of the lines can be longer than 79 characters. RED simply reports line truncated after feeding it a file with lines up to 255 characters wide. RED will prevent you from entering a line longer than 79 characters as well. If you enable line wrap, then it will move the last word on a line as a word processor would. Another technical innovation of RED is its interrupt-driven screen refresh. Screen refreshes most often occur when you scroll up or down an entire page of text. Rather than simply allowing MS-DOS to buffer keystrokes while the screen is refreshing, RED reads each keystroke and interrupts the refresh. The main benefit is that you can traverse the file much quicker than if you had to wait for every single screenful of text to be painted. Many commercial word processors use such a scheme to defer the reformatting of paragraphs until you are done typing, for example. Since RED is so compact, its command set is quite limited. (See Table 2 and Table 3.) The block move, copy, and delete functions are cumbersome because they require you to keyboard the actual line numbers affected. The undo handling is limited to the current line and disappears if you leave the line. However, RED does provide pattern matching search and replace. The patterns supported are a small subset of regular expressions. Specifically, you can set up patterns with ? for single character matches, ^ to match only at the beginning of a line, and $ to match only at the end of a line. Since RED does not have a Kleene * operator to match zero or more characters, it falls short of complete regular expression handling. I found RED to be in good working order with the exception of the quit command. The quit command, which is supposed to exit to MS-DOS, would report the error Too many ?s in mask and hang the system. I tried running it on both MS-DOS v3.3 and v5.0 and achieved identical results. With some detective work, I discovered #ifdefs and comments indicating that the quit command was only an experimental alternative to the exit command. As such, I only needed to comment out the #ifdef SUSPEND to remove the quit command. Since rebuilding RED.EXE correctly took some additional effort, I will report on the process. I first had to remove many hardcoded pathnames from the makefile and some of the #includes. The makefile expects a particular subdirectory structure but the files on the distribution disk were all in one flat directory. I corrected the makefiles and continued to build. Compiling RED with Microsoft C v6.00A uncovered numerous warnings about trying to use the NULL constant (OL) in conjunction with 16-bit int variables. I had to edit REDMAIN.C to prevent an error message where NULL was used as a case in an int switch statement. I encountered many long/short mismatch warnings where the compiler was forced to do a type promotion. Microsoft MASM v5.1 warned that the symbol NEAR in REDOUTA.ASM duplicated a reserved word. I changed the symbol NEAR to USENEAR to avoid the conflict. Upon completing the build, I soon discovered runtime stack errors on exiting RED. I eventually increased the stack from its initial value of 3,000 bytes up to 8,000 bytes to prevent stack overflows. After this final fix, RED performed all functions without complaint. The documentation for RED consists mainly of the 15-page Reference Guide provided in plain ASCII format. The Reference Guide covers each command and special keystroke in detail. The manual is organized well enough for even a novice to understand the behavior of the editor. The supplementary four page Technical Notes provide some porting hints, commentary on programming styles, and a description of how to use the SHERLOCK debugging macros. Since RED is freeware, one normally would not expect a formal support program. Rather than requiring a fee for support, Ream offers a bounty of $5.12 that is "cheerfully paid to the first person who finds any bug." He also offers a bounty of $1.28 for each typographical error found in documentation or source code. SE (CUG331) The Stack Editor (SE) by Gary Osborn (Yorba Linda, CA) provides complete command and text stacking rivalling commercial products such as Brief. SE also contains a powerful built-in virtual memory manager for editing files up to 3MB long. SE buffers very large files with as little as 6K additional RAM and as much as 500K RAM depending on available memory. SE also includes an embedded nroff dot command processor for added word processing functionality. CUG331 contains SE vl.0 as released on June 12th, 1990. The SE editor originates from a distinguished lineage of editors with a decade of development behind it. It all began with G. Nigel Gilbert's groundbreaking E editor for CP/M released between 1981 and 1983 (CUG133). In the days of CP/M, which had a 64K memory limit, the virtual-memory aspect was even more important than today. James W. Haefner built on Gilbert's initial design and eventually released Q.E.D. v4.6b for PCs in 1985 (CUG157). Mel Tearle in turn built on Haefner's code to produce GED vl.05 in 1986 (CUG199). The GED editor was the first of the line that could run on the new PC clones, such as Compaq. Last, Gary Osborn put his mark on the code in 1990 to produce the advanced command and text stacking capabilities comprising SE. SE provides a complete set of block editing operations: move, copy, delete, shift (indent/outdent), and print. SE is notable in that it actually highlights the block of text as you define it by moving the cursor. The block of text must begin and end on line boundaries. The shift operation is especially nice in that you move the block left and right with each keystroke. All block operations can be undone and redone in accordance with the stack metaphor. SE's stack operations provide an easy way of moving a line of code, of transposing lines, and of duplicating lines. Pushing several lines, then popping them elsewhere is a convenient way to move a small block. The text stack shares the same area as the command stack (used for undo and redo). Pops can be undone, but if the editing operation that did the push is undone, then the associated pop will find the stack empty. You can either pop directly from the stack or pop a copy of the top of the stack. A few editing operations other than delete (such as line concatenation) affect the stack also and can result in unexpected items on the stack. Don't do too much editing if items are to be popped from the stack. The nroff (or runoff) commands provide simple directives for basic word processing functions. You can imbed these commands in your plain ASCII text file or ignore them completely. These commands always start with a dot in column one and are abbreviated to two letters. Specifically, SE supports left margin (.in), right margin (.rm), top margin (.m1), disabled formatting (.nf), lines per page (.pl), page headers (.he), page footers (.fo), page breaks (.bp), page break override (.ne), center text (.ce), underline text (.ul), and line spacing (.ls). Many of these commands can be run from within SE, rather than requiring you to generate an output file to activate them. Building SE required some additional effort, as I will describe. Since SE was last released for Microsoft C v4.0, I had to adjust the BUILD.BAT file to call CL.EXE rather than MSC.EXE. Compiling the 15 source files produced more than 118 warning messages at the lowest warning level (/W1). Fortunately, about 100 warnings were due to implicitly defined int functions which returned no particular value. There were three warnings regarding case values of more than 1,000 being compared in a switch statement where the operand was an unsigned char. An unsigned char can only assume values from 0 to 255, so 1,000 is clearly out of range. Once compiled, I needed to adjust the linker response file to handle the format of Microsoft Linker v5.10. I quickly discovered a multiply-defined external called clock. The MSC run-time library provides a function called clock which returns the amount of time the application has been running. This clock function conflicted with a global int called clock defined in GED.H. The variable clock serves as a counter for the LRU paging mechanism in SE. I resolved the situation by renaming the clock variable to se_clock in GED.H, SWAP.C, and STORE.C. After resolving this conflict, I encountered no further problems. The Microsoft EXEHDR reported that SE.EXE was 43K in length and required 64K to run. All editing functions worked just as well as in the executable distributed on diskette. The SE documentation consists of two separate eight-page overviews entitled simply The SE Editor and SE Stack Editor. Both documents are formatted with some nroff dot commands, but are still quite readable in source form. The first document discusses the architectual tradeoffs taken in the virtual memory and screen refresh implementation. Other topics include the motivation for stack-based editing, handling partially binary files, and embedded nroff commands. The second document provides a keyboard quick reference, detailed dot command reference, and command-line parameters. Although there are no detailed discussions of algorithms or data structures, the C source code is fairly well documented. Since SE has been released into the public domain without compensation to the author, the absence of a formal support plan should not be surprising. Even though Osborn devotes several paragraphs to outlining ideas for future work, there is no release schedule attached. The documentation does not mention a host where the latest version may be found nor is there an e-mail address for the author. The Best Editor for the Job Choosing between the STEVIE (CUG313), RED (CUG318), and SE (CUG331) can be difficult. If you need cross-platform compatibility or prefer the vi command set, then STEVIE is your best choice. If you need a very compact editor that can handle reasonably large files (~700K) or require interrupt-driven screen refresh, then RED is your best choice. If you need a small editor with strong block operations, the ability to handle very large files (3,000K), or nroff support, then SE is your best choice. All are public domain with full source code and available at minimal cost directly from the C Users Group Library. Reference Lamb, Linda. 1988. Learning the vi editor. Sebastopol, CA: O'Reilly & Associates. Contact nuts@ora.uu.corn or uunet!ora!nuts. Table 1 Editors at a glance Item STEVIE RED SE ---------------------------------------------------------------------- Version 3.71 7.0 1.00 Release Date 08/06/89 01/26/90 06/12/90 Size of .C/.H files 224K 174K 195K Size of .ASM files N/A 5K 34K Free memory to run 129K 45K 64K .EXE on the disk No Yes Yes Max lines/file edit 16,384 * 32,767 16,384 Max chars/line edit > 1,024 79 253 Text display method ANSI.SYS BIOS INT 10h Direct Write Command emulation UNIX vi None WordStar Compilers Supported ** MSC and others MSC and Turbo C MSC Platforms Supported MS--DOS, OS/2, MS--DOS MS--DOS Atari ST, UNIX BSD, and SV * Limited by available memory. Usually closer to 8,192 due to 640K DOS limit. May be much higher on Unix and 32-bit systems. ** Note that unspecified compilers and versions might work, but have not been certified by the author of the package. Table 2 Editor advanced feature comparison Advanced Feature STEVIE RED SE ----------------------------------------------- 43-line mode editing Yes No No Edit multiple files Yes No No Auto-backup files Yes No Yes Show line numbers Yes No No Auto-indent mode Yes No No Global substitution Yes Yes Yes Regular expressions Yes Subset No CTAGS func. finder Yes No No Shell to DOS Yes Yes No Find matching ()[]{} Yes No No Block shift left/rt. Yes No Yes Block move/copy/del. Yes Yes Yes Highlights sel. block No No Yes Undo last command Yes Limited Stack * Repeat last command Yes Yes Stack Dump text to printer No Yes Yes Edit files memory No Yes Yes Runs without ANSI.SYS No Yes Yes Free roaming cursor No Yes Yes Debug build avail. No Yes No Interruptable display No Yes No Read Wordstar files No No Yes Embedded nroff No No Yes Scroll horizontally No No Yes Keys help window No No Yes *SE maintains a stack of the last 100 changes that can be undone. Table 3 Complete command set for RED Command Default args Description ------------------------------------------------------------- change 1 9999 Change all lines in clear Reset the editor copy Copy lines in after line delete Delete one or more lines exit Exit from the editor extract Create a file from a block find Search for a pattern findr Backward find g Enter edit mode at line help Print this message inject Add a line to the buffer list 1 9999 List lines to the printer load Replace the buffer with move Move lines of after line name Set filename nowrap Disable line wrapping resave Save the buffer to an existing file save Save the buffer to a new file search 1 9999 Search for pattern tabs 8 Set tabs to every columns wrap Enable line wrapping ! Shell to DOS Writing a UNIX Device Driver, Second Edition Sydney S. Weinstein Sydney S. Weinstein, CDP, CCP is a consultant, columnist, lecturer, author, professor, and President of Datacomp Systems, Inc., a consulting and contract programming firm specializing in databases, data presentation and windowing, transaction processing, networking, testing and test suites, and device management for UNIX and MS-DOS. He can be contacted care of Datacomp Systems, Inc., 3837 Byron Road, Huntingdon Valley, PA 19006-2320 or via electronic mail on the Internet/USENET mailbox syd@DSI.COM (dsinc!syd for those that cannot do Internet addressing). Janet Egan and Thomas Teixeira have updated their book Writing a UNIX Device Driver to encompass the newer versions of UNIX including BSD 4.3 and USL System V Release 4.0. Their goal was to lead the experienced programmer who has never written a device driver or one who has written device drivers for non-UNIX systems or older UNIX systems through the steps necessary to write both a character (raw) and a block (buffered) device driver for UNIX. They tried to accomplish this by explaining the input/output subsystem, describing a device driver, what it must handle, how the device driver and the hardware interact, and then how the device driver and UNIX interact. Writing a device driver has never been a simple task. There are many performance, reliability, and debugging obstacles to overcome. In this book, Egan and Eixeira have tried to show programmers experienced in writing device drivers for non-UNIX systems and for older UNIX systems, or experienced programmers who have never written a device driver before, how to write a device driver for the current UNIX systems. However, they fall short of that goal. Content The book is divided into 11 chapters and five appendices. All chapters include code examples to illustrate the chapter topic. The subject matter of the chapters is: Chapters 1 and 2--introduction to the UNIX I/O subsystem, with the emphasis on the underlying data flows through the operating system to satisfy the users I/O requests. Chapter 3--introduction to the UNIX device driver by discussing the required entry points. Chapter 4--how a driver interacts with the various types of I/O hardware. Chapter 5--configuring the UNIX operating system and installing a new device driver. Chapters 6 and 7--the device driver and its various options including Interrupt, Synchronous, and DMA drivers. Chapter 8--hints on debugging your new driver. Chapter 9--a template that can be used in writing your own driver. Chapter 10--the special aspects of SCSI drivers Chapter 11--streams The appendices provide a tour of the header files used for drivers, a list of the kernel support routines available, a template character device driver, a template block device driver, and a table of considerations in porting existing UNIX/XENIX drivers to the current UNIX versions. Using this book requires having a copy of a UNIX device driver manual handy. It does not present the information contained in that manual. For System V Release 4, having a copy of the DDI/DKI (Device Driver Interface/Driver-Kernel Interface) manual is essential because the options, arguments, and calling sequences to each kernel interface routine are not explained in the book. For other UNIX operating systems, access to the appropriate volume for your vendor is also necessary. Commentary The introductory text for each chapter flows well, but lacks detail, and sometimes correctness. When the authors discussed matters of the UNIX operating system outside the realm of writing device drivers, I found myself saying, "That's not how it works," several times as I read the book. However, these oversights and incorrect statements are only in the sidebar-type information on the functioning of the rest of the UNIX operating system around the device driver, and don't detract from the discussion of the device driver itself. The authors did not do a thorough job of revising the book. There are often subtle mistakes in some of the text, which sometimes is the older method that never got completely updated for the second edition. The examples offered in the book are short and present the bare minimum needed to explain each concept. Although explained action by action, the examples are so minimal that the explanations seem shallow and simplistic, presenting only the basics of the problems. The authors cover none of the useful and often required extra details, such as error recovery, timeouts, status tracking, or the interfaces into the systems performance monitoring routines. Overall, I think this book relies too heavily on prior knowledge of the reader, without explicitly stating special requirements for comprehension. First, it seems to require the reader to have read the first edition. Too many times the book refers to how things used to be or to the text in the first edition, but never presents the old way or text, leaving a description hanging without the sample code or a proper reference. In addition, the book seems to require the reader to have written a UNIX device driver, in spite of its stated goal of helping the programmer who has never written a UNIX device driver learn to write a UNIX driver. It would be easy for people to get lost if they have never written a UNIX driver before. The book meets its goal of bringing a prior-UNIX device driver writer up to speed on SVR4 and BSD4.3 (if one overlooks the editing mistakes). However, it falls short of its goal of teaching programmers not experienced with writing UNIX device drivers to write one. Conclusion If you have written a UNIX device driver for an older UNIX system and need to get up to speed for a newer version such as System V Release 4 or BSD 4.3, this book could be of some use to you. But if you have never written a UNIX device driver before, don't expect this book to help much. Title: Writing a UNIX Device Driver second edition Author: Janet I. Egan Thomas J Teixeira Publisher: John Wiley & Sons, Inc. 605 Third Avenue New York, NY 10158-0012 Price: $29.95 ISBN: 0-471-53574-5 Editor's Forum One of the livelier parts of this magazine, often enough, is the Letters to the Editor. It gives our readers a chance to sound off, and it gives the editorial staff (mostly me) and our authors a chance to reply. For my part, I appreciate the feedback on how we're doing. Even when it is negative. You'd think there'd be nothing easier than pasting together a bunch of letters and replies. After all, most of the copy is turned out by you folks. Those of us who craft readable sentences for a living are getting mostly a free ride. Unfortunately, life is never as simple as it seems. First, we have an obligation to filter letters for suitability. Robert Ward and I both tend to be generous in this regard. We feel that any reader who writes at all coherently on a topic relevant to this magazine deserves to be heard. We draw the line, however, at out and out libel or nonsense. CUJ does not have an obligation to be a soapbox for all comers. Then we owe it to our letter writers to present what they say in a fair light. Believe it or not, that means editing what they say. Errors of grammar and punctuation loom larger when typeset than when dashed off as a quick note or e-mail message. We often must trim long letters, or letters accompanied by lots of code. Partly that's for space considerations and partly that's to help focus the message for our readers. It is generally my job to ensure that any edits do not alter meaning or tone for the worse. We have a further obligation to our letter writers to respond to them appropriately. Again, Robert and I feel that most letters deserve some sort of answer. I am irrepressibly flip at times, but I try hard not to be cruel. It's distressingly easy to be offhand or snotty, particularly in answering the more naive letters. (They deserve the most careful attention.) Finally, we have a major obligation to our readers when letter writers make representations that can be damaging to others. In particular, CUJ tries hard to be fair to vendors, because they are so vulnerable to offhand attacks. The staff habitually checks out adverse product reviews or negative remarks by letter writers. If the affected vendor persuades us the charges are factually incorrect, we either print the counterclaim or drop the offending words from the outset. At the very least, it is my obligation in responding to a letter to put harsh statements in perspective. I must call attention to any possible bias, or label a negative statement as just one person's experience, not necessarily substantiated. To say nothing can easily be read as an implicit endorsement. I committed just such a gaffe recently. A reader wrote to criticize a vendor and suggest that CUJ serve as a forum for similar concerns. My response to the suggestion inadvertently appeared to affirm the criticism. This was a disservice to both our readers and our vendors. However strongly the letter writer felt, the letter presented only one side of the story. By presenting it unadorned, we biased the presentation against the vendor. I apologize. So you see, it's harder than you might think to present an open forum in a way that is fair to all parties involved. We won't always succeed, but we'll keep trying. P. J. Plauger pjp@plauger.com New Products Industry-Related News & Announcements MetaWare Compiler Available for UnixWare MetaWare Incorporated has released its High C compiler for Univel's UnixWare, supporting both the Personal Edition and the Application Server. High C supports code based on ANSI C and AT&T Portable C Compiler (pcc) standards. High C for UNIX SVR4 is $895. Contact MetaWare Incorporated, 2161 Delaware Ave., Santa Cruz, CA 95060-5706, (408) 429-6382; FAX: (408) 429-9273. STI Expands Universal Component System Software Transformation, Inc. (STI) has released the Connectivity Series, an expansion to its Universal Component System (UCS) multiplatform development system. The Connectivity Series, together with UCS's Foundation Series (operating system services) and Interface Series (graphical user interface), provide a broad, multiplatform API, with support that includes Windows, UNIX, NT, OS/2, and Macintosh, and compatibility with C, C++, and Pascal compilers. The Connectivity Series provides for interprocess communication, messaging, object linking and embedding (OLE), clipboard management, Apple Edition Manager (Publish and Subscribe), and AppleEvents. The components are hand-crafted on each platform. STI has also announced a UnixWare version of its Universal Component System. Contact Software Transformation, Inc., 1601 Saratoga-Sunnyvale Rd., Suite 100, Cupertino, CA 95014, (408) 973-8081; FAX: (408) 973-0989. Lattice Releases PenBase Lattice, Inc. has released PenBase, a database management library for pen-based computer applications. The library allows programmers to create, assess, and update files that are dBASE compatible by using C functions that supplement GRiD's PenRight! Pro Software Development Kit. The PenBase library includes 85 functions that can be called directly from a C program. PenBase functions were designed with three goals to support pen-based database applications in an MS-DOS environment: they allow programmers to build file management systems which use indexed sequential access method (ISAM), they integrate many of the tasks required to transfer data between the screen and database files, and they create and manage files which are compatible with the dBase III PLUS file structure. Specific functions that integrate the database with the pen application include the ability to link fields, controls and lists directly to the database, and the ability to load fields or controls with values from the database records. PenBase is $495 and does not require royalties or runtime licenses. Contact Lattice, Inc., 3010 Woodcreek Dr., Suite A, Downers Grove, IL 60515, (708) 769-4060; FAX: (708) 769-4083. Liant Software Releases Development Tools for UnixWare Liant Software has released three products, LPI-C, LPI-C++, and C++/Views, that support Univel's UnixWare, a Novell NetWare-ready operating system based on UNIX SVR4.2. LPI-C is an ANSI C compiler for developing UnixWare applications for 386, 486, and Sun SPARC platforms. LPI-C passes both the FIPS-160 Perennial and the Plum Hall ANSI C Validation Suites. LPI-C uses Liant's global optimizer that optimizes an entire compilation unit. LPI-C users can select either the standard UNIX header files and libraries or the LPI-C ANSI runtime libraries. LPI-C++ is a 32-bit compiler implementation of the C++ object-oriented development environment based on The Annotated C++ Reference Manual by Ellis and Stroustrup. LPI-C++ is compatible with cfront v1.2, v2.0, v2.1, ANSI C, and K&R C. LPI-C++ includes a complete implementation of the v2.0 iostream library and the complex class library. LPI-C++ v1.1 is bundled with the RogueWave tool.h++ class library. C++/Views is a C++ class library for GUIs supporting UNIX/Motif, Microsoft Windows, and OS/2 Presentation Manager. C++/Views provides compiler independence and includes C++/Browse, a source code browser and editor for building C++ applications and creating C++ classes. Liant also announced Fortran, COBOL, and PL/1 tools. The single user price is $595 for LPI-C, $1,295 for LPI-C++, and $1,495 for C++/Views UNIX/Motif version. Contact Liant Software Corporation, 959 Concord St., Framingham, MA 01701-4613, (508) 872-8700; FAX: (508) 626-2221. Blaise Computing Announces Control Palette/NC Blaise Computing Inc. has announced Control Palette/NC for Windows application development. Control Palette/NC lets programmers customize the appearances of the Windows nonclient area. Borders, title bars, and menu bars can be given a colorful, three-dimensional appearance. The capabilities of Control Palette/NC are embodied in a DLL, so it can be used with languages or environments that can access DLLs. Control Palette/NC is shipped with object-oriented libraries for OWL (both Borland C++ and Pascal) and for Microsoft MFC application frameworks. Control Palette/NC is packaged with a reference manual that describes customization of the nonclient area, with examples in both C and Pascal, bitmaps that can be used as is or modified, and all source code. Control Palette/NC is $169 and Blaise Computing provides "an unconditional, 60-day, money-back guarantee." Contact Blaise Computing Inc., 819 Bancroft Way, Berkeley, CA 94710, (510) 540-5441; FAX: (510) 540-1938. Symantec Upgrades Zortech C++ Symantec Corporation has upgraded two versions of its Zortech C++ compiler: Zortech C++ v3.1 for MS-DOS and Windows v3.1, and Zortech C++ v3.1 for OS/2 v2.0. Zortech C++ v3.1 is compliant with AT&T cfront C++ v3.0 and ANSI C, and provides full 32-bit and advanced numerics support. Zortech C++ v3.1 provides a true implementation of templates. Zortech C++ v3.1 conforms to the IEEE-754 floating-point standards and NCEG 91-015 draft for numerical extensions. Zortech C++ v3.1 for MS-DOS and Windows v3.1 comes with the MultiScope Debuggers and the Whitewater Resource Toolkit. MultiScope Debuggers are Windows-hosted, C++ debuggers. Whitewater Resource Toolkit allows users to create or customize dialogue boxes, cursors, icons, menus, and buttons. This feature allows programmers to design the look and feel of a Windows program. Zortech C++ v3.1 includes support for multimedia, Object Linking and Embedding (OLE), Dynamic Data Exchange (DDE), pen, and TrueType. Zortech C++ v3.1 for OS/2 v2.0 provides optimized, native 32-bit code generation. A mature, global optimizer aids users to produce tight code. The new versions of Zortech C++ are $499, with discounts for registered users. Contact Symantec Corporation, 10201 Torre Ave., Cupertino, CA 95014-2132, (408) 253-9600; FAX: (408) 253-4092 or (408) 252-4694. WNDX Corporation Upgrades WNDX Development Tools WNDX Corporation has upgraded its WNDX development tools for building portable Motif applications. WNDX's OPUS is an interactive visual resource builder that allows the user to build resource definitions for menus, dialogs, icons, cursors, color palettes, strings, and patterns without writing code. WNDX's Object with Attributes Lists attaches callback code to the GUI resources. WINDX's "superset" approach has lead to over 1000 functions and attributes, supported through the WNDX's API on several platforms. Other features of WNDX include interactive GUI design tools, open access to native services, user control over look and feel, and GUI portability. WNDX also allows users to build modularity from one application to another. Application components from existing WNDX applications can be reused in other applications. The cost is $695 and there are no runtime royalties. Platforms supported include Motif, WIndows, Macintosh, OPENLOOK, and MS-DOS. Contact WNDX Corporation, (403) 244-0995; BBS: (403) 244-1151. SES Announces SES/objectbench Scientific and Engineering Software, Inc. has announced SES/objectbench, an object-oriented analysis (OOA) toolset for developing object-oriented analysis models. SES/objectbench consists of a graphical modeling tool for creating OOA models, and an animated simulator for examining the dynamic behavior of OOA models. SES/objectbench is based on the Shlaer-Mellor method and offers on-screen animation and dynamic checking of OOA models. SES/objectbench's design capture capability provides for static and dynamic verification. SES/objectbench is available on Sun SPARC platforms. SES/objectbench Graphical Modeling is $4,900 and SES/objectbench Animated Simulation is $14,500. Contact Scientific and Engineering Software, Inc., 4301 Westbank Dr., Building A, Austin, TX 78746-6564, (512) 328-5544; FAX: (512) 327-6646. Reasoning Systems Ships REFINE/FORTRAN v1.2 Reasoning Systems and Interactive Development Environments, Inc. have upgraded REFINE/FORTRAN, a reverse engineering tool. REFINE/FORTRAN can be used to move FORTRAN code into design diagrams in Software through Pictures (StP), an analysis and design toolset from IDE. Developers can edit the design diagrams and generate standard C code templates from the completed design. REFINE/FORTRAN v1.2 costs $4,900 per user on the Sun SPARC workstations. Customers also need IDE's Structured Design, C navigator, and Design Generator for C, priced at $5,000, $2,000 and $6,000, respectively. Contact Reasoning Systems, 3260 Hillview Ave., Palo Alto, CA 94304, (415) 494-6201; FAX: (415) 494-8053. Computer Innovation Upgrades DEBUG*2000 Computer Innovation, Inc. has upgraded DEBUG*2000, its C/C++ debugger. DEBUG*2000 v3.1 features include shared library debugging, debug running process, and support for multiprocessor products such as the NCR 3450/3550 and Unisys U6000. DEBUG*2000 provides windows that allow multiple views of the C or C++ program being debugged, including Source, Local, Global, Stack, Dump, Register, Disassembly, Threads, Help, and a "Watch" window for user-selected data. The selected windows are updated as the program executes. Pop-up lists and menus are used on files, functions, types, variables, and objects. DEBUG*2000 provides C++ support, including classes and methods. DEBUG*2000 is available for UNIX 386/486 platforms. Contact Computer Innovations, Inc., 980 Shrewsbury Ave., Tinton Falls, NJ 07724, (800) 922-0169 or (908) 542-5920; FAX: (908) 542-6121. Softool Corp. Releases CCC/Pro Softool Corporation has released CCC/Pro, which integrates software change and configuration management with software problem management. CCC/Pro functions as a problem management tracking tool, allowing users to customize the management of incoming calls and software problems via electronic forms. CCC/Pro has the ability to continually interface with CCC/Manager, which manages software changes, from development through maintenance. CCC/Pro taps into e-mail packages for communication about problems and changes. CCC/Pro relies on commmercial RDBMS systems, allowing users SQL reporting capabilities. CCC/Pro supports UNIX, Digital, IBM, and PCs. Contact Softool Corporation, 340 S. Kellogg Ave., Goleta, CA 93117, (805) 683-5777; FAX: (805) 683-4105. The Khoros Group Upgrades Khoros The Khoros Group has announced plans for Khoros v2.0, its software development environment for data processing, graphics, and visualization. Khoros consists of programming toolkits, sample toolboxes and programming tools. Khoros includes programs which solve problems in specific areas such as medical imaging, remote sensing, and numerical analysis. The programming tools provide users with direct manipulation, graphical user interface building, automatic code generation, data flow visual programming, and interactive configuration management. Plans call for the upgrade to be released in the summer of 1993. Contact The Khoros Group, University of New Mexico, Rm 110, EECE Bldg., Albuquerque, NM 87131, (505) 277-6563; E-mail: khoros-request@chama.eece.unm.edu. AutoTester Releases AutoTester for OS/2 AutoTester, Inc. has released AutoTester for OS/2 which allows users to create and maintain automated test scripts for IBM OS/2 Presentation Manager-compatible applications. AutoTester's script language and approach allow test cases to be created in advance of code availability. Test cases and scenarios can be documented and stored in text files as they are developed. Prepared tests are executed at machine speeds. AutoTester for OS/2 is a structured, modular approach and operates at the Presentation Manager level. The cost is $5000 a copy. Contact AutoTester, Inc., 8150 N. Central Expressway, Suite 1300, Dallas, TX 75206, (800) 328-1196; FAX: (214) 750-9668. Softbridge Announces OS/2 Testing Product Softbridge, Inc. has announced its Automated Test Facility (ATF) WorkStation, a standalone version of its networked testing product. ATF WorkStation is designed for testing OS/2 Presentation Manager applications. Test plans codified as a set of test "scripts" can be run throughout the development cycle. AFT WorkStation supports a record-playback facility that captures all keystrokes and mouse movements in editable tapes. The built-in scripts can interact with an application through window controls to test the nonuser-interface aspects. Other features include text screen and window text file comparisons, handling timing issues in PC-mainframe interaction, and the use of existing data tables to test an application. Contact Softbridge, Inc., 125 CambridgePark Dr., Cambridge, MA 02140, (617) 576-2257; FAX: (617) 864-7747. SuiteSoftware Shipping SuiteTalk for OS/2 SuiteSoftware has begun shipping its Distributed Message Bus Environment, SuiteTalk, for the OS/2 operating system. TCP/IP will be supported initially, with plans for additional protocols. SuiteTalk is a message-based, interprocess communication environment which provides an API to encapsulate hardware and operating systems, network protocols and topography. SuiteTalk also provides a Name Service based on OSI's X.500 standard and a Security Service based on MIT's Kerberos work. Suite Talk supports Sun, HP, DEC, NeXT, and IBM workstations. A port to Microsoft's Windows is under development. SuiteTalk prices for OS/2 start at $375. Site licenses are available. Contact SuiteSoftware, 7777 Alvarado Rd., Suite 308, La Mesa, CA 91941, (619) 698-7550; FAX: (619) 698-7567. Sound Horizons Introduces SpeakEz Sound Horizons has introduced SpeakEz C++ Libraries, which add sound capabilities to Windows applications. With SpeakEz, programmers can build audio feedback and sound effects into applications, while avoiding the Windows MCI interface. SpeakEZ is a collection of C++ classes that provide transparent access to the Multi-media Extension in Microsoft Windows. The Media Control Interface (MCI) class provides support for manipulating WAVE, MIDI, and CD-Audio sound. A Librarian class is provided for managing the storage, data compression, and retrieval of media files using a RIFF file format. A sound Controller class provides an interface for playing and recording sound files. The Composition class provides support for the construction of complex sound function sequences. Timer and Joystick classes are also part of SpeakEz. SpeakEz includes sample applications that demonstrate typical uses for each class. SpeakEz C++ Class Libraries are compatible with Borland C++ v3.1 and Microsoft C/C++ v7.0. The cost is $99 for static and dynamic libraries. Royalty free source code licenses are available for $249. Contact Sound Horizons, P.O. Box 6625, Holliston, MA 01746, (508) 643-2882. PEER Networks Upgrades Multi-MIB SNMP Agent PEER Networks has upgraded its Multi-MIB SNMP Agent and Development Environment and has added an MS-DOS version. Added features for v1.5 include ANSI C compatibility for greater portability, increased trap generation flexibility, separately specifiable MIB extensions, and the ability to distribute agents and subagents across different processors. Multi-MIB SNMP Agent allows different programs to implement different portions of the MIB. If a managed application crashes, it won't take the rest of the MIB with it. Multi-MIB SNMP Agent includes the agent software, an interface library, and PEER'S MIB compiler, which transforms class definitions into management interface code. Multi-MIB SNMP Agent has been ported to environments including Sun O/S, Wind River's VxWorks, Apollo's Domain OS, and Lynx. Contact PEER Networks, 3375 Scott Blvd., Suite 100, Santa Clara, CA 95054, (408) 727-4111; FAX: (408) 727-4410. FlashTek Introduces FlashView FlashTek, Inc. has introduced FlashView, an upgraded version of the Zortech debugger, ZDB. FlashView adds support for X-32VM and Microsoft C7. FlashView retains the features of ZDM such as multiple debugging windows; automatic display of local variables; C++ name unmangling; expansion of structures; runtime memory protection; dual monitor debugging; runtime variable modification; and an "unlimited" number of breakpoints. When an exception occurs, FlashView places the cursor on the related source line in the program. Users can debug multi-megabyte executables with multi-megabytes of data. FlashView costs $250, or $150 for FlashTek's X-32VM customers, including support for both Zortech and Microsoft. Contact FlashTek, Inc., 121 Sweet Ave., Moscow, ID 83843, (208) 882-6893; FAX: (208) 882-7275; email: flashtek@proto.com. P&PI Releases Intelligent Null Modem Pryor and Pryor Inc. has released the Intelligent Null Modem (INM) for C or assembly language programmers. The INM uses a 386/486 PC to emulate a data link. Users can simulate a pair of modems connected via a telephone link, or any direct connect link between RS 232 ports. Users can control data error rates and line fault conditions on the simulated link. The emulation is programmable by means of a script control language and enables the users to test communication software under controlled error condition and normal condition. INM costs $229 with cable or $169 for the program only. Contact Pryor and Pryor Inc., 602-1230 Comox St., Vancouver, B.C., Canada, (604) 669-2609. ACCENT ToolKit Supports Motif and OpenWindows National Information Systems, Inc. has announced ACCENT ToolKit, a Motif to OPEN LOOK reengineering tool. ACCENT ToolKit is an Application Programming Interface (API) supporting both the OPEN LOOK and Motif Graphical User Interface (GUI) standards without application recoding. ACCENT ToolKit v1.0 provides both Motif v1.1 and Open Windows v3.0 support. Applications currently supporting Motif on Sun Microsystems workstations running Solaris v1.0 and v2.0 can support OPEN LOOK by compiling and linking the application with ACCENT ToolKit. With ACCENT ToolKit, maintaining a single version of source code for a Motif-based application suffices to produce versions for both GUIs. Support can be switched between either GUI standard at runtime, with an argument passed on the command line which starts the application. ACCENT ToolKit can substitute for a Motif license. Executable programs linked with ACCENT ToolKit do not require an on-going royalty commitment to NIS. Contact National Information Systems, Inc., 4040 Moorpark Ave., Suite 200, San Jose, CA 95117-1852, (800) 441-5758 or (408) 985-7100; FAX: (408) 246-3127. EAGLE Releases Disk Management Tool for UNIX EAGLE Software, Inc. has released DISK_PAK for UNIX, a disk defragmenter, optimizer, and management tool. DISK_PAK allows in-place clean-up of fragmented disks as well as clustering of frequently accessed information. DISK_PAK supports online file viewing, displaying file system fragmentation graphically without interrupting normal processing. Visual display of fragmentation information can be used to determine when file system optimization should be scheduled. DISK_PAK has an X-Window graphical user interface, and includes an online help utility. DISK_PAK supports character-based terminals and RAID 5 disk architecture. DISK_PAK supports these platforms: Sun SPARCstations with SunOS v4.1.x, SCO OPEN DESKTOP with SCO UNIX V/386, Data General AViiON with DG/UX v5.4.x. The price starts at $295. Contact EAGLE Software, 123 Indiana Ave., P.O. Box 16, Salina, KS, 67402-0016, (913) 823-7257; FAX: (913) 823-6185. MicroQuill Ships SmartHeap v1.5 Microquill Software Publishing, Inc. has begun shipping SmartHeap v1.5, a dynamic library link (DLL) for memory management in Microsoft Windows v3.x applications. SmartHeap provides a fixed-size allocator that manages standard, portable allocation and free-lists; APIs including ANSI malloc, calloc, realloc, and free for C; and operators new and delete for C++. SmartHeap costs $395, or $500 with source code. Contact MicroQuill Software Publishing, Inc., 4900 25th Ave. NE, Suite 206, Seattle, WA 98105, (206) 525-8218; FAX: (206) 525-8309. McCabe Ports Tool Set to Solaris McCabe & Associates have ported their Tool Set to run on Sun Microsystems Solaris v2.1. The McCabe tools are integrated software development tools for forward/reverse engineering, part of IBM's CASE Solution Workbench. The tools are built around Battlemap, which analyzes source code and calculates the McCabe complexity metrics. Five additional tools complete the set: BattlePlan, Analysis of Complexity Tool, CodeBreaker, McCabe Instrumentation Tool, and McCabe Slice Tool. The tools support symmetric multiprocessing and multithreading. Contact McCabe & Associates, 5501 Twin Knolls Rd., Suite 111, Columbia, MD 21045, (800) 638-6316; FAX: (410) 995-1528. ONTOS Ships ONTOS DMP ONTOS, Inc. has begun shipping ONTOS Distributed Multimedia Platform (DMP), for building distributed multimedia applications. ONTOS DMP includes a distributed DBMS based on ONTOS' object-oriented technology; low-end DOS machine connectivity; a multimedia class library; and extensions to support mainframe connectivity. ONTOS DMP also includes Distributed Multimedia Client (DMC), a C API, and Distributed Multimedia Server (DMS), and a class library. Contact ONTOS, Inc., 3 Burlington Woods, Burlington, MA 01803, (617) 272-7110; FAX: (617) 272-8101. Eyring Announces PXROM Eyring Corporation has announced PXROM Porting Environment for developing user-selectable modules into PDOS-based embedded, real-time applications. The integrated PXROM environment allows users to select and configure PDOS modules when building applications. PXROM supports the Microtec ANSI C compiler and includes PDOS modules, the XRAY debugger, and a library of POSIX and PDOS extensions. PXROM is available on various hosts. Runtime license pricing starts at $175. Contact Eyring Corporation, 1455 West 820 North, Provo, UT 84601, (801) 375-2434; FAX: (801) 374-8339. NetManage Adds NetControl for Chameleon TCP/IP products NetManage Inc. has announced NetControl, a utility for centralized management of its MS-Windows DLL-based Chameleon TCP/IP communications applications. NetControl can maintain configuration information and a master copy of NetManage software on a central server (and remotely install it over a LAN). NetControl can also protect configuration information. NetControl will be included with future releases of Chameleon products, or is available as an upgrade. Contact NetManage Inc., 20823 Stevens Creek Blvd., Cupertino, CA 95014, (408) 973-7171. EMS Adds Products to Library EMS Professional Software has added to its PD/Shareware C/C++ and C++ Utility libraries. The joint library has added 66 new programs, and now includes 854 products, compressed with PKZIP, and stored on multiple diskettes (39 1.44MB floppies for the joint library) or a single CD-ROM. The C++ Utilities library has grown 50% from its last release and now has 242 products. Library files are described and indexed in a database which accompanies each library. Users can find programs or routines by type, name, vendor, or free text search. Libraries are $100 for CD-ROM versions, $60 for the C++ Library on diskettes, and $150 for the C/C++ Library on diskettes. Contact EMS Professional Software, 4505 Buckhurst Ct., Olney, MD 20832-1830, (301) 924-3594; FAX: (301) 963-2708. We Have Mail Sirs, I felt impelled to write a response to Mr. Eberhardt's letter in the April '93 CUJ. As President of Strategic Software Designs, Inc. (SSDI), I was very dismayed that we had such an unhappy customer and somewhat surprised by Dr. Plauger's response to that person's letter. In the first half of 1992 we had a number of problems with Airmail and ground deliveries to Canada. Most of these problems were solved by re-shipping product via UPS International Air service where we could track each shipment exactly. Because of these shipping problems, we instituted a strict policy of only shipping via UPS Air for ALL international shipments. Since this policy has been in effect we have experienced no problems. In the process of re-shipping products, Mr. Eberhardt's product was not re-shipped. I cannot re-create exactly how we missed re-shipping this product but it was probably due to a mistake in our shipping department. I am personally calling Mr. Eberhardt to do whatever I can to remedy his situation. As with any small software company SSDI is not perfect but, in answer to Mr. Eberhardt's question, yes there are 300 (and more) satisfied users of _miniEd products and we are going to re-double our efforts to minimize mistakes and guarantee that we promptly respond to the needs of our current and future users. Paul Kovac Strategic Software Designs, Inc. Robert Ward, CUJ Publisher responds: Thanks for writing, I appreciate your candor. In all honesty, on second reading, we're not very proud of our response (see this month's Editorial Forum). I apologize to you and to our readers for not doing a better job of living up to our own standards. I would have preferred that our response had spoken in defense of your motives. Personally, I'm bothered by what appears to me to be a growing willingness to assume the worst of anyone who is in business. I've been running one kind of business or another for almost 20 years. I've dealt with lots of businesses, both as business person and as unknown consumer. In all that time, the only person who I know intentionally tried to cheat me, was an employee of a large department store. But, perhaps that's partly because I always try to assume the best of other people. Whether I'm dealing with one of my associates, another business person, or an unhappy customer, I always assume their motives are above reproach. I always assume that others intend, just as I do, what is best for all involved. Some would say that I'm naive. I would counter that after serving two years as a magistrate judge (as I have), and after running various newspapers for several years (as I have), one is usually cured of their naivete. Instead I believe a trusting approach affects my experience. The person on the other end is usually a lot more responsive when they sense that I trust and respect them. So, I'm on my soapbox. I know it's "politically correct" to assume that businesses are, by definition, greedy and inhuman. But, I would urge you to remember that businesses are composed of individuals. Some of those individuals are, like me, children of the sixties who care a great deal about being of service to their customers. Some, like our readers, are business people, or developers, or engineers--all people who are just trying to do the best they can. People make mistakes and their businesses make mistakes. Sometimes they compound one mistake with another. All that proves is their human frailty. Thus I always make it a point to question only the mistake--not the motives. That's my belief. That's my company's belief. Business would certainly be easier and more rewarding if others acted out of the same belief. Thus, I beg my readers, next time you contact a business about a mistake, question the mistake, not their motives--even if they compound one mistake with another. Again, Paul, I'm embarrassed and dismayed by our response. I hope you'll accept my sincere apology for any inconvenience or grief we may have caused. -- rlw Dear editor: The enclosed disk contains proposals for enhancements to the C programming language to be considered by the ANSI X3J11 committee. (The contents of the disk are available with the supplemental code disk for this issue.) Here is a thumbnail synopsis of each proposal. C-PACKED.TXT I consider this the most important of the proposals. It deals with adding a packed decimal (integer) data type to the language. I consider this an important goal to meet if C is to make any impact on the business computing world. Currently, the best we can get for numerical computations is 32-bit long ints, which are insufficient for meaningful dollar amounts, having a range up to only +21,474,836.47. Doubleprecision floating-point gives us around 52 bits, but the computations are an order of magnitude slower and least significant bits are too easily lost. Adding a packed decimal data type, especially a long packed integer type of at least 15 digits, would be appealing for business computations, since it extends the range of integers substantially (up to +9,999,999,999,999.99 dollars). Most of the issues involved in adding a new arithmetic type are addressed in the document, but a few (surrounded by brackets) still need to be ironed out. Also, the "usual arithmetic promotions" rules still need some work so they don't end up being pages, instead of lines, in length. C-MESSAG.TXT This is a feature that I've wanted, due to the nature of the business where I work. We port our UNIX/C software, unchanged, to a dozen or so platforms, each of which having its own quirks, bugs, and extensions. A #message directive would make porting an easier task, since the people doing the port to new and existing machines often have little clue as to what we, the developers, have done. C-FUNC.TXT This feature is useful for debugging. How many times have you and others you know reinvented a macro package for debugging that looked something like: int func(a) int a; { int x, y; dbg_enter ("func"); ... dbg("a=%d, x is now %d", a, x); ... dbg_leave(); return y; } In this fragment, the dbg_enter and dbg_leave functions/macros keep track of the current function nesting level and their names, while dbg is used like printf. Some of this work is tedious, especially remembering to add dbg_enter and dbg_leave calls to every function entry and exit point. A __FUNC__ macro would make things a little easier. Ideally, what I really want are functions that are called every time a function is entered and exited, such as: void func_enter(const char *name); void func_leave(); This could be done by an implementation class in C++, where func_enter is a constructor and func_leave is a destructor for an object representing function tracing information. Alas, it is C and not C++ that we are discussing. David R Tribble You offer several improvements that have been suggested in the past, in one form or the other. Unfortunately, right now the C standards committee WG14 has only a limited charter to extend the language. I expect the review process to begin sometime in 1995, at which point additions to the language can be again entertained. Keep reading these pages to find out the latest status of C and C++ standardization efforts. -- pjp Dear pjp, Ref: A Safer setjmp in C++, (C Users Journal, January 1993) I quote two sentences from the article: "If control has left the function that called setjmp, longjmp will usually crash" (the second sentence in the article) "This technique isn't foolproof. It can still fail..." (the first sentence under heading "Limitations" towards the end of the article). I skip the story, which can be found in between those two quotations. Not so long ago (perhaps ten years or so), when C was nothing but a character in the alphabet, I sold my first commercial program, which was written in ... BASIC. The main function had a label error. As soon as an error condition was encountered, which the corresponding routine couldn't handle, control returned to error, where all motors were turned off, all files closed....and the stack pointer was set back to zero. The program was bombproof. In the meantime, I have turned over a few million dollars in the software field (with the help of a few colleagues) and you are telling me, that there is no foolproof error handling in C, other than passing the control up the chain of nested function calls. The article mentions Borland "non-standard" methods, but fails to say, whether those methods are foolproof. All I know is that Turbo C 2.0 is knowledgeable about a stack pointer. I don't know about Borland C++. What if you simply remember the stack pointer in main, and restore the stack pointer on an error condition, which cannot be handled by lower-level functions? There are a few questions which are implicitly thrown up by the article: (so I dare to make them explicit here) 1) If the C Standard includes a setjmp and a longjmp, is it true that it is impossible to implement them? Or do compiler manufacturers simply not try hard enough? 2) Is it that the standards committee never paid attention to the question whether their "ideal" was possible to implement? I must say, I did consult your Standand C book which I turn to as the most authoritative book in the matter. (Thanks for a book well written). But it fails to mention the problems raised in the article. I understand that if control is passed to a caller of the function, in which the call to setjmp() appears... there is no help... but just what does the above-quoted "If control has left the function that called setjmp" mean? In any callee of the function in question we should be fine! So let's call setjmp in main and we are fine! Because main has no caller! Hasn't anyone ever implemented the standard? What is all this about? Have you been converted to C++? (The heading of the article seems to suggest that the answer is: use C++.) But then again the suggested solution isn't foolproof? Even in C++. I must admit the article left me 100 percent confused. Can you help me in this? I always wanted to ask you: Where can I find the C Standard? Who sells it? (I can pay with a Visa account number or a cheque drawn on a US bank). I appreciate your journal (I just renewed my subscription for another three years) and I appreciate your book Standard C, but just where do I find the publisher of the C Standard. Let's say, I want to implement a C compiler: now whoever wants to sue me saying that I didn't implement the standard, what book/brochure does he hold against me? Perhaps the author of your article simply fell victim to someone who didn't implement the standard? P.S.: Do you know about anyone who does all his programming in C++? I don't. Not in Europe. Windows is written in C (not C++). Everyone seems to write code in C, but then seems to write articles and publish ads on C++. I work for some 20 companies, but haven't found anyone who uses (uses!) C++. I haven't found anyone who used Prolog (when it was fashionable to use Prolog), nor have I found anyone who used AI (when it was fashionable to do so), nor have I found anyone who used CASE (when it was fashionable to do so). Is selling software nowadays just like selling clothes (keyword "fashion"). What is all this about? Did your advertisers convince you: better follow the C++ trend or else... We are all trying to make products obsolete by the magic of "fashion." Sellers unite! Caveat emptor. Your conversion to C++ seems to be somewhat faked. Who converted you? C++ is a bureaucrat's dream and a programmer's nightmare. And impossible to implement, going by your article. Neither C nor C++ provide for safe error handling. Come on! Tell me which language does and I will stop reading your magazine. Sincerely yours L. Engbert Engbert UB Taunusstrasse 8 D-6384 Schmitten 1 Germany Wow, you seem a little upset. I am answering these letters while on the road, so I can't reread the article you mentioned. I thought it simply warned about the known dangers of undisciplined use of setjmp/longjmp. A disciplined use, on the other hand, forms the solid basis for exception handling and error recovery in oodles of working code out there in the world. I was an active member of X3J11 when it hammered out the specifications of setjmp/longjmp. I have also implemented these functions commercially on at least half a dozen different computer architectures. I assure you they are reasonably thought out and they work. The BASIC error recovery you describe sounds like a yeoman attempt to retain control in the teeth of bad program errors. I and others often do the same thing in the C startup code, particularly for freestanding or embedded applications. It doesn't always rescue you, however. Curdling the stack can lead to too many kinds of craziness to assure that. Still, it's often worth a try. You can order the C Standard from ANSI, 11 West 42nd Street, New York NY 10036, USA. Call +1-212-642-4900 or fax +1-212-302-1286. You will find it rather expensive. (I paid at least $140 a few months ago.) ISO in Geneva also sells the same standard. The address is at the front of my book, The Standard C Library, but I don't have it with me. DIN should also sell it in Germany. Yes, there really are C++ programmers out there. I am currently attending the WG21/X3J16 meeting working on the C++ Standard. Hence, I am surrounded by C++ enthusiasts. Many others in the world are what we Americans call "tire kickers" (as when you inspect a car you think you might buy). But the number of serious C++ programmers is significant and growing. It is an inevitable artifact of journalism that we magazine editors focus more on what's new and developing than what's tried and true. So the mix of articles in this or any other trade magazine is seldom representative of what's actually happening in the world. We're not trying to distort reality, just give our readers what they need to know. -- pjp Dear William, Regarding William Smith, "An Essential String Function Library," CUJ, Jan. 1993: I read your article with much interest in The C Users Journal. I am in the awkward stage of learning C and C++ so I can covert an application from Pascal to C, which is obviously the future, for which I can make good use of your string functions. I am now working with THINK C for the Mac, but I assume that if I can understand your code, I can port it to THINK C, but for this I do need some help, if you would be so kind. For Example: 1. Your function str_insert(Str, Pos,Insrt) is comparable to the Pascal procedure insert(Insrt,Str, pos); {using your notation} but you cast Pos as a Char*, rather than an integer-type value as in Pascal. How does this work? I have not been able to make it function correctly in my test runs in C. Perhaps you could show me an example as your article assumes a higher understanding of C than I have at this time have. More annotating would have helped us C neophytes. 2. You use two functions in your code, main & max that are not include in THINK C libraries. What do these function do and where may I find examples of them? 3. Your function str_ljust is quite a bit different than one a have written called spacepad: /************** SpacePad ************ * routine to left pad a string with * spaces * If str s is no longer * than len, it is shortened to len * It also returns original len of * s, if wanted ************************************/ short SpacePad(char *s,short len) { short i; i = strlen(s); /* start pad/cut routine */ while(i < len) s[i++] = "; /* pad s to len */ s[len] = '\0'; return i; /* returns len of s */ } I must assume that yours handles memory better and is less inclined to cause crashes? True? False? I am enclosing a self addressed envelope for your reply, if you could find time to do so. Sincerely yours, Arne Bystrom 1617 Post Alley Seattle WA 98101 William Smith replies: 1) Pos is a pointer to a location within str. Pos in pointer form is equivalent to: Str + Pos if Pos is an integer index or &str [Pos]. 2) min and max are macros defined in the include file stdlib.h (Microsoft c). #define max(a,b) (((a) > (b)) ? (a) : (b)) #define min(a,b) (((a) < (b)) ? (a) : (b)) 3) Not necessarily true. They are functionally equivalent, and just represent different ways to do the same thing. Dear Dr. Plauger: Elaboration on the pros/cons of using #define's vs. const for string ID's, help contexts, etc., in both C and C++, would be appreciated. Also, more coverage of the Borland app. Frameworks (esp. Turbo Vision for C++) would be welcomed! Thanks for an ongoing fine job. Sincerely, Bryan D. Feldman, President Service Ware International 2376 Arrow Circle Atlanta, GA 30341 Thanks for the feedback. -- pjp Curve Fitting with Extrapolation Lowell Smith Lowell Smith is a retired aerospace engineer (Hercules Aerospace Company). He first programmed in FORTRAN in 1962 on the IBM 1620, then later on the IBM 1130 and various IBM 360/370 models. After getting an IBM PC in 1982, he began using Pascal and C. He has never worked as a programmer, but has programmed many engineering applications over the years. He can be reached via 73377.501@compuserve.com. Introduction Engineering models frequently require functional representations of empirical data that may be defined at arbitrary intervals. A least-squares fitted polynomial provides a simple approach to the problem, but may deviate from expected values between data points, particularly for polynomials of higher degree. Chebyshev polynomials can be used for a fit which very closely approximates the optimal minimax solution--minimizing the maximum deviations. Algorithms for implementing the method are well documented in Numerical Recipes in C (Press et al., 1992). Prince (December 1992) describes the use of this method for fitting analytical functions such as sin(x), cos(x), etc. The use of Chebyshev polynomials requires evaluation of the candidate function at specific values of x. This is no problem for continuous analytical functions, but engineering data to be fit are usually available at only a limited number of points. The algorithm described in Numerical Recipes in C for interpolation/extrapolation using rational functions provides a means for accurately estimating data at the abscissa values required for use of Chebyshev polynomials. It has the added benefit that it performs surprisingly accurate extrapolation for a wide variety of data. The RPFT code described here is based on the algorithms defined in Numerical Recipes in C. It fits both polynomials and rational functions to arbitrary sets of (x, y) data points. The rational function has the form: Click Here for Equation The extrapolation feature permits fitting somewhat beyond the limits of the available data. Operation The program requirements are defined in the help screen shown in Figure 1. The help screen is displayed whenever a command-line error is encountered and when the program is invoked with no command-line arguments. Checking is provided to trap most input errors. Optional keyboard input for calculating test results is permitted at the end of the run. Those data are echoed to the screen via stderr to avoid confusion when stdout is redirected. To illustrate the use of the program with extrapolation beyond the available data, I fit a fifth degree polynomial to the heat of formation data for HCl in water. The input data shown in Listing 1 was taken from Wagman, et al. (1992) with some data omitted. The logarithmic transformation of x values was used because of the extended range of the data. The upper limit for the curve fit was chosen well beyond the included data to illustrate the use of extrapolation. A screen dump for the run is given in Figure 2. The resulting polynomial is plotted in Figure 3 along with a conventional least-squares polynomial. Data points not included in the analysis are shown to verify the effectiveness of the extrapolation. The maximum error is 1.03 at x = 1.5. For a better fit I would use two polynomials, possibly with overlapping ranges, particularly if reasonable extrapolation beyond the lower limit were needed. Rational functions often give a better fit and also can be extrapolated for moderate distances beyond the range over which they are fit. For the second example, I fit a rational function to sin(x) for x between 0 and 3.14. The input data at intervals of 0.2 (except the last point) were provided with 16 significant digits. The option -DIG = 16 was used to get maximum accuracy in the coefficients. Fourth order polynomials were specified for both numerator and denominator (NN = 4 and ND = 4). The results are shown in Figure 4. Note that the error curve reflects the minimax nature of the fit between the lower and upper limits. Also, the rational function extrapolates reasonably well beyond the limits. The error at x = -0.5 is 0.00584 and that at 3.5 is 0.000031. The maximum error is 0.28 for x = -3, which is well outside the limits of the curve fit. When the analysis is redone with limits of -3 and +6 to take advantage of the extrapolation feature of the code, the maximum error is 0.00328 between those limits. Code Structure The code is given in Listing 2. It is normally compiled on a PC in the small model. It will compile and run without error in the large model, but that should never be necessary. It will accept over 1,000 data points using the small model on a machine with 640KB memory. The allocation failure error messages indicate the location of the error in the code if problems are encountered. All program options are specified on the command line as indicated in the help screen shown in Figure 1. main first calls commd to parse the command line and check for obvious errors. The inpt routine is then called to read the (x, y) data from the input file and check for obvious errors such as missing values or negative values with logarithmic transformation. inpt then calculates either the Chebyshev polynomials or the abscissa and function values required for the rational polynomial fit. The diagonal rational function interpolation routine ratint is used for interpolation or extrapolation to determine values between or beyond data points as required. The arrays required for the input data are freed prior to return. If NN is greater than 0, ratlsq is then called for the rational function analysis. The number of abscissa values used is npt, which is NPFAC times the total number of coefficients. Numerical Recipes in C recommends that you #define NPFAC to be 8, and that is used here. In some cases a better fit might be possible with a larger value. The analysis proceeds by iteration with the weighting factors wt[] and ee[] adjusted after each iteration to minimize the maximum error. Functions dsvdcmp for singular value decomposition and dsvbksb for back substitution are used for the solution in each iteration. The estimated maximum absolute error is displayed after each iteration. The coefficient set returned is that for the iteration with the smallest estimated error. If ND is zero, chebpc is called to transform the Chebyshev coefficients generated by inpt to the interval -1 to 1. pcshft then generates the polynomial coefficients. sprintf and sscanf are used to round the coefficients to the specified number of digits (default is 7). For the polynomial fit, a single column of coefficients is listed. For the rational function, a column is listed for both the numerator and the denominator. At the end of the run, optional input for trial x values is provided. This, like all error messages, is directed to stderr to avoid confusion when output is redirected. References Press, W.H., et al. 1992. Numerical Recipes in C, Second Edition, Cambridge, MA: Cambridge University Press. Prince, Tim. December, 1992. "Tuning Up Math Functions," C Users Journal. Lawrence, KS: R&D Publications. Wagman, D.D., et al. 1992. "The NBS Tables of Chemical Thermodynamic Properties: Selected Values for Inorganic and C1 and C2 Organic Substances in SI Units," Journal of Physical and Chemical Reference Data. Vol. 11, Supplement No. 2. National Bureau of Standards, Washington, CD 20234. Figure 1 RPFT help screen USAGE: rpft LL=a UL=b NN=n ND=m [-DIG=n -XLN -YLN] file LL=a a is the lower limit of theregion for fit UL=b b is the upper limit of the region for fit ND=m m>0: rational function, denominator degree m, NN=n numerator degree n. 1<=m<=10 1<=n<=10 m=0: Polynomial degree n. 1<=n<=20 -DIG=n (optional) n = number of significant digits for coefficients on output. Default=7. -XLN (optional) Transform X values to ln(X) -YLN (optional) Transform Y values to ln(Y) file File with input X Y data points, 1 per line All command line parameters except DIG, XLN and YLN are required. They may be in any order, upper or lower case Figure 2 Screen dump of analysis C:\>rpft hcl.dat ll:1 ul=100000 -xln nn=5 nd=0 Polynomial Coefficients -121.4214 -34.27512 10.335 -1.513737 0.1067046 -0.002897266 Enter E to quit or a trial X value: 5 For X=5 Y=-155.4404 Enter E to quit or a trial X value: 1e5 For X=100000 Y=-167.47783 Enter E to quit or a trial X value: e C:\> Figure 3 Fifth degree polynomials Figure 4 Rational function Listing 1 HCL.DAT 1 -121.55 1.5 -132.67 2 -140.96 2.5 -145.48 3 -148.49 4 -152.917 5 -155.774 6 -157.682 8 -160.005 10 -161.318 12 -162.18 15 -163.025 20 -163.845 30 -164.67 50 -165.356 100 -165.925 300 -166.423 500 -166.573 1000 -166.732 /* End of File */ Listing 2 RPFT. C /****************************************************** * RPFT.C -- Curve fitting with optional extrapolation. * Fits either polynomial or rational function. Based * on algoritms defined in Press, et al., "Numerical * Recipes", 2nd ed., Cambridge University Press, 1992 * Developed using the Borland C compiler. * * Lowell Smith * 3368 Crestwood Drive * Salt Lake City, UT 84109-3202 *******************************************************/ #include #include #include #include #include #include #include #define NPFAC 8 #define PI 3.141592653589793 #define PI02 (PI/2.0) #define SIGN(a,b) ((b) > 0.0 ? fabs(a) : -fabs(a)) void ratlsq(double xs[],double fs[],int npt,int mm, int kk, double cof[], double *dev); double *dvector(long nl, long nh, const char *ner); double **dmatrix(long nrl,long nrh,long ncl,long nch, const char *ner); void free_dvector(double *v, long ncl); void free_dmatrix(double **m, long nrl, long ncl); /******************************************************/ void main(int argc, char **argv) { double a,b,*xs,*fs,cof[42],dev=0.,sumd,x,yy; int i,j,nn,nd,ncof,dig,npt=0,xln,yln; char nar[90]; char filenam[80]; void commd(int argc, char**argv, double *a,double *b, int *nn, int *nd, int *dig, int *xln, int *yln, char *filenam); void inpt(int nn,int npt,double a, double b, int xln, int yln, double *xs,double *fs,char filenam[]); void pcshft(double a,double b,double d[],int n); void chebpc(double c[],double d[],int n); commd(argc,argv,&a,&b,&nn,&nd,&dig,&xln,&yln, filenam); if (xln) { a=log(a); b=log(b); } if (nd) /* Fit rational function */ { ncof=nn+nd+1; npt=NPFAC*ncof; xs=dvector(1L,(long)npt,"xs in main"); fs=dvector(1L,(long)npt,"fs in main"); inpt(0, npt, a, b, xln, yln, xs, fs, filenam); ratlsq(xs,fs,npt,nn,nd,&cof[0],&dev); fprintf(stderr,"Est. max error = %.7G\n",dev); free_dvector(xs,1L); free_dvector(fs,1L); } else /* Fit polynomial */ { ncof=nn+5; xs=dvector(0L,(long)ncof,"xs in main"); fs=dvector(0L,(long)ncof,"fs in main"); inpt(ncof, npt, a, b, xln, yln, xs, fs, filenam); chebpc(fs,cof,nn+1); pcshft(a,b,cof,nn+1); free_dvector(xs,0L); free_dvector(fs,0L); } for (i=0;i<=nn+nd;++i) { sprintf(nar,"%.*G ",dig,cof[i]); sscanf(nar,"%lf",&cof[i]); } if (nd) printf("\n Rational function coefficien" "ts\n Numerator Denominator\n\n"); else printf("\n Polynomial\n Coefficients\n\n"); for (i=0;i=0;j--) yy=yy*x+cof[j]; if (nd) { for (sumd=0.,j=nn+nd;j>=nn+1;j--) sum=(sumd+cof[j])*x; yy=yy/(1.0+sumd); } if (yln) yy=exp(yy); if (xln) x=exp(x); fprintf(stderr,"For X=%.8G Y=%.8G\n",x,yy); } } /******************************************************* * COMMD - Parses the command line ******************************************************/ void commd(int argc, char**argv, double *a,double *b, int *nn, int *nd, int *dig, int *xln, int *yln, char *filenam) { struct ffblk ffblk; int i,j,k,l; void help(char *msg); *a=1.11e300; *b=-1.11e300; *nn=-32768; *nd=-32768; *dig=7, *xln=0, *yln=0; if (argc<2) help(""); filenam[0]=0; for (i=1,k=0;i1 (!isdigit(argv[i][l-1]) && argv[i][l-1] !='.')) { fprintf(stderr,"Invalid command line parameter" "%s\n",argv[i]); help(""); } if (!strncmp(argv[i],"LL=",3)) { k=sscanf(&argv[i][3],"%lf",a); if (k<1) help("Invalid value for command line" "parameter LL\n"); continue; } if (!strncmp(argv[i] ,"UL=",3)) { k=sscanf(&argv[i][3],"%lf",b); if (k<1) help("Invalid value for command line" "parameter UL\n"); continue; } if (!strncmp(argv[i],"ND=",3)) { k=sscanf(&argv[i][3],"%d",nd); if (k<1) help("Invalid value for command line" "parameter ND\n"); continue; } if (!strncmp(argv[i],"NN=",3)) { k=sscanf(&argv[i][3],"%d",nn); if (k<1) help("Invalid value for command line" "parameter NN\n"); continue; } if (!strncmp(&argv[i][0],"-DIG=",5)) { k=sscanf(&argv[i][6],"%d",dig); if (k<1) help("Invalid value for optional" "command line parameter DIG\n"); continue; } fprintf(stderr,"Invalid command line parameter" "%s\n",argv[i]); help(""); } else { if (!strncmp(&argv[i][0],"-XLN",4)) { *xln=1; continue; } if (!strncmp(&argv[i][0],"-YLN",4)) { *yln=1; continue; } if (!filenam[0] && argv[i][0] != '-') { strcpy(filenam,argv[i]); if (findfirst(filenam,&ffblk,0)) { fprintf(stderr,"Input file %s doesn't" "exist\n",filenam); help(""); } } else { fprintf(stderr,"Invalid command line parameter" "%s\n",argv[i]); help(""); } } } k=0; if (*a>1.1e300) { fprintf(stderr,"Parameter LL undefined\n"); ++k; } if (*b<-1.1e300) { fprintf(stderr,"Parameter UL undefined\n"); ++k; } if (*nn==-32768) { fprintf(stderr,"Parameter NN undefined\n"); ++k; *nn=5; } if (*nd==-32768) { fprintf(stderr,"Parameter ND undefined\n"); ++k; *nd=5; } if (filenam[0]==0) { fprintf(stderr,"Input file is not defined\n"); ++k; } if (*nn<1 (*nd==0 && *nn>20) (*nd && *nn>10)) { fprintf(stderr,"Invalid value %d for " "command line parameter NN\n",*nn); ++k; } if (*nd<0 *nd>10) { fprintf(stderr,"Invalid value %d for " "command line parameter ND\n",*nd); ++k; } if (*a>*b) { fprintf(stderr,"Lower limit > upper limit\n"); ++k; } if (*dig<1 *dig>20) { fprintf(stderr,"Invalid value %d for " "command line option DIG\n",*dig); ++k; } if (*xln && *a<=0.) {fprintf(stderr,"Lower limit is <=0 with " "logarithmic transform of X values\n"); ++k; } if (k) help(""); } /******************************************************* * HELP - Prints the help screen ******************************************************/ void help(char *msg) { char *hlp[] = { "USAGE: rpft LL=a UL=b NN=n ND=m [-DIG=n -XLN -YLN" "] file",""," LL=a a is the lower limit of the" "region for fit"," UL=b b is the upper limit o" "f the region for fit",""," ND=m m>0: rational" " function, denominator degree m,"," NN=n " "numerator degree n. 1<=m<=10 1<=n<=10"," " " m=0: Polynomial degree n. 1<=n<=20",""," -DI" "G=n (optional) n = number of significant digits" ," for coefficients on output. Defaul" "t=7."," -XLN (optional) Transform X values to" "ln(X)"," -YLN (optional) Transform Y values" "to ln(Y)",""," file File with input X Y data" "points, 1 per line","","All command line paramete" "rs except DIG, XLN and YLN are ","required. They " "may be in any order, upper or lower case." }; int i,n=sizeof(hlp)/sizeof(char *)-1; fprintf(stderr,"%s\n",msg); for (i=0; i=0;i--) sumn=sumn*e+coff[i]; for (sumd=0.0,i=mm+kk;i>=mm+1;i--) sumd=(sumd+coff[i])*e; ee[j]=sumn/(1.0+sumd)-fs[j]; wt[j]=fabs(ee[j]); sum += wt[j]; if (wt[j] > devmax) devmax=wt[j]; } e=sum/npt; if (devmax <= *dev) { for(j=0;janorm) anorm=x; } for (i=n;i>=1;i--) { if (i < n) { if (g) { for (j=l;j<=n;j++) v[j][i]=(a[i][j]/a[i][l])/g; for (j=l;j<=n;j++) { for(s=0.0,k=l;k<=n;k++) s += a[i][k]*v[k][j]; for (k=l;k<=n;k++) v[k][j] += s*v[k][i]; } { for (j=l;j<=n;j++) v[i][j]=v[j][i]=0.0; } v[i][i]=1.0; g=rv1[i]; l=i; } for (i=min (m,n);i>=1;i--) { l=i+1; g=w[i]; for (j=l;j<=n;j++) a[i][j]]=0.0; if (g) { g=1.0/g; for (j=l;j<=n;j++) { for (s=0.0,k=l;k<=m;k++) s += a[k][i]*a[k][j]; f=(s/a[i][i])*g; for (k=i;k<=m;k++) a[k][j] += f*a[k][i]; } for (j=i;j<=m;j++) a[j][i] *= g; } else for (j=i;j<=m;j++) a[j][i]=0.0; ++a[1][i]; } for (k=n;k>=1;k--) { for (its=1;its<=30;its++) { flag=1; for (l=k;l>=1;l--) { nm=l-1; if ((fabs(rv1[l])+anorm) == anorm) { flag=0; break; } if ((fabs(w[nm])+anorm) == anorm) break; } if (flag) { c=0.0; s=1.0; for (i=l;i<=k;i++) { f=s*rv1[i]; rv1[i]=c*rv1[i]; if ((fabs(f)+anorm) == anorm) break; g=w[i]; h=dpythag(f,g); w[i]=h; h=1.0/h; c=g*h; s = -f*h; for (j=1;i<=m;j++) { y=a[j] [nm]; z=a[j] [i]; a[j] [nm] =y*c+z*s; a[j][i]=z*c-y*s; } } } z=w[k]; if(l == k) { if (z < 0.0) { w[k] =-z; for (j=1;j<=n;j++) v[j][k] = -v[j][k]; } break; } if (its == 30) { fprintf(stderr,"No convergence in 30" "dsvdcmp iterations\n"); exit(1); } x=w[l]; nm=k-1; y=w[nm]; g=rv1[nm]; h=rv1[k]; f=((y-z)*(y+z)+(g-h)*(g+h)) / (2.0*h*y); g=dpythag(f, 1.0); f=((x-z)*(x+z)+h*((y/(f+SIGN(g,f)))-h))/x; c=s=1.0; for (j=l;j<=nm;j++) { i=j+1; g=rv1[i]; y=w[i]; h=s*g; g=c*g; z=dpythag(f,h); rv1[j]=z; c=f/z; s=h/z; f=x*c+g*s; g=g*c-x*s; h=y*s; y *= c; for (jj=1;jj<=n;jj++) { x=v[jj][j]; z=v[jj][i]; v[jj][j]=x*c+z*s; v[jj][i]=z*c-x*s; } z=dpythag(f,h); w[j]=z; if (z) { z=1.0/z; c=f*z; s=h*z; } f=c*g+s*y; x=c*y-s*g; for (jj=1;jj<=m;jj++) { y=a[jj][j]; z=a[jj][i]; a[jj][j]=y*c+z*s; a[jj][i]=z*c-y*s; } } rv1[l]=0.0; rv1[k]=f; w[k]=x; } } free_dvector (rv1,1L); } /******************************************************* * DPYTHAG - Computes sqrt(a*a + b*b) without * destructive underflow or overflow. ******************************************************/ double dpythag(double a, double b) { double aba,abb,xx,yy; aba=fabs(a); abb=fabs(b); xx=abb/aba; yy=aba/abb; if (aba > abb) return aba*sqrt(1.0+xx*xx); else return (abb == 0.0 ? 0.0 : abb*sqrt(1.0+yy*yy)); } /******************************************************* * DSVBKSB - Solves A.X = B for a vector X, where A is * specified by the arrays u[1..m][1..n], w[1..n], * v[1..n][1..n] as returned by dsvdcmp. *******************************************************/ void dsvbksb(double **u, double w[], double **v, int m, int n, double b[], double x[]) { int jj,j,i; double s,*tmp; tmp=dvector(1L,(long)n,"tmp in dsvbksb"); for (j=1;j<=n;j++) { s=0.0; if (w[j]) { for (i=1;i<=m;i++) s += u[i][j]*b[i]; s /= w[j];} tmp [j]=s; } for (j=1;j<=n;j++) { s=0.0; for (jj=1;jj<=n;jj++) s += v[j][jj]*tmp[jj]; x[j]=s; } free_dvector(tmp, 1L); } /******************************************************* * CHEBPC - Transformation of Chebyshev coefficients * generated in INPT to the interval -1 to 1 ******************************************************/ void chebpc(double c[],double d[],int n) { int k,j; double sv,*dd; dd=dvector(0L,(long)n-1,"dd in chebpc"); for (j=0;j=1;j--) { for (k=n-j;k>=1;k--) { sv=d[k]; d[k]=2.0*d[k-1]-dd[k]; dd[k]=sv; } sv=d[0]; d[0] = -dd[0]+c[j]; dd[0]=sv; } for (j=n-1;j>=1;j--) d[j]=d[j-1]-dd[j]; d[0] = -dd[0] +0.5*c [0]; free_dvector(dd,0L); } /************************************************* * PCSHFT - Generate the polynomial coefficients * using the Chebyshev coefficients from CHEBPC *************************************************/ void pcshft(double a,double b,double d[],int n) { int k,j; double fac,cnst; cnst=2.0/(b-a); fac=cnst; for (j=1;j=j;k--) d[k] -= cnst*d[k+1]; } /****************************************************** * RATINT - Diagonal rational function interpolation in * the arrays xa[1..n] and ya[1..n]. ******************************************************/ void ratint(double xa[], double ya[], double *c, double *d, int n, double x, double *y) { int m,i,ns=1; double w,t,hh,h,dd; hh=fabs (x-xa [1] ); for (i=1;i<=n;i++) { h=fabs (x-xa[i]); if (h == 0.0) { *y=ya[i]; return; } else if (h < hh) { ns=i; hh=h; } c[i]=ya[i]; d[i] =ya[i]+1.e-50; } *y=ya[ns--]; for (m=1;m #include #include "funlib.h" void fun_unit_step( const long int wave_ary_len, double *waveform) { auto unsigned long int wave_index; for ( wave_index = 1; wave_index <= wave_ary_len; wave_index++) { waveform[wave_index] = 1.0 / (double)wave_ary_len; } return; } void fun_noise(const long int wave_ary_len, const double noise_width, double *waveform) { auto unsigned long int wave_index; for (wave_index = 0; wave_index < wave_ary_len; wave_index++) { waveform[wave_index] += noise_width * ((double)rand() / (double)RAND_MAX); } return; } void fun_step_real(const long int wave_ary_len, double *waveform) { auto unsigned long int i, temp_len = wave_ary_len / 2; fun_gaussian_density(temp_len, waveform); for (i = 0; i < wave_ary_len; i++) { waveform[i] = 1.0 - waveform[i]; } for (i = 1; i < temp_len; i++) { waveform[i] = waveform[i + 1 + (temp_len / 2)]; } for (i = temp_len / 2; i < wave_ary_len; i++) { waveform[i] = .999; } waveform[0] = 0.0; return; } void fun_gaussian_density(const long int wave_ary_len, double *waveform) { auto unsigned long int i; auto double x, mean, std_dev; for (i = 0; i < wave_ary_len; i++) { mean = (double)wave_ary_len / 2.0; std_dev = (0.7 * wave_ary_len) / 6.0; x = (double)i; waveform[i] = exp(- pow((x - mean), 2.0) / (2.0 * pow(std_dev, 2.0))); } waveform[0] = 0.0; return; } void fun_unit_gauss_dens (const long int wave_ary_len, double *waveform) { auto double area = 0.0; auto unsigned long int wave_index; fun_gaussian_density(wave_ary_len, waveform); for (wave_index = 0; wave_index < wave_ary_len; wave_index++) { area += waveform[wave_index]; } for (wave_index = 0; wave_index < wave_ary_len; wave_index++) { waveform[wave_index] /= area; } return; } /* End of File */ Listing 4 Header for deconvolution library in Listing 3 /* funlib.h */ #ifndef FUNLIB_LOADED #define FUNLIB_LOADED void fun_unit_step( const long int wave_ary_len, double *waveform); void fun_noise(const long int wave_ary_len, const double noise_width, double *waveform); void fun_step_real(const long int wave_ary_len, double *waveform); void fun_gaussian_density(const long int wave_ary_len, double *waveform); void fun_unit_gauss_dens(const long int wave_ary_len, double *waveform); #endif /* End of File */ Natural Language Expansions for Tense and Number Russell Suereth Russell Suereth has been consulting for over 12 years in the New York City and Boston areas. He started designing and coding systems on IBM mainframes and now also builds PC software systems. You can write to Russell at 84 Old Denville Rd, Boonton, NJ 07005, or call him at (201) 334-0051. This article expands the natural language processor presented in "A Natural Language Processor" (CUJ, April 1993) to include tense and number. Tense and number help determine the grammatical usage of auxiliaries and verbs, and derive meaning from the sentence. Tense indicates the time of the action or state: past, present, or future. Number indicates how many: one or more than one. These simple meanings help identify similar information between sentences. The processor uses similar information in sentences to generate an appropriate response. I added and expanded several processes to implement tense and number. This natural language processor can now accept input sentences with auxiliary combinations, identify tense and number, and use meaning to generate grammatical responses. These expansions enable the natural language processor to process tense and number, and to respond more correctly. Extracting Auxiliary and Verb Information The processor uses auxiliary and verb information from the dictionary to help identify tense and number. Auxiliaries, words such as is, have, and would, can be combined to create other auxiliaries such as would have and would have been. Underlying structures identify auxiliary combinations in the input sentence. Underlying structures define the kinds of input sentences that can be processed. The original processor had two underlying structures that defined two kinds of input sentences. In this article, I expand the underlying structures to accept auxiliary combinations. For instance, the structure NAME-AUX-VERB-PREP-DET-NOUN accepted Jim could run in the race. I expanded the structure to NAME-AUX-AUX-VERB-PREP-DET-NOUN and NAME-AUX-AUX-AUX-VERB-PREP-DET-NOUN. These structures accept Jim could be running in the race and Jim could have been running in the race. Listing 1 and Listing 2 show only the expanded code for the original processor. This expanded code can be a model for processing tense and number, or added to the original processor. You can get the complete processor code, including the expansions for tense and number from the various sources for CUJ online source code (see page 6). check_underlying contains the expanded structures. check_type matches the underlying structure to the input sentence. If the input sentence has the underlying structure, check_underlying concatenates the auxiliary words to the auxiliaries array. Then get_aux is called to find the auxiliary in the dictionary. The dictionary contains the auxiliary tense, number, meaning, and usage. get_aux reads each dictionary record and calls match_aux to match the auxiliary with the dictionary word. If the match is successful, match_aux extracts the tense, number, meaning, and usage for later use. The natural language processor uses this information to match the input sentence auxiliary with the verb, and to retrieve the correct verb when generating a response. The existence of a matching, underlying structure determines the processor's success in understanding an input sentence. The coded, underlying structures could be expanded further with code for phrase structures such as AUX, AUX-AUX, and AUX-AUX-AUX. Such an expansion would reduce the number of coded, underlying structures but would accept more kinds of input sentences. The processor also extracts verb and pronoun information from the dictionary to later help determine that an input sentence is grammatical, and retrieve the correct verb for a generated response. match_record extracts the verb and pronoun information according to the word type. That is, for the word type VERB, the processor extracts the verb root, usage, tense, and number. For the word type PRON, it extracts only the pronoun number. The dictionary in this expanded processor has new words and information for processing tense and number, and a new layout for the new information. Listing 3 shows the new dictionary. Figure 2 shows the dictionary layout. Determining Tense from Auxiliary and Verb The processor determines tense primarily by matching auxiliary tenses with verb tenses. For example, in the sentence Jim had run in the race, the processor matches the tenses of the auxiliary had with the tenses of the verb run. The auxiliary had is defined in the dictionary as past tense. The verb run is defined as past, present, and future. The past tense matches successfully and so the sentence is past tense. If no match is successful, then the auxiliary and verb don't agree and the sentence is in error. check_aux_verb matches the auxiliary with the verb. The routine first processes sentences that have an auxiliary. If the auxiliary and verb usage don't match, then the sentence tense and usage are unknown. If the auxiliary and verb usage match, then each auxiliary tense is matched with each verb tense. If the number of successful tense matches is one, then the matched tense and usage are assigned to the sentence tense and usage. The usage helps identify the correct verb that can be used with the auxiliary. Some kinds of verbs can't be used with a specific auxiliary. A correct verb is in was running and an incorrect verb is in was run. The verb run has a ROOT usage, was and running have an ING usage. The ING usage identifies a verb that ends with ing, NOAUX identifies a verb with no auxiliary, and ROOT identifies the main form of the verb. If the auxiliary and verb usage match, then that verb may be used with the auxiliary. Some auxiliary and verb combinations have more than one tense match. Consider, for example, the sentence Jim can run in the race. The auxiliary can is in the dictionary as present and future tense. The verb run is in the dictionary as past, present, and future tense. This auxiliary and verb combination may be present or future tense. More than one tense match causes the tense to be unclear. In this case, you must use an alternative method to determine tense. Determining Tense from a Previous Sentence Input sentence context determines tense when auxiliary and verb tenses have more than one match. The processor determines sentence context by analyzing information in previous sentences. Look at the sentences Jim will run in the finals, and Jim can run in the first lane. The second sentence tense is unclear when only that sentence is analyzed. The auxiliary can is in the dictionary as present and future tense, and run is in the dictionary as past, present, and future. The second sentence may be present or future tense. But when the tense of the first sentence is also analyzed, then the second sentence tense can be determined and becomes future tense. However, a previous sentence tense may be irrelevant or forgotten if that previous sentence occurred long ago. For example, the speaker says Jim will run in the finals and fifteen sentences later says Jim can run in the first lane. The listener may not be sure first lane refers to the finals. The sentence about the finals may even be forgotten. When no recent sentence indicates the tense, the listener assumes the tense is present. check_aux_verb looks at the previous three sentences when the number of successful tense matches is greater than one. The current sentence's subject, action, and possible tenses are matched with the previous three sentences' subject, action, and tense. If a match is successful, then the matching sentence tense is assigned to the current sentence tense. If no match is successful, then the current sentence tense is present. Determining Tense with No Auxiliary Some sentences don't have an auxiliary. In these sentences, the processor uses the verb to determine the input sentence tense. When the verb is past tense, then the sentence is past tense. The sentence Jim ran in the race is past tense. When the verb is not past tense, then the sentence tense is unclear. The sentence Jim runs in the race may be present or future tense. The previous three sentences are used to determine the tense when the tense is unclear. check_aux_verb also processes tense when the input sentence has no auxiliary. The current sentence's subject, action, and possible verb tenses are matched with the previous three sentence's subject, action, and tense. If a match is successful, then the previous sentence tense is assigned to the current sentence tense. If no match is successful, then the current sentence tense is present. Determining Number The sentence subject determines number. The number can be singular or plural depending on whether the subject refers to one or more than one. The sentence He runs in the race shows a singular subject, They run in the race shows a plural subject. The auxiliary, verb, and subject number in a grammatical sentence must be all singular or all plural. Many verb forms are both singular and plural. For example, running is singular in He is running in the race, and plural in They are running in the race. Many auxiliaries also can be singular and plural. For example, could be is singular in He could be running in the race, and plural in They could be running in the race. But some auxiliaries and verbs can be only singular or plural. check_number matches the auxiliary, verb, and subject number. If the match is successful, the matched number is assigned to the sentence number. If the match is unsuccessful, the sentence number is unknown. Number is used to identify the correct verb in a generated response. Using number enables the correct verb to be extracted from the dictionary. The sentence Jim runs in the race has a correct verb because Jim and runs are singular. A subject and verb that agree help make the response grammatical and effective. Meaning and the Response The original processor generated only two kinds of responses. It returned a simple OK when given a statement, and someone's location when given a question. The previous sentence that matched the same subject and action words gave the location for the response. Previous sentences were matched by the same words to identify similar information. But similar information is often determined by meaning rather than by the same words. The expanded response process uses tense and the auxiliary's meaning to help identify similar information, making the natural language processor sound more human. Figure 1 shows an example. The processor responded with correct information because it used tense and auxiliary meaning when it matched the sentences. The tense matched because the two sentences refer to past tense. The auxiliary meaning matched because did and had been mean PARTICULAR_POINT_OF_TIME. The processor currently assigns auxiliary meaning only when the auxiliary exists. Auxiliary meaning helps match similar information between sentences. A sentence without an auxiliary meaning can't be properly matched to another sentence. A further processor expansion can derive an auxiliary meaning from the sentence when no auxiliary exists. The derived auxiliary meaning will allow a sentence without an auxiliary to be properly matched. Matching Sentences for the Response When the processor generates a response, it searches in previous sentences for similar information. The processor uses the most accurate information in the response. There are four separate matches the processor uses to find information for the response. The first match has the highest probability that the information is accurate, the last match has the lowest probability that the information is accurate. People in conversation respond in a similar manner. A person may not have enough knowledge for a correct response. But that person may create an alternative response to show knowledge about the information. The first match criterion is subject, action, tense, and auxiliary meaning. Subject and action are always in the match criteria for the response. This ensures that all matched sentences have the same subject and action. The sentences Jim had been running in the race and Where did Jim run? match because had been and did have the same tense and meaning. make_response matches information in the current sentence with information in previous sentences. In the first match, the subjects, actions, tenses, and aux_meaning arrays match the current sentence to a previous sentence. When a match is successful, make_answer generates a response with information from the previous, matched sentence (see Figure 1). The second match criterion is subject, action, and tense. The subjects, actions, and tenses arrays match the current sentence to a previous sentence. Here, the sentences Jim had been running in the race and Where would Jim run? match because had been and would have the same tense. The third match criterion is subject, action, and auxiliary meaning. The subjects, actions, and aux_meaning arrays match the current sentence to a previous sentence. The sentences Jim should have run in the race and Where should Jim run? match because should and should have have the same meaning. The tense is different, and so the response may not give the correct information. Because of this, the processor prefaces the response with I'm not sure, but. The fourth match criterion is the same as in the original processor. Only subject and action are matched. The subjects and actions arrays match the current sentence to a previous sentence. The sentences Jim is running in the race and Where will Jim run? match because they have the same subject and action. But the tense and auxiliary meaning don't match so the response may be incorrect. Because of this, the processor prefaces the response with I'm not sure, but. When the four matches are unsuccessful, the processor can't find the similar information. Then make_response moves the statement I don't know to the response. Extracting the Correct Verb The correct verb helps ensure a grammatical response. make_answer generates a response by moving appropriate words to the response variable. The appropriate subject and auxiliary are first moved to the response. Then get_verb is called to extract the correct verb and move it to the response. get_verb reads each record in the dictionary and calls match_verb to find the correct verb. match_verb matches the passed tense, number, and usage with the tense, number, and usage of the current dictionary record. When a match is successful, the correct verb is extracted from the dictionary and moved to the response. After the verb is extracted, the place where the action occurred is also moved to the response. Processing the Pronoun The processor compares several word types to determine that the sentence agrees in tense and number. Another word type that must agree in number is the pronoun, words such as he, she, and they. A pronoun can replace the name in the sentence, and be the sentence subject. Pronouns must be defined in the underlying structures, be identified as singular or plural, and be used properly in a response. The structure PRON-AUX-VERB-PREP-DET-NOUN (Listing 1) is for a statement, and WH-AUX-PRON-VERB (Listing 1) is for a question. These structures allow the input sentences He is running in the race and Where did he run? Number is extracted from the dictionary and identifies that the pronoun is singular or plural. match_record extracts the pronoun number and assigns it to the subject number. Subject number is used to determine number agreement and to extract the correct verb. Pronouns in a generated response must have their first letter changed to upper or lower case. If the pronoun is the first word in the response, then it must be uppercase. If the pronoun is not the first word, then it must be lower case. make_answer has the code for the pronoun in a response. The subjects_type array has an entry for each sentence. Each entry identifies that the subject is a name or a pronoun. If the response's subject is a pronoun, then the subject's first letter is changed. check_subject assigns values to the subjects_type array. Pronoun letter change helps the processor use the pronoun properly in a response. Conclusion Several processes are required to process tense and number. The processes described in this article extract auxiliary and verb information; determine tense from the auxiliary and verb, a previous sentence, or with no auxiliary; determine number; match sentences for the response; extract the correct verb; and process the pronoun. Each of these processes is an expansion of the original natural language processor. Further expansions to the processor can process time features. These expansions identify time words such as last week, match time meaning with auxiliary meaning, generate a response from time meaning, and generate a response to explain time and number errors. The processes described in this article expand the natural language processor to include tense and number. Tense and number are used to extract the correct verb for a grammatical response. Tense and the auxiliary are used to match similar information between sentences. This match enables the processor to generate a response based on word meaning. These expansions provide the processor with more accurate responses. (C) Copyright 1993 Russell Suereth. Bibliography Liles, Bruce L. 1971. An Introductory Transformational Grammar. Englewood Cliffs: Prentice-Hall. Quirk, Randolph, and Sidney Greenbaum. 1973. A Concise Grammar of Contemporary English. San Diego: Harcourt Brace Jovanovich. Suereth, Russell. April 1993. "A Natural Language Processor." The C Users Journal. Lawrence, KS: R&D Publications. Figure 1 A processor session with tense and number Sentence: Jim should have run in the race Response: OK Sentence: Jim had been running in the race Response: OK Sentence: Jim is running in the race Response: OK Sentence: Where did Jim run Response: Jim had been running in the race Sentence: Where would Jim run Response: Jim had been running in the race Sentence: Where should Jim run Response: I'm not sure, but Jim should have run in the race Sentence: Where will Jim run Response: I'm not sure, but Jim is running in the race Figure 2 Dictionary Layout field columns ----- ------- word 1-24 word type 25-28 usage 30 tense 31-34 number 42-43 auxiliary meaning 45-47 verb root 47-60 Listing 1 The expansion code for tense and number /* Copyright (c) 1993 Russell Suereth */ #include "natural.h" void check_aux_verb(void); void check_number(void); void make_response(void); void make_answer(int); void get_verb(char, char, char); int match_verb(char, char, char); char response[200]; unsigned char verb_tense[5]; unsigned char verb_number[5]; unsigned char verb_usage; unsigned char aux_tense[5]; unsigned char aux_number[5]; unsigned char aux_usage; unsigned char subject_number; unsigned char tenses[20]; unsigned char numbers[20]; unsigned char usages[20]; unsigned char subjects_type[20]; unsigned char aux_meaning[20] [5]; char auxiliaries[20][25]; /*****************************************************/ /* Compare the passed word with the word in the */ /* current dictionary record. If they are the same, */ /* then extract the type (NOUN, VERB, etc.). If the */ /* type is PRON, then extract pronoun information. */ /* If the type is VERB, then extract verb */ /* information. */ /*****************************************************/ int match_record(char *pass_word, int types) { int i, j; char *root; char *dic_word; dic_word = extract_word(); /* Check if passed word equals dictionary word */ if (strcmpi(pass_word, dic_word) != 0) return(1); /* Word found, get the type */ for (i=24,j=0; i<28; i++) { if (isspace(dic_record[i])) break; type_array [word_ct] [types] [j++] = dic_record [i]; } /* Trim the type */ type_array[word_ct] [types][j] = '\0'; if (strcmp(type_array[word_ct][types], "PRON") == 0) subject_number = dic_record[41]; if (strcmp(type_array[word_ct][types], "VERB") == 0) { root = extract_root(); strcpy(root_array[word_ct], root); verb_usage = dic_record[29]; for (i=30,j=0; i<34; i++,j++) { if (isspace(dic_record[i])) break; verb_tense[j] = dic_record[i]; } verb_tense[j] = '\0'; for (i=41,j=0; i<43; i++,j++) { if (isspace(dic_record[i])) break; verb_number[j] = dic_record[i]; } verb_number[j] = '\0'; } return(0); } /*****************************************************/ /* Determine if the input sentence contains a known, */ /* underlying structure. If it does, then assign the */ /* correct types and phrases for the words. */ /*****************************************************/ int check_underlying() { int i = 0; /* Structure WH-AUX-PRON-VERB */ if ( (check_type("WH", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("PRON", i+2) == 0) && (check_type("VERB", i+3) == 0) ) { strcpy(prime_types[i], "WH"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "PRON"); strcpy(prime_types[i+3], "VERB"); strcpy(phrases[i], "WHQUESTION"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "NOUNPHRASE"); strcpy(phrases[i+3], "VERBPHRASE"); strcpy(auxiliaries[sentence], word_array[i+1]); get_aux(); return(0); } /* Structure PRON-AUX-VERB-PREP-DET-NOUN */ if ( (check_type("PRON", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("VERB", i+2) == 0) && (check_type("PREP", i+3) == 0) && (check_type("DET", i+4) == 0) && (check_type("NOUN", i+5) == 0) ) { strcpy(prime_types[i], "PRON"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "VERB"); strcpy(prime_types[i+3], "PREP"); strcpy(prime_types[i+4], "DET"); strcpy(prime_types[i+5], "NOUN"); strcpy(phrases[i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "VERBPHRASE"); strcpy(phrases[i+3], "PREPPHRASE"); strcpy(phrases[i+4], "PREPPHRASE"); strcpy(phrases[i+5], "PREPPHRASE"); strcpy(auxiliaries[sentence], word_array[i+1]); get_aux(); return(0); } /* Structure WH-AUX-NAME-VERB */ if ( (check_type("WH", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("NAME", i+2) == 0) && (check_type("VERB", i+3) == 0) ) { strcpy(prime_types[i], "WH"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "NAME"); strcpy(prime_types[i+3], "VERB"); strcpy(phrases[i], "WHQUESTION"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "NOUNPHRASE"); strcpy(phrases[i+3], "VERBPHRASE"); strcpy(auxiliaries[sentence], word_array[i+1]); get_aux(); return (0); } /* Structure NAME-AUX-AUX-AUX-VERB-PREP-DET-NOUN */ if ( (check_type("NAME", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("AUX", i+2) == 0) && (check_type("AUX", i+3) == 0) && (check_type("VERB", i+4) == 0) && (check_type("PREP", i+5) == 0) && (check_type("DET", i+6) == 0) && (check_type("NOUN", i+7) == 0) ) { strcpy(prime_types[i], "NAME"); strcpy(prime_types[i+l], "AUX"); strcpy(prime_types[i+2], "AUX"); strcpy(prime_types[i+3], "AUX"); strcpy(prime_types[i+4], "VERB"); strcpy(prime_types[i+5], "PREP"); strcpy(prime_types[i+6], "DET"); strcpy(prime_types[i+7], "NOUN"); strcpy(phrases [i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "VERBPHRASE"); strcpy(phrases[i+3], "VERBPHRASE"); strcpy(phrases[i+4], "VERBPHRASE"); strcpy(phrases[i+5], "PREPPHRASE"); strcpy(phrases[i+6], "PREPPHRASE"); strcpy(phrases[i+7], "PREPPHRASE"); strcpy(auxiliaries[sentence], word_array[i+1]); strcat(auxiliaries[sentence], " "); strcat(auxiliaries[sentence], word_array[i+2]); strcat(auxiliaries[sentence], " "); strcat(auxiliaries[sentence], word_array[i+3]); get_aux(); return(0); } /* Structure NAME-AUX-AUX-VERB-PREP-DET-NOUN */ if ( (check_type("NAME", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("AUX", i+2) == 0) && (check_type("VERB", i+3) == 0) && (check_type("PREP", i+4) == 0) && (check_type("DET", i+5) == 0) && (check_type("NOUN", i+6) == 0) ) { strcpy(prime_types[i], "NAME"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "AUX"); strcpy(prime_types[i+3], "VERB"); strcpy(prime_types[i+4], "PREP"); strcpy(prime_types[i+5], "DET"); strcpy(prime_types[i+6], "NOUN"); strcpy(phrases[i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "VERBPHRASE"); strcpy(phrases[i+3], "VERBPHRASE"); strcpy(phrases[i+4], "PREPPHRASE"); strcpy(phrases[i+5], "PREPPHRASE"); strcpy(phrases[i+6], "PREPPHRASE"); strcpy(auxiliaries[sentence], word_array[i+1]); strcat(auxiliaries[sentence], " "); strcat(auxiliaries[sentence], word_array[i+2]); get_aux(); return(0); } /* Structure NAME-AUX-VERB-PREP-DET-NOUN */ if ( (check_type("NAME", i) == 0) && (check_type("AUX", i+1) == 0) && (check_type("VERB", i+2) == 0) && (check_type("PREP", i+3) == 0) && (check_type("DET", i+4) == 0) && (check_type("NOUN", i+5) == 0) ) { strcpy(prime_types[i], "NAME"); strcpy(prime_types[i+1], "AUX"); strcpy(prime_types[i+2], "VERB"); strcpy(prime_types[i+3], "PREP"); strcpy(prima_types[i+4], "DET"); strcpy(prime_types[i+5], "NOUN"); strcpy(phrases[i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "VERBPHRASE"); strcpy(phrases[i+3], "PREPPHRASE"); strcpy(phrases[i+4], "PREPPHRASE"); strcpy(phrases[i+5], "PREPPHRASE"); strcpy(auxiliaries[sentence], word_array [i +1]); get_aux (); return(0); } /* Structure NAME-VERB-PREP-DET-NOUN */ if ( (check_type("NAME", i) == 0) && (check_type("VERB", i+1) == 0) && (check_type("PREP", i+2) == 0) && (check_type("DET", i+3) == 0) && (check_type("NOUN", i+4) == 0) ) { strcpy(prime_types[i], "NAME"); strcpy(prime_types[i+1], "VERB"); strcpy(prime_types[i+2], "PREP"); strcpy(prime_types[i+3], "DET"); strcpy(prime_types[i+4], "NOUN"); strcpy(phrases[i], "NOUNPHRASE"); strcpy(phrases[i+1], "VERBPHRASE"); strcpy(phrases[i+2], "PREPPHRASE"); strcpy(phrases[i+3], "PREPPHRASE"); strcpy(phrases[i+4], "PREPPHRASE"); return (0); } return(1); } /*****************************************************/ /* Compare the passed type with all the types for */ /* this word in the type_array. If the type is */ /* found, then return 0. The pass_number parameter */ /* identifies the word in the input sentence. */ /*****************************************************/ int check_type(char *pass_type, int pass_number) { int i; for (i=0; type_array[pass_number][i][0]; i++) { if (strcmp(type_array[pass_number][i], pass_type) == 0) /* Passed type is found in array */ return (0); } /* Passed type is not found in array */ return(1); } /*****************************************************/ /* If the correct type is "NAME" or "PRON" then the */ /* word refers to a subject so copy the word to the */ /* subjects array. */ /*****************************************************/ void check_subject() { int i; for (i=0; i 0) { if (aux_usage != verb_usage) { tenses[sentence] = UNKNOWN; usages[sentence] = UNKNOWN; return; } for (i=0,j=0,matches=0; aux_tense[i]; i++) { if ((result = strchr(verb_tense,aux_tense[i])) != NULL) { temp_tenses[j++] = *result; matches++; } } temp_tenses[j] = '\0'; if (matches == 0) { tenses[sentence] = UNKNOWN; usages[sentence] = UNKNOWN; return; } usages[sentence] = aux_usage; if (matches == 1) { tenses[sentence] = temp_tenses[0]; return; } for (i=sentence-1; i>=0 && i>=sentence-3; i--) { if ((strcmpi(subjects[i], subjects[sentence]) == 0) && (strcmpi(actions[i], actions[sentence]) == 0) && (strchr(temp_tenses, tenses[i]) != NULL) && (strlen(places[i]) > 0)) { tenses[sentence] = tenses[i]; return; } } tenses[sentence] = PRESENT; return; } /**************************************************/ /* No auxiliary in sentence */ /**************************************************/ usages[sentence] = verb_usage; if (strchr(verb_tense, PAST) != NULL) { tenses[sentence] = PAST; return; } /**************************************************/ /* No auxiliary, verb tense is present or future */ /**************************************************/ for (i=sentence-1; i>=0 && i>=sentence-3; i--) { if ((strcmpi(subjects[i], subjects[sentence]) == 0) && (strcmpi(actions[i], actions[sentence]) == 0) && (strchr(verb_tense, tenses[i]) != NULL) && (strlen(places[i]) > 0)) { tenses[sentence] = tenses[i]; return; } } tenses[sentence] = PRESENT; return; } /*****************************************************/ /* Match the subject, verb, and auxiliary number. */ /* If the match is successful, then the sentence */ /* number is the matched number. */ /*****************************************************/ void check_number() { if (strchr(verb_number, subject_number) == NULL) { numbers[sentence] = UNKNOWN; return; } if ((strten(auxiliaries[sentence]) > 0) && (strchr(aux_number, subject_number) == NULL)) { numbers[sentence] = UNKNOWN; return; } numbers[sentence] = subject_number; return; } /*****************************************************/ /* Read the dictionary to extract the auxiliary */ /* information. */ /*****************************************************/ void get_aux() { rewind(infile); fgets(dic_record, 80, infile); while (! feof(infile)) { if (match_aux() == 0) return; fgets(dic_record, 80, infile); } return; } /*****************************************************/ /* If the sentence auxiliary matches the word in the */ /* current dictionary record, then extract the */ /* auxiliary information from the dictionary. */ /*****************************************************/ int match_aux() { int i,j; char *dic_word; dic_word = extract_word(); if (strcmpi(auxiliaries[sentence], dic_word) != 0) return (1); aux_usage = dic_record[29]; for (i=30,j=0; i<34; i++,j++) { if (isspace(dic_record[i])) break; aux_tense[j] = dic_record[i]; } /* Trim the tense */ aux_tense[j] = '\0'; for (i=41,j=0; i<43; i++,j++) { if (isspace(dic_record[i])) break; aux_number[j] = dic_record[i]; } /* Trim the number*/ aux_number[j] = '\0'; for (i=44,j=0; i<47; i++,j++) { if (isspace(dic_record[i])) break; aux_meaning [sentence] [j] = dicrecord [i]; } return(0); } /*****************************************************/ /* Generate a response with information from a */ /* matching, previous sentence. */ /*****************************************************/ void make_response() { int i; /***************************************************/ /* Input sentence is not asking for information. */ /***************************************************/ if (strcmpi(word_array[0], "where") != 0) { strcpy(response, "OK"); return; } /***************************************************/ /* Match subject, action, tense, and meaning. */ /***************************************************/ for (i=sentence-1; i>=0; i--) { if ((strcmpi(subjects[i],subjects[sentence])==0) && (strcmpi(actions[i], actions[sentence]) ==0) && (strlen(places[i]) > 0) && (tenses[i] == tenses[sentence]) && (strpbrk(aux_meaning[i],aux_meaning[sentence]) != NULL)) { make_answer(i); return; } } /***************************************************/ /* Match subject, action, and tense. */ /***************************************************/ for (i=sentence-1; i>=0; i--) { if ((strcmpi(subjects[i],subjects[sentence])==0) && (strcmpi(actions[i], actions[sentence]) ==0) && (strlen(places[i]) > 0) && (tenses[i] == tenses[sentence])) { make_answer (i); return; } } /***************************************************/ /*Match subject, action, and meaning. */ /***************************************************/ for (i=sentence-1; i>=0; i--) { if ((strcmpi(subjects[i],subjects[sentence])==0) && (strcmpi(actions[i], actions[sentence]) ==0) && (strlen(places[i]) > 0) && (strpbrk(aux_meaning[i],aux_meaning[sentence]) != NULL)) { strcpy(response, 'I'm not sure, but "); make_answer(i); return; } } /***************************************************/ /* Match subject and action. */ /***************************************************/ for (i=sentence-1; i>=0; i--) { if ((strcmpi(subjects[i],subjects[sentence])==0) && (strcmpi(actions[i], actions[sentence]) ==0) && (strlen(places[i]) > 0)) { strcpy(response, 'I'm not sure, but "); make_answer(i); return; } } strcpy(response, "I don't know"); return; } /*****************************************************/ /* Move information from a previous sentence to the */ /* response. */ /*****************************************************/ void make_answer(int prev_sentence) { if (subjects_type[prev_sentence] == PRONOUN) { if (strlen(response) == 0) { subjects[prev_sentence][0] = (char) toupper(subjects [prev_sentence] [0] ); } else { subjects[prev_sentence] [0] = (char) tolower(subjects[prev_sentence][0]); } } strcat(response, subjects[prev_sentence]); strcat(response, " "); if (strlen(auxiliaries[prev_sentence]) > 0) { strcat(response, auxiliaries[prev_sentence]); strcat(response, " "); } get_verb(tenses[prev_sentence], numbers[prev_sentence], usages[prev_sentence]); strcat(response, places[prev_sentence]); return; } /*****************************************************/ /* Get the correct verb from the dictionary. */ /*****************************************************/ void get_verb(char pass_tense, char pass_number, char pass_usage) { rewind{infile); fgets(dic_record, 80, infile); while (! feof(infile)) { if (match_verb(pass_tense, pass_number, pass_usage) == 0) break; fgets(dic_record, 80, infile); } return; } /*****************************************************/ /* If the verb information in the current record */ /* matches the passed information, then move the */ /* correct verb to the response. */ /*****************************************************/ int match_verb(char pass_tense, char pass_number, char pass_usage) { int i; char *root; char *dic_word; root = extract_root(); /* Match verb with root */ if (strcmpi(actions[sentence], root) == 0) { /* Match verb with tense */ for (i=301 i<34; i++) { if (isspace(dic_record[i])) return(1); if (dic_record[i] -= pass_tense) break; } /* Match verb with number */ for (i=41; i<43; i++) { if (isspace(dic_record[i])) return(1); if (dic_record[i] == pass_number) break; } /* Match verb with usage */ if (dic record[29] == pass_usage) { dic_word = extract_word(); strcat(response, dic_word); return(0); } } return(1); } /* End of File */ Listing 2 natural.h /* natural.h Copyright (c) 1993 Russell Suereth */ #define UNKNOWN 200 #define PRONOUN 208 #define NAME 209 #define AUX_VERB_ERROR 210 /*****************************************************/ /* The following definitions are dictionary values */ /*****************************************************/ /* Usage */ #define ING 73 /* I */ #define NOAUX 78 /* N */ #define ROOT 82 /* R */ /* Tense */ #define PAST 48 /* 0 */ #define PRESENT 54 /* 6 */ #define FUTURE 57 /* 9 */ /* Number */ #define SINGULAR 64 /* @ */ #define PLURAL 36 /* $ */ /* Meaning for auxiliaries */ #define LIMITED_DURATION 65 /* A */ #define PARTICULAR_POINT_OF_TIME 66 /* B */ #define UP_TO_PRESENT 67 /* C */ #define NOT_COMPLETED 68 /* D */ #define CAN 69 /* E */ #define COULD 70 /* F */ #define MAY 71 /* G */ #define POSSIBILITY 72 /* H */ #define OBLIGATION 73 /* I */ #define WILL 74 /* J */ #define WOULD 75 /* K */ #define MUST 76 /* L */ #define FIXED_PLAN 77 /* M */ /* End of File */ Listing 3 diction -- the dictionary file a DET the DET house NOUN street NOUN store NOUN race NOUN we PRON $ he PRON @ she PRON @ it PRON @ they PRON $ run VERB R069 @$ run runs VERB N6 @ run running VERB I069 @$ run ran VERB NO @$ run walk VERB R069 @$ walk walks VERB N6 @ walk walking VERB I069 @$ walk walked VERB NO @$ walk to PREP in PREP on PREP where WH was AUX I0 @ D were AUX I0 $ D did AUX R0 @$ B had AUX R0 @$ B has AUX R06 @ C have AUX R06 $ C had been AUX I0 @$ B has been AUX I0 @ C have been AUX I0 $ C could AUX R09 @$ F would AUX R0 @$ K could have AUX R0 @$ F would have AUX R0 @$ K could have been AUX I0 @$ F would have been AUX I0 @$ K should have AUX R0 @$ I should have been AUX I0 @$ I may have AUX R0 @$ G might have AUX R0 @$ H may have been AUX I0 @$ G might have been AUX I0 @$ H must have been AUX I0 @$ L is AUX I69 @ AM be AUX been AUX are AUX I69 $ AM can AUX R69 @$ E can be AUX I69 @$ E will AUX R9 @$ J will be AUX I9 @$ J could be AUX I69 @$ F should AUX R9 @$ I should be AUX I9 @$ I would be AUX I9 @$ K must AUX R9 @$ L must be AUX I9 @$ L may AUX R9 @$ G may be AUX I69 @$ G might AUX R9 @$ H might be AUX I9 @$ H /* End of File */ Image Processing, Part 10: Segmentation Using Edges and Gray Shades Dwayne Phillips The author works as a computer and electronics engineer with the U.S. Department of Defense. He has a PhD in Electrical and Computer Engineering from Louisiana State University. His interests include computer vision, artificial intelligence, software engineering, and programming languages. Introduction to the Series of Articles This is the tenth in a series of articles on images and image processing. Previous articles discussed reading, writing, displaying, and printing images (TIFF format), histograms, edge detection, spatial frequency filtering, and sundry image operations. This article will discuss image segmentation using edges and gray shades. The last article (Phillips, February 1993) discussed image segmentation using histograms. That basic technique examined the histogram of an image, transformed the image into a 1-0 image, and "grew" regions. The results were acceptable given the simplicity of the approach. Segmentation Using Edges and Gray Shades There are powerful segmentation techniques that use the edges in an image, grow regions using the gray shades in an image, and use both the edges and gray shades. These techniques work well over a range of images because edges and gray shades are important clues to objects in a scene. Figure 1 shows the result of using edges to segment an image. The left side shows the output of an edge detector. The right side is the result of grouping the pixels "inside" the edges as objects--a triangle and rectangle. This idea is simple. Detect the edges and group the pixels as objects. Figure 2 illustrates growing objects using the gray shades in an image. You group a pixel with a neighboring pixel if their gray shades are close enough. You replace the two pixels with their average and then look at the neighbors of this two pixel object. If the gray shades of the neighbors are close enough, they become part of the object and their values adjust the average gray shade of the object. The left side shows the input, and the right side shows the result of growing objects in this manner. The 1s are the background object produced by grouping the 1s, 2s, and 3s. The triangle of 2s is a grouping of the 7s and 8s, and the rectangle of 3s is the 8s and 9s. Figure 3 combines the two techniques. The left side shows a gray shade image with the output of an edge detector (*s) superimposed. The right side shows the result of growing regions using the gray shades while ignoring the detected edges (*s). The result is the three objects produced in Figure 2 separated by the edges. These three simple techniques work well in ideal situations. Most images, however, are not ideal. Real images and image processing routines introduce problems. Problems You can encounter three potential problems using these segmentation techniques: the input image can have too many edges and objects, the edge detectors may not be good enough, and you may need to exclude unwanted items to effectively grow the region. The input image can be too complicated and have small, unwanted objects. Photograph 1 shows an aerial image of house trailers, roads, lawns, trees, and a tennis court. The white house trailers are obvious and easy to detect. Other objects (tennis court) have marks or spots that fool the edge detectors and region growing routines. Photograph 2 shows a house, and Photograph 3 shows its edges. Segmentation should detect the roof, windows, and door. The bricks, leaves, and shutter slats are real, but small, so you do not want to detect all of them. You need high quality edge detection to use these technisques. Figure 4 demonstrates how a small edge-detector error leads to a big segmentation error. On the left side of the figure, I poked a small hole in the left edge of the rectangle. The right side shows the terrible segmentation result. Edge detectors do not produce these 1-0 images without thresholding them (Phillips, November 1991). Photograph 4 shows the result of edge detection on Photograph 1. You need to threshold the strong (bright) and weak (faint) edges, to produce a clean 1-0 image. But you also need a consistent and automatic method to find the threshold point. Detected edges can be too thin and too thick. A surplus of stray, thin edges misleads segmentation, and heavy, extra-thick edges ruin objects. Figure 5 shows how the triple thick edges on the left side produce the distorted objects on the right side. The region-growing algorithm (Phillips, January 1992) must limit the size of objects and exclude unwanted pixels. The house in Photograph 2 contains objects of widely varying size. You may want to exclude the bricks to concentrate on large objects or vice versa. You may want to omit certain pixels such as edges. The left side of Figure 6 is a repeat of Figure 1 while the right side shows what happens when the region grower mistakes the edges for objects. Solutions You can solve the edge detection problems by preprocessing, better edge detection, and better region growing. Preprocessing Preprocessing involves smoothing the input image to remove noise, marks, and unwanted detail. The median filter (Phillips, October 1992), one form of smoothing, sorts the pixels in an nxn area (3x3, 5x5, etc.), and replaces the center pixel with the median value. High- and low-pixel filters, variations of the median filter, sort the pixels in an nxn area and replace the center pixel with either the highest or lowest pixel value. Figure 7 illustrates the median, high-pixel, and low-pixel filters. The left side shows the input, the image section. The right side shows the output for each filter processing a 3x3 area. The median filter removes the spikes of the larger numbers. The high-pixel filter output has many high values because the input has a large number in most of its 3x3 areas. The low-pixel filter output is all 1s because there is a 1 in every 3x3 area of the input. Photograph 5 and Photograph 6 show how the low-pixel filter reduces the clutter in the edge detector output. Photograph 5 is the result of the low-pixel filter applied to Photograph 2. The dark window shutters are larger, and the mortar lines around the bricks are gone. Photograph 6 is the output of the edge detector applied to Photograph 5. Compare this to Photograph 3. The edges around the small objects are gone. Listing 1 shows the high_pixel and low_pixel subroutines. They create the output image file if needed, read the input image, loop through the data, and write the output. The looping portion places the pixels in the nxn area into the elements array, sorts the array, and places the highest or lowest pixel value into the output image. Improved Edge Detection For effective segmentation you need accurate edge detectors with automatic thresholding of edges and the ability to thin edges. I improved the accuracy of the edge detectors in the C Image Processing System by adding two more edge detectors and correcting three others (Phillips, November 1991, January 1992). One of the new edge detectors, the variance operator, examines a 3x3 area and replaces the center pixel with the variance. The variance operator subtracts the pixel next to the center pixel, squares that difference, adds up the squares of the differences from the eight neighbors, and takes the square root. The other new edge detector, the range operator, sorts the pixels in an nxn area and subtracts the smallest pixel value from the largest to produce the range. Figure 8 shows the results of applying the variance and range operators to an array of numbers. Photograph 7 and Photograph 8 show the outcome of applying these operators to the house image of photograph 2. Listing 2 shows the source code for the variance and range operators. They create an output image if needed, read the input, loop through the image, and write the output. The looping structure performs the calculations described above. I corrected a major mistake in the directional edge detectors (Phillips, November 1991) that improved accuracy. The Sobel, Kirsch, and Prewitt edge detectors use eight different 3x3 convolution masks--one mask for each direction. Listing 3 shows the correction to the subroutine perform_convolution (Phillips, November 1991). A reader noticed I was setting the output to the answer of each direction convolution--not the strongest of all eight directions. To correct this, set the output to the convolution sum only if the sum is greater than the output. Photograph 9 and Photograph 10 show the effect of this correction. Photograph 9 is from the early article, and the edges are all on the right side and bottom of the objects. There are no edges on the upper and left sides. Photograph 10 shows how the correction produced edges without holes and having the quality needed for segmentation. For effective edge detection you need a technique for thresholding the edge detector output consistently and automatically. One technique sets the threshold point at a given percentage of pixels in the histogram. You calculate the histogram for the edge detector output and sum the histogram values beginning with 0. When this sum exceeds a given percent of the total, you have the threshold value. This method produces consistent results without any manual intervention. A good percentage to use is 50% for most edge detectors and images. Photograph 11 shows the thresholded edge detector output of Photograph 1. That is, I processed Photograph 1 with the edge detector (Photograph 4 shows the result) and set the threshold at 70 percent. Listing 4 shows the find_cutoff_point subroutine that looks through a histogram to find the threshold point. It takes in the histogram and the desired percent and returns the threshold point. This is a simple accumulate and compare operation. The erosion operation (Russ 1992) can solve the final problem with edge detectors, removing extra edges and thinning thick edges. Erosion looks at pixels turned on (edge detector outputs) and turns them off if they have enough neighbors that are turned off. Figure 9 and Figure 10 illustrate erosion. In Figure 9, the left side shows edges (1s) around the triangle and rectangle and then several stray edges. The right side shows the result of eroding or removing any 1 that has seven 0 neighbors. In Figure 10, the left side shows very thick edges around the triangle and rectangle. The right side shows the result of eroding any 1 that has three 0 neighbors. The edges are thinner, and the objects inside the edges are more accurate. Listing 5 shows the erosion subroutine erode_image_array. The looping structure examines every pixel in the_image that equals value. It counts the number of neighboring 0 pixels and sets the out_image to zero if this count exceeds the threshold parameter. The threshold parameter controls the erosion. Threshold was 6 in Figure 9 and 2 in Figure 10. Photograph 12 shows the result of eroding the thick edges of Photograph 6. Note how it thinned the thick edges and removed the stray edges in the house, lawn, and tree. The threshold parameter was 3 for this example. Improved Region Growing You need accurate region growing to implement the edge and gray shade segmentation techniques. Figure 11 shows the region-growing algorithm used in the last article (Phillips, February 1993). This worked for binary images containing 0s and a value. If the algorithm found a pixel equal to value, it labeled that pixel and checked its neighbors to see if they also equaled value (step 3). The region-growing algorithm needs improvements to work with any gray shades, limit the size of regions, and exclude pixels with special values (like edges). Figure 12 shows the new region-growing algorithm. The input image g contains gray shades and may contain special pixels equal to FORGET_IT. The output image array holds the result. There are three new parameters: diff, min_area, and max_area. diff specifies the allowable difference in gray shade for two pixels to merge into the same object. min_area and max_area specify the limits on the size of objects. The major differences in the algorithm begin at step 4. Instead of checking if g(i,j) == value, the algorithm performs three checks: g(i,j) cannot equal FORGET_IT output(i,j) must equal zero g(i,j) cannot differ from the target by more than diff The first two are simple. You want to exclude certain pixels, so you set them to FORGET_IT and ignore them. The output must not be part of an object, so it must be zero. The third test allows you to work with gray shade images. In step 3, you create a target equal to the average gray shade of the pixels in an object. You group neighboring pixels whose values do not differ by more than the diff parameter. The is_close routine at the bottom of Figure 12 tests for this condition. If the pixel g(i,j) is close enough to the target, you call the pixel_label_and_check_neighbor routine to add that pixel to the object and check its neighbors. The pixel_label_and_check_neighbor routine updates the target or average gray shade of the object. The new algorithm limits the size of objects in step 6. It tests count (the size of an object) against min_area and max_area. If the object fails the test, you set all pixels of g in the object to FORGET_IT and set all pixels in output to 0. This removes the object from output and eliminates the pixels from any future consideration in g. I've already discussed how the new algorithm excludes pixels with certain values via the FORGET_IT value. If you want to remove edges from consideration, you lay the edge detector output on top of the input image and set to FORGET_IT all pixels corresponding to the edges. Listing 6 shows the source code for the three subroutines outlined in Figure 12. They follow the algorithm closely. The only trick is how to implement the stack (Phillips, February 1993). The improved region-growing algorithm is the key to the new techniques. It ignores certain pixels and eliminates objects of the wrong size. These small additions produce segmentation results that are much better than those from Part 9. The Three New Techniques Now that I've laid all the ground work, I'll go on to the three new techniques. Edges Only The edge_region subroutine shown in Listing 7 implements this technique. The algorithm is 1. Create the output image file if needed 2. Read the input image 3. Edge detect the input image 4. Threshold the edge detector output 5. Erode the edges if desired 6. Set the edge values to FORGET_IT 7. Grow the objects while ignoring the edges Steps 1 through 6 should give you an image like that shown in Figure 1. Step 7 grows the objects as outlined by the edges. The edge_region subroutine calls any of the edge detectors from this and previous articles, the histogram functions from previous articles, and the find_cutoff_point, erode_image_array, and pixel_grow functions from Listing 4, Listing 5, and Listing 6. The edge_type parameter specifies which edge detector to use. min_area and max_area pass through to the pixel_grow routine to constrain the size of the objects detected. diff passes through to pixel_grow to set the tolerance on gray shades added to an object. diff has little meaning for this technique because the image in which you grow regions contains only 0s and FORGET_IT pixels. The percent parameter passes through to the find_cutoff_point routine to threshold the edge detector output. The set_value parameter is the turned on pixel in the threshold_image_array and erode_image_array routines. Finally, the erode parameter determines whether you perform erosion on the edge detector output. If erode is not zero, then it is the threshold parameter for erode_image_array. Gray Shades Only The short gray_shade_region subroutine in Listing 8 implements this technique. This subroutine creates an output image file if needed and calls the pixel_grow function of Listing 6. pixel_grow does all the work since it handles the gray-shade region growing and limits the sizes of the objects. The diff, min_area, and max_area parameters play the same role as in the edge_region routine described above. Edges and Gray Shade Combined The technique for combining edges and gray shades is implemented by the edge_gray_shade_region function in Listing 9. The algorithm is 1. Create the output image file if needed 2. Read the input image 3. Edge detect the input image 4. Threshold the edge detector output 5. Erode the edges if desired 6. Read the input image again 7. Put the edge values on top of the input image setting them to FORGET_IT 8. Grow gray shade regions while ignoring the edges The differences between edge_region and edge_gray_shade_region are in steps 6 and 7. At this point, edge_gray_shade_region reads the original input image again and overlays it with the detected edges. Step 8 grows gray shade regions while ignoring the detected edges. Steps 1 through 7 generate an image like the left side of Figure 3. Step 8 generates the right side of Figure 3. Photograph 13 through Photograph 17 illustrate these techniques on the aerial image of Photograph 1. Photograph 13 shows the result of the Sobel edge after erosion. The edges outline the major objects in the image fairly well. Photograph 14 shows the result of the edge-only segmentation of Photograph 1. It is result of growing the black regions of Photograph 13. This is a good segmentation as it denotes the house trailers, roads, trees, and parking lots. This is not just the negative image of Photograph 13. Regions too small and large were eliminated. Photograph 15 is the result of the gray-shade-only segmentation of Photograph 1. This segmentation also found the major objects in the image. The combination of edge and gray shade segmentation in Photograph 16 shows the edges of Photograph 13 laid on top of the input image of Photograph 1. Photograph 17 shows the final result of growing gray-shade regions inside these edges. This segmentation has better separation of objects than the gray-shade-only segmentation of Photograph 15. The edges between the objects caused this spacing. Which segmentation is best? That is a judgement call. All three segmentations, however, are better than those produced by the simple techniques in the last article. Photograph 18, Photograph 19, and Photograph 20 show the results of the three techniques applied to the house image of Photograph 2. The edge-only segmentation of Photograph 18 is fairly good as it denotes most of the major objects in the image. The gray-shade-only result in Photograph 19 is not very good because all the objects are right next to each other and hard to distinguish. The combination segmentation in Photograph 20 is an excellent result. It detected objects not found in the edge-only technique and also eliminated many of the unwanted bricks. Integrating the New Techniques Listing 10 shows the new code for the main routine of the C Image Processing System (CIPS). I added case 18 to allow segmentation using the three techniques discussed here. Given next are the changes to the CIPS main menu and the routine that interacts with the user to obtain all the options. Listing 11 shows a standalone application program for segmenting entire images using the new techniques. It is command-line driven and calls the functions given in the previous listings. Conclusions This installment in the series described three powerful image segmentation techniques that work on complicated ages. The techniques, however, are only combinations of existing tools and tricks. Given different images, you might have used different combinations of tools. Experiment, try different combinations, and modify existing tools to create new ones. References Phillips, Dwayne. November 1991. "Image Processing, Part 5: Writing Images to Files and Basic Edge Detection," The C Users Journal. Lawrence, KS: R&D Publicatons. Phillips, Dwayne. January 1992. "Image Processing, Part 6: Advanced Edge Detection," The C Users Journal. Lawrence, KS: R&D Publications. Phillips, Dwayne. October 1992. "Image Processing, Part 7: Spatial Frequency Filtering," The C Users Journal. Lawrence, KS: R&D Publications. Russ, John C. 1992. The Image Processing Handbook. Boca Raton, FL: CRC Press. Phillips, Dwayne. February 1993. "Image Processing, Part 9: Histogram-Based Image Segmentation," The C Users Journal. Lawrence, KS: R&D Publications. Figure 1 This figure shows how to use edges to segment an image. The left side shows the output of an edge detector. The right side is the result of grouping the pixels "inside" the edges as objects -- a triangle and rectangle. 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000001000000000000 -------------------- 00000010100000000000 -------*------------ 00000100010000000000 ------***----------- 00001000001000000000 -----*****---------- 00010000000100000000 ----*******--------- 00111111111110000000 -------------------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000111111111111000 -------------------- 00000100000000001000 -----**********----- 00000100000000001000 -----**********----- 00000100000000001000 -----**********----- 00000100000000001000 -----**********----- 00000100000000001000 -----**********----- 00000111111111111000 -------------------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- Figure 2 Growing objects using the gray shades in an image. The left side shows the input, and the right side shows the result of growing objects in this manner. the 1s are the background object produced by grouping the 1s, 2s, and 3s. The triangle of 2s is a grouping of the 7s and 8s, and the rectangle of 3s is the 8s and 9s. 12121212111122211221 11111111111111111111 13121312123121312312 11111111111111111111 12213121212131212122 11111111111111111111 22222218212113112122 11111112111111111111 12111187732121211122 11111122211111111111 12211788872221212211 11111222221111111111 22118778888212212212 11112222222111111111 12121222121222211212 11111111111111111111 21222122111222312123 11111111111111111111 32121321213221322121 11111111111111111111 22121122222121222122 11111111111111111111 22122298989999992212 11111133333333331111 12222298889998992112 11111133333333331111 12212189999898892122 11111133333333331111 12122299998999992122 11111133333333331111 22133389989988982123 11111133333333331111 12122212312321212212 11111111111111111111 13121213212132121321 11111111111111111111 11323212212121112222 11111111111111111111 12212121222222111122 11111111111111111111 Figure 3 Growing objects using gray shades and edges. The left side shows a gray shade image with the output of an edge detector (*s) superimposed. The right side shows the result of growing regions using the gray shades while ignoring the detected edges. 12121212111122211221 11111111111111111111 13121312123121312312 11111111111111111111 1221312*212131212122 1111111-111111111111 222222*8*12113112122 111111-2-11111111111 12111*877*2121211122 11111-222-1111111111 1221*78887*221212211 1111-22222-111111111 221*8778888*12212212 111-2222222-11111111 12***********2211212 11-----------1111111 21222122111222312123 11111111111111111111 32121321213221322121 11111111111111111111 22121************122 11111------------111 22122*9898999999*212 11111-3333333333-111 12222*9888999899*112 11111-3333333333-111 12212*8999989889*122 11111-3333333333-111 12122*9999899999*122 11111-3333333333-111 22133*8998998898*123 11111-3333333333-111 12122************212 11111------------111 13121213212132121321 11111111111111111111 11323212212121112222 11111111111111111111 12212121222222111122 11111111111111111111 Figure 4 A small edge detector error leads to a big segmentation error 00000000000000000000 ******************** 00000000000000000000 ******************** 00000001000000000000 *******-************ 00000010100000000000 ******-*-*********** 00000100010000000000 *****-***-********** 00001000001000000000 ****-*****-********* 00010000000100000000 ***-*******-******** 00111111111110000000 **-----------******* 00000000000000000000 ******************** 00000000000000000000 ******************** 00000111111111111000 *****------------*** 00000100000000001000 *****-**********-*** 00000100000000001000 *****-**********-*** 00000000000000001000 ****************-*** 00000100000000001000 *****-**********-*** 00000100000000001000 *****-**********-*** 00000111111111111000 *****------------*** 00000000000000000000 ******************** 00000000000000000000 ******************** 00000000000000000000 ******************** Figure 5 The triple thick edges on the left side produce the distorted objects on the right side. 00000000000000000000 -------------------- 00000001000000000000 -------------------- 00000011100000000000 -------------------- 00000111110000000000 -------------------- 00001110111000000000 -------*------------ 00011100011100000000 ------***----------- 00111111111110000000 -------------------- 01111111111111000000 -------------------- 11111111111111100000 -------------------- 00001111111111111100 -------------------- 00001111111111111100 -------------------- 00001111111111111100 -------------------- 00001111111111111100 -------------------- 00001110000000011100 -------********----- 00001110000000011100 -------********----- 00001110000000011100 -------********----- 00001111111111111100 -------------------- 00001111111111111100 -------------------- 00001111111111111100 -------------------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- Figure 6 The left side of this figure repeats Figure 1 while the right side shows what happens when the region grower mistakes the edges for objects. 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000001000000000000 -------1------------ 00000010100000000000 ------121----------- 00000100010000000000 -----12221---------- 00001000001000000000 ----1222221--------- 00010000000100000000 ---122222221-------- 00111111111110000000 --11111111111------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000111111111111000 -----333333333333--- 00000100000000001000 -----344444444443--- 00000100000000001000 -----344444444443--- 00000100000000001000 -----344444444443--- 00000100000000001000 -----344444444443--- 00000100000000001000 -----344444444443--- 00000111111111111000 -----333333333333--- 00000000000000000000 -------------------- 00000000000000000000 -------------------- 00000000000000000000 -------------------- Figure 7 Output of median, high-pixel, and low-Pixel filters 111212163 --------- 111212877 -1112123- 181212123 -1212123- 177212123 -1222122- 116212123 -1222122- Median Filter Output 111217123 -1112122- 111217123 -1112122- 111217123 -1112122- 111212123 --------- 111212163 --------- 111212877 -8822888- 181212123 -8772888- 177212123 -8872223- 116212123 -7777773- High Pixel Filter Output 111217123 -6667773- 111217123 -1227773- 111217123 -1227773- 111212123 --------- 111212163 --------- 111212877 -1111111- 181212123 -1111111- 177212123 -1111111- 116212123 -1111111- Low Pixel Filter Output 111217123 -1111111- 111217123 -1111111- 111217123 -1111111- 111212123 --------- Figure 8 The results of applying the variance and range operators to an array of numbers Input 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 5 5 5 5 10 10 10 10 20 20 20 20 Variance Output 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 0 0 0 7 7 0 0 14 14 0 0 0 Range Output 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 0 0 0 5 5 0 0 10 10 0 0 0 Figure 9 The left side shows edges (1's) around the triangle and rectangle and then several stray edges. The right side shows the result of eroding or removing any 1 that has seven 0 neighbors 00000000000000000000 00000000000000000000 01000000000000111000 00000000000000010000 00000001000000000000 00000001000000000000 01100010100000000010 00000010100000000000 00000100010000000010 00000100010000000010 00001000001000000010 00001000001000000000 00010000000100000000 00010000000100000000 00111111111110000000 00111111111110000000 00000000000000000110 00000000000000000000 00100000000000000000 00000000000000000000 01000111111111111000 00000111111111111000 00000100000000001000 00000100000000001000 00000100000000001000 00000100000000001000 00000100000000001000 00000100000000001000 00100100000000001000 00000100000000001000 00100100000000001000 00000100000000001000 00000111111111111000 00000111111111111000 00000000000000000000 00000000000000000000 00011000000000000000 00001000000000000000 00000100000000000000 00000000000000000000 Figure 10 The left side shows very thick edges around the triangle and rectangle. The right side shows the result of eroding any 1 that has three 0 neighbors 00000000000000000000 00000000000000000000 00000001000000000000 00000000000000000000 00000011100000000000 00000001000000000000 00000111110000000000 00000011100000000000 00001110111000000000 00000100010000000000 00011100011100000000 00001100011000000000 00111111111110000000 00011110111100000000 01111111111111000000 00111111111110000000 11111111111111100000 00011111111111000000 00001111111111111100 00000111111111111000 00001111111111111100 00000111111111111000 00001111111111111100 00000111000000111000 00001110000000011100 00000110000000011000 00001110000000011100 00000100000000001000 00001110000000011100 00000110000000011000 00001111111111111100 00000110000000111000 00001111111111111100 00000111111111111000 00001111111111111100 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 00000000000000000000 Figure 11 A region-growing algorithm for use with images containing zeroes and a value 1. Given an image g with m rows and n columns g(i,j) for i=1,m j=1,n g(i,j) = value for object = 0 for background 2. set g_label=2 this is the label value 3. for (i=0; i= min_area AND count <= max_area) g_label = g_label + 1 else remove object for all output(i,j) = g_label output(i,j) = 0 input(i,j) = FORGET_IT end else remove object end if end of checking jth element end of scanning ith row 7. The End -------------------------------------- procedure pixel_label_and_check_neighbor(g(r,e), output, count, sum, target, diff) output(r,e) = g_label count = count+1 sum = sum + g(r,e) target = sum/count for (R=r-1; r<=r+1; R++) for (E=e-1; e<=e+1; e++) if g(R,E) != FORGET_IT AND output(R,E) == 0 AND is_close(g(R,E), target, diff) push (R,E) onto the stack stack_empty = false end if end loop over E end loop over R end procedure pixel_label_and_check_neighbor --------------------------------------- procedure is_close(pixel, target, diff) if absolute value(pixel - target) < diff return 1 else return 0 end procedure is_close Photograph 1 Aerial image of house trailers Photograph 2 House image Photograph 3 Edge detector output from Photograph 2 Photograph 4 Edge detector output from Photograph 1 (not-thresholded) Photograph 5 Result of low pixel filtering performed on Photograph 2 Photograph 6 Edge detector output from Photograph 5 Photograph 7 Variance edge detector output from Photograph 2 Photograph 8 Range edge detector output from Photograph 2 Photograph 9 Directional edge detector output with error in perform_convolution Photograph 10 Directional edge detector output with correction in perform_convolution Photograph 11 Edge detector output from Photograph 1 thresholded at 70 percent Photograph 12 Result of eroding the edges in Photograph 6 Photograph 13 Sobel edge detector output from Photograph 1 (after erosion) Photograph 14 Result of edge-only segmentation of Photograph 1 Photograph 15 Result of gray-shade-only segmentation of Photograph 1 Photograph 16 Edges of Photograph 13 laid on top of Photograph 1 Photograph 17 Result of edge and gray shade segmentation of Photograph 1 Photograph 18 Result of edge-only segmentation of Photograph 2 Photograph 19 Result of gray-shade-only segmentation of Photograph 2 Photograph 20 Result of edge and gray shade segmentation of Photograph 2 Listing 1 High-pixel and low-pixel filters for image preprocessing /******************************************* * * high_pixel(.. * * This function replaces the pixel at * the center of a 3x3, 5x5, etc. area * with the max for that area. * *******************************************/ high_pixel(in_name, out_name, the_image, out_image, il, ie, ll, le, size) char in_name[], out_name[]; int il, ie, ll, le, size; short the_image[ROWS][COLS], out_image[ROWS][COLS]; { int a, b, count, i, j, k, length, sd2, sd2p1, ss, width; short *elements; struct tiff_header_struct image_header; sd2 = size/2; sd2p1 = sd2 + 1; /********************************************** * * Allocate the elements array large enough * to hold size*size shorts. * **********************************************/ ss = size*size; elements = (short *) malloc(ss * sizeof(short)); if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /*************************** * * Loop over image array * ***************************/ printf("\n"); for(i=sd2; i max) sum = max; out_image[i][j] = sum; } /* ends loop over j */ } /* ends loop over i */ /* if desired, threshold the output image */ if(threshold == 1){ for(i=0; i high){ out_image[i][j] = new_hi; } else{ out_image[i][j] = new_low; } } } } /* ends if threshold == 1 */ fix_edges(out_image, 1); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends variance */ /******************************************* * * range(.. * * This edge detector performs the * range operation. * It replaces the pixel at the center of a * 3x3, 5x5, etc. area with the max - min * for that area. * *******************************************/ range(in_name, out_name, the_image, out_image, il, ie, ll, le, size, threshold, high) char in_name[], out_name[]; int il, ie, ll, le, high, threshold, size; short the_image[ROWS][COLS], out_image[ROWS][COLS]; { int a, b, count, i, j, k, new_hi, new_low, length, sd2, sd2p1, ss, width; short *elements; struct tiff_header_struct image_header; sd2 = size/2; sd2p1 = sd2 + 1; /********************************************** * * Allocate the elements array large enough * to hold size*size shorts. * ***********************************************/ ss = size*size; elements = (short *) malloc(ss * sizeof(short)); if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); new_hi = 250; new_low = 16; if(image_header.bits_per_pixel == 4){ new_hi = 10; new_low = 3; } /*************************** * * Loop over image array * ****************************/ printf("\n"); for(i=sd2; i high){ out_image[i][j] = new_hi; } else{ out_image[i][j] = new_low; } } } } /* ends if threshold == 1 */ fix_edges(out_image, sd2); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); free(elements); } /* ends range */ /* End of File */ Listing 3 Corrected code used in the Sobel, Kirsch, and Prewitt directional edge detectors /*********************************************** * * perform_convolution (... * * This function performs convolution * between the input image and 8 3x3 masks. * The result is placed in the out_image. * ***********************************************/ perform_convolution(image, out_image, detect_type, threshold, image_header, high) short image[ROWS][COLS], out_image [ROWS][COLS]; int detect_type, high, threshold; struct tiff_header_struct *image_header; { int a, b, i, is_present, j, sum; short mask_0[3][3], mask_1[3][3], mask_213][3], mask_3[3][3], mask_4[3][3], mask_5[3][3], mask_6[3][3], mask_7[3][3], max, min, new_hi, new_low; setup_masks(detect_type, mask_0, mask_1, mask_2, mask_3, mask_4, mask_5, mask_6, mask_7); new_hi = 250; new_low = 16; if (image_header->bits_per_pixel == 4){ new_hi = 10; new_low = 3; } min = 0; max = 225; if(image_header->bits_per_pixel == 4) max = 16; /* clear output image array */ for(i-0; i max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ /**/ /**/ /**/ if(sum > out_image[i][j]) /**************/ out_image[i][j] = sum; /**************/ /**/ /**/ /**/ /* 1 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++) { sum = sum + image[i+a][j+b] * mask_1[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 2 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++) { sum = sum + image[i+a][j+b] * mask_2[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 3 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++){ sum = sum + image[i+a][j+b] * mask_3[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 4 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++){ sum = sum + image[i+a][j+b] * mask_4[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 5 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++){ sum = sum + image[i+a][j+b] * mask_5[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 6 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++){ sum = sum + image[i+a][j+b] * mask_6[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; /* 7 direction */ sum = 0; for(a=-1; a<2; a++){ for(b=-1; b<2; b++){ sum = sum + image[i+a][j+b] * mask_7[a+1][b+1]; } } if(sum > max) sum = max; if(sum < 0) sum = 0; /* Correction 12-27-92 see file header for details. */ if(sum > out_image[i][j]) out_image[i][j] = sum; } /* ends loop over j */ } /* ends loop over i */ /* if desired, threshold the output image */ if(threshold == 1){ for(i=0; i high) { out_image[i][j] = new_hi; } else{ out_image[i][j] = new_low; } } } } /* ends if threshold == 1 */ } /* ends perform_convolution */ /* End of File */ Listing 4 find cutoff_point - looks through a histogram to find the threshold point /******************************************* * * find_cutoff_point(.. * * This function looks at a histogram * and sets a cuttoff point at a given * percentage of pixels. * For example, if percent=0.6, you * start at 0 in the histogram and count * up until you've hit 60% of the pixels. * Then you stop and return that pixel * value. * ********************************************/ find_cutoff_point(histogram, percent, cutoff} unsigned long histogram[]; float percent; short *cutoff; { float fd, fsum, sum_div; int i, looking; long lc, lr, num=0, sum=0; sum = 0; i = 0; lr = (long)(ROWS); lc = (long)(COLS); num = lr*lc; fd = (float)(num); while(looking){ fsum = (float)(sum); sum_div = fsum/fd; if(sum_div >= percent) looking = 0; else sum = sum + histogram[i++]; } /* ends while looking */ if(i >= 256) i = 255; *cutoff = i; printf("\nCutoff is %d sum=%ld", *cutoff, sum); } /* ends find_cutoff_point */ /* End of File */ Listing 5 erode_image_array -- the erosion subroutine /******************************************* * * erode_image_array (.. * * This function erodes pixels. If a pixel * equals value and has more than threshold * neighbors equal to 0, then set that * pixel in the output to 0. * *******************************************/ erode_image_array(the_image, out_image, value, threshold) short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k, length, width; /*************************** * * Loop over image array * ****************************/ for(i=0; i threshold) out_image[i][j] = 0; } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ ) /* ends erode_image_array */ /* End of File */ Listing 6 Source code for the region-growing algorithm outlined in Figure 12 /********************************************** * * pixel_grow(... * * The function grows regions. It is similar * to the grow function in segment.c, but it * has several new capabilities. It can * eliminate regions if they are too large or * too small. * * It ignores pixels = FORGET_IT. This allows * it to ignore edges or regions already * eliminated from consideration. * * It adds pixels to a growing region only if * the pixel is close enough to the average gray * level of that region. * ***********************************************/ pixel_grow(input, output, diff, min_area, max_area) short input[ROWS][COLS], output [ROWS][COLS], max_area, min_area, diff; { char name[80]; int count, first_call, i, ii, j, jj, object_found, pointer, pop_i, pop_j, stack_empty, stack_file_in_use; short g_label, target, sum, stack[STACK_SIZE][2]; for(i=0; i= min_area && count <= max_area) ++g_label; /********************************** * * Remove the object from the * output. Set all pixels in the * object you are removing to * FORGET_IT. * **********************************/ else{ for(ii=0; ii found %d objects", g_label); } /* ends pixel_grow */ /*********************************************** * * pixel_label_and_check_neighbors(... * * This function labels a pixel with an object * label and then checks the pixel's 8 * neighbors. If any of the neigbors are * set, then they are also labeled. * * It also updates the target or ave pixel * value of the pixels in the region being * grown. * ***********************************************/ pixel_label_and_check_neighbor(input_image, output_image, target, sum, count, stack, g_label, stack_empty, pointer, r, e, diff, stack_file_in_use, first_call) int *count, e, *first_call, *pointer, r, *stack_empty, *stack_file_in_use; short input_image[ROWS][COLS], output_image[ROWS][COLS], g_label, *sum, *target, stack[STACK_SIZE][2], diff; { int already_labeled = 0, i, j; if (output_image[r][e] != 0) already_labeled = 1; output_image[r][e] = g_label; *count = *count + 1; if(*count > 1){ *sum = *sum + input_image[r][e]; *target = *sum / *count; } /*************************************** * * Look at the 8 neighors of the * point r,e. * * Ensure the points are close enough * to the target and do not equal * FORGET_IT. * * Ensure the points you are checking * are in the image, i.e. not less * than zero and not greater than * ROWS-1 or COLS-1. * ***************************************/ for(i=(r-1); i<=(r+1); i++){ for(j=(e-1); j<=(e+1); j++){ if((i>=0) && (i<=ROWS-1) && (j>=0) && (j<=COLS-1)){ if( input_image[i][j] != FORGET_IT && is_close(input_image[i][j], *target, diff) && output_image[i][j] == 0) { *pointer = *pointer + 1; stack[*pointer][0] = i; /* PUSH */ stack[*pointer][1] = j; /* OPERATION */ *stack_empty = 0; if(*pointer >= (STACK_SIZE - STACK_FILE_LENGTH)){ push_data_onto_stack_file(stack, pointer, first_call); *stack_file_in_use = 1; } /* ends if *pointer >= STACK_SIZE - STACK_FILE_LENGTH*/ } /* ends if is_close */ } /* end if i and j are on the image */ } /* ends loop over i rows */ } /* ends loop over j columns */ } /* ends pixel_label_and_check_neighbors */ /******************************************** * * is_close(... * * This function tests to see if two pixel * values are close enough together. It * uses the delta parameter to make this * judgement. * **********************************************/ is_close(a, b, delta) short a, b, delta; { int result = 0; short diff; diff = a-b; if(diff < 0) diff = diff*(-1); if(diff < delta) result = 1; return(result); } /* ends is_close */ /* End of File */ Listing 7 edge_region - implements edge-only segmentation /******************************************* * * edge_region(.. * * This function segments an image by * growing regions inside of edges. * The steps are: * ù detect edges * ù threshold edge output to a * percent value * ù remove edges from consideration * ù grow regions * *******************************************/ edge_region(in_name, out_name, the_image, out_image, il, ie, ll, le, edge_type, min_area, max_area, diff, percent, set_value, erode) char in_name[], out_name[]; float percent; int edge_type, il, ie, ll, le; short diff, erode, max_area, min_area, set_value, the_image[ROWS][COLS], out_image[ROWS][COLS]; { int a, b, count, i, j, k, length, width; short cutoff; struct tiff_header_struct image_header; unsigned long histogram[GRAY_LEVELS+1]; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /*************************** * * Detect the edges. Do * not threshold. * ***************************/ if(edge_type == 1 edge_type == 2 edge_type == 3) detect_edges(in_name, out_name, the_image, out_image, il, ie, ll, le, edge_type, 0, 0); if(edge_type == 4){ quick_edge(in_name, out_name, the_image, out_image, il ie, ll, le, 0, 0); } /* ends if 4 */ if(edge_type == 5){ homogeneity(in_name, out_name, the_image, out_image, il, ie, ll, le, 0, 0); } /* ends if 5 */ if(edge_type == 6){ difference_edge(in_name, out_name, the_image, out_image, il, ie, ll, le, 0, 0); } /* ends if 6 */ if(edge_type == 7){ contrast_edge(in_name, out_name, the_image, out_image, il, ie, ll, le, 0, 0); } /* ends if 7 */ if(edge_type == 8){ gaussian_edge(in_name, out_name, the_image, out_image, il, ie, ll, le, 3, 0, 0); } /* ends if 8 */ if(edge_type == 10){ range(in_name, out_name, the_image, out_image, il, ie, ll, le, 3, 0, 0); } /* ends if 10 */ if(edge_type == 11){ variance(in_name, out_name, the_image, out_image, il, ie, ll, le, 0, 0); } /* if 11 */ /* copy out_image to the_image */ for(i=0; i Enter input image name\n"); get_image_name (name); printf("\nCIPS> Enter output image name\n"); get_image_name (name2); get_parameters (&i1, &ie, &ll, &le); get_edge_region_options (er_method, &detect_type, &min_area, &max_area, &value, &diff, &percent, &erode); if(er_method[0] == 'e' er_method[0] == 'E') edge_region(name, name2, the_image, out_image, il, ie, ll, le, detect_type, min_area, max_area, diff, percent, value, erode); if(er_method[0] == 'g' er_method[0] == 'G') gray_shade_region(name, name2, the_image, out_image, il,ie, ll, le, diff, min_area, max_area); if(er_method[0] == 'c' er_method[0] == 'C') edge_gray_shade_region(name, name2 the_image, out_image, il, ie, ll, le, detect_type, min_area, max_area, diff, percent, value, erode); break; ù ù ù /************************************************* * * show_menu (.. * * This function displays the CIPS main menu. * **************************************************/ show_menu() { printf("n\nWelcome to CIPS"); printf("\nThe C Image Processing System"); printf("\nThese are you choices:"); printf("\n\t1. Display image header"); printf("\n\t2. Show image numbers"); printf("\n\t3. Print image numbers"); printf("\n\t4. Display image (VGA & EGA only)"); printf("\n\t5. Display or print image using " "halftoning"); printf("\n\t6. Print graphics image using " "dithering"); printf("\n\t7. Print or display histogram " "numbers"); printf("\n\t8. Perform edge detection"); printf("\n\t9. Perform edge enhancement"); printf("\n\t1O. Perform image filtering"); printf("\n\t11. Perform image addition and " "subtraction"); printf("\n\t12. Perform image cutting and pasting"); printf("\n\t13. Perform image rotation and " "flipping"); printf("\n\t14. Perform image scaling"); printf("\n\t15. Create a blank image"); printf("\n\t16. Perform image thresholding"); printf("\n\t17. Perform image segmentation"); printf("\n\t18. Perform edge & gray shade" " image segmentation"); printf("\n\t20. Exit system"); printf("\n\nEnter choice_\b"): } /* ends show_menu */ ù ù /******************************************** * * get_edge_region_options(... * * This function interacts with the user to * get the options needed to call the * edge and region based segmentation * routines. * ********************************************/ get_edge_region_options(method, edge_type, min_area, max_area, set_value, diff, percent, erode) char method[]; float *percent; int *edge_type; short *diff, *erode, *min_area, *max_area, *set_value; { int not_finished = 1, response; while(not finished){ printf("\n\nEdge Region Segmentation Options:"); printf("\n\t1. Method is %s", method); printf("\n\t Recall: Edge, Gray shade, " "Combination"); printf("\n\t2. Edge type is %d", *edge_type); printf("\n\t Recall: "); printf("\n\t 1=Prewitt 2=Kirsch"); printf("\n\t 3=Sobel 4=quick"); printf("\n\t 5=homogeneity 6=difference"); printf("\n\t 7=contrast 8=gaussian"); printf("\n\t 10=range 11=variance"); printf("\n\t3. Min area is %d", *min_area); printf("\n\t4. Max area is %d", *max_area); printf("\n\t5. Set value is %d", *set_value); printf("\n\t6. Difference value is %d", *diff); printf("\n\t7. Threshold percentage is %f", *percent); printf("\n\t8. Erode is %d", *erode); printf("\n\nEnter choice (0 = no change) _\b"); get_integer(&response); if(response == 0){ not_finished = 0; } if(response == 1){ printf("\n\t Recall: Edge, Gray shade, " "Combination"); printf("\n\t> "); read_string(method); } if(response == 2){ printf("\n\t Recall:"); printf("\n\t 1=Prewitt 2=Kirsch"); printf("\n\t 3=Sobel 4=quick"); printf("\n\t 5=homogeneity 6=difference"); printf("\n\t 7=contrast 8=gaussian"); printf("\n\t 10=range 11=variance"); printf("\n\t__\b"); get_integer(edge_type); } if(response == 3){ printf("\nEnter min area:_b\b"); get_integer(min_area); } if(response == 4){ printf("\nEnter max area:__b\b"); get_integer(max_area); } if(response == 5){ printf("\nEnter set value:__\b\b"); get_integer(set_value); } if(response == 6){ printf("\nEnter difference:_\b\b"); get_integer(diff); } if(response == 7){ printf("\nEnter threshold percentage:_\b\b"); get_float(percent); } if(response == 8){ printf("\nEnter erode:_\b\b"); get_integer(erode); } } /* ends while not_finished */ } /* ends get_edge_region_options */ /* End of File */ Listing 11 Standalone application program for segmenting entire images using edge detection and gray shades /*********************************************** * * file d:\cips\main2seg.c * * Functions: This file contains * main * * Purpose: * This is a calling program that calls * the three new segmentation techniques * discussed in Image Processing part 10. * * External Calls: * gin.c - get_image_name * numcvrt.c - get_integer * int_convert * tiff.c - read_tiff_header * segment2.c - edge_region * gray_shade_region * edge_gray_shade_region * * Modifications: * 5 December 1992 - created * ***********************************************/ #include "cips.h" short the_image[ROWS][COLS]; short out_image[ROWS][COLS]; main(argc, argv) int argc; char *argv[]; { char name[80], name2[80], low_high[80], type[80]; float percent; int count, i, ie, il, j, le, length, ll, looking = 1, lw, width; short value, value2, value3, value4, value5, value6; struct tiff_header_struct image_header; _setvideomode(_TEXTC80); /* MSC 6.0 statements */ _setbkcolor (1); _settextcolor (7); _clearscreen (_GCLEARSCREEN); /*********************************************** * * Interpret the command line parameters. * ************************************************/ if(argc < 4){ printf( "\n\nNot enough parameters:" "\n" "\n" usage: main2seg in-file out-file type" "[values ...]" "\n" "\n recall type: Edge-region edge-gray-grow (C)" "Gray-shade-grow" "\n" "\n main2seg in-file out-file R percent" "edge-type" "\n min-area max-area diff set-value erode" "\n main2seg in-file out-file E percent" "edge-type" "\n min-area max-area diff set-value erode" "\n main2seg in-file out-file G diff" "min-area max-area" "\n" "\n"); exit(0); } strcpy(name, argy[1]); strcpy(name2, argv[2]); strcpy(type, argv[3]); if(type[0] == 'a' type[0] == 'A' type[0] == 'c' type[0] == 'C' type[0] == 'e' type[0] == 'E'){ percent = atof(argv[4]); value = atoi(argv[5]); value2 = atoi(argv[6]); value3 = atoi(argv[7]); value4 = atoi(argv[8]); value5 = atoi(argv[9]); value6 = atoi(argv[10]); } else{ value = atoi(argv[4]); value2 = atoi(argv[5]); value3 = atoi(argv[6]); } il = 1; ie = 1; ll = ROWS+1; le = COLS+1; /****************************************** * * Read the input image header and setup * the looping counters. * *******************************************/ read_tiff_header(name, &image_header); length = (9O + image_header.image_length)/ROWS; width = (90 + image_header.image_width)/COLS; count = 1; lw = length*width; printf("\nlength=%d width=%d", length, width); /******************************** * * Edge only segmentation * *********************************/ if(type[0] == 'e' type[0] == 'E'){ for(i=0; i that declares all sorts of additional functions. My goal this month is to show you many of those new functions. I do so in the context of how you might actually use them, should the need arise. One day, your enterprise might feel moved to sell code into Japan, China, or the Arab world. You might find an advantage in writing code that is truly international in its handling of text. Or you might want to manipulate text in numerous fonts and colors. If so, you'll find no more sophisticated machinery than what's already in Standard C, at least not in any other standardized programming language. The additions described here make for a particularly rich environment for manipulating large character sets. I remind you that what I'm describing here is part of a "normative addendum" to the ISO C Standard. It is still subject to balloting within SC22, the parent committee of the C standards committee WG14. It may change in response to comments or criticism. But we've been reviewing it within WG14 for about three years now. I think it's mostly stable. Traditional Character Classification The existing header declares a number of functions for classifying (one-byte) characters. They have proved their worth in many a C program over the past two decades. An important part of processing text involves classifying characters in various ways. Some of those classifications occur so often that they're worth capturing in standard library functions. Thus, the functions declared in help you test quickly for characters in a number of common classes. These classes include lower-case letters, upper-case letters, digits, punctuation, and so forth. Not all the classes are exactly the same across implementations. You can be sure that isdigit returns true (nonzero) only for the ten decimal digits. You can be sure that ispunct returns true for all the punctuation required to write a C program. ispunct also often returns true for such characters as the at sign @, but it doesn't have to. It might return true for additional characters that you don't expect. In short, Standard C says that the punctuation class is to some extent implementation defined. That's not the end of it. When we put locales in Standard C, we allowed for additional changes to the character classification functions. A program starts out in the "C" locale, where the behavior is pretty traditional. You can call the function setlocale, however, to change to a new locale. If that locale specifies a new category LC_CTYPE, several of the functions declared in can begin behaving differently. They can recognize additional punctuation, or lower-case letters, or a few other critters. The proper style these days for classifying characters is thus to use the functions religiously. They are more likely to get the right answer regardless of execution character set and regardless of current locale. Wide Character Classification Now imagine the additional possibilities that come with a really large character set. You still want the old classifications. ASCII is a common subset of the various Kanji codes, for instance. And you still have occasion to parse numbers and names based on English-language rules. But you now face two additional considerations: The existing character classes may sensibly include characters not representable as single-byte characters. Some implementations may see fit to define whole new classes of characters. To address the first consideration, the new header provides parallels for all the functions delcared in : int iswalnum(wint_t wc); int iswalpha(wint_t wc); int iswcntrl(wint_t wc); int iswdigit(wint_t wc); int iswgraph(wint_t wc); int iswlower(wint_t wc); int iswprint(wint_t wc); int iswpunct(wint_t wc); int iswspace(wint_t wc); int iswupper(wint_t wc); int iswxdigit(wint_t wc); wint_t towlower(wint_t wc); wint_t towupper(wint_t wc); I described the new type wint_t last month. It represents all valid values of the wide-character type wchar_t, plus the value WEOF for wide-character end-of-file. These functions essentially behave the same as their older cousins for characters that are also representable as single-byte characters. (The language in the normative addendum is twisty, but that's what it amounts to.) Some can also return true for additional wide characters, by an extension of the same latitude granted their older cousins. The parallels break down in two cases, both involving the exclusion of whitespace. The older functions isgraph and ispunct are defined in terms of what other functions accept, minus the space character ' '. The newer functions iswgraph and iswpunct are defined in terms of what the analogous functions accept, minus the characters accepted by iswspace. That certainly includes the space character, but it might also include others, even single-byte white-space characters. We on WG14 couldn't resist the temptation to properly generalize these functions, even at the cost of some backward compatibility. What this means in practice is really rather simple, despite my nit-picking descriptions. You can change your programs to handle large character sets mostly by: replacing char declarations for text objects with wchar_t declarations replacing calls to the functions declared in with their analogs declared in If you run into problems, I've indicated a few subtle differences that can bite. Chances are, however, that you will have far more trouble getting the obvious code changes right than finding subtle changes in the definition of white space. New Classifications Look at all the ways we've found it convenient to classify one-byte characters over the years. Now imagine all the possibilities when you have a character set with thousands of elements. Actually, not even the Japanese could imagine the new classificationss they might want. The one thing they were sure of is that various people would want new ones. So what they proposed was an open-ended set of new classifications. An implementation publishes a list of property names. These are just null-terminated strings like so many other names in C. (They could be multibyte strings, but who knows or cares? All you have to do is read them in printed form and be able to reproduce them as string literals in a program you write.) To make use of one of these new classifications, you obtain its handle by calling the function wctype, declared in , as in: static wctype_t hirigana = wctype("hirigana"); I described the type wctype_t briefly last month. All you need to know is that it is declared in as some scalar type that you can use as an argument to iswctype (also declared in that header). Listing 1, for example, shows a possible function that tests whether an entire wide-character string is hirigana. It assumes, of course, that "hirigana" is a legitimate classification name, as in the example above. How many such classifications are there? That depends on the implementation. Every implementation must accept 11 names, corresponding to the 11 standard classification functions--"alpha", "alnum", "cntrl", etc. The promise is that an expression such as iswctype(wc, wctype("upper")) is entirely equivalent to iswupper(wc). An implementation may or may not accept additional names. The set of accepted classification names can even change with locale. A call to setlocale that changes the category LC_CTYPE can switch to a new set of character classifications. (It can even switch to a new encoding for wide characters, within limits, but that latitude is probably more theoretical than practical.) As some of you may know, I have an implementation of the Standard C library that includes support for extensible locales. (See my book, The Standard C Library, Prentice Hall, 1992.) I am currently adding the facilities in to that implementation. The new code supports added notation for locale files, so you can add an open-ended set of wide-character classifications to any locale. My suspicion is that it will take several years for people to agree on the commonest classifications for each large character set. In the meantime, there's little use to be made of this added capability. Remember, it's not portable, and it's not likely to correspond to anything in existing code. Just bear it in mind for future use. Revisited If you like manipulating null-terminated strings of char in C, you'll probably also like doing the same sorts of things with large character sets. So the header declares analogs for all our old friends from the standard header (Listing 2). Note the general substitution of wcs (Wide-Character String) for str (String), except that strstr became the more sensible wcsstr instead of the pedantic wcswcs. You might also note that the lack of any mem analogs in wide-character land. So far, the committee has heard no strong plea for them. You can mostly use the older functions even with the larger character types. A quick glance reveals that the new functions differ from the older ones in a simple way. Where the functions declared in have an argument or return value that involves type char, the newer functions instead use type wchar_t. Thus, you can convert an existing program that manipulates strings mostly by replacing pointer to char and array of char declarations for string objects with analogous wchar_t declarations replacing calls to the str functions declared in with their analogs declared in fixing calls to the mem functions, where necessary, by multiplying the size arguments by sizeof (wchar_t) WG14 mostly resisted the urge to "fix" the string functions, to ease this sort of migration. A closer glance, however, reveals that one of the functions has indeed been fixed. The function strtok has always been notorious in the C library. It is the only string function that requires the use of static memory, to retain state information between calls. strtok certainly has plenty of company throughout the rest of the Standard C library. Still, it's a pity that this one function could have been kept pure with just a bit more work, yet traditionally was not. So WG14 bowed to temptation in this area. The added argument wchar_t **ptr is the address of a pointer object that the caller must provide to the function wcstok. This pointer takes the place of the static memory used in the older strtok. By providing pointers to different objects, you can keep track of where you are in different wide-character strings. Thus, you can now parse multiple (wide character) strings at the same time. The cost is that you have more work to do in converting existing code. Revisited The header has three functions that produce null-terminated character strings. Two are the traditional functions asctime and ctime. The third is the function invented by X3J11, strftime. You can do essentially everything with the new function, and then some, that you could do with the older ones. Hence, WG14 decided to provide a wide-character version of only the new one: size_t wcsftime(wchar_t *s, size_t maxsize, const wchar_t *format, const struct tm *timeptr); wcsftime differs from strftime in two ways: Its format argument, which controls the formatting of the generated string, is a wide-character string. The generated string is also a wide-character string. Note that strftime treats its format argument as a multibyte string. Assuming the implementation is tidy enough, it can thus also generate well-formed multibyte strings. Hence, the existing C Standard already provides all the machinery needed to format times using large character sets. WG14 nevertheless elected to add wcsftime. The idea is to eliminate wherever possible the need to represent any multibyte strings within a program. That in turn eliminates the need to convert back and forth between multibyte and wide character forms. And that, in the short run, makes it easier to convert programs to manipulating text represented with a large character set instead of the current single-byte sets. In the long run, that will also keep programs cleaner. So to convert time strings to wide-character form: First convert any calls to asctime or ctime to their equivalents in terms of strftime. Replace pointer to char and array of char declarations for string objects with analogous wchar_t declarations. Replace calls to strftime with calls to wcsftime, declared in . Revisited A handful of the functions declared in convert arithmetic representations to text strings--strtod, strtol, and strtoul. These three functions now have wide-character analogs: double wcstod(const wchar_t *nptr, wchar_t **endptr); long int wcstol(const wchar_t *nptr, wchar_t **endptr, int base); unsigned long int wcstoul// (const wchar_t *nptr, wchar_t **endptr, int base); Again, the new versions don't add much new functionality. The characters they generate are all representable in single-byte form. (A possible exception is how an implementation chooses to print an overflow or other forms such as NaNs, but that is not likely to involve much Arabic or Kanji.) What you win, once again, is the ability to work purely with wide-character strings. On the other hand, several other functions declared in have been supplemented considerably. These are the ones that help you walk along multibyte strings, or that convert between multibyte and wide-character forms. We added them because experience to date says we should. Each fills some gap we found in the minimal set of conversion functions in the current C Standard. The first addition looks simple enough: int wctob(wint_t wc); This function takes a wide-character argument (or WEOF, as I discussed last month). It determines whether that wide character can be represented as a single-byte multibyte character in the initial shift state. If that is possible, the function returns the value of the single-byte representation. Otherwise, it returns EOF. (Note that the int return is really the "metacharacter" type used by fgetc and other functions, as I also discussed last month.) What does wctob buy you? It is certainly convenient, for one thing. It's a nuisance to do the equivalent using existing functions: Set up a wide-character string consisting of the argument wc followed by a null wide character. Convert it to a multibyte string in yet another buffer, by calling wcstombs. Check that the converted string consists of a single (non-null) character, followed by the null terminator. Pick up that character as the value of interest, then clean up the mess. But nuisancy as this sequence is, it is still not guaranteed to give you what you want. The C Standard does not require that wcstombs generate the most economical string (although it probably will). It might have redundant shift codes at the start and/or end of the converted string. Thus, wctob is both essential and convenient. It even turns out to simplify the description of other functions. Remembering Shift States I noted earlier that strtok had the bad grace to retain static memory. Well, X3J11 had the bad grace to indulge that same unfortunate practice in some of the functions we added. Among these were the functions that convert back and forth between wide-character and multibyte encodings. We did so to keep the external interfaces simpler, figuring that the functions wouldn't be used that often. We were wrong. So part of the proposed addition is a set of functions that do the same as existing conversion functions, but ask the caller to provide a pointer to the state memory. (Note the similarity to wcstok, described above.) The functions use the state memory to keep track of the current shift state, for multibyte encodings that are state dependent, that is. You provide a pointer to an object of (nonarray) type mbstate_t, declared as usual in . The functions decide how best to use this storage on your behalf. You can initialize the mbstate_t object to zero and be sure that it represents the initial shift state. Or you can call any of several functions (described below) in such a way that they enter the initial shift state. You can also test such an object to see whether it represents the initial shift state, by calling the function: int sisinit(const mbstate_t *ps); The functions that use this new state memory are analogous to existing functions, with an r (for Restartable) added in the middle of the name: int mbrlen(const char *s, size_t n, mbstate_t *ps); int mbrtowc(wchar_t *pwc, const char *s, size_t n, mbstate_t *ps); int wcrtomb(char *s, wchar_t wc, mbstate_t *ps}; size_t mbsrtowcs(wchar_t *dst, const char **src, size_t len, mbstate_t *ps); size_t wcsrtombs(char *dst, const wchar_t **src, size_t len, mbstate_t *ps); You can convert from using the existing functions mostly by providing your own state objects, but even that is optional. All these functions supply their own internal state, just like the bad old days, if you ask them. The harder part of the conversion involves a number of small changes in behavior, which I won't bother to describe here. I suspect very few readers of this column today have invested much in code that fiddles with large character sets already. Coming Soon There is still one batch of added functions that I have yet to describe. They let you read and write new creatures called wide-character streams. That is the topic of next month's column. Listing 1 Test for hirigana #include static wchar_t hirigana = wctype("hirigana"); int ishiri (wchar_t *wcs) { /* test each element of string */ while (*wcs) if (!iswctype(*wcs++, hirigana)) return (0); return (1); } /* End of File */ Listing 2 wchar.h analogs for string functions in wchar_t *wcscpy(wchar_t *s1, const wchar_t *s2); wchar_t *wcsncpy(wchar_t *s1, const wchar_t *s2, size_t n); wchar_t *wcscat(wchar_t *s1, const wchar_t *s2); wchar_t *wcsncat(wchar_t *s1, const wchar_t *s2, size_t n); int wcscmp(const wchar_t *s1, const wchar_t *s2); int wcscoll(const wchar_t *s1, const wchar_t *s2); int wcsncmp(const wchar_t *s1, const wchar_t *s2, size_t n); size_t wcsxfrm(wchar_t *s1, const wchar_t *s2, size_t n); wchar_t *wcschr(const wchar_t *s, wint_t c); size_t wcscspn(const wchar_t *s1, const wchar_t *s2); wchar_t *wcspbrk(const wchar_t *s1, const wchar_t *s2); wchar_t *wcsrchr(const wchar_t *s, wint_t c); size_t wcsspn(const wchar_t *s1, const wchar_t *s2); wchar_t *wcsstr(const wchar_t *s1, const wchar_t *s2); wchar_t *wcstok(wchar_t *s1, const wchar_t *s2, wchar_t **ptr); size_t wcslen(const wchar_t *s); /* End of File */ On the Networks Flip/Flop Sydney S. Weinstein Sydney S. Weinstein, CDP, CCP is a consultant, columnist, lecturer, author, professor, and President of Datacomp Systems, Inc., a consulting and contract programming firm specializing in databases, data presentation and windowing, transaction processing, networking, testing and test suites, and device management for UNIX and MS-DOS. He can be contacted care of Datacomp Systems, Inc., 3837 Byron Road, Huntingdon Valley, PA 19006-2320 or via electronic mail on the Internet/USENET mailbox syd@DSI.COM (dsinc!syd for those that cannot do Internet addressing). Well, last column's news is reversed this time. Once again, and with no explanation, comp.sources.unix seems to have gone quiet, and comp. sources.x picked up just as I started writing this column. Who knows why these things happen, but I vote for the moderators running out of spare time. I know I have lately. Before comp.sources.unix went quiet again, two postings were made in the final flurry that came after the previous column's deadline. First, dup, a utility for SCO Xenix, adds a special device driver to handle the dup system call from an open. Thus you can pass /dev/fd/0 to pass in the current unit 0 to a process. David J. Fiander contributed dup for Volume 26, Issue 87. Second, a set of two utilities, called faucet, extends the pipe concept over a network. faucet provides a sink to catch its input and send it over the network to a source on the remote machine, which then continues the pipeline. Robert Forsman contributed faucet for Volume 26, Issue 86. Too Many Files Programmers create many more files than they can possibly remember. You can easily forget the location of a file in the numerous subdirectories of a modern system. And the file names, while once meaningful, can become more and more cryptic as memories fade. Well, you cannot do much about losing track of the file itself, but while searching the disk for the files, perhaps describe, posted by Tim Cook as Volume 35, Issues 12-14 of comp.sources.misc, can help. To each directory, describe adds a database of file descriptions, placing the database in the dbm indexed file .desc. (With dbm this means you have .desc.pag and .desc.dir.) Each entry in the database is indexed by name and inode number (to allow for finding it after a file rename). Then the describe package adds a few utilities: dl--a replacement for the ls command, describe--to set or list the descriptive comment for a file, and patches--to support the descriptions directly with the common commands mv, compress, and ftpd. Highlights from the remaining postings in comp.sources.misc this time include: With the need to wrap text from within an editor, and the Berkeley tool fmt not always available, Chad R. Larson contributed fill for Volume 35, Issue 3. fill provides optional left and right justification, centering, paragraph indention or hanging indention, and boxed paragraphs. To use it from within an editor, you must be able to pipe a text object. If you are not running in Suns and don't have access to the OpenLook Ftptool, consider ncftp from Mike Gleason , posted as Volume 35, Issues 4-7. ncftp, a tool to automate ftp transfers and make them more convenient, handles automatic anonymous logins, use of your pager to view files and directory listings, wildcard support for get and put, and multi-site access in the same session. It even has a command-line interface to allow for noninteractive use. Angus Duggan has updated his psutils package to version 1.7 for Volume 35, Issues 8-11. This set of programs rearranges the pages of a PostScript file for printing or display, makes signatures, extracts pages or ranges of pages, prints multiple pages on a single page, and fits an encapsulated file within a given bounding box. It also contains utilities to add the proper Adobe Postscript Convention headers to Frame, WordPerfect, Microsoft Word, and Macintosh documents so they can be manipulated with the utilities in this package. Patch 1 was posted as Volume 35, Issue 110, to fix some bugs and better handle scaling for American Letter paper size. Stephen R. van den Berg's procmail was re-released at version 2.80 in Volume 35, Issues 21-32. procmail is a mail agent that sorts and processes incoming mail automatically. The most visible changes include full biff/comsat support, mailboxes in the home directories, better mailing list support, and numerous other enhancements. A new BBS package for UNIX, ix/MBox, was contributed as mbox for Volume 35, Issues 36-50 by Volker Schuermann . This BBS uses the underlying UUCP, MAIL, and NEWS transport software to move its files, but it provides: A News reader (or you can use your own UNIX newsreader), an internal mailer for intra-bbs mail, a hook to UNIX MTAs for intersystem mail, variable and configurable command set, user-defined commands, international support. The package uses SX/SZ/SB and RX/RZ/RB for upload/download protocols, supports its own line editor and can use full-screen editors, includes a full-screen MIME RTF editor, and much more. An updated release, and transfer of control, of the UNIX shell contains zsh was contributed for Volume 35, Issues 51-72 by The Zsh Mailing List .zsh is a shell designed for interactive use and based on bash, ksh, and tcsh. It also has a strong scripting language. This release, version 2.3.1, includes bug fixes, improvements in the vi line-editing mode, improved portability, RFS support, ksh compatibility mode, nested variables, array slices, and much more (the list goes on for about 100 lines). S. Manoharan contributed a C++ class for parsing command-line options as getlongopt for Volume 35, Issue 75. It supports long names and environment variables, but does not currently support abbreviations. The speed of the decompress side of freeze/melt caused a reposting of a complete 2.5 release by Leonid A. Broukhis for Volume 35, Issues 77-79. This supersedes the patches against version 2.4 that also appeared recently. freeze/melt is a file compression/decompression system. Hugh F. Mahon contributed ee, a simple to learn editor for e-mail and such. Easy Edit's goal was to provide an editor that needed no instruction to learn to use. Although based on System V, it should work on other platforms if they have the appropriate curses support. It also supports X/Open localization for international character sets (single octet only). It was posted in Volume 35, Issues 80-84. Philip D. Howard contributed a function library, bitio, that implements a bit stream on top of a byte stream (eight bits per byte). This bit stream is designed to be exchangeable between different platforms and retain the same bit order regardless of the endianness of the platforms. It includes macros and functions for writing from 1 through 32 bits per call and a complete test suite. It was posted as Volume 35, Issues 105-107. Wietse Venema updated the tcp wrapper program log_tcp, adding access protections. Contributed for Volume 36, Issues 4-6, tcp allows sites directly on the Internet, such as dsi.com, to control who accesses which services on our systems. New since the prior version are better protection against spoofed network addresses, extensions to the access control language, and portability changes. Patches On the patch front, the calendar/reminder program remind from David F. Skoll had three patches posted. Patch 2 in Volume 35, Issues 18-20 added support for non-English languages and generally internationalized the code, added ISO encoding to the PostScript output, and allows for more flexible formatting of calendars. Patch 3 in Volume 35, Issue 73 fixes a bug in the RUNOFF command and some problems with the ISO PostScript output. Patch 4 in Volume 36, Issues 1-3 allows for reminders to be sorted in the normal mode of operation, additional languages, and a new SCANFROM clause to make "safe movable OMITs" possible. Raphael Manfredi posted patches to mailagent, a competitor of procmail. Patch set 17-19, posted as Volume 35, Issues 33-35, contains many bug fixes and enhancements, including better bogus address recognition, auto reply based on keyword rules, NOTIFY (a utility that accepts a list of addresses), security enhancements, and enhanced support for the From: line. A rather large patch to the xvi editor was posted by John Downey in Volume 35, Issues 98-104. Patch 1 fixed xterm/vt100 updating, XENIX and System V compile problems, and removed the fixed limit on line lengths in input files. In addition, several other bugs were fixed. The patch takes version 2.15 to version 2.19. Tony Field submitted patch 4 to his psf3 package for Volume 35, Issue 109. This patch to his PostScript print filter system corrects problems with ctl-D processing and some of the usage of the ISO-LATIN encoding. A small patch was made to James Clark's lprps PostScript print filter for the lpr print spooler. This change, patch 2, makes lprps ignore a leading ctl-D in the input stream such as that produced by the Microsoft Windows print driver. X Is Back As I said, all of a sudden, comp.sources.x has started flowing again, so here are some highlights, mostly of games. xtmines, another version of the minefield game, where adjacent squares tell you how many mines are touching your square, was contributed by Timothy Tsai for Volume 19, Issues 4-5. His additions include providing you with three hand grenades that can be used to cause mines to explode (causing all adjacent mines to explode in a chain reaction). Jonny Goldman contributed a space invaders game, xinvaders, for Volume 19, Issues 6-8. It is based on Terry Weissman's roids program and requires at least X11R4 to run. Steven Grimm posted his latest version of workman, a graphical tool for controlling audio CDs in the system's CD-ROM drive. It provides all the standard features of a programmable CD player, but also includes a database that can store the artist's name, disk title, and track titles, can remember which tracks you like and don't like, and will automatically recall this information when the appropriate CD is inserted in the drive. workman is Volume 19, Issues 9-17. A searchable od octal(hex/ascii, etc.) dump utility was contributed as xod by Robert Starr for Volume 19, Issues 18-21. It provides the ability to search based on bytes and strings, and the ability to change display formats on the fly. Like ncftp above in the comp.sources.misc group and XView's ftptool, Salim Alam has written and contributed a Motif-based, screen-oriented FTP client called xgetftp. It provides an X/Motif user interface, remote directory caching, remote file viewing, and simple multiple file and directory retrieval. It was posted as Volume 19, Issues 22-25. One of the more unusual programs for X I've seen is xodometer from Stephen O. Lidie . Posted in Volume 19, Issues 37-40, xodometer is a tool that tracks the total distance your mouse cursor moves since startup, and since the trip reset button was pushed. A strange concept, but then, perhaps it fits in nicely with the next program. Mark Martin submitted xruler for Volume 19, Issue 42. It creates a shaped window in the form of a ruler, or as a transparent grid, which can be used to take measurements in other windows. Strangely, it uses shape, xbiff, and awk instead of C to perform its work. NetHack Updated The big news in comp.sources.games is the update to NetHack, one of the largest projects in the games group, nethack31. The 3.1 update is the product of two years work by over 30 team members from different countries. Submitted by Izchak Miller , it spans all of Volume 16, Issues 1-108. Yes, that's about 6.5MB of source. NetHack 3.1 is a new generation of NetHack. Quoting the announcement posting, in Volume 15, Issue INFO11: "Unlike 3.0, NetHack 3.1 is a multi-dungeon game. New dungeons branch off the main dungeon at different locations. A unifying mythology was adopted for the game. There are tasks which must be completed in various dungeons before the main task of capturing the Amulet can be undertaken. There are different quests for the different character classes. There is a new multi-level endgame full of surprises. Many of the monsters in NetHack 3.1 now can wield weapons, zap wands, read scrolls, and cast spells. Fighting them must take these special abilities into consideration." The game's display was completely rewritten. Now based on a line-of-sight principle, it gives a new feel to the game. The new display uses a standardized window interface that allows easy substitution and addition of windowing systems independent of operating system. For example, the UNIX port now supports X11 as well as the traditional tty. Numerous local changes were made, and many new monsters, objects, and intelligent artifacts were added. And, as they say, you will discover the details and the rest of the changes for yourselves. Dave Cooper officially released the xjewel game previewed in alt.sources a while back for Volume 17, Issues 1-5. xjewel is an X port of the jewel dropping game from Yoshihiro Satoh of HP. This revision fixes some bugs and adds monochrome display support. dinkum3 is the latest revision in the Australian text adventure game series from Gary Allen . Posted in Volume 17, Issues 6-12, version 2.12 fixes several bugs and includes extensive revisions. Patch 1, with portability fixes for Macs and Coherent, was issued in Volume 17, Issues 35 and 36. The other big release this time is a new revision of Galactic Blodshed, called gbp for Galactic Blodshed+. Contributed by John Deragon , it also entails a change in authors to the gb+ development team. Posted in Volume 17, Issues 13-33, the entire installation and configuration script has been redesigned for this release. Previews from alt.sources As usual, plenty in alt.sources, so here are just a few highlights of what's to come in the main stream groups. Al posted the ACS circuit simulator in 20 parts on January 25, 1993. ACS is an interactive circuit simulator for both analog and digital circuits. It performs nonlinear DC and transient analyses, fourier analysis, and AC analysis linearized at an operating point. It is fully interactive and command-driven. It can also be run in batch mode that is mostly Spice-compatible, so it is often possible to use the same file for both ACS and Spice. Spice-compatible models for the MOSFET (level 2) and diode are included in this release. Other models (MOSFET levels 1 and 3 and BJT) are in the testing phase. Lance Norskog posted his ST, sound tools, package in eight parts (0-7) on February 8, 1993. ST translates sound samples between different file formats, and performs various sound effects. This release understands raw files in various binary formats--Sound Blaster .VOC files, IRCAM SoundFile files, SUN Sparcstation .au files, mutant DEC .au files, Apple/SGI AIFF files, Macintosh HCOM files, Sounder files, and Soundtool (DOS) files. The sound effects include changing the sample rate, adding echo delay lines, applying low- and band-pass filtering, and the infamous Fender Vibro effect. Sam Lantinga posted splitsh on February 22, 1993 in one part. splitsh splits the screen into two windows, each with its own shell, so you can do two things at once. Questions & Answers Dynamic Program Suspension Kenneth Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All On C, and was a member of the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durham, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also receives email at kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. Q We enjoy reading The C Users Journal, more than other programmers' magazines. We think we have detected a lack of, or need for, a dynamic program suspender. Perhaps a reader of CUJ might have a solution. We do not know how to write one or even how to suggest one could write it, but here is a list of its operational parameters: 1. Activate by a hot-key or time clock event. 2. Active program, in-memory data, and other system requirements stored on disk. 3. Normal MS-DOS environment becomes active (as if the previous program had terminated normally). 4. At a later time (whenever required), the suspended program could be reloaded from either a normal MS-DOS environment, or even spawned/executed from within an active program. 5. On resumption, program should continue execution as if nothing had happened. 6. Repeat above as required. This utility has been sorely missed by us for a while now. We often write small programs to amuse ourselves; however, they sometimes require days to complete. In a normal working environment this is not a welcome feature of any program. If we could use the spare time available, usually at night, to run our toy programs, we would be able to complete our toying. Example toy programs: PI to a gazillion decimal places PENTOMINO placing FACTORIZATION of large numbers CROSSWORD creation We thank you from Australia, Jim Eadie George Leake Melbourne, Australia A There are several ways to perform the operations you have requested. The simplest is to get a multitasking overlay to MS-DOS, such as DESQview or Microsoft Windows, or a multitasking operating system that has multiple DOS windows, such as OS/2. These systems allow you to run multiple programs simultaneously. They vary in the control that you have over the percentage of time that each program is allowed. The simplest one to set up and use is DESQview. I have used it for a number of years to perform operations similar to the ones you requested. DESQview provides true timesharing between programs that can fit into memory. Each program runs in its own window. The window that receives the keyboard input is the foreground window. The remainder of the windows are background windows. You can switch the foreground window at any time using the Alt key. Every clock tick (about 18 times/second), the executive is called and decides which program to execute next. In the setup procedure, you specify the percentage of time to be devoted to the foreground program and the background programs. If you specify a very small percentage or even zero percentage for a background program, then you will hardly notice that it is present. DESQview will use extended memory as well as conventional memory to run programs. If all the programs you are attempting to run cannot fit into memory, DESQview will swap out background programs to disk. When you switch to a swapped-out program, it will swap in that program and swap another out to disk. You have a fair amount of control over each window. You can specify that the program is not to be swapped out. This would be the case for a program that needs to respond to interrupts, such as a communications program. You can prevent the window from being closed until the program terminates. I use this for word-processing windows to be sure my files are saved before exiting DESQview. You can prevent the CPU from being shared when the program is the foreground program. You can also say that the program should not run when it is in the background. To use DESQview in your application, you can open up a window and run your long-running program in it. The window should be set up to run only when it is the foreground window, to be not closable, and to be swappable. Then open another window or several other windows and run your daily programs in them. When you are through for the day, simply switch to the long-running program. Alternatively, you could purchase or obtain a routine like X-SPAWN, which swaps the current program out to disk and spawns another program. When that program terminates, the current program is swapped back in. The program that executes could be COMMAND.COM, so a DOS command prompt would appear. When you typed EXIT, the previous program would be resumed. The program performing the XSPAWN would contain the algorithm you are computing. It could check periodically or on a signal for keystrokes. If a keystroke occurred, it would call XSPAWN. I prefer using the DESQview approach. Programs do not need to be aware that they are sharing the machine with any other programs. (KP). Error Handling Q This response comments on your March 1993 column, wherein James Brown explained how he uses gotos to consolidate error handling near the bottom of a routine. I often use a similar technique, which is illustrated in Listing 1. The basic concept is similar in that the code falls through terminating procedures. However, for the benefit of some purists, this method does not use gotos although it does use multiple returns. The code sample also provides for an error message although this is not necessary. This method is particularly handy where, as in the code sample, the cleanup code is not called from within the same function. Andy Levinson Studio City, CA A Thank you for your contribution. It demonstrates a nice way of centralized error handling without gotos. It also spurred me to think more about error handling. The complexity of error handling can be reduced if each object, as a buffer or a file, contains its own internal state. The state is set when an initialization function is called and then tested when a termination function is called. These functions need to be called explicitly, rather than implicitly, as is done in C++. Listing 2 shows an example following James Brown's listing in the March issue. The Buffer_initialize and Buffer_terminate routines are fairly simple. The state of a Buffer variable is kept in the variable itself. If the value of the variable is NULL, then no memory is currently allocated. The only difference between this and using just malloc and free is that the value of pointer is reset to NULL when it is freed in Buffer_terminate. Note that each Buffer variable has to be initialized before it can be terminated. However, it can be allocated unsuccessfully (i.e. Buffer_allocate returned FAILURE) and Buffer_terminate can be called without causing a problem. Listing 3 is an example that follows along the lines of your code. The File structure includes the current state and the buffer pointer. Normally, I would write the open and read operations as separate functions. They are kept together here to parallel your example. I expanded your implicit states of 1 and 2 to the four that are listed in enum File_state, even though the operations upon termination are the same in two cases. As I do with Buffer_initialize and Buffer_allocate, I would separate the operations of initializing and opening the file. This would add a line of code to initialize each File variable, but it would simplify the testing of other operations' return values. The code in Listing 4 shows how multiple File objects could be used in the same function. Each File variable has to be initialized. However, File_terminate can be called without erroneous results for variables that have not been successfully initialized. These functions might seem like overkill for the simple errors that can be generated with these types. However, the same technique can be applied to more complex objects. Listing 1 Error handling using multiple returns main() { char *error_ptr; if ((error_ptr = initialize()) != NULL) { fprintf(stderr, "error during %s", error_ptr); exit(1) } // processing ... (void) cleanup(2, NULL); exit(0); } int fd; char *buf; char *initialized(void) { if((buf = malloc(BUFSIZ)) == NULL) return cleanup(0, "malloc"); // uses cleanup for consistency if ((fd = open("filename", 0_RDONLY)) < 0) return cleanup(1, "open"); if (read(fd, buf, sizeof(buf) < 0) return cleanup(2, "read"); return NULL; } char *cleanup(int start_from, char *where_failed) { switch (start_from) { case 2: close(fd); case 1: free(buf); } // could instead print error message here // if where_failed ! = Null return where_failed; } /* End of File */ Listing 2 Keeping a state in the variable itself typedef char * Buffer; void Buffer_initialize(char ** pointer); int Buffer_allocate(char ** pointer, size_t size, char * name); void Buffer_terminate(char ** pointer); int some_function(void) { Buffer buffer1, buffer2; int ret1, ret2; Buffer_initialize(&buffer1); Buffer_initialize(&buffer2); if (Buffer_allocate(&buffer1, 1024, "buffer1") == SUCCESS && Buffer allocate(&buffer2, 1024, "buffer2") == SUCCESS) { /* Do whatever processing is required. */ } Buffer_terminate(&buffer1); Buffer_terminate(&buffer2); } void Buffer_initialize(Buffer * buffer) { *buffer = NULL: return; } int Buffer_allocate(Buffer * pointer, size_t size, char * name) { int return code = SUCCESS; *pointer = malloc(size); if (*pointer == NULL) { fprintf(stderr, "Allocation failure for %s size %d", name, size); return_code = FAILUE; } return return_code; } void Buffer_terminate(Buffer * pointer) { if {*pointer != NULL) { free(*pointer); *pointer = NULL; } return; } /* End of File */ Listing 3 Keeping a state in a structure enum File_state { UNOPENED, BUFFER_ALLOCATED, FILE_OPEN, RECORD_READ}; char * File_state_strings[] = {"Unopened", "Buffer allocation", "File open", "File reading"}; typedef struct { enum File_state state; // Current state of file int file_id; // Id from open char *buffer; // Buffer for read } File; int File_initialize(File * file, char *filename, char *purpose); void File_terminate(File * file); #define SUCCESS 1 #define FAILURE 0 main() { File file; int return_code; if (File_initialize(&file, "filename", "purpose") == SUCCESS) { return_code = 0; /* Do whatever required */ } else return_code = 1; File_terminate(&file); exit(return_code); } int File_initialize(File * file, char *filename, char *purpose) { int read_count; int return_code = FAILURE; file->state = UNOPENED; if (Buffer_initialize(&file->buffer, BUFSIZ, "file buffer") == SUCCESS) file->state = BUFFER_ALLOCATED; if (file->state == BUFFER_ALLOCATED) { file->file_id = open(filename, 0_RDONLY); if (file->file_id >= 0) file->state = FILE_OPEN; } if (file->state == FILE_OPEN) { read_count = read(file->file_id, file->buffer, BUFSIZ); if (read_count > 0) { file->state = RECORD_READ; return_code = SUCCESS; } } if (return_code != SUCCESS) fprintf(stderr, "Filename %s purpose %s error at %s", filename, purpose, File_state_strings[file->state]); return return_code; } void File_terminate(File * file) { /*** Maintenance note - fall through required ***/ switch(file->state) { case RECORD_READ: case FILE_OPEN: close(file->file_id); case BUFFER_ALLOCATED: Buffer_terminate(&file->buffer); case UNOPENED: break; default: fprintf("Error in File_terminate -- illegal state"); } return; } /* End of File*/ Listing 4 Error handling using multiple file objects in the same function main() { File file1, file 2; int return_code; int ret1, ret2; ret1 = File_initialize(&file1, "filename1", "purpose1"); ret2 = File_initialize(&file2, "filename2", "purpose2"); if (ret1 == SUCCESS && ret2 == SUCCESS)) { return_code = 0; /* Do whatever required */ } else return_code = 1; File_terminate(&file1); File_terminate(&file2); exit(return_code); } /* End of File */ Stepping Up To C++ Recent Language Extensions to C++ Dan Saks Dan Saks is the founder and principal of Saks & Associates, which offers consulting and training in C ++ and C. He is secretary of the ANSI and ISO C++ committees. Dan is coauthor of C++ Programming Guidelines, and codeveloper of the Plum Hall Validation Suite for C++ (both with Thomas Plum). You can reach him at 393 Leander Dr., Springfield OH, 45504-4906, by phone at (513) 324-3601, or electronically at dsaks@wittenberg.edu. As part of the C Users Journal's expanding coverage of C++, I'll be writing periodic reports on the progress of the C++ standard. This is the first such report. For the last three years, I've been writing these reports for The C++ Report. Over that time, the standards committee added several extensions to the C++ language. This article summarizes most of those extensions. ANSI (the American National Standards Institute) chartered technical committee X3J16 way back in late 1989 to write a US standard for the C++ programming language. X3J16 started working in early 1990, and has been meeting three times a year ever since. About a year later, JTC1, the joint technical committee of ISO (the International Standards Organization) and IEC (the International Electrotechnical Commission) chartered SC22/WG21 to produce an international C++ standard. WG21 has joined X3J16 for all of its meetings since the summer of 1991. I refer to the joint C++ committee as WG21+X3J16. I'm reluctant to predict when the standard will be complete. Formal approval is at least three years away. But the essential features of the language and libraries should be pretty stable much sooner than that. As you may have noticed from the brief biography below, I'm the secretary of both the ANSI and ISO C++ standards committees. I try to remain detached when I talk about the committees, but I will occasionally lapse into talking about the joint committee as "we." Where We Started Language standards often start from one or more base documents. That is, rather than write the standard from scratch, the committee adopts an existing written description of the language and libraries as the first draft. The committee then spends years refining that draft. These refinements include: reconciling inconsistent statements improving the precision of the words adding missing features X3J16 selected two base documents: the AT&T C++ 2.1 Product Reference Manual (the PRM) the C standard C++, like C, began at AT&T Bell Labs. AT&T started distributing a C++ compiler called cfront in the mid-1980s. The 2.1 PRM describes the language more or less as AT&T had implemented it by early 1990. Most C++ programmers know the PRM as the Annotated C++ Reference Manual, or ARM (Ellis and Stroustrup 1990). The ARM includes the text of the PRM along with annotations and commentary that elaborate the language description, explain many language design decisions and suggest implementation techniques. However, AT&T (which holds the copyrights to both the PRM and the ARM) only granted the C++ committee the right to use the PRM, so the current draft of the standard does not include the annotations and commentary. The committee's first decisions brought the Working Paper closer to the ARM. The ARM includes chapters on templates and exception handling. cfront 2.1 did not include these features, so the PRM, and hence the base document, only had empty place holders for those chapters. The committee added these chapters to the Working Paper during its first year, so for all practical purposes, it's fair to say that the ARM is the base document. X3J16 selected the ANSI C Standard as its second base document in the hope of minimizing unnecessary differences between C++ and C. I think some of us on the committee expected that, whenever C++ and C share a feature, the C++ standard would use the same, or nearly the same, wording as the C Standard. But to date, the C++ draft still employs wording that is closer to the ARM, even when describing features from C. By the way, the politically correct term for the C++ draft is not the "draft," but the "Working Paper." The Working Paper won't be a draft until the committee submits it for public review. The first public review is at least a year away. Committee Groups The joint committee conducts most of its technical work in smaller ad hoc groups. These groups are: C Compatibility--This group compares the C++ Working Paper with the C Standard and makes recommendations for reconciling their differences. Core Language--This group resolves ambiguities and inconsistencies in the core features of the language as specified by the PRM. This group recently divided into smaller groups: the "hard" core group tackles the broader issues that take many meetings to resolve, and the "soft" core group addresses a numerous collection of smaller issues. Environments--This group focuses on requirements for the translation environment (including the preprocessor and linker) and the execution environment (including program startup, execution, and termination). Extensions--This group evaluates proposals to add new language features to C++. It makes recommendations for accepting some and provides rationale for rejecting others. (WG21+X3J16 considers templates and exceptions to be extensions.) Libraries--This group drafts the specification of standard library components, including iostreams and the C++ version of the Standard C library. Syntax--This group recommends refinements to the formal C++ grammar to make the language more precise and easier to parse. The New Stuff In addition to templates and exception handling, the committee added a few other new features to C++: support for European translation environments wchar_t as a keyword and a distinct type operator overloading for enumeration types overloading of new and delete operators for arrays relaxed restrictions on the return types for virtual functions runtime type identification None of these features is universally available. Of the MS-DOS compilers I own, only three provide templates (Borland, Comeau, and MetaWare), and only one supports overloading on enumerations (Microsoft). I will briefly summarize these extensions here, and cover them in detail in future columns. Templates Templates come in two flavors: function templates and class templates. A function template defines a family of overloaded functions employing the same algorithm, but applied to operands of different types. For example, template inline T abs(T x) { return x >= 0 ? x : -x; } defines a template for the absolute value function abs. The template itself doesn't generate any object code. Rather, the translator instantiates (creates) an abs function for a particular parameter type the first time you call abs with an argument of that type. For instance, if you call abs(i) for int i, the translator creates a definition for abs(int) as if you had defined inline int abs(int x) { return x >= 0 ? x : -x; } If you later call abs(f) for float f, the translator instantiates inline float abs(float x) { return x >= 0 ? x : -x; } Class templates provide a similar facility for classes. For example, in my recent columns on dynamic arrays (CUJ, November 1992) and operator[] (CUJ, January 1993), I wrote a class for a dynamic array containing elements of type float. A version of the float_array class definition appears in Listing 1. Rather than rewrite the class for each different element type, you can write a single class template for a dynamic array with an arbitrary element type. The template appears in Listing 2. Using that template, array ai (n); declares a dynamic array with n int elements, and typedef array float_array; defines type float_array as an instance of the class template. Exception Handling Exception handling provides an orderly way to respond to disruptive events that occur during program execution. C++ exception handlers can handle synchronous, rather than asynchronous, events. Synchronous events are the kinds of problems you can detect by conditional expressions (as in an if statement or the argumentto a call on the assert macro). For example, p = new T; if (p == 0) // you're out of memory An asynchronous event is one triggered by an external event, like a device interrupt or a hardware fault. The standard header defines the C library facilities for handling asynchronous events. Listing 3 sketches a simple exception handler. C++ reserves three new keywords for the syntax of exception handling: try, catch, and throw. A try-block is a compound-statement (a sequence of statements enclosed in brackets) followed by a sequence of one or more handlers, also known as catch clauses. The handlers "catch" exceptions "thrown" by throw expressions executed in the compound-statement or in functions called from within the compound-statement. Each catch clause in a try-block handles an exception of a different type. The exception-handling mechanism matches the type of the expression in the throw expression with the formal arguments in the catch clauses. Throwing an exception terminates the block throwing the expression and transfers control to the chosen catch clause. In addition, throwing an exception "unwinds" the runtime stack. That is, it terminates all active functions invoked from the try-block and deallocates their local variables. For example, function h in Listing 3 contains a try-block that calls function g. If g detects a problem, it throws an exception of type XXX which is caught by the third catch clause in h. Otherwise, g calls f. f may throw an exception of type int, which will be caught by the first catch clause in h. Some of you may have noticed that exception-handling behaves much like the Standard C functions setjmp and longjmp. The key difference is that throwing an exception invokes destructors for local objects as it unwinds the stack on the way back to the catch clause, whereas longjmp merely discards intervening stack frames as it returns to the setjmp point. longjmp works in C++ as long as it never terminates a function with local variables that have destructors. A European Representation for C++ WG14 (ISO C) recently approved a normative addendum to the ISO C standard. (See P. J. Plauger's "Formal Changes to C," CUJ, April 1993). The addendum includes a proposal from the Danish delegation to add a way for programmers to write C programs using only the invariant ISO 646 characters. This need arose because C uses the US ASCII character set as its alphabet. ASCII is the US national variant of the ISO 646 standard character set. Some or all of the ASCII characters [] { } ^ ~aren't available in non-US variants of ISO 646. Other national variants of ISO 646 replace some of these characters with native language characters, making C programs a bit harder to write and read. The normative addendum to the C Standard corrects the problem by adding new spellings as alternates for tokens that use ASCII-specific characters. The new spellings only use characters from the invariant ISO 646 character set. Recognizing that C++ shared this problem with C, the C++ committee also adopted a set of alternate spellings. However, C++ adopted a slightly different set. Table 1 lists the C and C++ alternatives, noting the differences. At one time the sets were the same, but they got out of sync. I believe the committees will try to reconcile the differences. In C, the alternate spellings that are identifiers (everything from bitand on down in Table 1) are macros defined in the new standard header . In C++, these new spellings are keywords (reserved for all uses by the language). C++ also provides for compatibility with C, but on many C++ implementations, it might just be empty. wchar_t as a Keyword Standard C provides wide characters and multibyte strings so that C programmers can manipulate very large character sets, like Japanese Kanji. Several headers in the Standard C library define wchar_t as the wide character type. wchar_t must be an integral type sufficiently large to represent all character codes in the largest character set among the supported locales. Defining wchar_t as a typedef poses a problem for C++. Members of the libraries group wanted to be able to overload functions with arguments of type wchar_t, such as int foo(int); int foo(wchar_t); In particular, they wanted to be able to overload the output operators ostream &operator<< (ostream &os, char c); ostream &operator<< (ostream &os, int i); ostream &operator<< (ostream &os, wchar_t w); so that given int i; wchar_t w; the expression cout << i; displays i as an int, and cout << W; displays w in its proper graphic representation. The problem is that, if wchar_t is a typedef, it is indistinguishable from at least one of the other integral types. In C+ +, a typedef is not a distinct type; it is an alias for another type. If the library defines typedef wchar_t int; then ostream &operator<<(ostream &os, int i); ostream &operator<<(ostream &os, wchar_t w); are the same function. That function will (most likely) display objects of type wchar_t as numbers. Thus, WG21+X3J16 made wchar_t a new keyword in C++ representing a distinct type. wchar_t is still represented the same way as one of the standard integral types (meaning it has the same size, alignment requirements, and "signedness" as one of the integral types), but it is now a distinct type for the purposes of overload resolution. Operator Overloading for Enumerations In C, enumerations are integral types. The ARM also states that enumerations are integral types in C++. However, the current C++ Working Paper says that "Enumerations are not integral, but they can be promoted to signed and unsigned ints." Thus, Standard C code such as enum color { RED, WHITE, BLUE }; enum color c = 0; is no longer C+ +. (I don't think this is any great loss; it was always poor style.) An assignment like int n = BLUE; which is valid C, is still valid C++. This change introduces a more serious incompatibility with C: the predefined arithmetic operators, most notably ++, no longer apply to enumerations. Therefore, a loop such as for (c = RED; c <= BLUE; ++c) no longer compiles as C++. (This is a loss because it can be good style.) The present wording of the Working Paper doesn't seem to allow c <= BLUE But I believe it is, in fact, allowed because both operands promote to int. WG21+X3J16 reduced the trauma of this incompatibility by extending C++ to permit overloading on enumerations. For example, you can write the for loop above by defining inline color &operator++(color &c) { return c = color(c + 1); } as the prefix form of ++ for objects of type color. Overloading new and delete for Arrays In C, you allocate dynamic memory using the standard library functions malloc (or calloc or realloc), and you deallocate memory by calling free. In C++, you use the new and delete operators instead. Both malloc and new allocate memory, but new also applies a constructor to the storage if the allocated object has a class type with a constructor. That is, if X is a class with a constructor p = new X; allocates and constructs an X object, but p = (X *)malloc(sizeof(X)); merely allocates storage and leaves the object unconstructed. Similarly, both free and delete deallocate storage, but only delete applies a destructor (if any) to the object just before deallocating it. Each C++ environment provides a default implementation for the new and delete operators. However, if this general-purpose allocator isn't right for your application, you can write you own versions of new and delete. For example, void *operator new(size_t n) { ... } defines a replacement allocation function, so that calling the new operator calls this allocator instead of the system-supplied allocator. Similarly void operator delete(void *p) { ... } defines a replacement deallocator for use by the delete operator. C++ not only lets you replace the global dynamic memory allocator, but even lets you define a different allocator for each class. That is, class Y { public: void *operator new(size_t); void operator delete(void *p); ... }; defines class Y with its own special-purpose versions of new and delete. A call such as q = new Y; allocates memory using Y::operator new, rather than the global operator new, and delete q; deallocates memory using Y::operator delete. Those classes that do not define their own new and delete use the global operators. The ARM states that, even for a class Y that defines its own operator new, allocating an array of Y objects always uses the global operator new. That is, q = new Y[n]; ignores Y::operator new and uses ::operator new. Consequently, deleting that array using delete [] q; ignores Y::operator delete and uses ::operator delete. The committee recently extended C++ to provide a separate set of dynamic memory management functions for arrays of objects: void *operator new[](size_t n); void operator delete[](void *p); With this extension, p : new X[m]; allocates memory using operator new[] instead of operator new, and delete [] p; uses operator delete[] instead of operator delete. You can even define operators new[] and delete[] for an individual class, such as class Y { public: void *operator new(size_t); void *operator new[] (size_t); void operator delete(void *p); void operator delete[](void *p); ... }; so that q = new Y[n]; uses Y::operator new[] instead of Y::operator new. Enhanced OOP Support The standards committee enhanced object-oriented programming (OOP) in C++ by relaxing restrictions on the return types for virtual functions adding runtime type identification C++ supports OOP by providing inheritance and virtual functions. I described inheritance in a recent two-part tutorial ("Inheritance, Part 1," CUJ, March 1993 and "Inheritance, Part 2," CUJ, May 1993). I will explain virtual functions, and these extensions, in future columns. Reference Ellis, Margaret A. and Stroustrup, Bjarne. 1990. The Annotated C++ Reference Manual. Reading, MA: Addison-Wesley. Table 1 Alternate spellings for tokens that use ASCII-specific characters Existing C C++ Different Token Alternate Alternate ------------------------------------------ [ <: <: ] :> :> { <% <% } %> %> # %: %% <- ## %:%: %%%% <- & bitand bitand && and and bitor bitor or or ^ xor xor ~ compl compl &= and_eq and_eq = or_eq or_eq ^= xor_eq xor_eq ! not not != not_eq ne <- Listing 1 A version of the float_array class definition // fa.h - a dynamic array of float #include class float_array { public: float_array(size_t n = 0); float_array (const float_array &fa); ~float_array(); float_array &operator= (const float_array &fa); float &operator[] (size_t i) const; size_t length() const; private: float *array; size_t len; }; /* End of File */ Listing 2 A class template for a dynamic array // da.h #include template class array { public: array(size_t n = 0}; array(const array &fa); ~array(); array &operator= (const array &fa); T &operator[](size_t i) const; size_t length() const; private: T *array; size_t len; }; /* End of File */ Listing 3 A simple exception handler #include class XXX { ... }; int f() { int i; // ... if (... something wrong ...) throw 2; } int g() { XXX x; unsigned long ul; // ... if (... something else wrong ...) throw x; return f() } int h() { try { // ... g() return 0; } catch (int n) { cerr<< "#"<< n <<" happened\n"; return n; } catch (char *s) { cerr << s << " went wrong\n"; return -1; } catch (const XXX &x) { cerr << x << " went wrong\n"; return -1; } } /* End of File */ Code Capsules File Processing, Part 2 Chuck Allison Chuck Allison is a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. Portability with POSIX In the early 1980s an organization called /usr/group (the UNIX Users' association, now called Usenix) began an effort to standardize both C and the C programming environment. Much of that they defined applied to all environments and became the basis for the Standard C library. Another part of their work resulted in a set of functions to access UNIX system services (UNIX was the most common C programming environment at the time). These functions constitute the C language bindings of what now is called POSIX (Portable Operating System Interface). Fortunately, many environments (including MS-DOS and of course, the many flavors of UNIX) provide most or all of these functions. POSIX compliance, therefore, can be important when moving applications from one platform to another. Here is a simple recipe for maximizing portability: 1. Program in Standard C. 2. If step 1 is too restrictive, use only POSIX-compliant functions. 3. If steps 1 and 2 aren't possible, isolate system-dependent code in separate modules. This will minimize how much code you'll have to rewrite when porting to another system. There are cross-platform tools available that do some of this work for you. There is of course much more to POSIX than C language bindings (bindings for other languages and a specification for a command shell, for example). This month's article illustrates most of the POSIX functions that pertain to file processing. I tested Listing 1, Listing 2, and Listing 3 with Borland C++ 3.1, Zortech C 3.0, Microsoft C 6.00A, and Mix Software's POWERC 2.20. (If you use Microsoft C/C++ 7.0, you must compile with OLDNAMES. LIB.) Of these four compilers, only Borland's supports the directory I/O functions required for Listing 4, Listing 5, Listing 6, and Listing 7. All seven listings should run on any UNIX platform (with some obvious modifications as described in the comments). POSIX File I/O The definition of the FILE structure in your stdio.h include file probably contains an integer that represents a file handle (or file discriptor). (Look for a member named something like fd, _file, or fildes). A file handle, a unique, non-negative integer assigned by the file system, identifies an access path into a file. POSIX file-access functions use file handles instead of file pointers to perform basic operations comparable to those offered in the Standard C library, but with typically less overhead. Table 1 contains a comparison of POSIX and Standard C functions. Other POSIX file access-functions are listed in Table 2. Copying Files cat.c (Listing 1) copies the files indicated on the command line to standard output. For example, the command cat file1 file2 >file3 combines file1 and file2 into a new file, file3. This program uses Standard C functions for reading and writing. The only POSIX functions are in the line FILE *std_out = fdopen (fileno(stdout), "wb"); This line enables writing to standard output in binary mode, in case one of the user files is not a text file. (See last month's "Code Capsule" for a discussion of binary mode.) It associates a new file pointer with standard output, without creating a new handle. (In other words, the file pointers stdout and std_out share the same file handle.) The function filecopy (Listing 2) opens an input and an output file in binary mode. The open system call returns -1 if the open fails. (In fact, most POSIX functions return -1 upon failure.) fcntl.h defines the flags used to define INPUT_MODE and OUTPUT_MODE. The third argument in the open of the output file specifies that if the file doesn't already exist, the newly created file should not be write-protected. (sys/stat.h defines S_IWRITE.) Any include file prefixed with sys/ is a POSIX include file (although many, such as io.h and fcntl.h, have no prefix). On most systems, you need to include sys/types.h before sys/stat.h. The read and write functions both return the number of bytes transferred. The program cp.c in Listing 3 uses filecopy to copy one or more files to a given directory. The stat function fills a structure with basic file information, including these members: set_mode file mode (directory indicator, file permissions) st_size file size in bytes st_mtime time of last data modification Testing st_mode with the mask S_IFDIR (from sys/stat.h) determines whether the file is a directory. The forward slash character is a directory separator for pathnames in all POSIX systems. (Note the sprintf statement in function cp.) Only the command line of the MS-DOS shell (COMMAND. COM) requires a backslash character as the separator character. Both slash characters are totally interchangeable within MS-DOS programs. For maximum portability, the names of files and directories should only use characters from the portable filename character set: alphanumerics, the period, and the underscore. Reading Directory Entries The most widely-used operating systems that support C development today have a hierarchical directory structure. POSIX defines functions to create, delete, and navigate among directories, as well as functions to read the entries in a directory (see Table 3). The program list.c in Listing 4 prints a listing of the current directory to standard output. To read a directory, you must first get a pointer to a DIR structure with the opendir function. Successive calls to readdir return a pointer to a struct dirent structure (which contains the entry name) or NULL when all entries have been read. These structures and functions are declared in dirent.h. As shown in Figure 1, list displays the name, permissions, size, and time of last modification of each file. The characters in the permissions column mean d--the entry is itself a directory r--the user has read permission w--the user has write permission The file system creates the first two entries: "." refers to the current directory and ".." to its parent. You can't alter them directly (hence no w permission). The token / in POSIX functions (and \ in MS-DOS's) refers to the root directory. Since the modification time is a standard time value (time_t), I use the Standard C function ctime to display it. (See the "Code Capsule" in the January 1993 issue for a discussion of time and date functions). The program findfile.c in Listing 5 searches a directory tree for all occurrences of a specified entry. On my MS-DOS system, the command findfile himem.sys \ searches the entire disk for the file himem.sys, and prints \dos\himem.sys \windows\himem.sys To restrict the search to a specific directory tree, change the second argument, for example, to findfile himem.sys \dos To make a UNIX version of this utility, replace the defined constants with #define SEPSTR "/" #define OFFSET 0 OFFSET skips over the disk identifier and colon (e.g., C:) that precede the full pathname returned by getcwd under MS-DOS. The function call getcwd(NULL,FILENAME_MAX) returns a pointer to a dynamically-allocated string that represents the current working directory (FILENAME_MAX is defined in stdio.h). If access(file,0) returns 0, then file exists. (file can also be a directory name.) chdir(dir) makes dir the current working directory. visit_dirs, a recursive function that visits all subdirectories in a directory tree, restores the original working directory when it returns. Redirecting Standard Error When you enter a command such as cat file1 file2 >file3 the command shell disconnects the internal file handle for standard output from the console and connects it to file3 before it loads the program cat. When cat terminates, the shell reconnects the handle to the console. Listing 6 shows how to do the same thing with standard error. The function redir_stderr gets a handle to the new destination by calling open. Then it creates a new handle to the original destination with dup (this is for restoring it later). Finally, it redirects the output by disconnecting it from the original destination and connecting it to the new one with dup2. When you don't need the redirection anymore, call restore_stderr, redirects standard error back to its original destination and discards the duplicate handle. A handle created with dup is "synchronized" with the original, so that if you change file position with lseek on one handle, the position is updated for the other handle also. You may be wondering why you can't just use a call to freopen to redirect standard error. This works fine within a single program, but freopen has no effect if you initiate another program from within your program (using system, say), because only the local file pointer changes. To have such changes persist across subprocesses you must use dup2. The program ddir.c in Listing 7 illustrates most of the concepts mentioned in this article. It deletes an entire directory tree by following these steps: 1. Make the root of the tree the current working directory. 2. Delete all files within that directory with a shell command (in this case the MS-DOS del command). 3. Any entry left in the directory is either a protected file or a subdirectory. For a protected file, lower the protection with chmod and delete the file explicitly with unlink. (You can use remove in MS-DOS, but you shouldn't in UNIX; unlink works for both.) For a subdirectory, recursively repeat the whole process starting with step 1) on the subdirectory. 4. Ascend to the parent directory and delete the directory in question with rmdir. All three of standard input, standard output, and standard error are redirected by the time the shell command passed to system is executed. When you issue a shell command to delete all files in a directory (del *. *), MS-DOS prompts you for a confirmation (Y or N). To bypass this prompt, I redirect standard output to the null device (the "bit bucket") so it doesn't appear, and create a file with the letter Y in it and redirect standard input to come from that file, so there is no pause for console input. If any files can't be deleted (or if there are none to delete), MS-DOS sends a message to standard error. I redirect standard error to the null device before calling system so these messages also will not appear. The functions in this article give you the control over your environment which is critical for robust applications. While these techniques aren't universally portable, they apply to the many platforms that are POSIX compliant (or soon will be, e.g., OpenVMS and Windows NT). Defensive Programming with assert At key points in well-organized programs you can make assertions, such as "the index points to the next open array element." You should test these assertions during development with the ANSI C assert macro, and document them for the maintenance programmer (who is often yourself). You could represent the assertion above, for example, as #include . . . assert(nitems < MAXITEMS && i == nitems); . . . If the condition holds, all is well and execution continues. Otherwise, assert prints a message containing the condition, the file name, and line number, and then calls abort to exit the program. Use assert to validate the internal logic of your program. If a certain thread of execution is clearly impossible, say so with a call assert (0): switch (color) { case RED: . . . case BLUE: . . . case GREEN: . . . default: assert (0); } Every function essentially has a contract with its users to "give me input that satisfies certain conditions and I'll give you this output." You can enforce the contract by validating parameters with assert. A function that takes a string argument, for example, should make sure the string really exists: char *f(char *s) { assert(s); . . . Assertions are for programmer errors, of course, not for end-user errors. Nothing a user does should create a NULL pointer; that's clearly your fault, so it is appropriate to use assert in such cases. My custom is to use asserts generously throughout my code and then replace assertions that might be affected by the user with more bulletproof exception handling. Any remaining assertions should not be needed in released code (because it's debugged, right?), so I define the macro NDEBUG to disable them. You can either include the statement #define NDEBUG in the beginning of the code, or define the macro on the command line if your compiler allows it. (Most support the -D option for this purpose.) With NDEBUG defined, all assertions expand to a null macro, but the text remains in the code for documentation. Figure 1 A sample directory listing . dr- 0 Sat Mar 06 14:29:34 1993 .. dr- 0 Sat Mar 06 14:29:32 1993 ddir.c -rw 2066 Thu Feb 18 08:26:14 1993 cp.c -rw 899 Thu Mar 04 21:11:58 1993 findfile.c -rw 1512 Thu Mar 04 21:13:22 1993 ls.c -rw 933 Tue Feb 16 10:57:36 1993 filecopy.c -rw 967 Tue Feb 16 21:17:02 1993 cat.c -rw 794 Mon Feb 15 19:05:16 1993 stderr.c -rw 562 Mon Feb 15 19:05:18 1993 stderr.h -rw 161 Mon Feb 15 19:05:18 1993 posix.wpf -rw 14870 Sun Mar 07 20:30:42 1993 ls.exe -rw 12820 Sun Mar 07 20:31:02 1993 Table 1 A correspondence between POSIX and Standard C file access functions POSIX Standard C ---------------------- close fclose creat fopen dup2 freopen lseek fseek open fopen tell ftell write fwrite Table 2 Other POSIX file access functions Function Description ------------------------------------------------------------ access Check file existence and permissions dup Duplicate a file handle (in-sync with original) chmod Change file permissions fdopen Associate a FILE * with a file handle fileno Extract the file handle from a FILE * fstat Get file info (from a FILE *) stat Get file info (from a filename) unlink Same as remove() on non-UNIX systems Table 3 POSIX directory access functions Function Description -------------------------------------------------------------- chdir Change the current working directory closedir Stop reading a directory getcwd Get name of the current working directory mkdir Create a new directory opendir Open a directory for reading readdir Get the next directory entry rewinddir Reset directory read pointer back to the beginning rmdir Delete a directory (must be empty) Listing 1 Concatenates files to standard output /* cat.c: concatenate files */ #include #include void copy(FILE *, FILE *); main(int argc, char **argv) { int i; FILE *f; FILE *std_out = fdopen(fileno(stdout),"wb"); assert(std_out != NULL); for (i = 1; i < argc; ++i) if ((f = fopen(argv[i],"rb")) == NULL) fprintf(stderr,"cat: Can't open %s\n",argv[i]); else { copy(f,std_out); fclose(f); } return 0; } void copy(FILE *from, FILE *to) { size_t count; static char buf[BUFSIZ]; while (!feof(from)) { count = fread(buf,1,BUFSIZ,from); assert(ferror(from) == 0); assert(fwrite(buf,1,count,to) == count); assert(ferror(to) == 0); } } /* End of File */ Listing 2 Copies files via handles /* filecopy.c: Low-level file copy */ #include #include #include #include #define BUFSIZ 512 #define INPUT_MODE (O_RDONLY O_BINARY) #define OUTPUT_MODE (O_WRONLY O_BINARY O_CREAT) int filecopy(char *from, char *to) { int nbytes; int status = -1; int fd1 = open(from,INPUT_MODE); int fd2 = open(to,OUTPUT_MODE,S_IWRITE); static char buffer[BUFSIZ]; if (fd1 >= 0 && fd2 >= 0) { status = 0; while ((nbytes = read(fd1,buffer,BUFSIZ)) > 0) if (write(fd2,buffer,nbytes) != nbytes) { /* Write error */ status = -1; break; /* Write error */ } /* Was there a read error? */ if (nbytes == -1) status = -1; } if (fd1 >= 0) close(fd1); if (fd2 >= 0) close(fd2); return status; } /* End of File */ Listing 3 Copies files to a directory /* cp.c: Copy files */ #include #include #include #include extern int filecopy(char *, char *); static void cp(char *, char *); main(int argc, char **argv) { int i; struct stat finfo; char *target = argv[argc-1]; /* Make sure target is a directory */ assert(argc >= 3); assert(stat(target,&finfo) == 0 && (finfo.st_mode & S_IFDIR)); /* Copy files */ for (i = 1; i < argc-1; ++i) cp(argv[i], target); return 0; } static void cp(char *file, char *target) { static char new newfile[FILENAME_MAX]; /*Combine target and source file for a full pathname */ sprintf(newfile,%s/%s",target,file); fprintf(stderr,"copying %s to %s\n",file,newfile); if (filecopy (file,newfile) != 0) fputs("cp: Copy failed\n",stderr); } /* End of File */ Listing 4 List entries in the current directory /* list.c: Print a directory listing */ #include #include #include #include #include #include #include static char *attr_str(short attr); main() { DIR *dirp = opendir("."); /* Current dir */ struct dirent *entry; struct stat finfo; assert(dirp); while ((entry = readdir(dirp)) != NULL) { stat(entry->d_name,&finfo); printf( "%-12.12s %s %8ld %s", strlwr(entry->d_name), attr_str(finfo.st_mode), finfo.st_size, ctime(&finfo.st_mtime) ); } closedir(dirp); return 0; } static char *attr_str(short attr) { static char s[4]; strcpy(s,"---"); if (attr & S_IFDIR) s[0] = 'd'; if (attr & S_IREAD) s[1] = 'r'; if (attr & S_IWRITE) s[2] = 'w'; return s; } /* End of File */ Listing 5 Searches a directory tree for a specific entry /* findfile.c: Search all directories for a file */ #include #include #include #include #include #include #include #include #include /* Change these for UNIX */ #define SEPSTR "\\" #define OFFSET 2 void visit_dirs(char *dir, char *file); main(int argc, char *argv[]) { if (argc > 1) { char *file = argv[1]; char *root = (argc > 2) ? argv[2] : SEPSTR; visit_dirs(root,file); } return 0; } void visit_dirs(char *dir, char *file) { DIR *dirp; struct dirent *entry; struct stat finfo; char *curdir = getcwd(NULL,FILENAME_MAX); /* Enter the directory */ assert(chdir(dir) == 0); /* Process current directory */ if (access(file,0) == 0) { char *tempdir = getcwd(NULL,FILENAME_MAX); char *sep = strcmp(tempdir+OFFSET,SEPSTR) ? SEPSTR : ""; printf("%s%s%s\n", strlwr(tempdir+OFFSET),sep,strlwr(file)); free(tempdir); } /* Descend into subdirectories */ assert((dirp = opendir(".")) != NULL); while ((entry = readdir(dirp)) != NULL) { if (entry->d_name[0] == '.') continue; stat(entry->d_name,&finfo); if (finfo.st_mode & S_IFDIR) visit_dirs(entry->d_name,file); } /* Cleanup */ closedir(dirp); chdir(curdir+OFFSET); free(curdir); } /* End of File */ Listing 6 Redirects standard error /* stderr.c: Redirect stderr */ #include #include #include #include #include #include static int old_handle = -1; int redir_stderr(char *fname) { int fd = open(fname,0_WRONLY0_CREAT0_TEXT,S_IWRITE); assert(fd >= 0); old_handle = dup(fileno(stderr)); dup2(fd,fileno(stderr)); close(fd); return fd; } void restore_stderr() { if (old_handle != -1) { dup2(old_handle,fileno(stderr)); close(old_handle); old_handle = -1; } } /* End of File */ Listing 7 Deletes a directory tree /* ddir.c: Remove directory tree */ #include #include #include #include #include #include #include /* All POSIX-compliant systems have this one */ #include /* Borland C also requires this one */ #include /* This is required to redirect stderr on DOS */ #include "stderr.h" /* DOS-specific macros - change for other OS */ #define CMD_FORMAT "del *.* <%s > nul" #define CMD_LEN 17 /* Change this to "/" for UNIX */ char Response_file[L_tmpnam+1] = "\\"; void rd(char *); main(int argc, char **argv) { FILE *f; char *old_path = getcwd(NULL,FILENAME_MAX); /* Create response file for DOS del command */ tmpnam(Response_file+1); assert((f = fopen(Response_file,"w")) != NULL); fputs("Y\n",f); fclose(f); /* Delete the directories */ while (--argc) rd(*++argv); /* Clean-up */ remove(Response_file); chdir(old_path); free(old_path); return 0; } void rd(char * dir) { char sh_cmd[L_tmpnam+CMD_LEN]; DIR *dirp; struct dirent *entry; struct stat finfo; /* Log onto the directory that is to be deleted */ assert(chdir(dir) == 0); printf("%s:\n",strlwr(dir)); /* Delete all normal files via OS shell */ hide_stderr(); sprintf(sh_cmd,CMD_FORMAT,Response_file); system(sh_cmd); restore_stderr(); /* Delete any remaining directory entries */ assert((dirp = opendir(".")) != NULL); while ((entry = readdir(dirp)) != NULL) { if (entry->d_name[0] == '.') continue; stat(entry->d_name,&finfo); if (finfo.st_mode & S_IFDIR) rd(entry->d_name); /* Subdirectory */ else { /* Enable delete of file, then do it */ chmod(entry->d_name,S_IWRITE); assert(unlink(entry->d_name) == 0); } } closedir(dirp); /* Remove the directory from its parent */ assert(chdir("..") == 0); assert(rmdir(dir) == 0); } /* End of File */ Get Quick, Professional Plots with GNUPLOT Roger T. Stevens After 42 years as an electronics and systems engineer, Dr. Roger T. Stevens is devoting full time to writing computer works. He has nine books on graphics and fractal programming for the PC currently in print, published by M&T Publishing, Inc. and Academic Press. Dr. Stevens holds a B.A. degree in English from Union College, an M.A. in Mathematics from Boston University, an M.Eng. in Systems Engineering from Virginia Tech, and a Ph.D. in Electrical Engineering from California Western University. Many fine software programs have been developed in our colleges and universities, but because of the academic fascination with large mainframe computers, or at the very least with high-powered workstations, little has trickled down that is of use to the PC community. GNUPLOT, a fine plotting program that has been tailored to work with IBM compatible PCs along with other kinds of machines, stands out as a clear exception. Let's hope that this will be only the beginning of a stream of high quality PC software from academia. GNUPLOT is a public domain function plotting program that will plot any equation of the form y = ƒ (x) or any parametric equation of the form x = ƒ1(t) y = ƒ2(t) GNUPLOT will also plot any curve that you can express by giving the coordinates of x and y for a number of points on the curve. GNUPLOT automatically provides axes and tics or a grid. It labels the grid lines or tics with the appropriate values and provides a heading for the graph. GNUPLOT control commands permit you to specify a range of values and otherwise control the nature of the graph that is drawn. GNUPLOT was originally written by Thomas Williams and Colin Kelley and has been modified by Russell Lang, John Campbell, and David Kotz of the Duke University Computer Science Department. Getting Started GNUPLOT comes on three floppy disks, which contain, among other things, all of the C source code for the GNUPLOT program together with make files that should enable you to compile and link the program. There are make files for Microsoft C and for Borland Turbo C. I used Borland C++ version 3.1 for my tests. I set up a directory on my hard disk called gnuplot and loaded the contents of the three floppies to it. The first thing that I discovered was that I had to modify the Turbo C make file before anything would run, as Turbo C uses tcc as the name of the compiler program, whereas in Borland C++ it is called bcc. The make file requires two subdirectories--term, which should have all the terminal definition files in it, and does, which should have several documentation files. Instruction Material The file GNUPLOT.DOC gives a detailed description of each GNUPLOT command. You'll find this file helpful when trying to produce a graph, but it doesn't really take the place of a full-fledged instruction book that would tell you how to run the program and give you a number of examples of how to use each feature. You'll need to do a lot of experimentation to discover what features are available in the program and how to use them most effectively. Using GNUPLOT GNUPLOT is an interactive program, which means that, as with BASIC, you can type in commands and have the program operate upon them immediately. The only problem with this is that after you tell GNUPLOT to plot a graph and it displays the graph, when you return to the program to type in another command, you are presented with a fresh screen with all of your previous commands gone. Thus if you have made some small error and ended with a graph that wasn't what you wanted, you have no idea of what was on the line that you typed in to produce this graph. Fortunately, if you begin GNUPLOT by typing gnuplot followed by a file name, GNUPLOT will read its input data from this file. You'll probably want to use this method, since you can use any text editor that can produce an ASCII output to create a file of commands that will be preserved and can be edited to correct any errors. I began by attempting to plot a simple sin(x)/x function. The command used was plot [0:8*pi] sin(x)/x The resulting plot is shown in Figure 1. The quantities in brackets following the plot statement give the range of values for x. Following this is the expression to be evaluated for y. Next, I got out my old analytical geometry book to find an interesting equation for the next try. What I found was the plot of two curves: y = x (x2 - 4) y2 = x (x2 - 4) I wrote the following GNUPLOT commands to plot these curves: set grid plot [-2.5:4] [-5:5] x*(x*x-4),sqrt(x*(x*x-4)), -sqrt(x*(x*x-4)) with line 2 The set grid command places a grid on the graph instead of just tic marks. The first two terms in brackets are the limits for x and y respectively. The next term evaluates the first equation given. You might expect the second term to give both the positive and negative square roots and thereby find the whole curve for y with respect to x, but unfortunately, GNUPLOT follows the C sqrt function in only producing the positive roots. Thus the third term is necessary if you want to produce the whole curve. Furthermore, GNUPLOT cycles through six line styles (from 1 to 6) as it draws curves on a single graph. We want terms 2 and 3 to have the same line style, since they are actually two parts of the same curve. The program automatically draws the second expression with line style 2; we force it to use the same line style for the third expression by adding with line 2 after the third expression. The result is shown in Figure 2. It looks just like the picture in my geometry book. Finally, I found another curve that makes use of parametric equations. The curve that results is shown in Figure 3. The equations are: x = t2(3 - 2t) y = t3(3 - 2t) The commands to draw this plot were set grid set parametric plot [t=-0.5:1.6] t*t*(3-2*t), t*t*t*(3-2*t) The graph limits are set up in terms of the parameter t. It took a little trial and error to find the proper limits for t so that the graph would look the way that I wanted it to. The set parametric expression is needed if you're going to use parametric equations. GNUPLOT then assumes that the first equation in the plot expression represents x and the second one represents y. As a final example, I created a data file called rdata that contained the coordinates of points that would make a big R (for my first initial). The resulting plot is shown in Figure 4. The data file listing is as shown in Listing 1. Note that a carriage return is required after each pair of coordinate values. The command to generate this graph from GNUPLOT is plot [-2.5:4][-5:5] 'rdata' with line 1 A Word about Printing The simple command files that I have used produced lovely graphs on the VGA screen. When it comes to printing these graphs, life gets a little more complicated. I have a Hewlett Packard LaserJet IIP printer with Pacific Page PostScript capability. Included with the GNUPLOT files is a driver for the Hewlett Packard LaserJet (generic). I tried printing out GNUPLOT graphs using this capability and the results were totally unacceptable. Fortunately, GNUPLOT also has a PostScript driver. When I used the Postscript driver, the printouts were beautiful. The commands to select Postscript are set terminal postscript set output "lpt1" The first line sets the terminal up as a PostScript terminal. The second command directs the output to the printer rather than the display screen. I also wanted to transfer my graphs to WordPerfect. I tried using Pizazz Plus to capture the screen images to disk files using the .PCX, .TIF, and .WPG (WordPerfect) file formats. I got images that looked good when displayed from the file onto the screen, but when I transferred them to WordPerfect, the results were not so good. First, WordPerfect dropped some bits so that there were gaps in the lines and missing portions in some of the characters. Second, the combination of colors in the original graphs was such that when I tried to convert so that the lines would be in black and the background in white, the whole graph completely disappeared. Fortunately, GNUPLOT has a driver that produces an output in standard Hewlett Packard plotter language. The commands are: set terminal hpgl set output "filename.pgl" where filename is any name that you want to assign to the disk file where the graph is to be stored. WordPerfect reads in, displays, and prints out Hewlett Packard graphics files perfectly, without any glitches whatsoever. A Few Problems Not everything about GNUPLOT is perfect. The graph shown in Figure 5 shows some examples of things that may go wrong. The commands used to produce this graph are set terminal postscript set output "lpt1" set grid set xtics -2,0.1 set ytics -2, 0.1 plot [-2:2][-2:2] sqrt(1-x*x), -sqrt(1-x*x) with line 1 What I wanted to do is draw a circle. I had already discovered that I had to separately plot the + and - square roots and to make sure that the same line style was used for each. The next problem to note is that although I specified plenty of range to plot the whole curve, GNUPLOT decided to stop before reaching the final point on each semi-circle, so that there is a gap in the circle. Next, I specified very close spacing on the grid lines (tics), causing the values along the x axis to overlap. Thus both on the display and the printout, the x values were unintelligible. Finally, although the shape of the curve is nearly circular on the display screen, the same scales for x and y are not used in printing the graph; as a result, on the printed version the curve appears more elliptical than circular. If the shape of the curve is important to you, you'll have to do some experimentation with setting the x and y ranges to get the curve to look right. Unfortunately, the GNUPLOT documentation doesn't mention how it determines scale factors for the printed versions. What GNUPLOT Won't Do When it comes to two-dimensional equations, GNUPLOT is a fast and highly effective program. There are, however, a few things that it can't do. First, GNUPLOT cannot plot equations in polar coordinates. If you have an equation in polar coordinates, you'll have to perform the conversion into rectangular coordinates before you can use GNUPLOT. Second, GNUPLOT won't permit you to break up a complicated equation into several lines of code. The whole equation must be incorporated into the plot statement. Third, you can't plot the results of iterated equations (where something in a for loop is repeated over and over to get a result). Such equations are used to draw fractal curves. Finally, GNUPLOT won't plot the 2-D projection of a 3-D equation. Summary GNUPLOT is an excellent program for plotting the curves generated by regular or parametric equations in two dimensions using rectangular coordinates. It requires some work to compile and run and some study to learn how to use it most effectively in the absence of the in-depth instructional data furnished with commercial programs. It lacks some capabilities that you might wish were available, but what it does, it does fast and accurately. As you have seen from the examples, the code lines needed to specify a graph using GNUPLOT are simple and short. There are one or two problems with GNUPLOT that should be corrected, but you can usually work around these fairly easily. At the price (for free) the program is just impossible to beat. When I thumb through my analytical geometry book and think of the hours of calculation with a slide rule or primitive calculator that went into determining all of the points on the curves that make up the many drawings in the book and the hundreds of hours spent by a skilled draftsman to draw up each figure in India ink, I am just amazed to think that with GNUPLOT I could create all of these drawings in finished form in about an hour's time. Note: GNUPLOT can be found in the C Users Group Library under catalogue number UG334. Figure 1 Sample sin(x)/x plot Figure 2 Sample plot of two curves Figure 3 Sample plot of a curve using parametric equation Figure 4 Sample plot from a data file Figure 5 Problematic circle plot Listing 1 Coordinates for plotting a big R -1 -4 -1 -3.5 -1 -2 -1 -1.5 -1 -1 -1 -0.5 -1 0 -1 0.5 -1 1 -1 1.5 -1 2 -1 2.5 -1 3 -1 3.5 -1 4 -0.5 4 0 4 0.5 4 1 4 1.1 4 1.2 3.94 1.3 3.9 1.4 3.82 1.5 3.74 1.6 3.6 1.7 3.4 1.8 3.2 1.9 2.84 2 2 1.9 1.16 1.8 0.8 1.7 0.6 1.6 0.4 1.5 0.26 1.4 0.18 1.3 0.1 1.2 0.06 1 0 0.5 0 0 0 -0.5 0 -1 0 0.5 0 1.2 -4 /* End of File */ Comprehensive C Kirk Thomas Kirk Thomas is currently working for Knight-Ridder Financial Services as Software Development Team Leader. He has been programming embedded systems and network applications in C for eight years. Comprehensive C, by David Spuler, attempts to serve a wide variety of programmers by dealing with as many C programming issues as possible. Spuler claims the book "will be useful to C programmers at all levels of experience," and will cover "all aspects of programming in ANSI C." By attempting to be so much to so many, the book fails to be truly useful in many areas. Contents The book is divided into two parts. Part 1, entitled "The C Language," includes 12 chapters on all of the topics one would expect: an introduction to C, variables, operators, control statements, functions, etc. In this part, the author places emphasis on touching on as many elements of C as possible for each of these topics. Coverage of each topic ranges from thorough (including an extensive example) to merely noting the presence of an element for completeness. The conceptual level of each topic ranges considerably as well. In the chapter on operators, paragraph one describes the basic arithmetic operators, but by paragraph three the discussion has already turned to implementation-specific issues of handling the MOD operator with negative numbers. Spuler notes in his introduction that he presumes fluency in another programming language, and this is absolutely true. However, the required fluency of other programming languages and programming issues in general ranges widely even between subsections of the same chapter. Perhaps the most unexpected chapter in Part 1 covers the preprocessor. For a programmer fluent in another language attempting to understand C, the preprocessor is likely to be one area that is relatively new. Spuler provides an excellent overview of the functionality of the preprocessor, providing explanations of replacement rules, macros, and conditional compilation constructs. As a very powerful and rather uncommon programming language element, it is easy for a programmer new to the C language to ignore the preprocessor as much as possible. Spuler not only brings the whole issue to the attention of the reader by giving it a separate chapter, but explains its functionality as well as generating an interest in exploiting its capabilities. Part 2, entitled "Advanced Issues," includes chapters on a wide range of issues such as large programs, efficiency, debugging techniques, program style, portability, and more. Obviously, whole books have been written on each of these topics individually, so Spuler's coverage is intended more to whet the appetite of the reader on these issues than to be the definitive source of information. Each chapter includes a further reading section. Again, the coverage of these issues are over a wide range of programming skill and experience levels. Although called "Advanced Issues," the examination of these issues is fairly elementary. The chapter on large programs mostly discusses how to declare variables and limit their scope, and what MAKE is and how to use it. The chapter on debugging techniques notes the existence of symbolic debuggers, briefly discusses the uses of LINT, and proceeds on through describing conditional debugging compilation, using cleverly placed printf statements, using redirected output, assertions, and signals. All of these topics have only a page or two devoted to them, so there isn't much that I would consider in-depth here. The chapter on efficiency actually wastes space describing the antiquated technique of unrolling loops. An embedded systems developer programming a four-bit micro might use this process, but in 1993, a typical new C programmer should not be spending his or her time learning a nearly obsolete technique with such highly specialized uses. Finally, the book contains three appendices: "C for Pascal Programmers," "Common errors in C," and "Complicated Declarations." These appendices again bring the focal point of the book to teaching C to a programmer fluent in another language. Commentary My question about a book that covers so much ground is "Who really benefits from reading this book?" But the corallary question, "Who would be most disappointed reading this book?" is probably easier to answer. Programmers new to C and hoping to learn it will be quite frustrated if Comprehensive C is the first book they pick up, or is the text book in a C programming course. Programmers new to C might become confused reading Comprehensive C because the book mixes so many programming language issues that make C such a great language with the basic issues a new C programmer would want answered first. For example, a programmer reading about variable usage in C for the first time, might not understand the fundamental concept of variable promotion and how it is handled by the compiler in its relatively early presentation in Chapter 2. Spuler does not provide enough foundation in the preceding text to justify introducing such a key concept so early. Furthermore, a reader wanting a lot of specific code examples of how to perform basic programming tasks using C can find better examples in other introductory C texts. On the other hand, Comprehensive C does offer many marvelous insights into a wide range of C programming issues that many books basically ignore. A programmer at any level can find items of interest in this book, and will genuinely learn from them. Instead of using it as a reference book, C programs occasionally reread whole chapters to further strengthen their development. Conclusion By trying to cover so many topics relating to the C programming language, many of which have had whole books devoted to them individually, Comprehensive C manages to touch on a lot of areas without definitively investigating any of them. However, by having plenty of insightful comments and ideas on many of these topics, the book do es have something to offer. If you're looking for a book to teach you how to program in C, this book will probably frustrate you. If, on the other hand, you want to augment your current library, don't mind reading a lot to pick up some good pointers, or want a text book that fills in around a class curriculum, this book deserves a look. Title: Comprehensive C Author: David Spuler Publisher: Prentice-Hall ISBN: 0-13-156514-1 Editor's Forum Dan Saks has been writing for The C Users Journal for quite some time now. He has shown an exacting grasp of the C++ language, coupled with an ability to explain thorny topics clearly. (And we all know that C++ has more than its share of thorny topics.) As Secretary of WG21/X3J16, Dan is also responsible for producing extensive minutes for three standards meetings a year. That makes him a principal chronicler of the development of the C++ language standard. Thus, I am pleased to announce that Dan has agreed to "go monthly" with The C Users Journal. Starting this month, you can expect to find his column, "Stepping up to C++," in every issue. We can all look forward to more frequent doses of wisdom on a topic of steadily growing importance. Several of his annual installments will be reports on the progress of the C++ language standard. WG21/X3J16 are still considering significant additions to the language, besides tying up numerous loose ends. From my experience as Secretary of X3Jll while C was being refined, I know well the importance of keeping the user community informed of committee deliberations. Dan, as Secretary of WG21/X3J16, has carried out that function for C++ in various trade publications. From now on, you can keep an eye on the C++ standardization effort just by reading CUJ. I plan to keep my column "Standard C" focused primarily on the C language proper. The standards activity in that arena seems never to quite run down, however much it has slowed in the past year or two. Besides, I confess that I still don't "get" C++ with anywhere near the depth that I understand C. I suspect I never will. Dan Saks, on the other hand, has clearly "got" C++. We can depend on him to broaden our coverage to include both C and C++, while still keeping that coverage deep. Despite my obvious preoccupation with standards, I realize they're only part of the picture. We also need tutorials, style guides, and examples of pragmatic code to help us improve as programmers and designers. Dan has promised to continue, and even extend, his offerings in these areas as well. Chuck Allison's "Code Capsules" have been giving us gentle introductions to basic topics in C and C++. "Stepping up to C++" will help those of us who are ready to be more ambitious. My not-so-secret goal is to make The C Users Journal the magazine of choice for practicing C and C++ programmers. I'm content to let others keep telling you why you should be programming a certain way. But when it comes to learning how to get the job done, you can read about it here. P.J. Plauger pjp@plauger.com New Products Industry-Related News & Announcements Koyn Software Releases New Library Koyn Software has released Exceptions for C, a library that provides exception handling for ANSI C compilers. Exceptions for C supports a uniform exception handling mechanism, resembling that of Ada, that allows users to remove the bulk of error handling code from the code handling normal cases. To use Exceptions for C, a programmer compiles and links XCEPT.C into the program and #includes XCEPT.H in source files that use Exceptions for C. Three keywords are used to delineate use of exceptions: xc_with_handling, xc_on_exception, and xc_end_exception. Code for normal cases is placed after xc_with_handling, and exception handling code is placed after xc_on_exception. Handlers that don't recognize a particular exception can invoke xc_reraise(), to pass the exception ot the next handler in the calls stack. xc_cleanup and xc_end_cleanup, can be used to mark code that executes normally and whether there has been an exception or not. The package includes the library (provided as ANSI C source code) and documentation in the header file. Exceptions for C is available in MS-DOS and Macintosh formatted disks, at $30 for a single copy. A site license is available for $30 for the first user plus $6 for each additional user, minimum 5 users. Contact Koyn Software, 1754 Sprucedale, St. Louis, MO 63146, (314) 878-9725. Software Migration Ports Xlib to DOS Software Migration has released X/DOS, a DOS implementation of Xlib, the C interface for X. X/DOS allows X developers to port UNIX X clients to DOS. The programming interface of X/DOS is identical to Xlib. No changes to Xlib GUI calls are required when a program is ported to DOS. X/DOS is distributed as a DOS-linkable library. A UNIX C client can be recompiled and linked with X/DOS to create a standard DOS application than runs on a standalone PC. No special operating environment or X server is required. X/DOS provides a cross-development tool to create graphic applications that are portable between DOS and UNIX. X/DOS supports all X drawing capabilities and X events. Several fonts are supplied with X/DOS, and standard X fonts may be used. Prices for X/DOS begin at $295. Educational version of X/DOS are available for $99. Contact Software Migration, 9 Commodore Dr., Suite 404, Emeryville, CA 94608, (510) 658-5436. Media Cybernetics Releases Imaging Library Media Cybernetics has released the HALO Imaging Library, a platform-independent library that lets C programmers add imaging capabilities to their applications. HALO Imaging Library gives developers 100 imaging functions and commands, allowing them to develop C programs that can read and store image files in several file formats, and perform various processing functions on images in memory. The HALO Imaging Library is divided into several functional groups: Image Management Functions to create and manage images in memory; Image Conversion Functions to convert images from one class to another; Image Enhancement Functions to adjust and refine brightness, contrast, and gamma characteristics, and to enhance an image with special effects such as blur, pixelize, sculpt, emboss, and overlay; Image Transformation Functions to rotate, transpose, flip, or spatially distort and image; Image File Format Functions to read and write image files including TIFF, JPG, GIF, TGA, PCX, BMP, CUT, EPS (write only), and PICT. HALO Imaging Library is available for MS-Windows, OS/2, Macintosh, UNIX/Motif, and UNIX/OPEN LOOK. The library contains an application program and source code samples that demonstrate the functions, along with platform-specific image display and print routines. A single workstation license for HALO Imaging Library for Windows is $595. Other platform versions are planned for release. Contact Media Cybernetics, 8484 Georgia Ave., Silver Spring, MD 20910, (301) 495-3305; FAX: (301) 495-5964. Alydaar Offers Translation Service Alydaar Software Corporation has offered a service for translating source code between programming languages. Alydaar can translate C to Ada, COBOL to C/C++, Jovial to Ada, Pascal to C++, FORTRAN to C, RPG2/3 to COBOL, FORTRAN to Ada, and 370 BAL to C. Alydaar also permits migration between operating systems. Alydaar's translation technology is language independent and Alydaar claims typical translation rates of a million lines in three weeks. Alydaar technology is based on Algebraic and functional programming systems and is modeled after the symbolic manipulators designed to facilitate mathematic manipulations. Alydaar lets the users construct descriptions of each portion of the source language. This model accommodates information such as left/right associativity of operators and also operator precedence, adicness (monadic, dyadic, and triadic), operands, data conversion issues, type conversion issues, structural and elementary levels, and language addressing differences present in the implied abstract machine. The translator augmentation process uses a high-level, rule-based language. Alydaar supports: macro preprocessing, white space management, and comment processing; identification of expressional issues, description of statement level topics, and execution-time statements; resolution of declaration issues; and teaching the translator context syntax and semantics. Other features of Alydaar include: detailed specification of optimization and code-generation issues; saving interactive context for later use or compiling it into a C or Ada application; and target source code. Alydaar translation costs range from 25 to 50 cents per line translated. Contact Alydaar Software Corporation, 216 Crown St., Suite 502A, New Haven, CT 06510, (203) 787-2544; FAX (203) 787-1440. Hyperception Releases C Code Generator Hyperception, Inc. has released Hypersignal-Windows Block Code Generator v1.0, a C code generator designed to work with the visually-programmed Hypersignal-Windows Block Diagram program. Using the code generator, users can simulate and test algorithms, and generate C code to run the algorithms outside the Block Diagram. To implement a DSP design, Block Diagram proves the algorithm, debugs it, and sets up "what-if" situations in a visual environment. The code generator then produces C code for the algorithm, which may be compiled with the DSP chip manufacturer's C compiler. This enables the program to run real-time on a DSP. Previously, such capability was typically found on workstations. Hypersignal-Windows Block Code Generator is available for $2995. Contact Hyperception, Inc., 9550 Skillman, LB 125, Dallas, TX 75243, (214) 343-8525; FAX: (214) 343-2457. DataDraw Releases Graphical Data Structure Toolkit DataDraw has released a graphical data structure toolkit for Windows and DOS developers using C or C++. Using DataDraw, developers design data structures by drawing schemas. The DataDraw schema editor runs under MS-Windows v3.0 and can define objects along with their properties and relationships. Once a set of data structures has been represented, the editor will create C files that define and support all objects and their methods. Constant time memory management and data access through pointers are supported through an interface that hides implementation-specific details. Besides creating methods for building and traversing relationships, DataDraw also creates binary-file load and save functions. DataDraw includes code-generation source code, so users can customize DataDraw's output. DataDraw sells for $179 and comes with a 30-day, money-back guarantee. Contact DataDraw, 1613 Butano Dr., Milpitas, CA 95035, BBS: (408) 956-0576. Software & Consulting Releases DMalloc v1.0 Software & Consulting has released DMalloc v1.0, a heap debugging and quality assurance tool for Microsoft C. DMalloc provides an overview and integrity checks of dynamically-allocated memory. DMalloc can be used to check memory allocations, without the need to recompile source code, by relinking the program with the DMalloc object file and the DMalloc library. DMalloc features include: automatically checking the heap when an allocation function is called; surrounding the allocated memory unit with "safeguards", so memory overwrites are detected but don't destroy the heap; cooperating with debuggers like CodeView and SoftICE in setting breakpoints; and identifying allocated memory units by source file and line of allocation. (Source modules must be recompiled to use this feature.) DMalloc can be popped up with the [PrtScreen] key to view status and examine or edit memory. DMalloc can be configured to reflect the working environment, during debugging, or by inserting calls to the DMalloc API into the program. DMalloc works with Microsoft C/C++ v5.1 or higher. The large memory model is required. DMalloc is marketed as shareware, and is available on CompuServe, through ASP-approved BBSes, and ASP-approved shareware vendors. The registered version of DMalloc is $47 and an unregistered evaluation disk is $5. Contact Software & Consulting, Hietzinger Hauptstrasse 40c, A-1130 Vienna, Austria, FAX: (+43) 1 876 46 094; CompuServe: [100015,551]. Pure Software Upgrades Error Detection Tool Pure Software Inc. has upgraded their runtime error detection tool for C and C++ UNIX developers. Purify 2 adds checking instruction to an application's executable through its object code insertion (OCI) technology. OCI parses the object code and adds commands around memory functions to monitor memory usage at runtime and report illegal memory accesses and memory leaks. OCI allows Purify 2 to detect runtime errors in the entire application, including shared and third-party libraries. Purify 2 includes an incremental linker for rapid turnaround and a maid mode for remote error reporting. Purify 2's mail mode remote error reporting feature transparently detects bugs in software as it is run by other developers throughout an organization. Error diagnostics are emailed back to a designated developer. Purify 2 supports Sun's SPARC platforms. Purify 2 is $4,000 per floating network license. Contact Pure Software Inc., 1309 South Mary Ave., Sunnyvale, CA 94087, (408) 720-1600; FAX: (408)720-9200; email: info@pure.com. ImageSoft Releases C++ Public Domain Source Code Library ImageSoft Inc. has released ISCL, a collection of public domain and shareware C++ software. ISCL consists of more than 100 volumes organized into various categories. ImageSoft describes these volumes as a public service providing C++ source files for educational use and as a collection of "illustrative examples" of various C++ applications. The categories of software in ISCL include: tutorials, compilers, communications, matrix/numerical, graphical user interfaces, memory management, database/ISAM, neural networks, and source utilities. Some example volumes are: GNU 386 C++ compiler, Hypertext tutorial for C++, GOOD C++ comm lib, C++ math matrix class with overloaded operators, GUI TesSe-Ract Development Tools Shareware, SNAV (the Screen Navigator) 1.21 library of reusable objects, Btrieve class for C++, Paradox C engine C++ class definitions, ANSI C and C++ declaration composer and decoder, and Prototype generator for C++. All ISCL volumes are available as individual volumes or as a complete set, and may be freely copied and distributed. All material is in the public domain, or is available as shareware. Volumes are available on 3 1/2 inch and 5 1/4 inch disks, as well as other media, including magnetic media, from forums such as BBSs. Contact ImageSoft Inc., 2 Haven Ave., Port Washington, NY 11050, (516) 767-2233; FAX: (516) 767-9067. We Have Mail Editor: CUJ With regard to Dwayne Phillips' article histogram-based image segmentation last year (CUJ January 1992), I faced the problem of changing gray-scale video images to black-and-white. Here are a couple of tricks that worked quiet well: The first trick does automatic, histogram thresholding--it finds the threshold value that separates "black" from "white": The threshold value is the intensity value which is closest to the mid-point between these two values: 1) The average of all pixels equal or darker than it. 2) The average of all pixels brighter (higher) than it. This trick finds a good threshold even when the histogram does not have distinct peaks. Note that the dark average (not the light average) includes the "weight" of the intensity value under consideration. Probably that was caused, not by deep thought, but by how the code was typed in. Someday, I will have time to try recursing on this trick to do histogram-equalization. (That is, once you find the first threshold, find finer threshold values inside "white" and "black." And so on. Then, re-map the image according to the found thresholds.) It seems to me that whatever method you use to threshold an image ought to be usable in this way to get a best-case histogram-equalization mapping. The second trick is a 3 X 3 filter that helps clean up the image before you threshold it. Filter/copy the image like this: For each pixel, find the highest and lowest intensity values for the pixel and its eight neighbors. Make the new image's pixel equal to either the highest or lowest intensity value of the nine pixels, depending upon which is closest to the current, center pixel's intensity value. If the pixel's intensity value is exactly in the middle between the highest and lowest values, leave it be. Examples: 34 44 56 Low value is 34, high is 56. 35 40 43 Change the 40 to a 34. 39 39 37 34 44 45 Low value is 34, high is 45. 35 40 43 Change the 40 to a 45. 39 39 37 34 44 46 Low value is 34, high is 46. 35 40 43 Leave the 40 alone. 39 39 37 With my images, this high-low filter, followed by an average filter, followed by this high-low filter again gave the best results. The 3 X 3 average filter was: 1 2 1 2 4 2 1 2 1 (That is, multiply the center pixel by 4, each of the pixels diagonal to it by 1, and each pixel beside it, above it, and below it by 2. Add the results together, divide by 16, and the result is the new center pixel value. This filter is fast on 80x86 CPUs.) Applying first the high-low filtering, then histogram thresholding, did a good job of black-and-whiting video images of an LCD screen. As a side note, I got pleasing histogram-smoothing results with a more complex method than the article used: Do exponential smoothing of the histogram from the bottom up and from the top down. Average the two resultant values. That is: #define INTENSITY_LEVELS 256 int histogram[INTENSITY_LEVELS] = the values... int up[INTENSITY_LEVELS]; int v, i; for (v = i = 0; i < INTENSITY_LEVELS; ++i) up[i] = v = (histogram[i] + v) / 2; for ( v = 0, i = INTENSITY_LEVELS; i----;) { v = (histogram[i] + v) / 2; histogram[i] = (up[i] + v) / 2; } This little riff worked out real well when done before histogram- equalizing one image to match the histogram of another. Thank you, B. Alex Robinson 22126 270th Ave SE Maple Valley, WA 98038 Thanks for the tips. -- pjp Dear Sirs; I enjoyed David Burki's "Date Conversions" article (CUJ, February 1993), plus his "A Brief History of the Calendar" sidebar. For simple uses (in this century and in this country) the information is excellent. But danger lurks if you expand past that for astronomical, historical, and perhaps other types of databases. Here's some more information which any programmer working with dates must have: 1. According to English-usage columnist William Safire, B.C. goes after the year but A.D. (anno Domini, "in the year of the Lord") goes before. Thus, it's 46 B.C. and A.D. 46. 2. The Gregorian calendar was adopted at different times in different countries: Date Adopted by 1582 France, Italy, Spain, Portugal, Luxembourg 1583-4 most German Catholic states, Belgium, part of Switzerland, part of Netherlands 1587 Hungary 1699-1700 rest of Netherlands, Denmark, German Protestant states 1752 British Government incl. all colonies, incl. American colonies 1753 Sweden 1873 Japan 1875 Egypt 1912 China 1917 Turkey 1918 Russia 1923 Greece Thus, many history books refer to dates during this transition period as either "old style" (O.S.) or "new style" (N.S.) depending on whether a country was following the older Julian or new Gregorian calendar at the time. 3. In 1752, the British Government decreed that the day after Sept. 2, 1752 would become Sept. 14. They also moved New Year's Day from Mar. 25 to Jan. 1 (under the old style, Mar. 24, 1700 had been followed by Mar. 25, 1701). "O.S." and "N.S." were used to distinguish between the Old Style dates and New Style dates. George Washington's birth date, which was Feb. 11, 1731, O.S., became Feb. 22, 1732, N.S. 4. In the Roman calendar (on which ours is based), March was the first month of the year. September gets its name from the Latin "septem" meaning "seven," because it is the seventh month when March is the first. This is why February gets that extra day--it's the last month of the year. 5. 46 B.C., incidentally, was the year 709 of Rome. The Roman calendar was based on the date April 21, 0753 B.C. when, according to legend, Rome was purportedly founded. I know 46 plus 709 doesn't equal 753; I haven't figured that one out yet. And for human interest, it was in A.D. 730 that the Venerable Bede, an Anglo-Saxon monk, announced that the 365-1/4 day Julian year was 11 min., 14 sec. too long, creating a cumulative error of about one day every 128 years or 3 days every 400 years. Nothing was done about it until 1582. Bibliography and References: Scientific American, May 1982, pp. 150-151, "The Gregorian Calendar," by Gordon Moyer. Astronomical Formulae for Calculators by Jean Meeus, 1982, Willmann-Bell, Inc. The Discoverers by Daniel J. Boorstin, 1983, chapter 1, for an excellent historical discussion on calendars. Practical Astronomy With Your Calculator by Peter Duffett-Smith, 3rd edition 1988, Cambridge University Press. Calendrical Calculations by Edward M. Reingold and Nachum Dershowitz, 1989, University of Illinois at Urbana-Champaign. The 1989 Information Please Almanac, pp. 549-550. (Or any of the various publishers' annual almanacs.) Sincerely, Peter Skye 550 North Brand Boulevard, 7th Floor Glendale, CA 91203-1900 I find calendars to be a bottomless pit of detail. They are as Byzantine as politics--and rightly so. The whims of emperors often influence the shape of calendars far more than reason. Besides, how can you form a logical system in human terms that is based on solar, sidereal and lunar cycles that refuse to divide neatly into days? I thought I knew a lot of esoterica, but you taught me a few new twists. As an aside, the UNIX cal program used to make a point of printing September 1752 "right," by American colonial standards. Give it a try. -- pjp Dear Mr Plauger, C programmers frequently need to select an integer of a particular size; eight bit, 16 bit, 32 bit, etc. With the current ANSI/ISO standard, there is no way to do this in a portable manner. There also seems to be a potential problem in that some particular integer sizes may not be available at all. I am thinking particularly of what happens on large word-size machines. Taking a 64-bit machine as an example, the current standard would permit, say, eight-bit chars64-bit shorts and 128-bit longs. How would I declare a 16 or 32 bit integer? Am I overlooking something? I would be interested to know. I wonder if the ANSI C committee has considered the addition of a set of keywords for fixed-size integers. e.g. int8, int16, int32, uint8, uintl6, uint32, etc The standard could stipulate, for example, that a compiler should provide a minimum of all integer sizes up to, say, twice the machine size. This would give programmers precise control over all integer data sizes, a facility which I would find most useful, particularly when working with embedded systems. Yours sincerely, Ian Cargill 54 Windfield Leatherhead SURREY KT22 8UQ England cargill @exnet.co.uk Committee X3J11 did discuss proposals similar to yours and chose not to include them in the C Standard. You're right that any given implementation gives you a maximum of four different size integers. (Yes, int can differ from the other three sizes.) If a given implementation doesn't give you the sizes you need, you have to write some moderately ugly code to manipulate your data. The Numerical C Extensions Group (NCEG or X3J11.1) has considered several proposals for specifying the exact size of integers, but I don't recall whether they've recommended any yet.--pjp Dear Mr. Smith, I read with interest your article "Debugging with Macro Wrappers" in The C Users Journal, October, 1992. I would be very glad if you could help me with the following question: You recommend to enclose each statement sequence generated by a macro with do { ... } while(0) I cannot understand in what respect this is superior to using braces alone: { ... } By the way: I believe that an ANSI-conforming way to create individual labels from a macro is the macro MAKE_LABEL below: #define MAKELAB_(NR) _LAB_ ## NR #define MAKE_LABEL MAKELAB(_LINE_) The trick is to use an auxiliary macro MAKELAB_ which separates macro expansion for __LINE__. from token pasting in the correct way. Sincerely yours, Ronald Fischer Haberl Strabe 12 #W 8000 Munchen 2 Germany e-mail: 100042.2363@compuserve.com CompuServe: [100042,2363] Greetings: Just a quick note to say I read the article "Debugging with Macro Wrappers" in the October, 1992 CUJ. Thanks much for some very helpful ideas. I also noted with interest your problems in smoothing out the macro for default. Being relatively new to C programming, your discussion of possible fixes went mostly over my head, but it occurred to me that the problem may be unnecessary in the first place. It may not be appropriate in this instance to be quite so elegant with the macro (as much as I admire an elegant programming solution). Would the following code work just as well for you? (Or for me, for that matter...) #ifdef NDEBUG #define BUGCHECK break #else #define BUGCHECK\ . . //debug stuff . . #endif Rather than redefine default at all, and use the following in your source: default: BUGCHECK; Perhaps there is some hideous, hidden reason why this won't work, but it looks reasonable, anyway. Plus it has the advantage (for my personal programming style at least) that the debug macro is visible in the source code, yet still adds no overhead when debugging is turned off. Anyhow, thanks again for the helpful instruction--meanwhile, have fun. Sincerely, Bruce V. Edwards P.O. Box 1922 Billings, MT 59103 Dear Mr. Smith, Thanks very much for your article on wrappers. Here is a label generator: #define UniqueLabel GUL(__LINE__) #define GUL(line) GULI(line) #define GULI(line) L ## line Bill McKeeman Digital Equipment Corporation 110 Spit Brook Road Nashua, New Hampshire 03062-2698 (Bill McKeeman is the notoriously competent writer of compilers and explainer of parsers whose books and articles you have doubtless read.--pjp) Dear Mr. Smith, I recently read your article entitled "An Essential String Function Library" in the January 1993 issue of The C Users Journal. I am a professional programmer and I am currently involved in a fairly large C programming effort which has many applications for your functions. My plan is to add your functions to a directory of reusable C software which is accessible by the entire project. I have enclosed a diskette in the hope that an electronic copy of your functions is available. Please return the diskette to the address below with a note listing any costs you incur in the process. I will be happy to reimburse you. I also have a question about your use of va_start in str_vcat. va_start takes the last fixed parameter name as its second parameter. From the description given for str_vcat I understand that it has a minimum of two parameters. Used in this way it would set Dest to the null string. If this is correct, shouldn't the call to va_start be va_start(VarArgList, Strl)? Thank you for your interesting and insightful article. Sincerely, Elizabeth D. Bennington 12501 Carter Avenue Fredericksburg, VA 22407 William Smith replies to all: Thanks for the interest and insightful ideas! Dear Mr. Plauger: I would like to propose a new financial vehicle for software production called the Premium Bond. This Bond would have a price range from $25 to $100. Your subscribers and staff would recommend software that they would like to have developed, and the Journal would publish a listing together with the Bond price. For example, I would like to have a Borland C++ genetic algorithm (GA) so I would purchase a %50 GA Bond. Let us say the 2,000 subscribers purchased this bond, then thefund would have $100 for software development. Based on a reasonable proposal, a programmer or team of programmers would be given the software contract. By purchasing the software, the bond subscribers would have the right to use the routines for their own purpose without worrying about copyright infringement. On the CompuServe network a news release from Borland indicated that they sold at least 800,000 copies for the C++ Framework software. Immediately you can see that Premium Bond funds could reasch upwards of $80,000,000 which wowuld attract the best C language programmers in the business. Taiwan and Singapore are now in the process of constructing entire cities devoted to developing new software using advanced desktop computers networked to supercomputers. The Premium Bond would help us remain competitive in the years to come by provinding the funds needed to develop world calss software. Cordially yours, John St. Clair 52 Kings Couurt, 4A San Juan, Puerto Ricoo 00911 You have done a marvelous job of describing a financial market for software. Several stock exchanges do this sort of thing at a coarser level by how they value various software companies. To do so at the level of detail you desire requires two new players, administrators and bond traders. I assure you that they will get their share of the take first. It would be interesting to see whether programmers could learn to interact on a regular basis with folks of either stripe. -- pjp. A Windows and MS-DOS C++ Output Stream for a Secondary Monitor Kevin Pinkerton Kevin Pinkerton is a Senior System Programmer for AlliedSignal Technical Services in southern Maryland. He has been programming for sixteen years. Much of that time involved VAX/VMS systems. More recently he has been writing C+ + Windows programs and drivers for specialized data acquisition boards. He can be reached at 301-862-8862. How often have you used output statements to try to find that elusive program bug? Carefully placed output statements can help you follow your program's execution flow or watch the state of a variable. If only they didn't tend to clutter up your screen and interfere with the program's user interface. I have developed an output stream compatible with MS-DOS and Windows that will write to a secondary monitor. The mstream MDA Output Stream mstream (Listing 1 and Listing 2), the output stream class I developed based on the ostream class, inherits all the overloaded insertion operators (<<) from ostream.mstream's class constructor creates an instance of the mstreombuf class and uses it for mstream's buffer. The mstreambuf class is the heart of the MDA output stream. Each character displayed on the MDA monitor uses two bytes of video memory. The low-order byte contains the character and the high order byte contains the display attribute. The video memory is mapped directly to the monitor's display. Each character and display attribute written to video memory is immediately displayed on the MDA monitor. I also defined a global instance of the mstream class called mout. mstream's header file defines an external reference to mout. You must include the mstream header file in your program and link to the appropriate MS-DOS or Windows link libraries before you can use mout in your program. Once you have linked mstream with your program, you can use mout just as you would use the C++ cout standard output stream. You also can use the standard stream manipulators defined in iomanip.h. Here are some mout usage examples: mout << "XMAX = "<< xmax <<" in module XYZ"; mout << "bit code =" << (hex) << bitcode << endl; An external reference to mout is included in mstream.h so that everyone can use mstream without having to create their own instance of it. The global instance in mstream's source file works well for the MS-DOS and Windows static link libraries. However, defining a global instance in the DLL and trying to get the linker to find it when you link your program won't work. (More on this later.) mstreambuf uses the screen member to point to the video memory. screen is an unsigned integer array where array position 0 points to row 1, column 1, array position 1 points to row 1, column 2, and so on. mstreambuf uses the row and column class members to form an index into the screen array. The MS-DOS version of mstreambuf creates a far pointer to memory address 0XB000 and assigns it to screen. The Windows version of the mstreambuf assigns _BOOOH, a global selector constant, to screen. The constant_BOOOH is one of several global selector constants that Windows provides for accessing common BIOS memory areas. screen, row, and column are all static class members so only one copy of each of them exists for all instances of the class. This will keep multiple instances of mstream from overwriting each other. The mstream class constructor sets the stream's unit buffered (unitbuf) format flag. The unitbuf flag causes ostream::osfx to flush the stream after each insertion by calling streambuf::sync. In mstream's case, ostream::osfx will end up calling mstreombuf::sync. mstreambuf::sync takes each character out of the buffer and writes it (along with a display attribute byte) to the MDA video memory. The mstreambuf class also redefines the overflow virtual function. Other stream functions frequently call the overflow function when they sense that the put area is full. overflow flushes the stream and resets the put area pointers. The Link Libraries I compiled, linked, and tested the source code using Borland C++ v3.1. Listing 3 through Listing 7 show sample command procedures to compile and build large memory model versions of the MS-DOS and Windows static link libraries and the Windows dynamic link library (DLL). I prefer to use the DLL library when working with Windows. A single copy of a Windows DLL and all of its static and local variables are used by all the programs that link to it. In mstream's case, this means that multiple programs can send output statements to the MDA monitor at the same time and not overwrite each other. The linker needs to know where to find the references to the DLL routines that your program uses. The easiest way to do this is to create a special import library version of the DLL. The import library contains all the exported (callable by Windows) functions and class names in the DLL and allows the linker to construct a reference table in the .EXE file. When Windows loads the program, it uses this table to resolve calls to the DLL. Unfortunately, you cannot export any of the DLL's static variables (or in this case, class instances) into an import library. So I created a separate source file (Listing 8) that contained the mstream mout global instance statement. I then compiled it using non-DLL compiler switches and added it to the import library. This portion of the import library will be statically linked to your program. This is not a good practice if you plan on distributing your DLL, but as long as the make or batch file automates the process I feel comfortable with it. A DLL also requires two additional routines: LibMain and WEP. Windows calls LibMain when it loads the DLL into memory and WEP when it unloads it. The version of these routines used by the library are empty shells that do the minimum amount required by Windows. Important Note: The DLL library is compiled with the DLL version of the C Runtime Library. If you use the DLL library, your program must also use the DLL version of the C runtime library. Otherwise the linker will not be able to resolve your references to mstream. I find this MDA output stream to be an invaluable tool for debugging programs, especially Window programs. There are several enhancements that would increase its usefulness. One of them would be adding a file stream to mstream that would mirror the MDA output to a permanent log file. Another might be to allow exact row and column positioning similar to the conbuf streambuf. Listing 1 mstream.h -- header file for using the MDA output stream, mout // File : mstream.h // Header file for using the MDA output stream, mout. #ifndef __MSTREAM_H #define __MSTREAM_H #include // for ostream class #include // for streambuf class #include <_defs.h> // for _CLASSDEF and _CLASSTYPE _CLASSDEF(mstreambuf) // create typedefs for the class class _CLASSTYPE mstreambuf : public streambuf { private: static unsigned int far* screen; // monochrome memory area pointer static int column; // current column position static int row; // current row position void newline(); // generates a new line on monitor public: mstreambuf(char* s, int n); ~mstreambuf(); virtual int sync(); // Empties the put area virtual int overflow(int); // Also empties the put area } _CLASSDEF(mstream) // create typedefs for the class class _CLASSTYPE mstream : public ostream { private: enum {bsize=128}; // stream buffer size char msgs[bsize]; // stream buffer Pmstreambuf strbuf; // pointer to mstreambuf public: mstream(); ~mstream(); }; extern mstream mout; // reference to global instance #endif // End of File : MSTREAM.H /* End of File */ Listing 2 mstream.cpp -- C++ output stream classes // MSTREAM.CPP // Defines the classes required to // access a secondary monitor for // debugging. #ifdef_Windows #include extern WORD _BOOOH; // pointer to global selector #else #include #endif #include "mstream.h" #define CPL 80 // Characters Per Line #define LPP 25 // Lines Per Page #define ATTR 0x700 // white on black character attribute mstreambuf::mstreambuf(char* s, int n) : streambuf(s,n) { setp(base(),ebuf))); // Initialize the put pointers if ( screen == 0 ) { // screen pointer needs to be initialized #ifdef _Windows // Windows will use the predefined global selector screen = (unsigned int far*) MAKELONG( O, &_BOOOH ); #else // DOS will point directly to the MDA memory area screen = (unsigned int far*) MK_FP( OxBOO0, 0 ); #endif // first time thru we clear the screen for ( int i = O; i < (CPL*LPP); i++ ) screen[i] = ' ' + ATTR; column = row = 0; // reset the current screen character position } } mstreambuf::-mstreambuf() { sync(); // flush the put buffer before we leave. } void mstreambuf::newline() // This routine generates a new line on the mda monitor. It will scroll all of the lines on // the screen up one line and erases the bottom line if it needs it. { column = 0; // reset the current column position row++; // increment the current row pointer if ( row == LPP ) { // are we at the bottom of the screen? // move lines 2 thru 25 (LPP) up one line _fmovmem( &screen[CPL], &screen[O], (CPL*(LPP-1))*2); // clear the last line for ( int i = CPL*(LPP-1); i < (CPL*LPP); i++ ) screen[i] - ATTR + ' '; row--; // decrement the current row pointer } } int mstreambuf::overflow( int nChar ) // This routine empties the put area and consumes the "nChar" character if it is not EDF. { sync(); // flush the buffer if ( nChar != EOF ) // is there a character to consume? return sputc( nChar ); // store it in the put area, returning the results else return 0; // return success otherwise } int mstreambuf::sync() // This routine flushes the put area. { int count = out_waiting(); // get the number of characters in the put area if ( count ) { for ( int i=O; i< count; i++ ) { char c = pbase()[i]; // get each character and send to mda if ( column == CPL ) // are we at the end of the line? newline(); // then generate a CR/LF if ( c == '\n' ) // is the character a CR/LF? newline(); // then generate a CR/LF here as well else // send character and attribute to monitor screen[(CPL*row) + column++] = c + ATTR; } setp(base(),ebuf()); // reset the put area pointers } return 0; } mstream::mstream(): ostream() // The class constructor. It will create a new streambuf using the classes character buffer. { strbuf = new mstreambuf(msgs,bsize); // do the actual initialization and associate strbuf with ostream ostream::init(strbuf); // set flag so that the stream to be flushed after each insertion setf(ios::unitbuf); } mstream::-mstream() // The class destructor. { delete strbuf; // delete the streambuf used by the class. } // static class member allocation. This allocates space and initializes. unsigned int far* mstreambuf::screen=0; // mda screen memory area int mtreambuf::column=0; // current screen column position int mstreambuf::row=0; // current screen line position #ifndef _DLL_____LINEEND____ mstream mout; // create a global instance of mstream for everyone to use. #endif /* End of File */ Listing 3 libmain.cpp -- defines the DLL startup and shutdown procedures that Windows requires // LIBMAIN.CPP // Define the DLL startup and shutdown procedures that // Windows requires. #include extern "C" int FAR PASCAL _export LIBMAIN( HANDLE, WORD, WORD wHeapSize, LPSTR ) { // Initializes the local heap (if there is one) // with a call to LocalInit which locks the data // segment. if ( wHeapSize != 0 ) UnlockData( 0 ); return 1; } int FAR PASCAL _export WEP( int ) { return 1; } /* End of File */ Listing 4 mdadll.bat -- builds the Windows DLL rem Builds: MDAWIN.DLL rem Dynamic link library for Windows. rem bcc -c -WD -D_RTLDLL -ml! mstream.cpp libmain.cpp tlink x/c/C/Twd cOdl mstream libmain,mdawin.dll,, import crtldll,mdawin.def implib mdawin.lib mdawtn.dll bcc -c -WS -D_RTLDLL -ml moutinst.cpp tlib mdawin.lib +moutinst.obj Listing 5 mdados.bat -- builds the large memory model static link library for MS-DOS rem Builds: MDADOSL.LIB rem Large memory model, static link library for DOS. rem bcc -c -ml mstream.cpp del mdadosl.lib tlib mdadosl.lib +mstream.obj Listing 6 mdawin.def LIBRARY MDAWIN DESCRIPTION 'C++ Output Stream for a Secondary Monitor' EXETYPE WINDOWS CODE PRELOAD MOVEABLE DISCARDABLE DATA PRELOAD MOVEABLE SINGLE HEAPSIZE 4096 Listing 7 mdawinl.bat -- builds the large memory model static link library for Windows rem Build: MDAWINL.LIB rem Large memory model, static link library for Windows. rem bcc -c -WS -ml mstream.cpp del mdawinl.lib tlib mdawinl.lib +mstream.obj Listing 8 moutinst.cpp -- builds a static object for DLL // MOUTINST.CPP // Used to build a static object // to be stored in the Windows // .LIB file for the DLL. #include "mstream.h" // static instance of class mstream mout; /* End of File */ A Revision Control System for MS-DOS Dwayne Phillips The author works as a computer and electronics engineer with the U.S. Department of Defense. He has a PhD in Electrical and Computer Engineering from Louisiana State University. His interests include computer vision, artificial intelligence, software engineering, and programming languages. Have you ever wanted to go back to a version of a program you wrote six months ago? You've changed the program, thrown out your printouts, written over your backup floppies, and you have no hope of retrieving the version you want. A revision control system can solve this problem for you. A revision control system (RCS) allows you to save source code as you develop it and retrieve any past version later. This article discusses and implements a simple revision control system for MS-DOS. It shows how an RCS can help you in your work, demonstrates the basic commands, and looks at the source code for the RCS. The RCS written by Walter Tichy (Sept. 1982, July 1985) for UNIX systems inspired this work. Benefits of an RCS Programmers and authors need a revision control system because source code and documentation change constantly. We are always adding or removing a feature from a program, then often change our minds about a revision. We sometimes need to go back to a feature we had months ago, but have since deleted. Unfortunately, that feature is gone forever and will take a week to recreate. Revisions cannot be tracked by regular backups. Backups use the same name for the copy and the original, but you need different identifiers for different versions. Backups do not help if your source changed several times between backups. You lose the intermediate versions. Backups do not explain why and how your source changed. And the greatest deficiency, backups lose the versions that are older than your oldest backup. An RCS is a simple answer to these problems. An RCS keeps all of the old versions of your source, and records when and why you made the changes. It keeps these in a separate RCS file in a separate directory out of harm's way. You can retrieve any previous version of your source and use it. When you do your weekly backup, backup the current source code files and the RCS files. Processing a File The RCS presented here has six separate programs. They are: checkin checkout revlog revdiff revup prunerev I will process an example, a_file (Figure 1), to illustrate how each command works and helps keep track of revisions. You begin by checking a_file into the RCS. Type: checkin a_file checkin prompts you to enter comments or header lines and then puts the first version of a_file into its own RCS file. The RCS file is in an RCS directory below the current directory as shown in Figure 2. checkin inserts the letter v before the filename and uses va_file as the name for a_file's RCS file. checkin removed a_file from the current directory (moved it out of harm's way), so before revising, you must check it back out. Check out a_file by typing: checkout a_file checkout reads the RCS file for a_file and brings back the original text. You can edit a_file (such as the small change shown in Figure 3), then check the changed a_file back in by typing: checkin a_file checkin prompts for header lines and stores the new version of a_file in its RCS file. checkin retains the prior versions and header lines for a_file. The revlog command displays the header lines and dates of all revisions for a file. Figure 4 shows the output of typing revlog a_file. It lists the information for three versions of a_file. The revdiff command displays the difference between the current copy of a file and any version that you checked into the RCS file. Figure 5 shows the difference between the current file and the latest version in the RCS file after typing revdiff a_file. The next two commands--revup and prunerev--help you maintain your RCS files. revup allows you to move up the latest version to any number you desire. If you use a_file with several other files and you wanted them all to be at version 10, you would type revup a_file 10. Figure 6 shows how the revision log would appear after using revup. This permits you to keep related files coordinated at a desired version number. The prunerev command deletes unwanted versions from the RCS file. The RCS file could grow too large over time. prunerev prunes or reduces the RCS file by deleting versions you select. Figure 7 shows the revision log for a_file after pruning the initial version with prunerev a_file 1. Use prunerev with caution. Once you prune a version from the RCS file it is gone and you cannot retrieve it. Implementation Figure 8 shows the format of the RCS files. The first line is a FIRST_LINE constant. The next lines hold the version number and the date you checked the version into the RCS system. The next section holds the header lines you enter when you call checkin. The header lines section ends with a line containing a DELlMETER constant. The next part contains the complete text of the source code or document and ends with another line containing the DELIMETER constant. The next version begins with the FIRST_LINE constant and continues like the first. Figure 9 shows the RCS file for the a_file example given earlier. It holds three versions of a_file. You can see the FIRST_LINE constant, version number, date, header lines, DELIMETER constant, and so on for each of the three versions. Note, this RCS only works for text files because it reads and writes lines of text. It will not work with executables or data files in its present form. I wrote the programs in C using Microsoft's version 6.0 compiler. The code does not contain any compiler dependent statements so it should port to other C compilers easily. The include file (Listing 1) includes the needed .h files and defines the FIRST_LINE and DELIMETER constants. The FIRST_LINE and DELIMETER constants would probably not appear in any normal text. Listing 2 contains the checkin program. The command line is checkin [-l] source-file. checkin asks you for header lines, then writes these and the text of the source file into the RCS file. If you enter checkin a_file, checkin checks in a_file and removes it from the current directory (moves it out of harm's way). If you enter checkin -l a_file, checkin checks in a_file and leaves it for further editing. The first section of checkin interprets the command line. The call to create_rcs_file_name (Listing 8) constructs the name of the RCS file. (rcsutil.c in Listing 8 holds functions common to several of the RCS programs.) If the RCS file does not exist (result == -1), checkin creates it, gets the header lines from the user, and copies the input file to the RCS file. If the RCS file does exist (result == 0), checkin reads the latest version number from the RCS file, increments the version number, and copies the input file to the RCS file. checkin copies via a temporary file. checkin copies the input file to the temporary file by calling copy_source_to_rcs, and then the RCS file to the end of the temporary file. checkin closes the input, RCS, and temporary files, uses DOS to copy the temporary file to the RCS file, and then deletes the temporary file. If desired (leave_source == 0), checkin removes the input file. The other functions in Listing 2 are short utilities used by checkin. copy_source_to_rcs copies lines of text from one file to another. get_latest_ revision_number reads two lines of text and converts the second from a string to a number. Listing 3 shows the checkout program. The command line is checkout [-r#] source-file [output-file]. checkout will pull any version already in the RCS file and place it in either the source file or an output file of a different name. For example, checkout -r3 a_file b_file pulls the third version of a_file from the RCS file and places it in b_file. If you do not specify -r#, checkout pulls the newest version from the RCS file. If you do not specify an output file, checkout puts the version in the source file. The first section of checkout interprets the command line to determine which version to check out and which destination to use. After this, checkout calls create_rcs_file_name to construct the name of the RCS file and then opens the necessary files. The next section of checkout copies the desired version from the RCS file to a destination file. If the user specified an output file (extra_file == 1), checkout copies the version to the output file. If the user did not specify a particular version (rev == 0), checkout copies the latest version by calling copy_latest_rcs_to_source (see Listing 8). If the user specified a particular version, checkout finds that version by calling go_to_correct_rev (see Listing 8) and then copies it by calling copy_rcs_to_source (also in Listing 8). If the user did not specify an output file (else use source_file), checkout performs the same steps with the source file as the destination. Listing 4 shows the revlog program. The command line is revlog sourcefile.revlog reads through the entire RCS file and prints out all the version numbers, dates, and header lines. The bulk of revlog reads through the RCS file looking for header lines. A FIRST_LINE constant indicates the following lines down to a DELIMETER constant should be printed. revlog prints these lines and continues reading until the end of the file. Listing 5 shows the revdiff program. The command line is revdiff [-r#] [-r#) source-file. revdiff uses the DOS utility fc to show the difference between the current source file and any previous version or between any two previous versions. revdiff a_file will show the difference between the current version of a_file and the last version checked in. revdiff -r2 a_file will show the difference between the current version of a_file and the second version checked in. revdiff -r2 -r4 a_file will show the difference between the second and fourth versions of a_file that were checked in. The first section of revdiff interprets the command line to determine which versions of the source file to compare. revdiff then acquires the name of the RCS file and opens it. revdiff compares versions of the source file by using temporary files. It opens one or two temporary files (output_file and output2_file) and copies versions from the RCS file to these for comparison. If the user wants to compare the source file with one version from the RCS file (extra_file == 0), revdiff finds and copies the desired version from the RCS file to a temporary file output_file). Otherwise, revdiff finds and copies two versions from the RCS file to two temporary files. revdiff closes all the files and compares them by calling the DOS fc utility. Listing 6 shows the revup program. The command line is revup sourcefile #.revup obtains header lines from the user and then pushes up the latest version in the RCS file to the version number entered by the user. If the user enters revup a_file 10, revup obtains header lines and copies the latest version from the RCS file up to version 10. Just like the other programs, revup interprets the command line, obtains the name of the RCS file name, and opens it. revup uses a temporary file to copy up the latest version of the RCS file. revup calls get_header_lines (Listing 8) to obtain header lines and puts them into the temporary file. revup then calls copy_latest_rcs_to_source to move the latest version from the RCS file to the temporary file. Next, revup seeks back to the beginning of the RCS file and copies all of it to the end of the temporary file. The final section uses DOS to copy the temporary file back to the RCS file and delete the temporary file. Listing 7 shows the prunerev program. The command line is prunerev source-file rev-number. prunerev deletes a version from the RCS file. You must use it with caution because once you delete a version you cannot recover it. Typing the command prunerev a_file 1 removes version 1 from the RCS file. prunerev interprets the command line, gets the RCS file name, opens the RCS file, and opens a temporary file. prunerev well copy all but the unwanted version from the RCS file to a temporary file and then copy back to the RCS file. The bulk of the program is in the function copy_wanted_revs__to_temp_file. It has a nest of whiles, ifs, and elses that read through the RCS file and copy selected lines to the temporary file. The logic is to read and copy until you hit a FIRST_LINE constant. If the version number is the one you want to prune, then read through it without copying. If the version number is not the one you want to prune, then read and copy it. After this finishes, prunerev closes the files, uses DOS to copy the temporary file to the RCS file, and deletes the temporary file. Listing 8 shows the file rcsutil.c. This file contains functions used by several of the RCS programs. create_rcs_file_name constructs the name of the RCS file by inserting rcs/v into the proper place. It turns a_file into rcs/va_file and c:/text/a_file into c:/text/rcs/va_file. rev_number takes in a string and returns the integer value of the number. If the input is the string 12 the output is the number 12. The function replace_slash replaces slashes (/) with back slashes (\). The RCS programs need this because the DOS utilities they use interpret all slashes as options. go_to_correct_rev searches through an RCS file and stops when it finds the desired version. It is similar to copy_wanted_revs_to_temp_file in how it looks for the FIRST_LINE constants and compares the version numbers. go_to_correct_rev reads past the version number, date, and header lines and quits when it points to the start of the text for the desired version. copy_rcs_to_source and copy_latest_rcs_to_source are very similar. copy_rcs_to_source copies text from an RCS file to another file. It assumes it begins reading at the desired text and stops copying when it hits a DELlMETER constant. copy_latest_rcs_to_text a assumes it begins reading at the start of an RCS file. It reads down until it hits the first DELIMETER. The line after this first DELIMETER is the first line of text in the latest version. The function copies until it hit the next DELIMETER and then quits. get_header_lines obtains header lines from the user and writes them to the RCS file. It reads from the keyboard until it hits a line whose first character is a period. Conclusions Programmers and authors need a revision control system to help manage the changing nature of their work. The simple RCS presented here works effectively for basic text files and enables you to save and retrieve versions of your work. Since writing it, I have used it for several months on various projects. It has paid for itself in time and peace of mind. References Tichy, Walter F. September 1982. "Design, Implementation, and Evaluation of a Revision Control System," Proceedings of the Sixth International Conference on Software Engineering. IEEE Computer Society Press, pp. 58-67. Tichy, Walter F. July 1985. "RCS--A System for Revision Control," Software--Practice and Experience, Vol. 15, No. 7, pp. 637-654. Figure 1 A sample text file This is the initial version of a text file. Figure 2 The RCS moves the file to its own RCS file and puts this first version into an RCS subdirectory below the current directory. Figure 3 Version two of the text file in Figure 1 This is the second version of a text file. Figure 4 Output from the revlog command ---------------------------------- 3 Wed Oct 30 17:41:30 1991 I made another change and created the 3rd version of the file. ---------------------------------- 2 Wed Oct 30 17:40:26 1991 I made a simple change and checked in the 2nd version of the file. ---------------------------------- 1 Wed Oct 30 17:39:41 1991 First version of an example file ---------------------------------- Figure 5 The difference between the current file and the latest version in the RCS file Comparing files RCSS and (((( ***** RCSS 1: This is the third version 2: of a text file. ***** (((( 1: This is the second version 2: of a text file. ***** Figure 6 The revision log after using revup ---------------------------------- 10 Sat Nov 02 08:24:36 1991 I revved this up to version 10. ---------------------------------- 3 Wed Oct 30 17:41:30 1991 I made another change and created the 3rd version of the file. ---------------------------------- 2 Wed Oct 30 17:40:26 1991 I made a simple change and checked in the 2nd version of the file. ---------------------------------- 1 Wed Oct 30 17:39:41 1991 First version of an example file ---------------------------------- Figure 7 The revision log for a_file after pruning the initial version with prunerev a_file 1 ---------------------------------- 10 Sat Nov 02 08:24:36 1991 I revved this up to version 10. ---------------------------------- 3 Wed Oct 30 17:41:30 1991 I made another change and created the 3rd version of the file. ---------------------------------- 2 Wed Oct 30 17:40:26 1991 I made a simple change and checked in the 2nd version of the file. ---------------------------------- Figure 8 Format of RCS files FIRST_LINE constant revision number revision date header lines DELIMETER constant the source text . . . DELIMETER constant FIRST_LINE constant revision number revision date header lines DELIMETER constant the source text . . . DELIMETER constant and so on ... Figure 9 The RCS file for the a_file example in Figure 1 &^%$#@! 3 Wed Oct 30 17:41:30 1991 I made another change and created the 3rd version of the file. !@#$%^& This is the third version of a text file. !@#$%^& &^%$#@! 2 Wed Oct 30 17:40:26 1991 I made a simple change and checked in the 2nd version of the file. !@#$%^& This is the second version of a text file. !@#$%^& &^%$#@! 1 Wed Oct 30 17:39:41 1991 First version of an example file !@#$%^& This is the initial version of a text file. !@#$%^& Listing 1 Defines FIRST_LINE and DELIMETER /* rcs.h This is the include file for the Revision Control System Dwayne P. November 1991 */ #include #include #include #include #include #include #define FIRST_LINE "&^%$#@!\n" #define DELIMETER "!@#$%^&\n" /* End of File */ Listing 2 checkin.c -- enters a file into the revision control system /* checkin.c The Revision Control System Dwayne Phillips November 1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[]; { char rcs_name[80], source_name[80], string[80]; FILE *rcs_file, *source_file, *xxx_file; int leave_source = 0, one = 1, result, rev; struct stat rcs_file_status; if( (argc < 2) (argc > 3)){ printf("\n\n\tusage: checkin [-l] source-file\n"); exit(1); } if(argc == 2){ strcpy(source_name, argv[1]); if((source_file = fopen(source_name, "r")) == '\0'){ printf("\ncheckin>> " "cannot open the source file >>%s<<", source_name); exit(-1); } } if(argc == 3){ if( (strncmp("-l", argv[1], 2) == 0)){ leave_source = 1; strcpy(source_name, argv[2]); if((source_file = fopen(source_name, "r")) == '\0'){ printf("\ncheckin>> " "cannot open the source file >>%s<<", source_name); exit(-1); } } else{ printf("\n\n\tusage: checkin [-l] source-file\n"); exit(3); } } /* ends if argc == 3 */ create_rcs_file_name(source_name, rcs_name); /* does an rcs file already exist? */ result = stat(rcs_name, &rcs_file_status); if(result == -1){ /* rcs file does not yet exist */ if((rcs_file = fopen(rcs_name, "w")) == '\0'){ printf("\ncheckin>> " "cannot create the rcs file >>%s<<", rcs_name); exit(-1); } else{ get_header_lines(rcs_file, one); copy_source_to_rcs(source_file, rcs_file); fclose(rcs_file); fclose(source_file); if(leave_source == 0) unlink(source_name); } /* ends else we can create the rcs file */ } /* ends if result == -1 */ if(result == 0){ /* rcs file already exists */ if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\ncheckin>> " "cannot open the rcs file >>%s<<", rcs_name); exit(-1); } get_latest_revision_number(rcs_file, &rev); rev++; fseek(rcs_file, 0L, SEEK_SET); if((xxx_file = fopen("RCS/((((", "w")) == '\0'){ printf("\ncheckin>> " "cannot create the temp file (((("); exit(-1); } get_header_lines(xxx_file, rev); copy_source_to_rcs(source_file, xxx_file); while( fgets(string, 80, rcs_file) != '\0') fputs(string, xxx_file); fclose(xxx_file); fclose(rcs_file); fclose(source_file); unlink(rcs_name); if(leave_source == 0) unlink(source_name); sprintf(string, "copy RCS/(((( %s", rcs_name); replace_slash(string); system(string); unlink("RCS/(((("); } /* ends if result == 0 */ } /* ends main */ */ copy_source_to_rcs(FILE *source_file, FILE *rcs_file) /* copy_source_to_rcs(FILE *source_file, FILE *rcs_file) { char line[80]; while( fgets(line, 80, source_file) != '\0'){ fputs(line, rcs_file); } fputs(DELIMETER, rcs_file); } /* get_latest_revision_number(FILE *rcs_file, int *rev) */ get_latest_revision_number(FILE *rcs_file, int *rev) { char *line; fgets(line, 80, rcs_file); fgets(line, 80, rcs_file); *rev = atoi(line); } /* End of File */ Listing 3 checkout.c -- pulls the previous version of a file from the revision control system /* checkout.c The Revision Control System Dwayne Phillips November1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[]; { char output_name[80], rcs_name[80], source_name[80], string[80]; FILE *output_file, *rcs_file, *source_file; int extra_file = 0, rev = 0; if(argc < 2 argc > 4){ printf("\n\n\tusage: checkout [-r#] source-file" " [output-file]\n"); exit(1); } if(argc == 2){ strcpy(source_name, argv[1]); rev = 0; } if(argc == 3){ /* checkout -r# source-name */ if( (strncmp(argv[1], "-r", 2) == 0)){ strcpy(string, argv[1]); strcpy(source_name, argv[2]); rev = rev_number(string); } /* checkout source-name output-name */ if( (strncmp(argv[1], "-r", 2) != 0)){ extra_file = 1; rev = 0; strcpy(source_name, argv[1]); strcpy(output_name, argv[2]); } } /* ends if argc == 3 */ /* checkout -r# source-name output-name */ if(argc == 4){ if( (strncmp(argv[1], "-r", 2) != 0)){ printf("\n\n\tusage: checkout [-r#] source-file" "[output-file]\n"); exit(1); } else{ extra_file = 1; rev = rev_number(argv[1]); strcpy(source_name, argv[2]); strcpy(output_name, argv[3]); } } create_rcs_file_name(source_name, rcs_name); if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\ncheckout>> " "cannot open the rcs file >>%s<<", rcs_name); exit(-1); } if(extra_file == 1){ /* open output_name for writing */ if((output_file = fopen(output_name, "w")) == '\0'){ printf("\ncheckout>>" "cannot open the output file >>%s<<", output_name); exit(-1); } } /* ends if extra_file == 1 */ else{ /* else open source_name for writing */ if((source_file = fopen(source_name, "w")) == '\0'){ printf("\ncheckout>> " "cannot open the source file >>%s<<", source_name); exit(-1); } } /* ends else extra_file == 0 */ if(extra_file == 1){ /* use output file */ if(rev == 0) copy_latest_rcs_to_source(rcs_file, output_file); else{ go_to_correct_rev(rcs_file, rev); copy_rcs_to_source(rcs_file, output_file); } } else{ /* else use source_file */ if(rev == 0) copy_latest_rcs_to_source(rcs_file, source_file); else{ go_to_correct_rev(rcs_file, rev); copy_rcs_to_source(rcs_file, source_file); } } fclose(rcs_file); if(extra_file == 1) fclose(output_file); else fclose(source_file); } /* End of File */ Listing 4 revlog.c - print out version numbers, dates, and header lines /* revlog.c The Revision Control System Dwayne Phillips November 1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[] { char rcs_name[80], source_name[80], string[80]; int printing; FILE *rcs_file; if(argc < 2 argc > 2){ printf("%\n\n\tusage: revlog source-file\n"); exit(1); } strcpy(source_name, argv[1]); create_rcs_file_name(source_name, rcs_name); if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\nrevlog>> " "cannot open the rcs file >>%s<<", rcs_name exit(-1); } printf("\n----------------------------------\n"); while(fgets(string, 80, rcs_file) != '\0'){ if( strncmp(string, FIRST_LINE, 5) == 0){ printing = 1; while(printing){ fgets(string, 80, rcs_file); if(strncmp(string, DELIMETER, 5) != 0){ printf("%s", string); } /* ends if != DELIMETER */ else /* else you hit DELIMETER stop printing */ printing = 0; } /* ends while printing */ printf("\n----------------------------------\n"); } /* ends if FIRST_LINE */ } /* ends while reading rcs file */ fclose(rcs_file); } /* End of File */ Listing 5 revdiff.c - shows differences between two files or versions of a file /* revdiff.c The Revision Control System Dwayne Phillips November 1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[]; { char output_name[80], rcs_name[80], source_name[80], string[80]; FILE *output_file, *output2_file, *rcs_file, *source_file; int extra_file = 0, rev = 0, rev2 = 0; if((argc < 2) (argc > 4)){ printf("\n\n\tusage: revdiff [-r#] [-r#] source-file\n"); exit(1); } /* copy latest revision to a temp file and run fc on the source file and temp file */ if(argc == 2){ strcpy(source_name, argv[1]); rev = 0; } /* revdiff -r# source-file */ if(argc == 3){ if( (strncmp(argv[1], "-r". 2) == 0)){ strcpy(string, argv[1]); strcpy(source_name, argv[2]); rev = rev_number(string); } else{ printf("\n\n\tusage: revdiff [-r#] [-r#] source-file\n"); exit(1); } } /* revdiff -r#1 -r#2 source-file */ if(argc == 4){ if( (strncmp(argv[1], "-r", 2) == 0) && (strncmp(argv[2], "-r", 2) == 0)){ extra_file = 1; strcpy(string, argv[1]); rev = rev_number(string); strcpy(string, argv[2]); rev2 = rev_number(string); strcpy(source_name, argv [3]); } else{ printf("\n\n\tusage: revdiff [-r#] [-r#] source-file\n"); exit(1); } } create_rcs_file_name(source_name, rcs_name); if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\nrevdiff>> " "cannot open the rcs file >>%s<<", rcs_name); exit(-1); } if((output_file = fopen("((((", "w")) == '\0'){ printf("\nrevdiff>> cannot open the temp file (((("); exit(-1); } if(extra_file == 1){ if((output2_file = fopen("((((2", "w")) == '\0'){ printf("nrevdiff>> cannot open the temp file ((((2"); exit(-1); } } if(extra_file == 0){ if(rev == 0) copy_latest_rcs_to_source(rcs_file, output_file); else{ go_to_correct_rev(rcs_file, rev); copy_rcs_to_source(rcs_file, output_file); } } else{ go_to_correct_rev(rcs_file, rev); copy_rcs_to_source(rcs_file, output_file); fseek(rcs_file, 0L, SEEK_SET); go_to_correct_rev (rcs_file, rev2); copy_rcs_to_source(rcs_file, output2_file); } fclose(rcs_file); fclose(output_file); if(extra file == 1) fclose(output2_file); /* Use DOS's fc to compare files */ if(extra_file == 0){ replace_slash(source_name); sprintf(string, "fc /L /N %s ((((", source_name); system(string); } else{ sprintf(string, "fc (((( ((((2"); system(string); } unlink("(((("); if(extra_file == 1) unlink("((((2"); } /* End of File */ Listing 6 revup.c -- advances version number to next level up /* revup.c The Revision Control System Dwayne Phillips 1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[]; { char rcs_name[80], source_name[80], string[80]; FILE *output_file, *rcs_file; int leave_source = 0, one = 1, result, rev; struct stat rcs_file_status; if(argc < 3 argc > 3){ printf("\n\n\tusage: revup source-file #\n"); exit(1); } strcpy(source_name, argv[1]); create_rcs_file_name(source_name, rcs_name); rev = atoi(argv[2]); if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\nrevup>> " "cannot open the rcs file >>%s<<", rcs_name); exit(-1); } if((output_file = fopen("((((", "w")) == '\0'){ printf("\nrevup>> cannot open the temp file (((("); exit(-1); } get_header_lines(output_file, rev); copy_latest_rcs_to_source(rcs_file, output_file); sprintf(string, "%s", DELIMETER); fputs(string, output_file); fseek(rcs_file, 0L, SEEK_SET); while(fgets(string, 80, rcs_file)){ fputs(string, output_file); } fclose(rcs_file); fclose(output_file); sprintf(string, "copy (((( %s", rcs_name); replace_slash(string); system(string); unlink("(((("); } /* ends main */ /* End of File */ Listing 7 prunerev.c -- deletes a version from the RCS file /* prunerev.c The Revision Control System Dwayne Phillips November 1991 */ #include "rcs.h" main(argc, argv) int argc; char *argv[]; { char rcs_name[80], source_name[80], string[80]; FILE *rcs_file, *source_file, *xxx_file; int prune = 0, leave_source = 0, one = 1, result, rev; if(argc < 3 argc > 3){ printf("\n\n\tusage: " "prunerev source-file rev-number\n"); exit(-1); } prune = print_warning(); if(prune == 0) { printf("\nExiting prunerev\n"); exit(1); } strcpy(source_name, argv[1]); strcpy(string, argv[2]); rev = rev_number(string); create_rcs_file_name(source_name, rcs_name); if((rcs_file = fopen(rcs_name, "r")) == '\0'){ printf("\nprunerev>> " "cannot open the rcs file >>%s<<", rcs_name); exit(-1); } if((xxx_file = fopen("((((", "w")) == '\0'){ printf("\nprunerev>> " "cannot open the temp file (((("); exit(-1); } copy_wanted_revs_to_temp_file(rcs_file, xxx_file, rev); fclose(rcs_file); fclose(xxx_file); unlink(rcs_name); sprintf(string, "copy (((( %s", rcs_name); replace_slash(string); system(string); unlink("(((("); } /* copy_wanted_revs_to_temp_file(rcs_file, xxx_file, rev) */ copy_wanted_revs_to_temp_file(FILE *rcs_file, FILE *xxx_file, int rev) { char *result, second_string[80], string[80]; int found_it = 0, new_rev, reading = 1, still_reading = 1; reading = 1; while(reading){ /* read file */ result = fgets(string, 80, rcs_file); if( strncmp(string, FIRST_LINE, 5) == 0){ result = fgets(second_string, 80, rcs_file); new_rev = atoi(second_string); if(rev == new_rev){ still_reading = 1; while(still_reading){ found_it = 1; result = fgets(string, 80, rcs_file); if( strncmp(string, FIRST_LINE, 5) == 0){ fputs(string, xxx_file); still_reading = 0; found_it = 1; } /* ends if FIRST_LINE*/ if(result == '\0') still_reading = 0; } /* ends while still_reading */ } /* ends if rev == new_rev */ else{ /* else rev != new_rev */ fputs(string, xxx_file); fputs(second_string, xxx_file); } } /* ends if FIRST_LINE */ else /* not FIRST_LINE so fputs it out */ fputs(string, xxx_file); if(result == '\0') reading = 0; } /* ends while reading */ if(found_it == 0){ printf("\n\nprunerev>> Did not find the" " desired revision\n"); fclose(rcs_file); exit(-5); } } /* ends copy_wanted_revs_to_temp_file */ /* print_warning() */ print_warning() { char *string; printf("\n" "\nWARNING: If you continue, you will delete part of" "\n your RCS file. You will never be able to" "\n retrieve those deleted revisions." "\n" "\n Do you want to continue (y/n) ? _\b"); fgets(string, 10, stdin); if(string[0] == 'y' string[0] == 'Y') return(1); else return(0); } /* End of File */ Listing 8 rcsutil .c -- functions used by several of the RCS programs /* rcsutil.c The Revision Control System Dwayne Phillips November 1991 This file contains functions that are used by several of the RCS programs. They are held here to avoid having multiple copies of the same function. create_rcs_file_name rev_number replace_slash go_to_correct_rev copy_rcs_to_source copy_latest_rcs_to_source get_header_lines */ #include "rcs.h" /* create_rcs_file_name(char source_name[], char rcs_name[]) */ create_rcs_file_name(char *source_name, char *rcs_name) { char *place, temp[80]; int i, j, slash, slot; slash = 47; place = strrchr(source_name, slash); slash = 92; if(place == '\0') place = strrchr(source_name, slash); if(place == '\0'){ strcpy(rcs_name, "RCS/v"); strcat(rcs_name, source_name); } else{ slot = place - source_name; strncpy(temp, source_name, slot); temp[slot] = '\0'; strcat(temp, "/RCS/v"); slot++; j = strlen(temp); for(i=slot; source_name[i]!='\0'; i++){ temp[j] = source_name[i]; j++; } temp[j] = '\0'; strcpy(rcs_name, temp); } } /* rev_number(char * string) */ rev_number(char *string) { char *new_string; int doit = 0, i, j, result; i = 0; while(doit == 0){ j = string[i]; doit = isdigit(j); i++; } i--; for(j=0; string[i] != '\0'; i++, j++) new_string[j] = string[i]; result = atoi(new_string); return(result); } /* replace_slash(char string[]) */ replace_slash(char string[]) { int slash = 47, back_slash = 92, i, j; j = strlen(string); for(i=0; i Did not find the" " desired revision\n"); fclose(rcs_file); exit(-5); } } /* copy_rcs_to_source(FILE *rcs_file, FILE *source_file) */ copy_rcs_to_source (FILE *rcs_file, FILE *source_file) { char string[80]; int reading = 1; while(reading){ fgets(string, 80, rcs_file); if( strncmp(string, DELIMETER, 5) == 0) reading = 0; else fputs(string, source_file); } } /* copy_latest_rcs_to_source(FILE *rcs_file, FILE *source_file) */ copy_latest_rcs_to_source(FILE *rcs_file, FILE *source_file) { char string[80]; int reading = 1; while(reading){ fgets(string, 80, rcs_file); if( strncmp(string, DELIMETER, 5) == 0){ while(reading){ fgets(string, 80, rcs_file); if(strncmp(string, DELIMETER, 5) == 0) reading = 0; else fputs(string, source_file); } /* ends while reading */ } /* ends if DELIMETER */ } /* ends while reading */ } /* get_header_lines(FILE *the_file, int version) */ get_header_lines(FILE *the_file, int version) { char string[80]; int entering = 1; time_t ltime; time(<ime); fputs(FIRST_LINE, the_file); sprintf(string, "%d\n", version); fputs(string, the_file); sprintf(string, "%s", ctime(<ime)); fputs(string, the_file); printf("\n\nEnter your header lines"); printf("\nStop the header lines by entering"); printf("\na . on a line by itself."); printf("\n"); while(entering){ printf(">>"); fgets(string, 80, stdin); if(string[0] != '.') fputs(string, the_file); else entering = 0; } /* ends while entering */ fputs(DELIMETER, the_file); } /* End of File */ Extending C for Object-Oriented Programming Gregory Colvin Although trained in cognitive psychology, Dr. Colvin has been happily hacking computers for twenty years. He has programmed professionally in C for the last ten years and in C++ for the last five, and is a member of X3J16, the ANSI C++ Standards committee. Greg is Scientist for Systems Development at Information Handling Services in Englewood, Colorado, 80150 (303-397-2848) and is on the Internet at gregc@ihs.com. Three years ago my team and I set out to create two new programs for access to industry standards, vendor information, and catalog page images on CD-ROM. These programs were to be just the first of many, and though most of our customers were using only MS-DOS workstations we would soon need X Window, Microsoft Windows, and MacIntosh versions. We were less than six months from market. We met our goals with an object-oriented approach to C programming that may well serve others making the transition from C to C++. In this article I present a redesigned and simplified version of the object macros of our tool kit kernel. Using this kernel it is possible to write object-oriented programs that can be compiled without change by either a Standard C compiler or a C++ compiler. If you need to implement an object-oriented design, an object-oriented language like C++, Smalltalk, or Eiffel will make your job much easier. But if you have compelling reasons to use Standard C this kernel may be of help. To motivate the presentation I will present a design for a simple file access facility and show how the design can be implemented in four different ways: traditional C, C++, object-oriented C, and C extended with object macros. Standard C My team was convinced that only an object-oriented design could make the rapid creation of so many applications, across so many platforms, reasonably painless. But the object-oriented languages available at that time were all deficient in either performance or portability--C was the language we all knew and loved. After much debate we chose to design our systems as an object class hierarchy and do the first implementations in Standard C, with the intention of porting to C++ as soon as quality compilers were supported on all of our target platforms. Standard C does not support object-oriented programming. According to Bjarne Stroustrup, to support a style of programming means to make it "convenient (reasonably easy, safe, and efficient) to use that style." It is certainly possible to implement a class hierarchy in C, as the Xt Intrinsics and the Motif and Open Look widget sets demonstrate. But even a cursory reading of "Inside A Widget" will reveal just how inconvenient C can be. Fortunately, the C preprocessor does allow the syntax of C to be extended, and Standard C blesses and codifies the necessary stringizing (operator #) and tokenizing (operator ##), which required dirty tricks in older C dialects. We were thus able to extend Standard C to provide a more convenient object-oriented syntax in which to implement our design. Although not perfect, it allowed us to get our products to market on schedule, and has eased our transition to C++. File Access: A Simple Example in C and C++ Let's say we need to implement a File facility that allows files to be opened either by the Standard C ƒopen call or by Unix system calls, but accessed by the same Read, Write, and Seek methods, without needing to know how the file was opened. I have sketched out the design of such a facility in Figure 1. The File class declares Read, Write and Seek methods with no defining functions. The derived StdFile and UnixFile classes define these methods with the appropriate functions, and extend the File interface with appropriately defined Construct and Destruct methods. In a traditional C implementation, the File object would be defined by a structure with a tagged union, which could contain either a standard file handle or a UNIX file descriptor: typedef struct { union { FILE *hdl; int dsc; } u; enum { STD, UNIX } tag; } File; The File methods would be defined by access functions that take a pointer to this structure and switch on the tag to call the appropriate functions. For instance: int FileRead(File*p,void*buf,int n) { switch (p->tag) { case STD: if (fread(buf,n,1,p->u.hdl)<1) return -1; return n; case UNIX: return read(p->u.dsc,buf,n); } return -1; } The disadvantage of this approach becomes evident once you attempt to extend the file object by adding more access functions (e.g., character I/O) or other ways of opening files (e.g., calls for other operating systems). Although our design calls for three distinct classes, each with its own methods, this implementation defines only one data structure for all three classes, and one function for each method. Thus, every new function you write must have a switch on the type tag and code for all classes. Even worse, adding a new type means adding new members to the File structure, adding new cases to every switch in every function, and rebuilding and retesting the entire facility and every other piece of code that uses this facility. Forget a case in one of your switches and your compiler may not complain, but some user of your facility eventually will. If you want to further extend your class hierarchy, say to derive a class of UnixFile with special buffering, then your structures and switches get even more complex. These difficulties can be managed in a small program, but can easily become overwhelming in larger programs. Thus the traditional C approach is most appropriate for small programs, or for programs whose design is well-specified before implementation and not likely to change in the future. The C++ Approach C++ can provide a solution that maps much more closely to our design. The abstract File interface is declared in a single structure that serves as an abstract base class. No data members are declared, just the common access methods, which are defined as null by default: struct File { virtual int Read(void*buf,int n)=0; virtual int Write(void*buf,int n)=0; virtual long Seek(long off,int pos)=0; } This declaration would typically be in a separate header, so that users of the access functions need include only the common interface. The StdFile class is derived from File, with a class declaration in a header file: class StdFile::File { FILE *hdl; public: virtual int Read(void*buf,int n); virtual int Write(void*buf,int n); virtual long Seek(long off,int pos); StdFile(const char*nam,const char*acc); ~StdFile(); }; The method definitions would go in a separate source file: int StdFile::Read(void*buf,int n)=0; { if (fread(buf,n,1,hdl)<1) return -1; return n; } //...other methods... The StdFile and ~StdFile functions are special. The StdFile function is the constructor, which acquires any resources needed by the object, in this case a file handle. The ~StdFile function is the destructor, which releases those resources. All C++ classes must have a constructor and destructor. The compiler will provide default implementations if the programmer doesn't. The UnixFile class is also derived from File, with a class declaration in a header file: class UnixFile::File { int dsc; public: virtual int Read(void*buf,int n); virtual int Write(void*buf,int n); virtual long Seek(long off, int pos); UnixFile(const char*nam, int omode,int cmode=0); ~UnixFile(); }; Again, the method definitions go in a separate source file: virtual int Read(void *buf,int n) { return read(dsc,buf,n); } //...other methods... Class declarations and definitions are typically placed in separate files so that they can be modified and extended independently, with minimal recompilation. In C++, virtual functions provide for polymorphism, without the need for switching on tag fields. Thus the function int GetNextBlock(File*fp) { return fp->Read(BlkBuf,BlkSz) } might call either StdFile::Read or UnixFile::Read depending on the type, at the time of the call, of the actual ƒp parameter. Implementing Classes in C Stroustrup's original C++ compiler, CFront, is a portable front-end that emits efficient and notoriously cryptic C code. The Xt Intrinsics implement classes with similar, if slightly less cryptic, techniques. You too can use these techniques to implement objects directly in C. A picture of one possible implementation is shown in Figure 2. This implementation is rooted in a method table (called a vtable in CFront), which is a structure of pointers to functions, one pointer for each method: typedef struct { int (*Read)(void*obj,void*buf,int n); int (*Write)(void*obj,void*buf, int n); long (*Seek)(void*obj, long off,int pos); } FileVTable; Each base class includes a pointer to a table of this type as a member of its data structure: typedef struct { FileVTable *methods; } File; Using this approach, you can derive StdFile from File by nesting structure declarations: typedef struct { FileVTable base; void (*Construct)(void*obj, const char*nam, const char*acc); void (*Destruct)(void*obj); } StdFileVTable; typedef struct { File base; FILE *hdl; } StdFile; You can derive UnixFile from File in the same way: typedef struct { FileVTable base; void (*Construct)(void*obj, const char*nam, int omode,int cmode=0); void (*Destruct)(void*obj); } UnixFileVTable; typedef struct { File base; int dsc; } UnixFile; Because a pointer to a structure (in Standard C) has the same value as a pointer to its first member, any pointer to a StdFile or to a UnixFile also points to a File. (The first member of each is a File.) Thus a pointer to a StdFile or a UnixFile can be safely cast to a pointer to File. Of course, the methods pointer of each StdFile object must be initialized to an appropriate table of functions for the StdFile type: int StdFileRead (void*obj,void*buf,int n) { StdFile *this=obj; if (fread(buf,n,1,(this->hdl)<1) return -1; return n; } /*...other functions...*/ StdFileVTable StdFileVT = { { StdFileRead, StdFileWrite, StdFileSeek }, StdFileConstruct, StdFileDestruct, }; and each UnixFile object must be initialized to an appropriate table of functions for the UnixFile type: int UnixFileRead(void*obj, void*buf,int n) { UnixFile* this= obj; return read(this->dsc,buf,n); } /*...other functions...*/ UnixFileVTable UnixFileVT = { { UnixFileRead, UnixFileWrite, UnixFileSeek }, StdFileConstruct, StdFileDestruct, }; Given these definitions, a C++ expression for calling a virtual function, such as fp->Read(buf, n), can be implemented as the C expression ((File*)fp)->methods->Read(fp,buf,n). This expression will call a different function, depending on whether fp is a StdFile* or a UnixFile*, since these structures are initialized to point to different method tables. Notice that with this implementation the cost, in time and space, of a virtual function call is comparable to calling a function which must switch on a type tag. The time cost is just accessing and calling the function via the method table, versus making a normal function call followed by accessing and switching on the tag. The space cost is just one table of pointers for each class and one extra pointer in each object, versus the switch logic in each function and one type tag in each object. It is far from convenient to directly implement objects in this way, but it can be done. To implement a large class hierarchy you must properly declare, define, and initialize a host of class data structures and method tables. Using the resulting objects requires a cryptic syntax with lots of unsafe type casting. If you do choose to implement objects in this way I recommend you follow the lead of the Xt Intrinsics: establish and follow strict conventions for laying out and naming your data structures, and hide the ugly details of object creation and method invocation behind a wall of access functions. Extending C Syntax with Macros The features of C++ which I have chosen to support with my extensions are type safety, public classes, single inheritance, virtual functions (with a pointer to this), constructors, destructors, runtime type identification, and separate declaration and implementation of classes. These features were sufficient to meet our needs, and I was unable to go much farther within the constraints of the C preprocessor. I chose not to support multiple inheritance, and cannot see how to provide operator overloading, function overloading, default arguments, or access specifiers using macros. The industrial strength predecessor to this extension did provide simple templates and exception handling, but I have omitted those features from this version. The syntax presented here did not spring full grown from my forehead. It has evolved over six years of trial and error. I have not been able to provide as concise a syntax or as much type safety as C++, but I have tried to ensure that incorrect syntax will cause compiler or runtime diagnostics. I use runtime type checking where compile-time checking is impractical. Listing 1 (objects.h) and Listing 2 (objects.c) provide the full source code for my extension. The code disk for this article also includes a simple File facility implemented in my extension. Declaring Classes Just as with a C structure, you must declare a class before it is used. As the syntax shows, a class is declared by separately declaring first the methods and then the members of the class, using the macro DCL_METHODS( class-name, base-name, constructor- parameters ) DCL_MEMBERS( class-name, base-name ) and associated macros. Methods are declared as pointers to functions using the DCL.METHOD( method-name, method-parameters, return-type) and REDCL_METHOD macros, and members are declared just like C structure members. Note that the base-name for the methods and members declarations must be the same. A single base class, Object, is provided to serve as the starting point for other classes. For example, the StdFile class is declared as: DCL_METHODS (StdFile, File, (const char *nam, const char *acc)) REDCL_METHOD(Seek,(long off, int pos),long); REDCL_METHOD(Read,(void *buf, int n),int); REDCL_METHOD(Write,(void *buf, int n),int); END_METHODS DCL_MEMBERS(StdFile,File) FILE *hdl; END_MEMBERS The class declaration macros expand to declarations of two structures, one for the class method table and one for the class data structure. The names for these structures are derived from the class-name via token pasting. Each of these structures includes the corresponding structure from its base class by token pasting with the base-name. Defining and Implementing Classes The macro METHOD( class-name, method-name, (parameter-list), return-type) begins the implementation of a method, the body of which is pretty much the same as for a C function. Within a method implementation the keyword this is defined as a pointer to an object of the same class as the method, just as in C++. The object pointers for each active method invocation are pushed on a separate stack (rather than being passed as parameters), which allows for runtime type checking and casting. This separate stack does increase the space and time for method invocation. In the MS-DOS version, I avoid this overhead by using Borland's inline 8086 assembler to pass the object pointer in a register. For instance, the Read method for the StdFile class is implemented as: METHOD(StdFile,Read,(void*buf,int n),int) if (fread(buf,n,1,this->hdl) < 1) return -1; return n; END_METHOD Two special methods must be implemented for each class: the constructor and the destructor. Constructors begin with the CONSTRUCTOR( class-name, method-parameters) macro, followed by the CONSTRUCT(base-name, constructor-arguments) macro to invoke the base class constructor and set the methods pointer to the class-name method table. At the entry to each constructor the methods pointer points to the method table for that class. Construction is from the base class out to the derived classes. The constructor for the StdFile class is implemented as: CONSTRUCTOR(StdFile, (const char*nam,const char*acc)) CONSTRUCT (File,()); this->hdl = fopen(nam,acc); assert (this->hdl ); END_CONSTRUCTOR Destructors begin with the DESTRUCTOR(class-name) macro. At the entry to each destructor the methods pointer points to the method table for that class. At the exit of each destructor the object destruction function automatically invokes the base class destructor. Destruction is from the derived class back to the base classes. The destructor for the StdFile class is implemented as: DESTRUCTOR(StdFile) fclose(this->hdl); END_DESTRUCTOR Once you have implemented a class's methods you must define the class itself. The class definition begins with the DEF_CLASS( class-name, base-name ) macro, and ends with the END_DEF macro. Within the class definition each new class method is defined with the DEF_METHOD( base-name, method-name ) macro, and inherited methods are redefined with the REDEF_METHOD( base-name, class-name, method-name ) macro. The REDEF_METHOD macro can, in the C version of the macro, be used at any tune to change the behavior of a class. (This is arguably a bug rather than a feature.) The StdFile class is defined as: DEF_CLASS (StdFile,File) REDEF_METHOD(StdFile,File,Read); REDEF_METHOD(StdFile,File,Write); REDEF_METHOD(StdFile,File,Seek); END_CLASS The definition of a class is actually implemented as the definition of a method table and a runtime function to initialize the table with pointers to functions. This function is called by the USE(class-name) macro, which must be invoked before any actual use of the class. Creating and Destroying Objects With these macros, you can construct objects in two places: on the stack or on the heap. Objects on the stack can exist only within the scope of their declaration, whereas objects on the heap can exist until explicitly destroyed. To create an object on the stack, use the PUSH(object-pointer, class-name, constructor-arguments) macro. (This statement declares object-name as a pointer to an object of the specified class.) Destroy an object on the stack with the POP(object-pointer) macro beƒore it goes out of scope. To create an object on the heap, use the NEW(class-name, constructor-arguments) macro. This expression evaluates to a pointer to an object of the specified class. Destroy an object on the heap with the DELETE(object-pointer) macro (when it is no longer needed). Both of these macros invoke the class destructor method, which should release any resources used by the object. Using Objects You use objects primarily by invoking their methods, and also by accessing their data. You can invoke object methods polymorphically with the SEND macro, or monomorphically with the CALL macro. You can access object data through the object pointer. The SEND( object-pointer, base-name, method-name, method-arguments ) macro invokes the named method of the specified base class as redefined for the class of the object-pointer. The SEND macro calls the function pointed to by the method-name member of the method table pointed to by the first member of the object pointed to by object-pointer. Thus the runtime type of object-pointer determines which function is called. The CALL( object-pointer, class-name, base-name, method-name, method-arguments ) macro invokes the named method of the specified base class as redefined for the specified class. The CALL macro calls the function pointed to by the method-name member of the method table of the specified class. Thus the compiler determines which function to call. Dynamic binding with SEND is the most common form of method invocation. Indeed, I created all of this macro machinery primarily to make dynamic binding possible. Static binding with CALL is most useful when a redefined class method must invoke an earlier definition of that method. Discipline As the implementer of a class library you should always provide a public interface which defines access macros for every SEND, CALL, and data access operation needed by a user of a class. If enforcing data-hiding is more important than performance, then access functions, rather than macros, can be declared in a separate set of interface headers, and the library can be provided as compiled code only. As the user of a class you should never directly access the methods and data of that class. You should always treat a class as a black box, using only the public interface, so that the implementer is free to change the private implementation details. C compilers cannot automatically enforce this discipline, as can C++ compilers, but discipline is needed nonetheless. Porting C Classes to C++ Even if you choose to implement your classes in C, you may want or need a C++ implementation in the future. You can accomplish this port by simply using my macros to generate C++ instead of C. For many of the macros, like POP and END_METHOD, the translation amounts to a no-op. In others, like METHOD, SEND, and CALL, simple C++ code is substituted for more cryptic C code. Listing 2 provides both C and C++ versions of all these macros. Interestingly enough, Borland's Turbo C++ compiler produces smaller and faster output than does its Turbo C compiler for the example File classes, so don't be fooled into thinking that C++ is necessarily less efficient than C. When we moved our project to C++, I ran our code through a Standard C preprocessor and a code beautifier, and followed that with some hand editing. Using this approach I was able to port several thousand lines of our C class code to C++ in a few days. The resulting code did not, of course, take full advantage of the more powerful C++ features, but did provide a clean, working starting point for further development. Conclusion C++ remains my first choice for implementing an object-oriented design. Nonetheless, there can be good reasons to choose C, including include contractual obligations, the need to use an industry standard language, severe portability or performance constraints, and the availability of trained staff. Although Standard C does not support object-oriented programming, it can be extended--using its own preprocessor--to support the essentials of object-oriented programming in a way that is upwardly compatible with C++. What is an Object? A Quick Review Object-oriented design means modelling a problem domain with related objects. Objects have state and are eligible to participate in behaviors which can change that state. Object-oriented programming means implementing an object-oriented design with data structures and functions. Data structures represent the state of an object. Functions associated with an object define the methods for accessing and changing the state of an object, that is, for making the object behave. Simple objects are common in C libraries. For example, the Standard I/O library provides simple file objects consisting of a FILE data structure and a set of associated functions that operate on a pointer to a FILE structure. Inheritance is the relation of derivation among different types, or classes, of objects. If class Z is derived from class Y, and class Y is derived from class X, then X and Y are called base classes of Z. Derivation of a class from a base class means extending the base class by adding new object data, adding new object methods, and redefining existing object methods. Polymorphism, "having many forms," means that an object of a derived class can be referenced wherever an object of a base class is called for, but will behave as appropriate to its type. That is, if an object method has been redefined, then the redefining function, not the original function, will be called. Dynamic binding is a means of achieving polymorphism. With dynamic binding, when a method is invoked the function to call is determined at runtime. Dynamic binding is supported in C++ by virtual functions. C provides no equivalent facility, but switch statements or pointers to functions can be used to imitate it. Static binding is monomorphic. When a method is invoked, the function to call is determined at compile time. Static binding is supported by ordinary C function calls. Extended C Syntax This syntax is an extension of the Standard C syntax and uses the same notation as the C Standard. Symbols are shown in italics, and those defined in the C Standard are also shown in bold face. Undefined terminals are shown in typewriter face. external-declaration: function-definition declaration class-declaration method-implementation class-definition class-declaration: class-methods-declaration class-members-declaration class-methods-declaration: DCL_METHODS ( Class--name , base--name , constructor--parameters ) method--listopt END_METHODS constructor-parameters: method-parameters method-list: method-declaration method-listopt method-declaration: DCL_METHOD ( method-name , method-parameters , method-return-type ) DCL_ABSTRACT ( method-name , method-parameters , method-return-type ) REDCL_METHOD ( method-name , method-parameters , method-return-type ) class-members-declaration: DCL_MEMBERS ( class-name , base-name ) member-listopt END_MEMBERS member-list: struct-declaration-list method-implementation: METHOD (method-specififier) method-body END_METHOD CONSTRUCTOR ( class-name , method-parameters ) constructor-body END_CONSTRUCTOR DESTRUCTOR ( class-name ) method-body END_DESTRUCTOR method-specififier: class-name , method-name , method-parameters , method-return-type method-parameters: ( parameter-type-list ) method-return-type: declaration-specifiersopt constructor-body: CONSTRUCT ( base-name , constructor-arguments ) method-body method-body: declaration-listopt statement-listopt class-definition: DEF_CLASS ( class-name , base-name ) method-definition-list END_CLASS method-definition-list: method-definition method-definition method-definition-list method-definition: DEF_METHOD ( base-name , method-name ); DEF_ABSTRACT ( base-name , method-name ); REOEF_METHOD ( base-name , class-name , method-name ); declaration: declaration-specifiers init-declarator-listopt PUSH ( object-pointer, class-name , constructor-arguments ); primary-expression: identifier constant string ( expression ) USE ( class-name ) SEND ( object-pointer , base-name , method-name , method-arguments ) CALL ( object-pointer , class-name , base-name , method-name , method-arguments ) NEW ( class-name , constructor-arguments ) DELETE ( object-pointer ) POP ( object-pointer ) IS_A ( object-pointer , class-name ) CLASS_NAME ( object-pointer ) constructor-arguments: method-arguments method-arguments: ( argument-expression-list ) base-name: class-name Object class-name: identifier object-pointer: identifier method-name: identifier The Hazards of Macros What I really wanted was a clean, safe, readable syntax for declaring, defining, and using classes in Standard C. I achieved this by using C preprocessor macros to "automagically" generate and intialize structures like those described in the section "Implementing Classes in C." Using macros to provide new syntax is frowned on by many experts, and for good reasons. For one, the C preprocessor is quite primitive, and macros which are easy to use can be amazingly difficult to read and understand. Indeed, my teammates often rib me about "the macros from hell." Another disadvantage is that brain dead tools will not expand macros during source debugging. But the most compelling argument is that once you extend the C syntax, you are no longer programming in C. Instead you are programming in the new language defined by your macros. So be sure that you really need to extend C. If you do, then you are facing a language design task as much as a programming task, so carefully consider the features you really need, and carefully lay out the syntax of your new language. Figure 1 File facility design Figure 2 File facility implementation Listing 1 Macros which add object-oriented extensions to Standards C. /******* OBJECTS.H Copyright 1992 Gregory Colvin ******** This program may be distributed free with this notice. **********************************************************/ #ifndef OBJECTS_H #define OBJECTS_H #include #include #include #ifndef _cplusplus /* Not C++, assume C */ /if !defined(_MSDOS_) defined(_STDC_) #define near /* near pointers are DOS extension */ #endif /* Base class for all objects: */ typedef struct{ char near *name; int initialized; void (*destroy)(void); int (*isA)(const char *className); } ObjectMethods; extern ObjectMethods ObjectTable; void ObjectInitClass(void); typedef struct Object { ObjectMethods near *methods; } Object; void Object_construct(void); void Object_destruct(void); /* Manage "this" pointer stack for methods: */ extern Object * near *pThis; #define MAX_PUSH 512 #if 1 #define PREV_THIS()(*(pThis-1)) #define PUSH_THIS(pObj) (*--pThis = (Object*)(pObj)) #define GET_THIS(Class) ((Class*)*pThis) #define GET_THIS_SAFE(Class) \ (assert(IS_A(*pThis,Class)),GET_THIS(Class)) #define POP_THIS()(*pThis++) #define POP_THIS_SAFE(Class) \ (assert(IS_A(*pThis,Class)),(Class*)POP_THIS()) #else #define PREV_THIS() (*(Object**)_DI) #define PUSH_THIS (pObj) ((* (Object**)_DI) = pObj) #define GET_THIS(Class) (*(Class**)_DI) #define GET_THIS_SAFE(Class) \ (assert(IS_A(GET_THIS(Class),Class)),GET_THIS(Class)) #define POP_THIS() (*(Object**)_DI) #define POP_THIS_SAFE(Class) GET_THIS_SAFE(Class) #endif /* Begin class methods declaration: */ #define DCL_METHODS(Class,Base,ConstructorArguments) \ typedef struct Class##DataStruct Class; \ void Class##_construct ConstructorArguments; \ void Class##_destruct(void); \ struct Class##MethodStruct { \ Base##Methods base; /* Declare and redeclare methods: */ /define DCL_METHOD(Name,Arguments,Return) \ Return (*Name)Arguments /define DCL_ABSTRACT(Name,Arguments,Return) \ Return (*Name)Arguments #define REDCL_METHOD(Name, Arguments, Return) \ void dummy_##Name /* End class methods or class type declaration: */ #define END_METHODS }; /* Begin class members declaration: */ #define DCL_MEMBERS(Class,Base) \ typedef struct Class##MethodStruct Class##Methods; \ extern Class##Methods Class##Table; \ extern void Class##InitClass(void); \ struct Class##DataStruct { \ Base base; #define END_MEMBERS }; /* Begin class type definition: */ #define DEF_CLASS(Class,Base) \ Class##Methods Class##Table = { 0, 0 }; \ int Class##_isA(const char *className) { \ int yes= !strcmp(#Class,className); \ if (!yes) \ yes = ((ObjectMethods*)(&Base##Table))->isA(className); \ return yes; \ } \ void Class##_destroy(void) { \ Class##_destruct(); \ ((ObjectMethods*) (&Base##Table))->destroy(); \ } \ void Class##InitClass(void) { \ ObjectMethods *objTable= (ObjectMethods*)&Class##Table; \ Base##Methods *baseTable= (Base##Methods*)&Class##Table; \ Base##InitClass(); \ if (objTable->initialized) \ return; \ *baseTable = Base##Table; \ objTable->name = #Class; \ REDEF_METHOD(Class,Object,isA); \ REDEF_METHOD(Class,Object,destroy); /* Set and reset methods within a class definition: */ #define DEF_ABSTRACT(Base,Method) \ (Base##Table.Method = 0) #define DEF_METHOD(Base,Method) \ (Base##Table.Method = Base##_##Method) #define REDEF_METHOD(Class,Base,Method) \ (((Base##Methods *)(&Class##Table))-->Method \ = Class##_##Method) /* End class type definition: */ #define END_CLASS } /* Initialize a class before use: */ #define USE(Class) Class##InitClass() /* Begin class method implementation: */ #define METHOD(Class,Name,Arguments,Return) \ Return Class##_##Name Arguments { \ Class *this = POP_THIS_SAFE(Class); /* End class method implementation: */ #define END_METHOD } /* Begin class constructor implementation: */ /define CONSTRUCTOR(Class,Arguments) \ void Class##_construct Arguments { \ Class *this= GET_THIS(Class); \ ObjectMethods *Methods= \ (ObjectMethods *)&Class##Table; /* Must construct Base first in class constructor: */ #define CONSTRUCT(Base,Arguments) \ Base##_construct Arguments; \ ((Object *)this)->methods = Methods; { /* End class constructor implementation: */ #define END_CONSTRUCTOR }} /* Begin class destructor implementation: */ #define DESTRUCTOR(Class) \ void Class##_destruct(void) { \ Class *this = GET_THIS_SAFE(Class); \ ((Object *)this)->methods = \ (ObjectMethods *)&Class##Table; { /* End class destructor implementation: */ #define END_DESTRUCTOR }} /* Monomorphic access to a method of a class: */ #define CALL(pObj,Class,Base,Name,Arguments) \ ((PUSH_THIS(pObj), \ (Base##Methods *)(&Class##Table))->Name Arguments \ ) /* Polymorphic access to a method of a class: */ #define SEND(pObj,Base,Name,Arguments) \ (((Base##Methods*) \ (PUSH_THIS(pObj)->methods))->Name Arguments) /* Construct a new object on the heap: */ #define NEW(Class,Arguments) \ (PUSH_THIS(malloc(sizeof(Class)))), \ Class##_construct Arguments, \ POP_THIS_SAFE(Class))) /* Construct a new object on the stack: */ #define PUSH(ObjName,Class,Arguments) \ Class auto_##ObjName; \ Class *ObjName = (Class*) \ (PUSH_THIS(&auto_##ObjName), \ Class##_construct Arguments, \ POP_THIS_SAFE(Class)) /* Destroy an object on the heap: */ #define DELETE(pObj) \ (SEND(pObj,Object,destroy,()), free(POP_THIS()) /* Destroy an object on the stack: */ #define POP(pObj) SEND(pObj,Object,destroy, ()) /* Test whether object is a member of named class: */ #define IS_A(pObj,Class) \ ((((Object *)pObj)->methods)->isA(#Class)) /* Access to the class name of an object: */ #define CLASS_NAME(pObj) \ (((ObjectTable*)(((Object *)(pObj))->methods))->name) #else /* C++ */ /* Base class for all objects: */ struct Object { virtual int isA(const char *className); virtual const char *classname(); }; /* Begin class methods declaration: */ #define DCL_METHODS(Class,Base,ConstructorArguments) \ struct Class : Base { \ Class ConstructorArguments; \ virtual ~Class(); \ virtual int isA(const char *className); \ virtual const char *classname(); /* Declare and redeclare methods: */ #define DCL_METHOD(Name,Arguments,Return) \ virtual Return Name Arguments #define DCL_ABSTRACT(Name,Arguments,Return) \ virtual Return Name Arguments=0 #define REDCL_METHOD(Name,Arguments,Return) \ virtual Return Name Arguments /*End class methods declaration: */ #define END_METHODS /* Begin class members declaration: */ #define DCL_MEMBERS(Class,Base) /* End class type declaration: */ #define END_MEMBERS }; /* Begin class type definition: */ #define DEF_CLASS(Class,Base) \ const char *Class::classname() { \ return #Class; \ } \ int Class::isA(const char *className) { \ if (strcmp(#Class,className)) \ return Base::isA(#Class); \ return 1; \ } \ /* Set and reset methods within a class definition: */ #define DEF_ABSTRACT(Base,Method) /define DEF_METHOD(Base,Method) #define REDEF_METHOD(Class,Base,Method) /*End class type definition: */ /define END_CLASS /*Initialize a class before use: */ #define USE(Class) /* Begin class method implementation: */ #define METHOD(Class,Name,Arguments,Return) \ Return Class::Name Arguments { /*End class method implementation: */ #define END METHOD } /* Begin class constructor implementation: */ #define CONSTRUCTOR(Class,Arguments) Class::Class Arguments: /*Must construct Base first in class constructor: */ #define CONSTRUCT(Base,Arguments) Base Arguments { /* End class constructor implementation: */ #define END_CONSTRUCTOR } /* Begin class desstructor implementation: */ #define DESTRUCTOR(Class) Class::~Class(){ /* End class destructor implementation: */ #define END_DESTRUCTOR } /* Monomorphic access to a method of a class: */ #define CALL(pObj,Class,Base,Name,Arguments) \ (pObj->Class::Name Arguments) /* Polymorphic access to a method of a class: */ /define SEND(pObj,Base,Name,Arguments) \ (pObj->Name Arguments) /*Construct a new object on the heap: */ #define CREATE(Class) new Class /* Construct a new object on the stack: */ #define PUSH(ObjName,Class,Arguments) \ Class auto##ObjName Arguments; \ Class *ObjName= &auto##ObjName /* Destroy an object on the heap: */ #define DELETE(pObj) \ delete(pObj) /* Destroy an object on the stack: */ #define POP(pObj) /* Test whether object is a member of named class: */ #define IS_A(pObj,className) \ SEND(pObj ,Object,isA,(className)) /* Access to the class name of an object: */ #define CLASS_NAME(pObj) (pObj->classname() #endif /* C vs. C++ */ #endif /* End of File */ Listing 2 Default methods and other runtime support for the macro package. /******* OBJECTS.C Copyright 1992 Gregory Colvin ******** This program may be distributed free with this notice. ********************************************************/ #include #include #include "objects.h" #ifndef_cplusplus /* Not C++, assume C */ /* Object Stack: */ static Object *ObjectStack[MAX_PUSH]; Object * near *pThis = ObjectStack + MAX_PUSH; /* Object Class definition: */ void Object_destroy(void) { GET_THIS_SAFE(Object); } int Object_isA(const char *className) { int result = !strcmp("Object",className); return result; } void Object_construct(void) { } void Object_destruct(void) { } ObjectMethods ObjectTable; void ObjectInitClass(void) { if (ObjectTable.initialized) return; ObjectTable.name = "Object"; ObjectTable.initialized = 1; DEF_METHOD(Object,destroy); DEF_METHOO(Object,isA); } #else /* C++ */ const char *Object::classname() { return "Object"; } int Object::isA(const char *className) { return !strcmp("Object",className); } #endif /* C vs. C++ */ /* End of File */ The Alpha-Beta Filter Robert Penoyer Robert Penoyer has an M.S. in electrical engineering from the University of Southern California. He has 17 years experience in electronic circuit and system design and simulation. He has been programming in C for five years and working with alpha-beta filters for four years. Bob can be reached on GEnie at RPENOYER. or on Compuserve at 71603,1335. Data collected or received in the real world is nearly always corrupted by errors, or noise. With relatively consistent data--information that varies more slowly than the noise--you can look at more than one measurement at a time and more accurately separate the data from the noise. To minimize the effects of noise, the measured data can be filtered, or smoothed. In an ideal world you could remove all the noise and be left with an exact duplicate of the original information. (But in an ideal world, the information would not have been corrupted in the first place.) The best you can hope to accomplish in the real world is to remove as much noise as possible while retaining as much of the original information as possible. The a-b filter is a classic option for smoothing a corrupted signal and tracking discrete data. Child of the Kalman Filter The a-b filter is a derivative of the Kalman filter. The Kalman filter, a so-called optimal filter, performs better than any other linear filter, under specific conditions (Gelb 1974, Meditch 1969, Mendel 1974). You pay for such excellent performance with extreme complexity and computationally-intensive software. The a-b filter resembles the Kalman filter in its nature, structure, and performance, but without as much complexity. The Kalman filter automatically varies its coefficients to optimize its performance as a function of the statistics of the incoming corrupted data. The a-b filter, on the other hand, uses either fixed coefficients or coefficients that vary as the programmer requires them to vary. Thus, the a-b filter avoids entirely, or almost entirely, the computational overhead necessary for the Kalman filter to perform properly. The a-b filter is called suboptimal. But don't interpret that to mean it performs poorly. On the contrary, the a-b filter can be a highly effective smoother. By varying the coefficients of this filter, you can control the length of the data history used to estimate the data's true value from the most recent measurement. The Classic a-b Filter The a-b filter has about as many forms as there are sources that describe it (Benedict and Bordner 1962, Lewis 1984, Morris 1986). The following equation set is based on one presented by Morris. Equation 1 Equation 2 Equation 3 Equation 4 where: s = position v = velocity a = multiplier on the position measurement error (0œaœ1) b = multiplier on the velocity measurement error (0œbœ1) T = time between measurements n parameter value for this time increment pn predicted parameter value for this time increment p(n+1) predicted parameter value for the next time increment m measured parameter ^ estimated value (corrected prediction) Though these equations use the terms position and velocity, any parameters that are related by the derivative operator (e.g., d/dx) can be substituted. The equations form a recursive relationship and are used formally as follows: Given a new position measurement, an estimate is made of the true position and velocity for the current time increment using Equation 1 and Equation 2. The estimates from Equation 1 and Equation 2 are used to predict the position and velocity for the next time increment using Equation 3 and Equation 4. When the next time increment occurs, i.e., when a new position measurement is made, the process is repeated. This method requires that T be constant or nearly so. The a and b terms give the filter its name. These fixed coefficients replace the optimized coefficients of the Kalman filter. Simplifying the a-b Equations While fine as given, Equation 1 through Equation 4 can be massaged to make the programming effort simpler. However, beware that any time a derivative is used on noisy measurements, the derivative tends to multiply the effects of the noise. Because the velocity term in the a-b filter is related to the position term by the derivative operator, the velocity estimate can be very poor. Therefore, I assumed that the velocity estimate of this a-b filter would not be used to provide a velocity estimate outside the equations. That is, I used the a-b filter only to estimate position from some set of position measurements. (If you should want to use the velocity term as an estimator of true velocity, consider smoothing it with a second a-b filter.) Because I was not going to use the velocity term outside the equations, I could afford to modify it. First, I rearranged Equation 2 by multiplying through by T: Equation 5 I further modified Equation 5 such that T was absorbed into v. This is a valid simplification since I do not use v directly. Consider these facts about T: T operates only on v Every place v occurs, T occurs (with the understanding that Equation 4 is also multiplied by T) Given these facts, the absorption of T into v has no effect on the predicted and estimated positions. Each of the v variables is simply scaled by T. Consider, also, these facts about T: T is the time between position measurements T depends upon the operating speed of the computer system (assuming its speed determines the time between measurements) The filter's characteristics (i.e., its predictions and estimations of s) are not affected by T. That is, though a faster computer yields a faster filter, the damping of the filter is not affected. This will be made clear in the geometric presentation that follows. When T is absorbed into v, Equation 5 becomes: Equation 6 As noted above, Equation 6 simply causes v to be scaled by T. To begin the recursive estimation sequence, the initial estimate of position, s^o, is taken to be the first measured position, sm0. Set the initial velocity estimate, v^o, to 0 unless you have a good reason for doing otherwise. Given these starting conditions, the equations can be rearranged, placing Equation 3 and Equation 4 first. The indices in these two equations then become pn rather than p(n+1) and n-1 rather than n so that these equations are used to predict the parameters for the current time increment based on the previous time increment. (Note that, as originally formulated, the equations were used to predict the parameter value for the next time increment based on the current time increment.) Also, since Equation 4 provides no new information, it can be ignored--the old estimate of velocity becomes the new predicted velocity. Following the elimination of Equation 4 and the replacement of vpn with v^n-1 in Equation 6, the reformulation of the original set of equations and their operating sequence now becomes: Equation 7 Equation 1 Equation 8 When the above sequence begins, the entry point is Equation 7 with s^o set equal to the first value of smn (i.e., sm0) and v^o set equal to 0. This set of equations and conditions are the basic smoothing algorithm. The actual programming implementation adds one more step. The parenthetical term in both Equation 1 and Equation 8 is the difference between the measured position and the predicted position. This difference is the measurement error. Instead of calculating it twice, once for each equation, a single measurement error term can be calculated following Equation 7: emn = smn - spn Equation 9 Thus, the actual implementation of the a-b filter is: Equation 7 Equation 9 Equation 1 Equation 8 This formulation is very simple. It lends itself to simple coding and to a simple geometric explanation of the a-b filter's operation. A Geometric Explanation of the a-b Filter Assume that the filter has been running. That is, the starting procedure has been completed and, when a new measurement is made, the filter runs through the sequence of equations. The equations use the estimates from the previous time increment and the current position measurement to estimate the true current position. Figure 1 shows that, given the starting point, s^n-1, and the velocity, v^n-1, a straight line predicts the new position, spn. This is the operation performed by Equation 7. Since v^ includes the T multiplier, the units of v^ (i.e., Tv^) are distance, not velocity. In general, the measured position will not coincide with spn. Equation 9 quantifies the measurement error. This error is due primarily to noise. However, some of the measurement error legitimately may be due to a change in velocity. Therefore, since some of the difference may be valid, all the difference cannot be ignored. Thus, a, a positive multiplier that generally has a magnitude smaller than 1, is used to add a fraction of the measurement error to the predicted position to improve the position estimate. Equation 1 yields the final estimate of position. The function of Equation 1 is illustrated in Figure 2. From the geometry of Figure 1 and Figure 2, it can be seen that if v is varied inversely as T is varied, then none of the details in the figures (i.e., positions or slopes) will change. That is, if the data and measurements are speeded up equally, the details of the figures will not be affected. Hence, variations in T affect only the filter's speed; damping is not affected. Relating b to a Just as a determines the fraction of the measurement error applied to the position prediction, b determines the fraction of the measurement error applied to the velocity prediction. Equation 8 uses b for this purpose. Benedict and Bordner have derived a relationship between a and b that optimizes the filter's performance: Equation 10 This relationship is based on the condition that the best estimate be provided for a system that is tracking noisy information that is changing at a constant velocity. This may or may not be satisfactory for your particular application. Equation 10 yields a slightly underdamped system. Being underdamped implies that the system will "ring" somewhat. That is, if the data makes a sudden step in position, the estimate of position will overshoot the true position at least once before settling down to an accurate estimate. A more heavily damped system might perform better in your application. Benedict and Bordner offer an a-b relationship for a critically damped system. If their relationship is rearranged to solve for b, it will be found that damping is not critical. To get closer to critical damping, Equation 11 multiplies the rearranged relationship by 0.8. Equation 11 Equation 11 yields a system that is nearly critically damped. (In a critically-damped system, the output agrees with the input as quickly as possible with no ringing.) By using Equation 11 instead of Equation 10, you can reduce--but not eliminate--the ringing response. Because a is never permitted to exceed the range 0œaœ1, the quantity under the radical is never negative. Consider Equation 10 and Equation 11 merely as reference points. As long as the limits 0œaœ1 and 0œbœ1 are not violated, you are free to select the values of a and b to suit your needs. A Programming Example in C I begin by estimating measured data uncorrupted by noise. Figure 3 contains a profile of the data that begins at a steady state, ramps to a second steady state, then steps to a third steady state. (Normally, you would not filter clean, uncorrupted data like this for an estimated value, but I am using it to demonstrate the performance of the a-b filter against a ramp and a step.) The profile might represent a sequence of measured voltages, currents, distances, altitudes, volumes of storage tank contents, disk-drive head positions, highway traffic volumes, percentage of x in a vat full of y, etc. I wrote and compiled ALPHABET.C (Listing 1) using the Borland C++, Version 3.1 compiler configured for MS-DOS. I verified it on a system running MS-DOS, version 5.0. The plotting functions assume a standard VGA monitor (640 x 480 pixels). If your monitor is different, modify the #define statements for PLOTx and STEADY_STATEx by simple scaling. profile (Listing 1) generates the data profile of Figure 3. ALPHABET .C is a simple demonstration of the performance of the a-b filter for two sets of a-b values. In each case, I give a an arbitrary value of 0.25; thus, 25% of each measurement error is used as a correction to the predicted position. Over the first part of the program, b is determined by Equation 10, the so-called optimal value. (It may not be optimal for your application.) The second part of the program uses the b value determined by Equation 11, the so-called near-critically damped value. First, the program plots the data profile. Once it plots the profile, the program pauses until you press a key. After you press a key, the program plots the response of the a-b filter to the data profile over the profile. You will see slight overshoots as the profile begins and ends the ramp. You will notice a significant overshoot following the step. Pressing another key plots the filter's output by itself. Press another key to produce the signal corrupted by noise. gauss_noise generates a mean-zero, gaussian (normal) noise function. To do so, it uses the Box-Muller method (Press, et al. 1988, MathSoft, Inc. 1988). The noise is given an arbitrary standard deviation of s = 10. The mean is set to m = 0 to comply with the requirement that the mean of the corrupting noise be zero. If the mean were non-zero, the a-b filter's response would be offset by the mean value of the noise. In gauss_noise the variable mean is retained, despite being set to zero, to clarify its part in the computation of the noise function. Continue pressing a key to run through all the combinations of clean signal, noisy signal, and smoothing provided by Listing 1. Filter Performance Several figures illustrate the filter's performance. Figure 4 compares the original data profile with the resultant output due to the a-b relationship of Equation 10. The filter tracks the ramp well. It tracks the step well, too, except for the overshoot following the step. The ringing of the underdamped filter is apparent here. Figure 5 illustrates the data profile corrupted by noise. The noisy data remains the same throughout a single run of the program so that each step of the program operates on the same noise. The noise is different each time the program runs so that the noise you will see if you run it will be different from that shown in Figure 5. Figure 6 compares the filtered noisy data with the original, uncorrupted data. Keep in mind that the intent is to recover the original, uncorrupted data as accurately as possible, thus the comparison. Figure 7 and Figure 8 use the more heavily damped filter due to the a-b relationship of Equation 11. You might find it useful to modify Listing 1 by supplying your own values for a and b. Just be sure that neither is permitted to be negative or to exceed a value of 1.0. alpha_beta performs the actual a-b filter algorithm. Three things should be said about this function. First, if it were used in real time, all variables could be scalar. No arrays would be needed. As each estimated position (or whatever) value was calculated it could be returned to the calling function. Only the most recent values of variables s_est and v_est need to be preserved. Therefore, if the algorithm were to be modified to filter measurements one call at a time, s_est and v_est should be made static variables. Second, keep in mind that the method used here has absorbed the term Tv into the single variable, v. If you want to make use of v as well as s, be sure to divide v (actually, Tv) by T, the fixed period over which each measurement is made. As implemented, the b/T term of Equation 2 has been change to simply b, thus eliminating any need for a division. If you would rather preserve the true scaling of v, you can absorb T into the calculation of b, avoiding repeated divisions by T. Finally, you will have to make your choice of a based on the nature of the data you are attempting to smooth. A large a (near 1.0) results in very little smoothing and very little noise reduction. A small a (near 0.0) results in a lot of smoothing, i.e., a lot of noise reduction and very rounded estimates of data steps. Your choice must be based as much on your good judgment as it is on the data to be smoothed. Equation 10 or Equation 11 will usually provide a suitable b value, though you might want to experiment with this too. Let experience and testing be your guide. Tracking Tracking, another important application of the a-b filter, differs from smoothing by centering a tracking window on the predicted position of the measured data as determined by Equation 7. When the measured data falls inside the window, that data is taken as a valid measurement and used to update the filter's position estimate. Data that falls outside the window is ignored. When the data is ignored for failing to fall inside the track window, the calculation of Equation 9 cannot be completed. Instead, ignore Equation 9 and assign 0 as the error value. Then carry out the other calculations. Because the a-b filter incorporates a velocity term, you use it to predict where the data should have been. When the error value is zero, the velocity estimate of Equation 8 is not changed so that the velocity estimate remains constant. That is, the velocity estimate, based on the history of calculations up to the data that failed to fall within the track window, is used to update the position prediction of Equation 7 and, hence, the position of the track window. If the data falls inside the track window once again, the normal calculation of measurement error, Equation 9, is resumed and tracking updates continue. If the system goes through too many iterations without the data falling inside the track window, the program must declare loss of track. You must determine for your application what constitutes too many misses. Once you determine that track has been lost, you need a search-and-acquire algorithm to reacquire the data. During acquisition, you might find it useful to scale a (and b) from a larger value (near 1.0) when the target is first acquired, down to a more suitable value (e.g., 0.25 or so) to maintain track. (Note that initially setting s^o = smo is equivalent to starting with an alpha symbol value of 1.0.) The down scaling can be accomplished over a few iterations. This strategy can speed the convergence of the position estimate to the true data value. You will have to determine how wide the track window should be for your application. You will also have to figure out how to determine that track has been lost and how the data is to be reacquired. References Benedict, T.R., and G.W. Bordner. July 1962. "Synthesis of an Optimal Set of Radar Track-While-Scan Smoothing Equations," IRE Transactions On Automatic Control, pp. 27-32. New York, NY: The Institute of Radio Engineers. Gelb, Arthur. 1974. Applied Optimal Estimation, Chapter 4. Cambridge, Massachusetts: The M.I.T. Press. Lewis, Frank L. 1986. Optimal Estimation: With an Introduction to Stochastic Control Theory, pp. 88-89. New York, NY: John Wiley & Sons. Meditch, J.S. 1969. Stochastic Optimal Linear Estimation and Control, Chapter 5. New York, NY: McGraw-Hill, Inc. Mendel, Jerry M. 1973. Discrete Techniques of Parameter Estimation, pp. 159-162. New York, NY: Marcel Dekker, Inc. Morris, Guy V. 1988. Airborne Pulsed Doppler Radar, Section 12.7.1. Norwood, MA: Artech House, Inc. Press, William H., et al. 1988. Numerical Recipes in C, pp. 216-217. New York, NY: Cambridge University Press. MathSoft, Inc. Summer 1988. "Industrial Engineering Prof. Uses MathCAD for Simulations, Statistics, and Teaching," MathCAD User's Journal, Vol. 2, No. 3, p. 4. Cambridge, MA: MathSoft, Inc. Figure 1 The last position estimate is used to predict the current position. The prediction is based on the last estimate of rate. Figure 2 A fraction of the measurement error is added to the current position prediction to improve the position estimate. Figure 3 Clean data profile Figure 4 Clean data profile with underdamped alpha-beta smoothing Figure 5 Noisy profile Figure 6 Underdamped alpha-beta smoothing of noisy profile with clean data profile Figure 7 Clean data profile with near-critically damped alpha-beta smoothing Figure 8 Near-critically damped alpha-beta smoothing of noisy profile with clean data profile Listing 1 Generates the data profiles of Figure 3 through Figure 8 /* ALPHABET.C Demonstrate the alpha-beta filter with a particular data profile, with and without gaussian noise, and with different alpha-beta values. The graphic features here assume the presence of a VGA monitor. This code was written for Borland C++, Version 3.1, coded for MS-DOS. */ #include #include #include #include #include #include #include #define PLOT1 120 /* range of x for first part of plot */ #define PLOT2 310 /* range of x for second part of plot */ #define PLOT3 430 /* range of x for third part of plot */ #define PLOTEND 640 /* end of x range */ #define STEADY_STATE1 120.0 /* first steady state y value */ #define STEADY_STATE2 400.0 /* second steady state y value */ #define STEADY_STATE3 70.0 /* third steady state y value */ #define STEADY_STATEBOT 480.0 /* bottom limit for steady state values */ #define ALPHA 0.25 /* arbitrary alpha value */ void alpha_beta(double *y,double *y_smoothed, double alpha,double beta); void profile(double *y); void gauss_noise(double *noise); void plot_it(double *y); void message(char *string1,char *string2, char *string3); void main(void) { int x,graphdriver=DETECT,graphmode,errorcode; double *y_pure,*y_noisy,*y_smoothed,*noise; double alpha,beta; /**********************************/ /* allocate memory for the arrays */ /**********************************/ if((y_pure = (double *)malloc(PLOTEND*sizeof(double)))== NULL) { printf("\a"); cprintf("Unable to allocate space for array "); cprintf("y_pure[]\n\r"); exit(1); } if((y_noisy = (double *)malloc(PLOTEND*sizeof(double)))== NULL) { printf("\a"); cprintf("Unable to allocate space for array "); cprintf("y_noisy[]\n\r"); exit(1); } if((y_smoothed = (double *)malloc(PLOTEND*sizeof(double)))== NULL) { printf("\a"); cprintf("Unable to allocate space for array "); cprintf("y_smoothed[]\n\r"); exit(1); } if((noise = (double *)malloc(PLOTEND*sizeof(double)))==NULL) { printf("\a"); cprintf("Unable to allocate space for array "); cprintf("noise[]\n\r"); exit(1); } /****************************/ /* initialize graphics mode */ /****************************/ registerbgidriver(EGAVGA_driver); initgraph(&graphdriver,&graphmode,""); errorcode=graphresult(); if(errorcode != grOk) { printf("\a"); cprintf("Graphics error: %s\n\r", grapherrormsg(errorcode)); exit(1); } /*************************************************/ /* generate the pure and corrupted data profiles */ /*************************************************/ /* generate the clean profile */ profile(y_pure); /* generate the noise */ gauss_noise(noise); /* corrupt the clean data with noise */ for(x=0;x Redefining floating-point types has repercussions on the standard math function prototypes in . Using a generic type also impacts these standard math library functions. Changing floating-point types generally requires macro wrappers for all the math functions in . real_t.h, takes care of these problems by creating macros for all the floating-point functions. The macro wrappers ensure that a program using a math function calls the right function. In the case where the same function supports more than one precision, the macros in real_t.h cast the return value and parameters correctly. In addition to ensuring that the proper precision version of a particular function is called, macros wrappers for all the functions in yield an opportunity for embedding debugging code in an application. (See "Debugging with Macro Wrappers", CUJ, October 1992.) Pinpointing what line of source code caused a floating-point exception is easy by using the predefined macros __FILE__ and __LINE__, along with parameter checking of the standard math functions. You can also make safe versions of the standard math functions by checking for out-of-range parameters. One fixup for such a parameter is to substitute a default result that is in range. This will hide the problem, of course. But you can also trap the error and report it, then use such a substitution to recover gracefully. Effects on Just as the function prototypes in needed some adjustment, the constants defined in also need adjustment to account for different precision. real_t.h defines a new set of constants that begin with REAL_. You use these constants with the generic type real_t. real_t.h sets these constants to the appropriate values of FLT_, DBL_, or LDBL_, depending on the desired precision. real t.h also redefines all the FLT_, DBL_, and LDBL_ constants if you are not using mixed precision. Manifest Constants When dealing with different floating-point types, there are even caveats concerning constant data and how to declare constants, so you can modify the precision at which the compiler stores constants in a program. To prevent conversion and possible loss of accuracy, the precision of manifest constants should match the precision of any floating-point variables that use them. Casting constants upon assignment may eliminate a compiler's warning that a conversion is taking place, but it does not ensure the program stored the constant in the program file the way you may desire. For example, long double x = (long double)0.1; is not the same as nor as accurate as: long double x = 0.1L; To ensure that a compiler stores constants in the same precision as the floating-point type, real_t.h uses a string-pasting macro. The version for storing manifest constants as floats is: #define RC( x ) ( x##F ) And the version for storing manifest constants as long doubles is: #define RC( x ) ( x##L ) The macro RC(x) provides a mechanism for adjusting the precision of constants as you change the type. real_t.h defines the macro RC to match the desired floating-point type. The only limitation to RC (actually just an inconvenience) is that you have to use the macro. In a similar fashion, you may wish to define a format code for the printf family of functions. This format code would change based upon the precision of the generic type real_t. Parameter Passing A compiler may employ different mechanisms for passing and returning floating-point values from functions, possibly depending on compile-time options and data sizes. In the case of MSC, the compiler uses a global location called the floating point accumulator (_fac) for functions that return either floats or doubles. MSC uses the numeric data processor (NDP) stack for functions that return long double. BC uses the NDP stack regardless of the type of floating-point number. MSC further complicates matters by employing a hidden pointer parameter for functions that you declare with the _pascal calling convention instead of _cdecl. Most of the time this behavior does not matter and is hidden from the programmer. About the only time this will become a concern is when you are creating a dynamic link library (DLL) for Microsoft Windows and are exporting functions from the DLL that return floating-point types. Since DLLs have a different data segment than the calling application, but share the same stack, there can be a bit of confusion. It can be hard to guess, for example, what segment the hidden pointers point to, or contains the floating point accumulator, or contains the NDP stack. The best way to avoid problems for Windows DLLs that export functions returning floating-point types is to use _pascal calling convention and the large memory model. Using real_t.h To use real_t.h with an existing program, just add the line: #include "real_t.h" You should include real_t.h after you list all the standard library headers that a module needs. If your program makes any calls to the standard library math functions declared in , the program must include before real_t.h. If your program uses any of the manifest constants defined in , the program must include before real_t.h. real_t.h reacts to and modifies its behavior in response to six different constants. You can define these constants on the compiler command line or inside your program. Either way, you must define them before you program includes the file real_t.h. The first three are REAL_????, where ???? is either FLT, DBL, or LDBL. These three constants control what the default floating-point precision is. REAL_FLT is float, REAL_DBL is double, and REAL_LDBL is long double. If an application does not define one of these three constants and still includes real_t.h, REAL_DBL is assumed. This results in designating double as the default floating-point type real_t. The constant REAL_MP determines if existing standard types get redefined. If you define REAL_MP, your program desires mixed precision. The existing standard types do not get redefined to the desired floating-point type. Defining REAL_MP also prevents the constants in float.h from getting redefined. The constant REAL_NL controls whether the standard keyword long gets redefined. To convert the long double type to another type requires defining long to nothing. If you define REAL_NL and do not define REAL_MP, then long gets defined to nothing. /* kludge for long double */ #if defined ( REAL_NL ) && !defined ( REAL_MP ) #define long #endif The constant REAL_SAFE causes real_t.h to wrap the log, log10, pow, and sqrt functions in safe parameter-checking macros. This demonstrates how to implement a safe version of some (not all) of the math functions. Comparing Precisions There are two test programs (REAL_L2.C in Listing 2 and REAL_L3.C in Listing 3) that demonstrate the effects of different precision. They give some "real world" indications of the effects that floating-point precision has on speed and accuracy. The size differences between the three different floating-point types are obvious, so the programs do not demonstrate differences in size. Table 2 summarizes the results of running Listing 2 for float, double, and long double types. Table 3 lists similar results for running Listing 3. I compiled the programs using MSC with the compiler options /FPi87 /0d /DREAL_???? The first option forces the use of a math co-processor. The second prevents code optimization. The third defines either the constant REAL_FLT, REAL_DBL, or REAL_LDBL. This is how you adjust the precision at compile time. You will probably see different results using a different compiler or different compiler options, such as for a co-processor emulator version. Although interesting, analyzing all the possible combinations would soon balloon into a project larger than I can concisely summarize in a single article. For both Listing 2 and Listing 3, the error value represents the accumulated error due to lack of accuracy. Ideally, the error should be 0.0. For simplicity and to facilitate timing, I obtained the value of error using a debugger instead of using printf. The time is in seconds. Times are for testing on a 20MHz 386 with a math co-processor. Smaller numbers represent faster better performance. Speed The speed differences are interesting. The results of testing Listing 2 indicates the performance degrades as the precision increases. Listing 3 results shows the opposite--the performance increases as the precision increases. This may seem like conflicting data, but I came up with a explanation that makes sense. At least to me. My interpretation of the results is that the larger the size the more overhead in basic assignment operations. Listing 2 emphasizes simple operations and asassignments. The float version is the fastest because floats are smaller, resulting in less physical memory being pushed around. In the case of Listing 3, the long double version is the fastest. This is due to the fact that long double is the native type for the math co-processor. Using the native long double when calling the math functions eliminates the need to convert from one type to another. Type float is the slowest because the program must not only repeatedly convert from long double to double but also from double to float. Consequently, a program that makes a lot of use of the math functions in may see some speed improvement with the long double type. I base these conclusions solely upon the data generated by Listing 2 and Listing 3. To get a clear perspective on how a specific compiler treats different types would entail disassembling the compiled code. Accuracy Accuracy improves with precision. This is no surprise. In the case of Listing 2, the difference in magnitude between the error for float versus double is about the same as the difference between FLT_EPSILON and DBL_EPSILON. This means that FLT_EPSILON divided by DBL_EPSILON is about the same value as the error for the float version divided by the error for the double version. The same is true for the long double type compared to the double or the float. The difference in magnitude is about the same as the difference in magnitude between LDBL_EPSILON and DBL_EPSILON or FLT_EPSILON. In the case of Listing 2, accuracy is directly related to precision. This relationship does not hold true for Listing 3. Listing 3 tests the accuracy of one of the math functions, sqrt. I am tempted to conclude that the loss of accuracy in Listing 3 occurs mainly in the sqrt function and in any type conversion in calling the sqrt function. Hence, the differences between the error results are due to the differences between the two functions sqrt and sqrtl. Since there is not a float version of the sqrt function, both the float and the double version use the same function, sqrt. It is not surprising then that the corresponding versions generated the same amount of error. The long double type has its own function sqrtl. It would be interesting, from the standpoint of both accuracy and speed, to see the effect of using the long version with the other precisions. In the case of Listing 3, accuracy did not improve when going from type float to the higher precision type double. It did improve by going to the long double type. Conclusions Real numbers in the world of computers are an oxymoron. The type double can represent roughly 264 different real numbers. These numbers are spread out between the limits--DBL_MIN and DBL_MAX. This is rather limited. By definition, there are actually an infinite number of different numbers between any two different real numbers you can name--even if they are very close together. Likewise, a binary representation of some real numbers, (0.1, 1/3 etc.) is truly just an approximation. So by definition, working with real numbers in the binary world is limited in accuracy. Fortunately, imperfect accuracy is "good enough" most of the time. And for situations that require varying "goodness," there are three different floating-point types in C that one can use. Deciding which type is the best match for a specific application requires an assessment based upon the size, speed, and accuracy needs of the application. The best way to determine what type is best is to test different precision versions of an application. The include file real_t.h provides the tools to easily switch floating-point types at compile time. It uses some aggressive and uncommon macro and preprocessor tactics, but it can even help existing applications change precision at compile time. So instead of just using the default type double, try experimenting your code with the three different floating-point representations that most modern C and C++ compilers offer. There are usually advantages of one type over the others when addressing specific applications needs Table 1 Comparison of floating-point sizes and precisions (on a PC) Type Size (bytes) Significant Epsilon Digits ------------------------------------------------------------------ float 4 7 1.192092896e-07 double 8 15 2.2204460492503131e-016 long double 10 19 5.4210108624275221706e-020 Table 2 Comparison of accuracy performance for different versions of Listing 2 Type Error Time ------------------------------- float 6.70556 1.16 double 2.49808e-008 1.21 long double 4.83773e-012 1.48 Table 3 Comparison of accuracy and performance for different versions of Listing 3 Type Error Time ------------------------------- float 3.85273e-008 3.85 double 3.85272e-008 3.46 long double 1.80687e-011 3.35 Listing 1 real_t.h -- contains all the needed preprocessor gymnastics to make different precision versions of the same application /* REAL_T.H */ #if !defined ( REAL_T_INCLUDED ) #define REAL_T_INCLUDED /* Minimum numbers before sqrt()fails */ #define FLT_SQRT_MIN 1.0e-19F #define DBL_SQRT_MIN 1.0e-154 #define LDBL_SQRT_MIN 1.0e-2466L #if defined ( REAL_FLT ) /* Type and constant macro for float */ typedef float real_t; #define RC( x ) ( x##F ) /* float.h constants */ #define REAL_DIG FLT_DIG #define REAL_EPSILON FLT_EPSILON #define REAL_GUARD FLT_GUARD #define REAL_MANT_DIG FLT_MANT_DIG #define REAL_MAX FLT_MAX #define REAL_MAX_10_EXP FLT_MAX_10_EXP #define REAL_MAX_EXP FLT_MAX_EXP #define REAL_MIN FLT_MIN #define REAL_MIN_10_EXP FLT_MIN_10_EXP #define REAL_MIN_EXP FLT_MIN_EXP #define REAL_NORMALIZE FLT_NORMALIZE #define REAL_RADIX FLT_RADIX #define REAL_ROUNDS FLT_ROUNDS #define REAL_SQRT_MIN FLT_SQRT_MIN #if !defined ( REAL_MP ) #undef DBL_DIG #undef DBL_EPSILON #undef DBL_MANT_DIG #undef DBL_MAX #undef DBL_MAX_10_EXP #undef DBL_MAX_EXP #undef DBL_MIN #undef DBL_MIN_10_EXP #undef DBL_MIN_EXP #undef DBL_RADIX #undef DBL_ROUNDS #undef DBL_SQRT_MIN #undef LDBL_DIG #undef LDBL_EPSILON #undef LDBL_MANT_DIG #undef LDBL_MAX #undef LDBL_MAX_10_EXP #under LDBL_MAX_EXP #undef LDBL_MIN #undef LDBL_MIN_10_EXP #undef LDBL_MIN_EXP #undef LDBL_RADIX #undef LDBL_ROUNDS #undef LDBL_SQRT_MIN #define DBL_DIG REAL_DIG #define DBL_EPSILON REAL_EPSILON #define DBL_MANT_DIG REAL_MANT_DIG #define DBL_MAX REAL_MAX #define DBL_MAX_10_EXP REAL_MAX_10_EXP #define DBL_MAX_EXP REAL_MAX_EXP #define DBL_MIN REAL_MIN #define DBL_MIN_10_ EXP REAL_MIN_10_EXP #define DBL_MIN_EXP REAL_MIN_EXP #define DBL_RADIX REAL_RADIX #define DBL_ROUNDS REAL_ROUNDS #define DBL_SQRT_MIN REAL_SQRT_MIN #define LDBL_DIG REAL_DIG #define LDBL_EPSILON REAL_EPSILON #define LDBL_MANT_DIG REAL_MANT_DIG #define LDBL_MAX REAL_ MAX #define LDBL_MAX_10_EXP REAL_MAX_10_EXP #define LDBL_MAX_EXP REAL_MAX_EXP #define LDBL_MIN REAL_MIN #define LDBL_MIN_10_EXP REAL_MIN_10_EXP #define LDBL_MIN_EXP REAL_MIN_EXP #define LDBL_RADIX REAL_RADIX #define LDBL_ROUNDS REAL_ROUNDS #define LDBL_SQRT_MIN REAL_SQRT_MIN typedef long double long_double; #define long_double real_t #define double real_t #endif /* #if !defined ( REAL_MP ) */ /* math.h functions */ #define acos( x ) ( (float)acos((double)(x) ) ) #define asin( x ) ( (float)asin((double)(x) ) ) #define atan( x ) ( (float)atan((double)(x) ) ) #define atan2( y, x ) ( (float)atan2((double)(y), \ (double)(x) ) ) #define ceil( x ) ( (float)ceil( (double)(x) ) ) #define cos( x ) ( (float)cos( (double)(x) ) ) #define cosh( x ) ( (float)cosh( (double)(x) ) ) #define exp( x ) ( (float)exp( (double)(x) ) ) #define fabs( x ) ( (float)fabs( (double)(x) ) ) #define floor( x ) ( (float)floor( (double)(x) ) ) #define fmod( x, y ) ( (float)fmod( (double)(x), \ (double)(y) ) ) #define sin( x ) ( (float)sin( (double)(x) ) ) #define sinh( x ) ( (float)sinh( (double)(x) ) ) #define tan( x ) ( (float)tan( (double)(x) ) ) #define tanh( x ) ( (float)tanh( (double)(x) ) ) #if defined ( REAL_SAFE ) #define atof( x )\ ( ( (x) == NULL ) ? \ RC( 0.0 ) : (float)atof( (x) ) ) #define log( x ) \ ( ( fabs( (x) ) ) < REAL_MIN ? \ (float)log( (double)REAL_MIN) : \ (float)log( (double)fabs( (x) ) ) ) #define log10( x ) \ ( ( fabs( x ) ) < REAL_MIN ? \ (float)log10((double)REAL_MIN) : \ (float)log10( (double)fabs( (x) ) ) ) #define pow( x, y ) \ ( fabs( (y) ) < REAL_MIN ? \ RC ( 1.0 ) : \ ( fabs( (x) ) < REAL MIN ? \ ( (y) #include void main( void ) { int i; real_t x, error; x = RC( 1.0 ); error = RC( 0.0 ); for ( i = 0; i < 30000; i++ ) { error += fabs( x - RC( 0.1 ) * x * RC( 10.0 ) ); x++; } error = error; } /* function main */ /* End of File */ Listing 3 REAL_L3.C -- demonstrates the effects of different precision. This code uses the native type double long for speed. /* REAL_L3.C */ #include #include void main( void ) { int i; real_t x, error; x = RC( 1.0 ); error = RC( 0.0 ); for ( i = 0; i < 30000; i++ ) { error += fabs( x - sqrt( x ) * sqrt( x ) ); x++; } error = error; } /* function main */ /* End of File */ Standard C Wide Character Streams P.J. Plauger P.J. Plauger is senior editor of The C Users Journal. He is convenor of the ISO C standards committee, WG14, and active on the C++ committee, WG21. His latest books are The Standard C Library, published by Prentice-Hall, and ANSI and ISO Standard C (with Jim Brodie), published by Microsoft Press. You can reach him at pjp@plauger.com. Introduction This is the third (and last) in a series of columns on the facilities being added to Standard C. (See "State of the Art: Large Character Set Support," C Users Journal, May 1993 and "State of the Art: Large Character Set Functions," C Users Journal, June 1993.) By the time you read this, the first round of balloting should be complete on the "normative addendum" that adds these facilities. At least two more rounds must occur before they become a part of the C Standard. I have so far described: why Standard C needs more support for large character sets how existing functions change in the Standard C library what's in the new header how many of the added functions work I conclude this month by describing the largest single group of those functions, the ones that support wide-character input and output. I remind you again that what I'm describing here can still change in some details, as a result of comments during balloting. I expect, however, that those changes should be small. Problems with Large Character Sets Remember that the overall goal of these additions is to ease the handling of text expressed with a very large character set. Japanese, Koreans, Chinese, Arabs, and typesetters all face a similar problem. They regularly work with "alphabets" containing thousands, or even tens of thousands, of characters. And that presents a host of problems: how to key in text from a huge character set how to display a huge character set on a screen how best to represent text inside a program how best to represent text in external data streams and files We learned a long time ago to divorce the first two issues from programs and programming languages. Operating systems change keystrokes into a stream of bytes for us. Similarly, they convert an output stream into suitable dots on a screen or printer. Yes, the separation breaks down from time to time, but it is an important ideal to keep striving toward. Put another way, MS-DOS and UNIX may have to worry about how to display Kanji or tenpoint Times Roman--C and C++ should not. We have learned more recently to represent characters inside a program as fixed-size integers. A 16-bit integer can support distinct codes for up to 65,535 characters (setting zero aside as a null terminator). A 32-bit integer can support billions of distinct codes. Such characters in C are called wide characters, as opposed to the traditional one-byte versions of type char (and its signed and unsigned varieties). That's why the C Standard introduced the defined type wchar_t as a synonym for some integer type. Wide-character constants such as L'' have this type, and wide-character strings are arrays of this type. C++ has gone further, making wchar_t a distinct type, so you can overload functions on one-byte and wide-character arguments. That's also why the normative addendum adds so many new functions to manipulate wide characters. As I discussed last month, it is relatively easy to convert programs that manipulate char data to deal with wchar_t data instead. Having a fairly complete set of analogous functions can only help. A number of code sets exist for wide characters, by the way. Even Kanji has more than one in popular use within Japan. True, the new ISO 10646 standard promises to unite all the character sets of the world. But the jury is still out on just how that "universal" code set will best be used as a wide-character representation inside computer programs. Multibyte Encoding Wide characters are not nearly as much at home in external data streams. It's an eight-bit-byte world out there. Chop a wide character into a sequence of bytes and you immediately face problems: Byte order differs among computers for multibyte integers of the same size. You need some way to distinguish wide-character streams from conventional streams of one-byte characters. Raw binary data offers few if any clues for resynchronizing the parse of a corrupted data stream. For all these reasons, we have learned over the years to encode large character sets differently for data streams. This alternative form is called a multibyte encoding. No, there is no one universal multibyte code (just as there is still no universal wide-character code). But the rules for making multibyte codes follow a few basic patterns. In all cases, the number of bytes used to represent a character varies for different members of the set. Codes based on ASCII, for example, represent the most used ASCII characters unchanged, as one-byte sequences. (Files containing mostly ASCII code are hence economical of disk space.) Longer codes are signaled one of two ways, either by prefix codes or by shift encoding. With a prefix code, you can tell just by looking at a byte whether it stands alone as a single-byte code or whether it is the first byte of a longer sequence. You may also have to look at bytes that follow to decide when the sequence stops. But the major virtue is that each code sequence defines itself in all contexts. At worst, you may have to know that a given byte marks the start of a new code sequence. A popular Kanji encoding is Shift JIS. Any code in the interval [0x81, 0x9F] or [0xE0, 0xFC] is the first byte of a two-byte code. Any other first byte is a single-byte code. The second byte of a two-byte code must be in the interval [0x40, 0xFC]. With shift encoding, you have more flexibility. A shift code (sometimes called an escape sequence) signals a change of rules. Once the byte sequence enters an alternate shift sequence, it might, for example, group two bytes at a time to define each subsequent character. The first byte might look like an ASCII A, but in this context it is not treated as such. Only another shift code behaves differently. It can put the byte stream back in the initial shift state, where A is A once again. Another popular Kanji encoding is JIS. The three-byte shift code "\33$B" shifts to two-byte mode. In that mode, both first and second bytes must be in the interval [0x21, 0x7E]. The three-byte shift code \33(B" returns to the initial shift state. Still a better example is a typical word-processor file format. One code shifts text to italic (or bold), another shifts it back to normal. Shift encoding exacts a price for its flexibility. To parse a multibyte string, you need even more context than for prefix codes. You have to know where you stand within a given character, as before. You also have to know the current shift state. The parsing job is that much harder. The prospects for getting out of sync are that much greater. Multibyte Streams So here is the quandary. You want to work with wide characters inside a program. You need to convert them to and from multibyte characters as you read and write I/O streams. Yet opportunities abound for mucking up the parse as you read streams. And opportunies abound for generating flawed sequences, or redundant shift codes at least, as you write streams. Now consider the way a typical program reads and writes data. Occasionally, all reads occur in one place and/or all writes in another. More likely, reads and writes are sprinkled throughout the code. That makes it hard for a program to coordinate parsing a stream as multibyte characters, or generating characters in the proper shift state. The only thing in common to all reads or writes is the object of type FILE that controls the stream. But operations on C streams are defined in terms of byte-at-a-time input or output. (The defining primitives are fgetc and fputc.) Streams know nothing about possible shift codes or any other multibyte structure. It is altogether too easy to create mayhem using the conventional C facilities to read and write multibyte streams. The Japanese tried several ways to "fix" the existing I/O functions. They failed repeatedly. scanf, for example, is troublesome enough reading streams a byte at a time. Any attempt to make the function aware of multibyte syntax leads to parsing rules that are impossibly complex. printf is less troublesome, but still a nuisance. It has no obligation to produce shift codes and multibyte codes that are optimal, or even correct. One approach (as always) is to add another layer of code. Define a "wide input stream" that reads bytes from another stream and assembles them into a stream of wide characters. Also define a "wide output stream" that accepts wide characters and turns them into multibyte sequences that it writes to another stream. Using them gives you more of the pure wide-character environment we now know is desirable within a program. Sequential I/O works fine this way, but random access is rather more of a problem. You think you're positioning a stream at a given wide character. The actual underlying stream has to be positioned at the start of a given multibyte character. Not only that, you may have to restore the remembered shift state at the start of that multibyte character. The job is not impossible, but it is a nuisance. One way to make the job of file positioning easier is to constrain it somewhat. The C Standard already lets an implementation do this with text files. You can find out where you currently are in a text file (ftell or fgetpos). If that query succeeds, you can later return the stream to that position (fseek or fsetpos). But you can't just position the file to read byte 17,239 next, as you can with a binary file. So you impose a similar constraint on positioning within a wide-character stream. You can find out where you currently are in the wide-character stream, memorizing the position as for a text file. The only additional problem is that you may need additional bits to memorize a shift state. That's a real nuisance for ftell, which returns a long. Take one bit for the shift state and you can only memorize positions half as far into files. Need more bits for the shift state? The capability of ftell is more drastically reduced. You have no additional problems with fgetpos, however. It returns a value of type fpos_t, which an implementation can define as a structure containing all the fields, of whatever size, it might need. Given a file position, you can later find your way back to it by calling fseek or fsetpos, as appropriate. You can also position a file at its start with equal ease. (The implementation may have to decree that all streams start in the initial shift state, a not unreasonable constraint.) To position a file at its end can be tougher, if it uses a state-dependent encoding. The only sure way to get the shift state right is to read the file from a known position to the end. (The implementation can always end files in a known shift state, but that makes it hard to avoid writing files containing redundant shift codes, a desirable if not mandatory property.) Of course, you have to impose similar constraints on rewritting a file. You can't just drop a few wide characters in the middle of nowhere. It's too hard for an implementation to splice in an arbitrary multibyte sequence. But the same problem already exists for text files on many operating systems. They freeze record boundaries when you first write them. Rewriting the file cannot alter those boundaries. So as a general rule, once you rewrite part of a file, its contents from the end of what you just wrote to the old end of file are now in an undefined state. Even with all these constraints, however, you can get a lot of work done. People have been writing C code for years that runs with highly structured text files. Imposing multibyte syntax is just more of the same, not an entirely new problem. Wide-Character Streams The proposed addendum to the C Standard introduces the concept of wide-character streams. These behave much like the extra layer of code I hypothesized above, with one important difference. The new functionality is essentially stuffed into existing FILE objects. You don't interpose a second stream to convert to and from streams of wide characters. Instead, you convince a conventional FILE object that it is mediating a wide-character stream instead of a conventional unstructured byte stream. How do you convince a FILE that it is mediating a wide-character stream? By the operations you perform on it. The header declares analogs for many of the functions declared in . (See Listing 1.) Like their older cousins, each of these new functions operates on a stream, either implied or explicitly named as an argument. But the new functions read and write wide characters instead of one-byte characters. Note that fopen does not change. The difference is that now a stream does not know, when first opened, what its orientation will be, whether it will become wide-oriented or byte-oriented. As soon as you call any of the functions in Listing 1 for that stream, it becomes wide-oriented. All subsequent reads, writes, and positionings will be in terms of wide characters. If you instead call any of the older analogs to these functions, the stream becomes byte oriented. All subsequent operations will be in terms of single bytes, just like in the C library you know and love. A few functions, are ecumenical. You can, for example, call setvbuf or fgetpos for a stream and still make no commitment. You can also call these (and their related) functions for a stream of either orientation, once it is established. The one thing you cannot do is try to have it both ways. Mix calls of different orientations for the same open stream and you get undefined behavior. (That means that an implementation can do something sensible if the mood strikes it, but it doesn't have to.) The only way to alter the orientation of a stream, in fact, is to call freopen on the FILE object that mediates it. WG14 chose this approach over adding a mode qualifier to fopen (and freopen, of course). One reason for doing so was simplicity. The other was to simplify the use of stdin and the other standard streams with wide orientation. If the first operation determines orientation, a standard stream can go either way with no additional considerations. Whither C++ An important open issue is how best to include this new capability into C++. X3J16/WG21, the C++ standards committee, has already committed to tracking the C Standard as it evolves. You can be sure that all these new functions will become a part of the C++ library, just as the entire Standard C library already is. But is that the best way to manipulate large character sets in C++? Probably not. Serious C++ programmers already disdain the use of the C I/O machinery. They favor the classes declared in . (The continued presence of a trailing .h is currently being debated.) They like to use operator >> and operator << to do the actual input and output. Every new class they write also overloads these operators, if it makes sense at all to read or write objects of that class. So can we expect iostreams to be further overloaded on wide characters? The decisions have not yet been made, but I'd be astonished if that didn't happen. I just worry a bit about complexity overload. C streams were already pretty complex. C++ iostreams add considerable complexity atop C streams. Wide streams add even more complexity within C streams. How well it all hangs together will be interesting to observe. As always, I have to warn that all this stuff is new. We have very little experience with wide character manipulation in general. We have even less with the newly proposed wide-character functions I've described in the previous two columns. And we have next to no experience with the wide-character streams described here. We can only hope that it all will indeed simplify our lives as programmers in the years to come. Listing 1 Wide-character I/0 functions int fwprintf(FILE *stream, const wchar_t *format, ...); int fwscanf(FILE *stream, const wchar_t *format, ...); int wprintf(const wchar_t *format, ...); int wscanf(const wchar_t *format, ....); int swprintf(wchar_t *s, size_t n, const wchar_t *format, ...); int swscanf(const wchar_t *s, const wchar_t *format, ...); int vfwprintf(FILE *stream, const wchar_t *format, va_list arg); int vwprintf(const wchar_t *format, va_list arg); int vswprintf(wchar_t *s, size_t n, const wchar_t *format, va_list arg); wint_t fgetwc(FILE *stream); wchar_t *fgetws(wchar_t *s, int n, FILE *stream); wint_t fputwc(wint_t c, FILE *stream); int fputws(const wchar_t *s, FILE *stream); wint_t getwc(FILE *stream); wint_t getwchar(void); wint_t putwc(wint_t c, FILE *stream); wint_t putwchar(wint_t c); wint_t ungetwc(wint_t c, FILE *stream); /* End of File */ Code Capsules C++ Streams Chuck Allison Chuck Allison is a regular columnist with CUJ and a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3J16, the ANSI C ++ Standards Committee. Chuck can be reached on the Internet atallison@decus.org, or at (801)240-4510. Like C, C++ does not provide language facilities for input or output. They are, instead, part of the standard library. Although you can use C's stdio functions for input and output in C++, the stream classes are an efficient way to read and write data in C++. The stream classes of C++ offer three advantages to C's stdio functions. First, streams are type-safe. The printf/scanf family of functions depends solely on a format string that the compiler cannot validate. Since the type of an object in C++ determines which action follows, you just can't do the wrong thing without hearing about it at compile time (unless you try really hard). Second, streams provide a uniform interface for all input and output operations. Whether the source or destination is a file, a string, or a standard device, the operations are identical ("A stream is a stream is a stream..."). Finally, streams are extensible--and I don't just mean that you can derive new stream classes. Almost always, just overloading standard stream operators is all you need. Standard Streams There are four pre-defined streams: cin (standard input), cout (standard output), cerr (standard error), and clog (standard error). All but cerr are fully-buffered streams. Like stderr, cerr behaves as if it is unbuffered, but it is actually unit-buffered, meaning that it flushes its buffer automatically after processing each object, not after each byte. For example, with unit-buffering, the statement cerr << "hello"; buffers the five characters and then flushes the buffer. An un- buffered approach sends a character at a time all the way through to its destination. Simple I/0 The following program copies standard input to standard output: // copy1.cpp: Copy standard input to standard output #include main() { char c; while (cin.get (c)) cout.put(c); return 0; } get, called an extractor, reads a single character or a string from a stream, depending on how it's called. (In this case, get reads a single character from the cin stream.) put, called an inserter, sends the character, or string, to an object. (In this case, put sends a character to the cout object.) The get extractor stores the next byte from a stream into its char parameter. Like most stream member functions, get returns the stream itself. When a stream appears in a boolean context (like in the while loop above) it tests true if the data transferred successfully, and false if there was an error, such as an attempt to read beyond end-of-file. Although such simple boolean tests suffice most of the time, you can query the state of a stream anytime with the following member functions: bad--severe error (stream is corrupted) fail--conversion error (bad data but stream is OK) eof--end-of-file good--none of the above The program in Listing 1 copies a text file line-by-line. The getline extractor reads up to BUFSIZ-1 characters into s, stopping if it finds a newline character, and appends a null byte. (It discards the newline.) Output streams use the left shift operator as an inserter. Any object, built-in or user-defined, can be part of a chain of insertions into a stream. (However, you must overload << yourself for your own classes.) Listing 2 contains a program that illustrates the extraction operator >>. Since in C you normally use stderr for prompts (because it is unbuffered), you might be tempted to use cerr in C++: cerr << "Please enter an integer: "; cin >> i; However, this is not necessary in C++ because cout is tied to cin. An output stream tied to an input stream is automatically flushed when input is requested. If necessary, you can use the flush member function to force a flush. Addresses print in an implementation-defined format (usually hexadecimal). Character arrays are, of course, an exception--the string value is printed, not the address. To print the address of a string, cast it to a void *: cout << (void *) s << '\n'; // prints address The >> extraction operator skips whitespace by default. The program in Listing 3 uses this feature to count the words in a text file. Extracting into a character string behaves like the %s edit descriptor in scanf. It is possible to turn off the skipping of whitespace when reading characters (see Listing 4). Beyond Simple I/O Formatting ios::skipws in Listing 4 is an example of a format flag. Format flags are single-bit values that you can set with the member function setf, and reset with unsetƒ. (See Table 1 for a description.) The program in Listing 5 illustrates numeric formatting. The member function precision dictates the number of decimal places to display for floating-point values. Unless the ios::show-point flag is set, however, trailing zeroes will not appear. ios::showpos prints positive numbers with a leading plus sign. ios::uppercase displays the x in hexadecimal values and the e in exponential in upper case. Some formatting options can take on a range of values. For example, ios::basefield, which determines the numeric base for displaying integers, can be set to decimal, octal, or hexadecimal. (See Table 2 for a description of the three format fields available.) Since these are bit fields and not single bits, you set them with a two-parameter version of setf. For example, the program in Listing 6 changes to octal mode with the statement cout.setf(ios::oct,ios::basefield); With the flag ios::showbase set, octals print with a leading 0 and hexadecimals with a leading 0x (or 0X if ios::uppercase is also set). Manipulators endl, of a manipulator, inserts a newline character into the output stream and then flushes the stream. Instead of inserting an object into the stream, a manipulator changes a stream state. Table 3 contains built-in manipulators declared in iostream.h. The program in Listing 7, although functionally equivalent to the one in Listing 6, uses manipulators instead of explicit calls to setf. Manipulators often allow for more streamlined code. Other manipulators take parameters (see Table 4). The program in Listing 8 uses the setw(n) manipulator to set the output width directly in the insertion sequence, so you don't need a separate call to width. The special field ios::width is reset to 0 immediately after every insertion. When ios::width is 0, values print with the minimum the number of characters necessary. As usual, numbers are not truncated, even if you don't allow enough space for them. You could replace the statement cout.fill('#'); with the in-sequence manipulator ... << setfill('#') << ... but it seems cumbersome to do so in this case. Extractors usually ignore width settings. An exception is string input. You should set the width field to the size of a character array before extracting into it in order to avoid overflow. When processing the input line nowisthetimeforall the program in Listing 9 produces nowisthet,im,eforall Remember that extractors use whitespace as a delimiter, so if the input is now is the time for all then the output is now, is, the You can create your own manipulators by simply defining how a function takes a stream reference parameter and returns that same reference. For example, here is a manipulator that rings the bell at the console when you insert it into any output stream: // A manipulator that beeps #include ostream& beep(ostream& os) { os << (char) 7; // ASCII BEL return os; } To use this, just insert it: cout << ... << beep << ... Beyond the Standard Streams String Streams To read from and write to strings in memory (like sscanf and sprintf do), use the string stream classes declared in strstream.h. The program in Listing 10 uses an anonymous output string stream to write to the string s. The line ostrstream(s,sizeof s) << "You entered " << i << ends; is a safe alternative to the C statement sprintf(s,"You entered %d",i); If you want the stream to persist beyond the statement, declare an object: ostrstream os(s,sizeof s); os << "You entered " << i << ends; Listing 11 contains a program that counts both the number of lines and the number of words in a text file. It reads a line at a time and then attaches a string stream to the line to extract each word. NOTE: the current release (v3.1) of the Borland C++ compiler requires a slight change in the while loop: while (line >> word && word[0]) ++wc; (For some reason the string stream extractor doesn't set the state properly). The getline function normally extracts characters up to a newline. You can use another character as a delimiter by sup- plying the optional third argument. For example, the statement sstr.getline(s,sizeof s,','); reads characters into s up to the next occurrence of a comma, then discards the comma. This is useful for reading database files with delimited fields. The program in Listing 12 reads a comma-delimited file with fields representing name, address, city, state, and zip code. For example, with the following input: Bill CIinton, 1600 Penn. Ave.,Washington,DC,21234 Ross Perot,1234 Feedback Dr.,Big Ears,TX,5678g Annymous,, Nowhere,, 00000 it produces this output: Bill Clinton1600 Penn. Ave.WashingtonDC21234 Ross Perot1234 Feedback Dr.Big EarsTX5678g AnonymousNowhere00000 File Streams The program in Listing 13 copies one file to another by explicitly opening file streams. You must enter both file names on the command line. The definition ifstream inf (argv [1] ) attempts to open the file named in the first command-line argument and connects the input stream inf to it. If all went well, inf will test true as a boolean expression (analogously for outf).Both files close automatically when the file streams are destroyed as they go out of scope. Listing 14 contains a more flexible copy utility. It allows you to either name the files on the command line: copy fi1el fi1e2 or to use redirection: copy file2 or to leave one or both files connected to a standard device: copy file1 {displays file1 on the console) If you supply a file name it opens the file, otherwise it uses standard input and/or standard output. In order to attach a file stream to an open file you need a file handle, such as ƒileno returns. (See last month's "Code Capsule" for information on fileno and other POSIX functions). When a file stream goes out of scope, its destructor closes the associated file, unless attach made the connection (otherwise standard I/O would shut down!). The program in Listing 15 creates a file of fixed-length records, and then reads back and displays selected records. An fstream object allows simultaneous input and output on the same file. The flag ios::bin enables binary mode, which is necessary for accurate file positioning on non-UNIX systems. (Some compilers use ios::binary or ios: :notranslate, but the current working paper for standard C++ specifies ios::bin). The member functions read and write perform binary input and output, similar to their C counterparts ƒread and ƒwrite (except that the former process only one record at a time). seekg moves the file-read pointer to a specified byte position in the file. Its second argument controls positioning in the usual way: ios::beg--forward from the beginning of the file ios::cur--relative to the current position ios::end--backward from the end of the file Note: Classes that specifically support simultaneous input and output on a stream (namely, ƒstrearn, strstream, and iostream) are not part of the proposed standard for C++. The committee decided to simplify the standard library by eliminating the use of multiple inheritance. (Multiple inheritance is one of those necessary evils that you should only use when absolutely.) To open a file for both input and output in a future conforming implementation, you will have to replace int mode = ios::bin ios::trunc ios::out ios::in; Fstream f("recs.dat',mode); with int mode = ios::bin ios::trunc ios::out ios::in; ofstream f("recs.dat", mode); ifstream g(f.rdbuf(),ios: :bin); which causes streams ƒ and g to share the same internal buffer. (See Table 5 for descriptions of the file open modes.) Streams and User-Defined Objects To read or write a user-defined object usually reduces to processing built-in types which comprise the object (or its components, etc.). For example, suppose you wanted to define an inserter for a complex number class: class complex { double real, imag; public: friend ostream& operator<<(ostream&, const complex&); // Details omitted }; To display a complex number in coordinate form (e.g., (2,7.5)), you simply use the built-in inserter on the components: ostream& operator<<(ostream& os, const complex& c) { os << '{' << c.real << ',' << c.imag << ')'; return os; } The semantics of operator<< require returning the stream to allow chaining multiple insertions in a single statement. (For more detail on overloading inserters and extractors, see Dan Saks' column "Stepping Up to C++" in the May 1992 and July 1992 issues of CUJ.) There is more machinery to streams than what I have presented here. (If you're interested in what's under the hood, look up the streambuƒ family of classes in your compiler's documention). I have illustrated the high-level features that should suffice for most non-systems applications. If things appear slightly more involved that what you're used to in C, that merely reflects the careful design that makes streams safer and more robust than the stdio functions. Table 1 Format flags Flag Meaning Default ------------------------------------------------- showbase shows octal or hex prefix off showpos shows plus sign when positive off showpoint shows trailing zero decimals off uppercase OX for hex, E for scientific off unitbuf enables unit buffering off skipws > > skips whitespace on Table 2 Format fields Field Values Default ------------------------------------------- adjustfield left, right, internal right basefield dec, oct, hex dec floatfield fixed, scientific fixed Table 3 Simple manipulators(iostream.h) Manipulator Meaning ----------------------------------------------- dec Change numeric base to decimal oct Change numeric base to octal hex Change numeric base to hexadecimal ws Skip over whitespace in input endl Insert a newline and flush ends Insert a `\0' flush Flush the stream Table 4 Parameterized manipulators (iomanip.h) Manipulator Meaning ------------------------------------------------------- resetiosflags(n) Reset all the flags that are set in n setios flags (n) Set all the flags that are set in n setbase(n) Same as setf (n, ios::basefield) setfill (c) Same as fill (c) setprecision(n) Same as precision(n) setw(n) Same as width (n) Table 5 File open modes Mode Meaning (stdio equivalent) --------------------------------------------------- in Open for input ("r") out Open for output ("w") ate Position at end of file bin Binary mode ("b") trunc Discard existing contents app Position for output at end of file ("a") nocreate Fail if doesn't already exist noreplace Fail if already exists (The last three are not part of draft standard C++) Listing 1 Copies standard input lines to standard output // copy2.cpp: Copy input lines # include # include main() { const size_t BUFSIZ = 128; char s[BUFSIZ]; while (cin.getline(s,BUFSIZ)) cout << s << '\n'; return 0; } // End of File Listing 2 Prompts for a integer, echoes its value and address // int.cpp: Prompt for an integer # include main() { int i; cout << ''Please enter an integer: ''; cin >> i; cout << ''i == '' << i << '\n'; cout << ''&i == '' << &i << '\n'; return 0; } // Sample Execution: // Please enter an integer: 10 // i == 10 // &i == Oxfff4 // End of File Listing 3 Counts the words in a text file // wc.cpp: Display word count #include #include main() { const size_t BUFSIZ = 128; char s[BUFSIZ]; size_t wc = 0; while (cin >> s) ++wc; cout << wc << '\n'; return 0; } // Output from the command "wc> #include main() { char c; // Don't skip whitespace cin.unsetf(ios::skipws); while (cin >> c) cout << c; return 0; } // End of File Listing 5 Illustrates numeric formatting // float.cpp: Format real numbers #include main() { float x - 12345.6789, y = 12345; cout << x << ' ' << y << '\n'; // Show two decimals cout.precision(2); cout << X << ' ' << y << '\n'; // Show trailing zeroes cout.setf(ios::showpoint); cout << x << ' ' << y << '\n'; // Show sign cout.setf(ios::showpos); cout << x << ' ' << y << '\n'; // Return sign and precision to defaults cout.unsetf(ios::showpos); cout.precision(0); // Use scientific notation cout.setf(ios::scientific,ios::floatfield); float z = 1234567890.123456; cout << z << '\n'; cout.setf(ios::uppercase); cout << z << '\n'; return 0; } // Output // 12345.678711 12345 // 12345.68 12345 // 12345.68 12345.00 // +12345.68 +12345.00 // 1.234568e+09 // 1.234568E+09 // End of File Listing 6 Shows the bases of integers //base1.cpp: Shows the bases of integers #include main() { int x, y, z; cout << "Enter three ints: "; cin >> x >> y >> z; cout << x << ',' << y << ',' << z << endl; // Print in different bases cout << x << ','; cout.setf(ios::oct,ios::basefield); cout << y << ','; cout.setf(ios::hex,ios::basefield); cout << z << endl; // Show the base prefix cout.setf(ios::showbase); cout << X << ','; cout.setf(ios::oct,ios::basefield); cout << y << ','; cout.setf(ios::hex,ios::basefield); cout << z << endl; return 0; } // Sample Execution // Enter three ints: 10 010 0x10 // 10,8,16 // 10,10,10 // 0xa,010,0x10 // End of File Listing 7 Changes numeric base with manipulators // base2.cpp: Shows the bases of integers // (Uses manipulators) #include main() { int x, y, z; cout << "Enter three ints: "; cin >> x >> y >> z; cout << x << ',' << y << ',' << z << endl; // Print in different bases cout << dec << x << ',' << oct << y << ',' << hex << z << endl; // Show the base prefix cout.setf(ios::showbase); cout << dec << x << ',' << oct << y << ',' << hex << z << endl; return 0; } // End of File Listing 8 Sets the output field width setw() // adjust.cpp: Justify output #include #include main() { cout << '' << setw(10) << "hello" << '' << endl; cout.setf(ios::left,ios::adjustfield); cout << '' << setw(10) << "hello" << '' << endl; cout.fill('#'); cout << '' << setw(10) << "hello" << '' << endl; return 0; } //Output // hello //hello //hello##### // End of File Listing 9 Controls the wiidth of input strings // width.cpp: Control width of input strings #include #include main() { char s1[10], s2[3], s3[20]; cin >> setw(10) >> s1 >> setw(3) >> s2 >> s3; cout << s1 << ',' << s2 << ',' << s3 << endl; return 0; } // End of File Listing 10 Writes to a string stream // incore.cpp: Incore formatting #include #include #include main() { const size t BUFSIZ = 128; char s[BUFSIZ]; int i; cout << "Please enter an integer: "; cin >> i; ostrstream(s,sizeof s) << 'You entered "<< i << ends; cout << "s == \"" << s << "\"\n"; return 0; } // Sample Execution // Please enter an integer: 10 // s == "You entered 10" // End of File Listing 11 Reads words from a string stream // lwc.cpp: Display count of lines and words #include #include #include main() { const size_t BUFSIZ = 128; char s[BUFSIZ]; size_t lc = 0, wc = 0; while (cin.getline(s,BUFSIZ)) { ++1c; istrstream line(s); char word[BUFSIZ]; while (line >> word) ++wc; } cout << "Lines: "<< lc << ", words: "<< wc << endl; return 0; } // Output from the statement "lwc < lwc.cpp" // Lines: 24, words: 63 // End of File Listing 12 Reads a comma-delimited file // comma.cpp: Extract comma-delimited tokens #include #include #include main() { const_size t BUFSIZ = 128; char s[BUFSIZ]; while (cin.getline(s,BUFSIZ)) { char name[16], addr[26], city[16], state[5], zip[6]; istrstream sstr(s); sstr.getline(name,sizeof name,','); sstr.getline(addr,sizeof addr,','); sstr.getline{city,sizeof city,','); sstr.getline(state,sizeof state,','); sstr.getline(zip,sizeof zip); cout << name << '' << addr << ,, << city << '' << state << '' << zip << endl; } return 0; } // End of File Listing 13 Uses file streams to copy files // copy4.cpp: Copy one file to another #include #include // Required for file streams #include main(int argc, char *argv[]) { if (argc == 3) { ifstream inf(argv [1] ); ofstream outf(argv[2]); if (inf && outf) { char c; while (inf.get(c)) outf.put(c); return EXIT_SUCCESS; } // Streams destroyed by this point } return EXIT_FAILURE; } // End of File Listing 14 Navigates a file of fixed-length records // records.cpp: Illustrate file positioning #include #include #include #include struct record { char last[16]; char first[11]; int age; }; main() { const size_t BUFSIZ = 81; int nrecs = 0; char buf[BUFSIZ]; struct record recbuf; int mode = ios::bin ios::trunc ios::out ios::in; fstream f("recs.dat",mode); for(;;) { // Prompt for input line cout << "Enter LAST,FIRST,AGE: "; if (!cin.getline(buf,BUFSIZ)) break; // Extract fields istrstream line(buf); line.getline((char *)&recbuf.last,sizeof recbuf.last,','); line.getline((char *)&recbuf.first,sizeof recbuf.first,','); line >> recbuf.age; // Write to file f.write((char *) &recbuf, sizeof recbuf); ++nrecs; } /* Position at last record */ f.seekg((nrecs-1)*sizeof(struct record),ios::beg); if (f.read((char *) &recbuf,sizeof recbuf)) cout<< "last: "<< recbuf.last << ", first: ' << recbuf.first << ", age: ' << recbuf.age << endl; /* Position at first record */ f.seekg(0,ios::beg); if (f.read({char *) &recbuf,sizeof recbuf)) cout << "last: " << recbuf.last << ", first: ' << recbuf.first << ", age: " << recbuf.age << end1; return 0; } // Sample execution // Enter LAST,FIRST,AGE: Lincoln,Abraham,188 // Enter LAST,FIRST,AGE: Bach,Johann,267 // Enter LAST,FIRST,AGE: Tze,Lao,3120 // Enter LAST,FIRST,AGE: ^Z // last: Tze, first: Lao, age: 3120 // last: Lincoln, first: Abraham, age: 188 // End of File Listing 15 Copies files with or without redirection // copy5.cpp: Copy one file to another #include #include #include #include main(int argc, char *arg[]) { ifstream inf; ofstream outf; // Open optional input file if (argc> 1) inf. open (argv [1] ); else inf.attach(fileno(stdin)); // Open optional output file if (argc > 2) outf.open(argv[2]); else outf.attach(fileno(stdout)); if (inf && outf) { char c; while (inf.get(c)) outf.put(c); return EXIT_SUCCESS; } else return EXIT_FAILURE; } // End of File Questions & Answers Internationalization and Localization Kenneth Pugh Kenneth Pugh, a principal in Pugh-Killeen Associates, teaches C and C++ language courses for corporations. He is the author of C Language for Programmers and All on C, and was member of the ANSI C committee. He also does custom C programming for communications, graphics, image databases, and hypertext. His address is 4201 University Dr., Suite 102, Durhan, NC 27707. You may fax questions for Ken to (919) 489-5239. Ken also email at Kpugh@dukemvs.ac.duke.edu (Internet) and on Compuserve 70125,1142. The questions in the past column regarding internationalization of programs have spawned a number of replies. These replies reveal that some software companies receive over half their revenue from overseas. You can serve this important market, even if you are not concerned with internationalization, by structuring your programs to separate strings from the code, thus making the program more maintainable. I had a phone conversation with Mr. Shae Murphy of Network Dynamics in Williamsburg, VA. They produce an internationalization toolkit for C and C++. He sent me a white paper on internationalization, from which some of these remarks are extracted. I'll briefly describe how their toolkit works, as a basis for comparison with some other approaches. The toolkit extracts strings from your text files and stores them in an external file. It replaces each string with a macro call that is translated to a pointer to a char. You make a call in your program to a function that initializes the pointers. The function loads the external file into dynamic memory. The external files can be appended to your executable program, so that you don't have lots of files to worry about. The strings can also be processed into a file to be included in a Microsoft Windows program's resource file. Unlike the OSF internationalization files, the external file does not contain any string identifiers in it. The line number of each string in the file implicitly identifies the string. If you edit the file, you need to be sure to keep the lines in the same sequence. The toolkit has a few limitations. It cannot handle implicitly concatenated strings, as each part is turned into a character pointer. This was a rather large problem for the code I ran it on. Ever since ANSI C developed the feature, I use it a fair bit to keep my code indented nicely. The total size of the extracted strings cannot exceed 64KB. However, multiple external files can be used to increase this number. If you wanted to, you could simply use the string parsing program to replace strings in your source files with a call to your own function. Your function could perform the accessing of the strings in any manner you choose. There are a couple of other methods for creating internationalization. Each has its tradeoffs. Microsoft Windows has Language Pre-Processor. It scans the source code and places the strings into a database. A human translator changes the strings to the new language. Then the FLPP replaces the strings in the source file. There is no string processing overhead at runtime, but this approach requires multiple copies of the executable. Translation time is cut down by two factors. Duplicate uses of the same string map to the same entry in the database and the same string database can be used for multiple applications. Another approach is that of OSF, which stores the strings in an external file, compiled into a compact form. OSF provides functions to access the strings. You have to write your program specifically to use these functions, although the string parsing program could help in converting a currently written program. There can be a bit of overhead in these function calls. Whichever of these approaches you use does not automatically guarantee that you will have an internationalized program. The length of the translated strings may vary dramatically. The English phrase not found translates into French as fichier non trouve. A simple message popup turns into Nachrichtenuberlagerrungsfenster in German. In general, short strings (ten characters or less) may double in size, and longer strings (over 70 characters) may increase by 30 percent. You need to plan for string sizes. If you are using a window system, your windows should be self-sizing. If you are simply using printf, then your sizing requirements may be non-existent. Menu and button string labels may need to be shortened. During the translations, you need to watch for sentence order. Other languages place subjects, predicates, and modifiers in sundry orders. Format statements may need to be altered to place the values in the proper location for a particular sentence. If the sequence of the format specifiers changes, you will need to alter the calling sequence in your source code. I haven't discussed translations to languages which are not supported by eight-bit character sets. Conversion to these languages implies using a graphical interface to represent the characters. A program which uses printf for output will probably need a good deal of work for it to handle 16-bit wide characters. printf does not have a format specifier for wide characters. Making your program international involves not only string translation, but also localization. Localization means conforming to the local culture. This includes using date and monetary formats specific to the particular market. The ANSI C library provides some features that aid in writing a localizable program. I'll describe a few of them here. The setlocale function sets up a program to operate within a specific environment. The prototype is: char *setlocale(int category, const char *locale); The category can be LC_ALL to change all features in the locale. The other options will be discussed shortly. The locale affects the way some other ANSI functions operate and some of the information that they return. The locale variable is a string which identifies a particular locale. The only two strings that are universally supported are "C" and "", which represent a locale for translation of C programs and an implementation-defined native locale. This function provides the framework for altering the locale, without defining the specific values. The locale variable could be something like "English.USA" or "English.England". Compiler manufacturers may or may not provide other locales. If they do not, it is difficult to make use of this function, as different locales alter the operation of other ANSI functions, for which you do not have source. The localeconv function returns a pointer to a structure of type lconv. This structure contains various values that are specific to the current locale. They include the decimal-point character, currency symbols (local and international), and information on how the currency symbol and positive and negative values are displayed. For example, in Norway, a positive currency is shown as "kr1.234,56" and a negative currency as "kr1.234,56-". Italian lira amounts are shown as "L1.234" and "-L.1.234". The currency amounts are not automatically formatted for you. You will need to write or obtain a function that can use the information returned by localeconv to produce the correct output. You can modify particular parts of the locale by using different options for the first parameter. The LC_NUMERIC option affects the decimal point character in other functions, such as printf and scanf and the information returned by localeconv. LC_MONETARY alters the information returned by localeconv. LC_TIME affects the output of strftime. LC_COLLATE alters the operation of strcoll and strxfrm. LC_TYPE changes the operation of the character functions. Character Functions The strftime function converts the values in a time structure (of type tm) to a string. Its prototype looks like: size_t strftime ( char *output_string, size_t output_string_size, const char *format, const struct tm *p_time); The format parameter may contain specifiers that are replaced in output_string with characters corresponding to the time. For example "%a" represents the abbreviated weekday name in the locale. "%X" is the locale's time representation. strcoll and strxfrm are used to compare and translate strings that contain characters other than the standard seven-bit ASCII set. They may also be used to sequence ASCII characters in non-ASCII order. strcoll works like strcmp, but it uses character comparisons that are based on the locale. For example, characters with accent marks, e.g. ˆ, may be interpreted as greater than the non-accented character e and less than the character f. This comparison involves some sort of table lookup or other algorithmic method, rather than a simple subtraction of two chars. Using this in place of strcmp for qsort may increase the sort time substantially. Its prototype looks exactly like strcmp: int strcoll(const char *string_1, const char *string_2); To avoid the overhead of doing a translation every time you want to compare two strings, you can use strxfrm. This function transforms a string into a strcmp comparable string, based on the locale. The prototype is: size_t strxfrm (char *transformed_string, const char *input_string, size_t input_string_size); This is a one-way process and the transformed_string is good only for comparing with another transformed_string. Since you would normally perform the transformation only once for each string, this function can make a sort faster at the expense of some memory and a little setup time. The character functions, such as isalpha and isupper, return true or false values, depending on the value of the character passed to them. Regardless of whether you have ever considered going international, you should be using these functions. You may have coded if (c >= 'A' && c <= 'Z') to test for upper case characters. That is essentially non-portable to non-ASCII computers. Although only an estimated .1 percent of computers use a code other than ASCII, for completeness, consider the possibility. Additionally, using isalpha and other functions makes your code more readable and possibly faster. These functions use the locale information to determine the type of characters. So isalpha would return true for ˆ. Function Pointers I've had a number of questions in my classes regarding the use of function pointers. A function pointer is a variable whose contents is the address of a function. The syntax for declaring a function pointer looks a bit odd the first time you see it: int (*pointer_to_function)(); The parentheses surround the name of the variable, along with the asterisk. This declaration can be read from inside out. This statement declares pointer_to_function to be a variable. This variable is a pointer (because of the *). The pointer variable will be used to point to functions (because of the ()). The function that it points to returns an int. For example, suppose you had functions whose prototypes were: int a_function(int one_parameter); double another_function (int first_parameter, int second_parameter); int still_another_function (int first_parameter, int second_parameter); Remember that the name of a function all by itself represents the address of the function. You can code: pointer_to_function = a_function; or pointer_to_function = still_another_function; but you cannot code: pointer_to_function = another_function; since another_function returns a double. The ANSI standard suggests (and C++ requires), that when you declare a pointer to a function, you must also specify the types of the parameters. For example: int (*pointer_to_function_____LINEEND____ with_one_int)(int); Now you can code: pointer_to_function_with_one_int = a_function; but not: pointer_to_function_with_one_int = still_another_function; since still_another_function has two int parameters. The types of the parameters of the function being pointed to must match exactly. If you had a function: a_fourth_function (double one_parameter); you could not set: pointer_to_function_with_one_int = a_fourth_function; even though doubles are automatically converted to ints if the function is called directly. You would have to have a variable as: int (*pointer_to_function_with_double)(double); and set pointer_to_function_with_double = a_fourth_function; You could use a cast to permit the previous assignment, as: pointer_to_function_with_int = (int (*)(int)) a_fourth_function; This cast changes the type of the value of a_fourth_function, which is a "pointer to function which expects a double and returns an int" to a "pointer to a function which expects one int and returns an int". You should not call a function using this pointer as it will be passed an int, but the actual function will expect a double. The cast of the function pointer does not alter the parameter types. Given that background in pointers to functions, there are at least three common places where they are used in code. The first is a parameter to qsort. The qsort function sorts an array of values based upon a comparison function. Its prototype is: void qsort(void *array, size_t number_elements, size_t size_of_element, int (*compare_function)( const void *, const void *)); It assumes that array points to the beginning of an array that is number_elements long, each element of which is size_of_element bytes. It sorts the array according to its internal algorithm, which is documented in a number of data structure books and in Knuth's books. When it needs to compare two elements to determine whether to swap them or not, it calls the function whose address you have passed it. This function is passed two addresses which point to the two elements which it is currently comparing. Your function needs to return a value less than zero, equal to zero, or greater than zero, depending on how the elements compare. For example: your_compare_function(int * first, int * second) /* Quick compare - may not work for all int values */ { return *first - *second; } #define NUMBER_ELEMENTS 10 int int_array[NUMBER_ELEMENTS] = {5,3,4,6,7,1,2,3,4,7}; qsort (int_array, NUMBER_ELEMENTS, sizeof(int), (int (*)(const void *, const void *)) your_compare_function); The cast is necessary to match the parameter types. Note that qsort is a generic sort. It cannot pretend to be able to sort any kind of array. You supply the comparison function to make it specific to your particular array. The same comparison function can be used with the bsearch function to search a sorted array for a particular value of the same type. The cast of the function pointer does not change the parameter types. In this case, the parameters are pointers to data. It is possible that on some implementations that the representation of a void * is different than an int*. To be strictly ANSI-compliant and absolutely portable, you have to declare your comparison function to take two void * and then typecast them to ints*. The comparison function should look like your_comparison_function(void *first, void *second) { return *(int*)first - *(int*)second } Note that regardless of what you may be comparing, your function header will look exactly the same. So only use this function to pass to qsort and bsearch and not as a general comparison function. The next area in which function pointers have been used extensively is with device drivers. I'll cover this in general, since they vary from operating system to operating system. A device driver responds to requests from the operating system for opens, reads, and writes. To install a device driver, you usually set up a system defined structure or array that looks like: struct s_device_driver_interface { int (*open)(char *filename, char *mode); int (*read)(char *buffer, int count); int (*write)(char *buffer, int count); .... int (*close)(void); }; You would code your functions that performed each of the operations, and then pass those values to the operating system, along with some identification of the device. For example, my_device_open(char *filename, char *mode); my_device_read(char *buffer, int count); struct s_device_driver_interface my_device = {my_device_open, my_device_read, ...); install_device_driver("MY_DEVICE", &my_device); When the user opens a device, the operating system determines which device it is from the name. If it is "MY_DEVICE", then it would call my_device_open. The third major place function pointers are used are in window programming. Both Microsoft Windows and X-Window have callback functions, though their granularity differs. When an event, such as a mouse-click, occurs when the mouse cursor is in your window, the window system calls the function whose address you have passed. This is termed a callback function since the system will literally call back to your function. The system passes along some information as to what event occurred and additional details on some of the events. In a sense, your program is made of functions which respond to the user input, rather than the more traditional procedural approach. One major difference between the systems is how many callback functions are used. Microsoft Windows has a single callback per window (parent, child, or dialog box). Under X-Window, you can specify a callback for every object on the screen (pushbutton, radio button, etc.). The latter may be easier to comprehend, as it eliminates a large number of events being passed to a single callback. For those interested in the full treatment, you can read the thousands of manual pages on both Microsoft Windows and X-Window. Naming Conventions In previous columns, I have discussed using actual words for names of variables and functions. It appears that widespread use of abbreviations such as hwnd may affect the spellchecking ability for normal text. For example, a major manufacturer of a window software system has a CD-ROM describing many of its features. One of the articles contains the sentence, "It also means that using the same command that invoked it again, you mght [sic] become reentrant and can really mes [sic] up your application." Stepping Up to C++ Nested Classes Dan Saks Dan Saks is the founder and principal of Saks & Associates, which offers consulting and training in C++ and C. He is secretary of the ANSI and ISO C++ committees. Dan is coauthor of C++ Programming Guidelines, and codeveloper of the Plum Hall Validation Suite for C++ (both with Thomas Plum). You can reach him at 393 Leander Dr., Springfield OH, 45504-4906, by phone at (513)324-3601, or electronically at dsaks@wittenberg.edu. Last month, I summarized most of the extensions to C++ introduced by the joint C++ standards committee WG21 + X3J16 (see "Recent Extensions to C++," CUJ, June, 1993). I neglected to mention one other extension--forward declaration of nested classes. Nested classes were a relatively late addition to the C++ language. They were introduced in 1989, shortly before the committee embarked on drafting a standard. Nested classes added considerable complexity to the language's lookup rules, that is, the rules for matching a reference to an identifier in a translation unit with a declaration for that identifier. The rules stated in the ARM (Ellis and Stroustrup, 1990) seem to work fine for most common simple cases, but they are incomplete and inconsistent in handling complex cases. The standards committee's Core Language working group has spent much of its time these last three years ironing out name lookup problems. This month's column explains nested classes and types, and explains the extension to allow forward declaration of nested classes. Next month, I'll describe the problems posed by nested classes and the new lookup rules designed to solve those problems. Nested Classes A nested class is a class declared inside another class, as in: class outer { public: class inner { /* ... */ }; void foo(int); // ... private: inner i; // ... }; Here, class inner is nested within class outer. Hence, the name inner is in the scope of class outer. Within class outer (and the bodies of its member functions), you can refer to inner simply as inner. For example, void outer::foo(int i) { inner *p; // ok, inner in scope of outer // ... } But outside class outer, you must refer to inner by its fully-qualified name, outer::inner, as in int main() { outer::inner *q = new outer::inner; // ... } Nested class names are subject to access control. That is, you cannot reference outer::inner outside a member or friend of class outer unless inner is a public (or protected) member of outer. Nested classes reduce the need for global names in C++ programs, making it easier to avoid name conflicts between different parts of a large program. The following example shows how. Applying Nested Classes Threaded data structures, like lists and trees, typically are implemented using at least two distinct types of objects: node objects that store a value along with one or more pointers to other nodes in the thread, and a root object that stores a pointer to the beginning (and possibly another pointer to the end) of each thread. For instance, Figure 1 illustrates the structure of a singly-linked list of integers. Each node object contains a single integer value and a pointer to the next node in the list. The root contains a pointer to the first node in the list, and another pointer to the last node in the list. When you build an application using a variety of threaded structures, you will probably define several different kinds of node objects--one for each different threaded structure. If you don't use nested classes, you must give each node type a different name, like listnode or treenode, so they don't conflict. For example, Listing 1 and Listing 2 contain header files that define different threaded structures. Listing 1 defines the node class (called listnode) and the root class (called list) for a singly-linked list of (unsigned) integer values. Listing 2 defines the node and root classes (treenode and tree, respectively) for a binary tree of dynamically-allocated character arrays. (These classes resemble the list and tree classes I used in a cross-reference generator many moons ago. See "Your First Class," CUJ, May, 1991, or "Rewriting Modules as Classes," CUJ, July, 1991.) This ad hoc approach to avoiding name conflicts isn't so bad in small projects with only one programmer (or maybe even a few programmers), but such an approach quickly becomes unmanageable in a large project with many programmers. Avoiding global name conflicts gets even harder when you start using libraries from various sources in your applications. For example, lists can have many different implementations. They can be singly- or doubly-linked, and linear or circular. If your application uses, or your library supports, both singly- and doubly-linked lists, then you must use longer names to distinguish the different root and node classes--names like singly_-linked_list_node, SinglyLinkedListNode, SLlistnode. Juggling long names like these takes some of the fun out of programming. Long names for the node types don't have to be pretty. Users of the list class are not supposed to use the node type name in their own code; they are only supposed to use the name of the root class. The node class is simply part of the (supposedly hidden) implementation of the list. Thus, you could argue that the more ugly and obscure a node type name is, the less chance there is that anyone else will use it. Making a name ugly and obscure doesn't really hide it. Users must include the header list.h to gain access to the list class. But the header also contains the declaration for the list node class, so the user can access it too. When you implement a list class, you should really put the node class out of the user's reach. Nesting the node class inside the list class does just that. Listing 3 shows the list class definition with the list node class as a nested class. Since the list node class is no longer global, I shortened its name to just node. As a member of class list, its fully-qualified name, list::node, is sufficiently distinct. Keeping the name as list::listnode is overkill. Listing 4 shows an implementation for the list class. It implements four simple operations: list(unsigned n) (a constructor) initializes a list with n in it. ~list (the destructor) destroys a list. add(unsigned n) adds n to the end of the list. print prints the contents of the list. Notice that the implementation uses the name node without qualification. It never has to refer to the node class as list::node, because all the function bodies are in the scope of the list class, and node is a member of the class. Making Do without Nested Classes C does not support nested structs. If you write a struct within a struct, C treats the inner struct as if it has been written outside the outer struct. That is, C treats struct outer { //... struct inner { /* ... */ }; }; more or less like struct outer { // ... }; struct inner { /* ... */ }; Before it supported nested classes, C++ treated nested classes and structs the same as does C. Thus, even if you had defined class node inside class list as in Listing 3, an early C++ translator would treat node as a non-nested (global) name. Listing 5 illustrates a technique that C++ programmers used to prevent users of a list class from accessing the list node class, even when that list node class is global. Listing 5 is essentially the same as Listing 1, except that it defines listnode as a class, rather than as a struct. When listnode is a struct, its members are public. When listnode is a class, its members are private. Remember, in C++, a struct is a class. A struct, like a class, can contain function as well as data members. A struct can also contain the access specifiers public, protected, and private. The only real difference between a struct and a class is that struct members are public by default, and class members are private by default. In Listing 5, not only are listnode's data members private, but so is its constructor. When the constructor is private, users cannot access it. Whenever a user tries to create a listnode object (such as by a declaration or a new-expression), the translator complains that the user cannot access a private constructor. How, then, does the list class use the listnode class? The listnode class declares list as a friend. This means that every member of class list is a friend of class listnode. Thus, members of class list can invoke the listnode constructor when creating listnode objects. Although this friend class technique prevents users from accessing the listnode objects directly, it doesn't prevent the name listnode from conflicting with other global declarations for the name listnode. Thus, I recommend using nested classes (as in Listing 3). Nested classes are now widely, but still not universally, available. Thus many of you probably won't need to use this friend class technique, but others may find it in older code that you have to use or maintain. You may also encounter it in older C++ books and articles. Declaring Out-of-Line Members The nested node class in Listing 3 defines only a single member function--a constructor. That constructor has only a short initializer list and an empty body, so it's convenient to just define the function inline inside the class. In general, however, I discourage defining member functions inside class definitions. In all but the most trivial classes, it leads to cluttered, hard-to-read class definitions. You can define member functions for nested classes outside the class definition. You write the definitions at file scope and refer to each function using its fully-qualified name. For example, Listing 6 shows list.h rewritten with the list::node constructor defined outside the class. The function definition identifies the constructor as list::node::node. If list::node had a destructor, it would be defined as list::node::~node() { // ... } Nested Types and Constants Not only can you nest classes inside classes, you can also nest typedefs and enumerations. Like nested classes, nested types and enumerations are subject to access control. For example, class shape { public: enum palette { RED, GREEN, BLUE }; shape(palette c) : color(c) { } palette color() { return _color; } // ... private: palette _color; // ... }; defines type palette as a nested type (a type member) of class shape. It also defines the constants RED, GREEN, and BLUE as member constants. Since the enum declaration is public, you can access the type name or any of the constants outside class shape using their fully-qualified names. For example, shape s(shape::GREEN); shape::palette c = shape::RED; or even if (s.color() != shape::BLUE) There's a subtle difference between a member constant and a const data member. An enumeration constant like shape::BLUE is a member constant, but MAX declared in class X { public: const size_t MAX; // ... }; is a const data member. You can use shape::BLUE in a constant expression (like the dimension of an array, or a case label), but you can't use MAX that way. Furthermore, you can't specify a value for MAX by writing class X { public: const size_t MAX = 100; // error X(); // ... }; You must initialize MAX using an initializer in a constructor, as in X::X() : MAX(100) { } If you need to use MAX in constant expressions, simply define MAX as an enumeration constant, as in class X { public: enum { MAX = 100 }; // clever // ... }; so that X::MAX is a member constant rather than a const data member. Static Members C++ defines rules by which a nested class may refer to identifiers declared in enclosing classes. These rules involve static data and function members, which I'll explain by the following example. Suppose you want to track the number of list objects in existence at any given time in the execution of your program. Simply define a counter called list_count, initialized to zero, that counts the number of objects. Then, add a statement to the list constructor to increment the counter, and add another statement to the destructor to decrement the counter, as shown in Listing 7. Notice that list_count is a global variable that exists outside class list. As always, you should avoid using global variables, but you can't make the counter an ordinary data member of the list class. If you did, you'd get a copy of a different counter in each list object. That just won't work. The variable must be statically allocated and separate from every list object so that there's one--and only one--counter for all list objects. But it should be in the scope of the class so it doesn't conflict with other global names. Static data members solve this dilemma. A static data member is not part of each class object; it's a separate data member shared by all objects of its class. A static data member is in the scope of its class and is subject to access control. For class list, you write class list { public: list(unsigned n); ~list(); // ... static unsigned count; private: // ... }; which declares count as a static member of list. The list constructor and destructor can refer to that static member as just count. count is public, so non-member functions can access it using its fully-qualified name, list::count, as in printf("# of list objects = %u\n", list::count); Unfortunately, since it is public, non-member functions can also modify list::count and invalidate the count. Thus, you should declare count private, and write a public member function, howmany, that returns the current count: unsigned list::howmany() { return count; } Hence, a non-member function can only inspect the count by calling L. howmany, where L is some list object. This protects list::count from unauthorized access, but forces users to keep an extra list object lying around just so they can call howmany. The problem is that howmany is an ordinary member function. An ordinary member function always has a hidden extra argument--the object addressed by its this pointer. But howmany doesn't need a this pointer to locate list::count because list::count is not in a list object. Rather, declare howmany as a static member function, as shown in Listing 8. A static member function does not have a this pointer, so it cannot access ordinary data members, but it can access static data members. Thus, you don't need a list object to call howmany. You simply call it by its full name, as in printf("# of list objects = %u\n", list::howmany()); If you wish, you can still call L.howmany (where L is a list object). The translator only uses L to determine howmany's class type; it does not bind a this pointer to L's address. The declaration of a static data member inside a class is only a declaration. The definition (and initialization) of the static member appears elsewhere, typically in a source file along with other members of the class. For list::count, the definition looks like unsigned list::counter = 0; as shown in Listing 9. For completeness, Listing 9 also contains the member function definitions for the list class definition in Listing 8. Accessing Members A nested class is in the scope of its enclosing class. This means that the name of the nested class is local to the enclosing class, and that the nested class may refer to identifiers declared in the enclosing class. If an unadorned identifier referenced in a nested class member function refers to a name in an enclosing class, that name must be a type name (a class, typedef, or enumeration type), a member constant (an enumerator), or a static data member. By an unadorned identifier, I mean an identifier not preceded by x. (where x is an object or reference), or by p-> (where p is a pointer). An unadorned name in a nested class member function cannot refer to an ordinary data or function member. For example, function X::Y::f in Listing 10 cannot access X::i because it doesn't have any objects of type X handy. The translator tries to interpret an unadorned reference to a non-static class member as a reference via a this pointer. But f's this pointer points to an object of type X::Y, not X. A nested class is not necessarily part of its enclosing class. For example, X::Y xy; declares xy as an X::Y object, separate from any X object. The call xy.f only provides f with access to an X::Y object. It has no X object in which to find non-static member i. On the other hand, X::s is static. It exists apart from all X objects, so X::Y::f needs no objects of type X to access X::s. A nested class has no special access rights to members of its enclosing class, and an enclosing class has no special access rights to members of a nested class. In other words, unless granted friendship by its enclosing class, a nested class cannot access private or protected members in the enclosing class. Similarly, unless granted friendship by its nested class, an enclosing class cannot access private or protected members in the nested class. Forward Declarations With all this as background, I now describe the extension to allow forward declaration of nested classes. In and of itself, the extension is quite simple. As described in the ARM, a nested class must be completely defined inside its enclosing class. But, a single enclosing class with many nested classes is usually difficult to read and maintain. This extension lets you simply declare the nested classes inside the enclosing class, and then define them later. For example, when this feature becomes available, you should be able to unravel class X { class Y { // definition of class X::Y ... }; class Z { // definition of class X::Z ... }; // ... }; into the somewhat cleaner form class X { class Y; // declaration of X::Y class Z; // declaration of X::Z // ... }; class X::Y { // definition of class X::Y... }; class X::Z { // definition of class X::Z ... }; References Ellis, Margaret A. and Stroustrup, Bjarne. 1990. The Annotated C++ Reference Manual. Reading, MA: Addison-Westley. Figure 1 A singly-linked list Listing 1 Defines the node class and the root class for a singly-linked of unsigned integer values // // list.h - list interface using global classes // struct listnode { listnode(unsigned n, listnode *p} : number(n}, next(p} { } unsigned number; listnode *next; }; class list { public: list(unsigned n}; ~list(}; void add(unsigned n}; void print(}; private: listnode *first, *last; }; // End of File Listing 2 Defines the node and root classes for a binary tree of dynamically-allocated character arrays // // tree.h - tree interface using global classes // struct treenode { treenode(const char *w); char *word; treenode *left, *right; }; class tree { public: tree(); ~tree(); void add(const char *w); void print(); treenode *root; }; Listing 3 The list class definition with the node class as a nested class // // list.h - list interface using a nested class // class list { public: list(unsigned n); ~list(); void add(unsigned n); void print(); private: struct node { node(unsigned n, node *p) : number(n), next(p) { } unsigned number; node *next; }; node *first, *last; }; // End of File Listing 4 An implementation of the list class // // list.cpp - list implementation // #include #include "list.h" list::list(unsigned n) { first = last = new node (n, 0); } list::~list() { node *p; while ((p = first) != 0) { first = first->next; delete p; } } void list::add(unsigned n) { if (last->number != n) last = last->next = new node (n, 0); } void list::print() { for (node *p = first; p != 0; p = p->next) printf("%4u ", p->number); } // End of File Listing 5 Defines listcode as a class that grants friendship to the list class // // list.h - list interface using a friend class // class listnode { friend class list; listnode(unsigned n, listnode *p) : number(n), next(p) { } unsigned number; listnode *next; }; class list { public: list(unsigned n); ~list(); void add(unsigned n); void print(); private: listnode *first, *last; }; // End of File Listing 6 Defines list_node constructor outside the nested class // // list.h - list interface using a nested class // with member definitions outside the nested class // class list { public: list(unsigned n); ~list(); void add(unsigned n); void print(); private: struct node { node(unsigned n, node *p); unsigned number; node *next; }; node *first, *last; }; inline list::node::node(unsigned n, node *p) : number(n), next(p) { } // End of File Listing 7 Tracks the number of list objects in existence at any given time in the execution of the program // // list.cpp - list implementation using a nested class // with global variables counting the objects // #include #include "list.h" unsigned list_count = 0; list::list(unsigned n) { first = last = new node (n, 0); ++list_count; } list::~list() { node *p; while ((p = first) != 0) { first = first->next; delete p; } --list_count; } void list::add(unsigned n) { if (last->number != n) last = last->next = new node (n, 0); } void list::print() { for (node *p = first; p != 0; p = p->next) printf("%4u ", p->number); } // End of File Listing 8 Defines a static data member to prevent access by non-member data functions to access count // // list.h - list interface using a nested class // with a static data member for counting the object // class list { public: list(unsigned n); ~list(); void add(unsigned n); void print(); static unsigned howmany(); private: struct node { node(unsigned n, node *p); unsigned number; node *next; }; node *first, *last; static unsigned count; }; inline unsigned list::howmany() { return count; } // End of File Listing 9 Implementation using static data member // // list.cpp - list implementation using a nested class // with a static data member for counting the objects // #include #include "list.h" unsigned list::count = 0; list::list(unsigned n) { first = last = new node (n, 0); ++count; } list::~list() { node *p; while ((p = first) != 0) { first = first->next; delete p; } --count; } void list::add(unsigned n) { if (last->number != n) last = last->next = new node (n, 0); } void list::print() { for (node *p = first; p != 0; p = p->next) printf("%4u ", p->number); } // End of File Listing 10 Illustrates access rules for nested classes class X { public: int i; static char *s; class Y { public: void f(); }; }; void X::Y::f() { s = 0; // ok, X::s is static i = 0; // error, X::i is non-static } // End of File The Elements of C Programming Style Chuck Allison Chuck Allison is a regular columnist with CUJ and a software architect for the Family History Department of the Church of Jesus Christ of Latter Day Saints Church Headquarters in Salt Lake City. He has a B.S. and M.S. in mathematics, has been programming since 1975, and has been teaching and developing in C since 1984. His current interest is object-oriented technology and education. He is a member of X3Jl6, the ANSI C++ Standards Committee. Chuck can be reached on the Internet at allison@decus.org, or at (801)240-4510. The Elements of C Programming Style gives advice to C programmers on how to create good programs. It contains tips, rules of thumb and sample programs that illustrate principles and techniques that take years to master. The authors hope to provide readers a short-cut on their journey to programming maturity and productivity. Audience The authors tried to put something for just about everyone into this book. There are specific rules and conventions for the novice. The intermediate programmer will likely gain a deeper understanding of the philosophy of C, and may have occasion to unlearn some bad habits. Experts will have an opportunity to compare their opinions with the authors', and maybe alter their own style just a little. The authors assume familiarity with basic C syntax and program structure. Content The book contains five main parts: Part I The Project Cycle Part II General Programming Advice Part II C Usage Part IV C Review Part V Miscellaneous Each part has several chapters and each chapter contains a set of rules or principles to program by. Each rule is stated concisely, then followed by a discussion. With rare exception, each rule and its discussion occupy only one or two pages, making for easy reference. Part I discusses the concepts of design, testing, debugging, and optimizing that apply to any programming language. Part II covers visual organization, commenting, data objects, numeric considerations, and input and output. Part III explores the particulars of C usage, including expressions and statements, functions, arrays and pointers, and the preprocessor. Part IV examines each operator in the language and points out key features of the Standard C Library. Part V has an unusual chapter--"Coding for non-C Programmers." This chapter doesn't quite fit the book's motif. It attempts to gently introduce non-C programmers to the language by soft-pedal-ling the "hard-core" use of C that the rest of the book encourages. The very next chapter swings the pendulum to the other extreme with idioms that are unique to C. Commentary There is much good advice in this book. I especially liked the fact that the authors do not apologize for C. (It has a bad rap in many quarters: terse, dangerous, unmaintainable, etc.) In their own words: "The C language, like many other notations (mathematical, musical, etc.), is difficult for a beginner or outsider to understand. Once mastered, however, it is easier to read than equivalent text. Readability should therefore be measured not by how similar the program is to text, but rather how effectively the notation is used to express the operations clearly. "It is a key concept of this book that this notation (the C language) should be mastered and exploited." The authors demonstrate that although C has its quirks, it is a language that works. I have very few nits to pick with this book. In one case (section 17.8), the text mentions that a signal handler should only write to variables of type volatile sig_atomic_t, but the sample code leaves off the volatile keyword. The authors also seem to think that wide characters and multibyte characters are the same thing (which they are not). Two nits in 340 pages ain't bad. Since this is book on style, it treads on subjective ground. What is elegant or aesthetically-pleasing to one may be unclear or offensive to another. For example, they prefer this while (condition_0) { statement_1; statement_2; if (!condition_1) continue; statement_3; statement_4; if (!condition_2) continue; statement_5; statement_6; } to this while (condition_0) { statement_1; statement_2; if (condition_1) { statement_3; statement_4; if (condition_2) { statement_5; statement_6; } } } They also think for (i=1, e=t=1.0; i<=n; t/=++i) e += t; is clear. However, many find e = t = 1.0; for (i = 0; i <= n; ++i) { e += t; t /= (i + 1); } is clearer. But this brings us to the most important point of all. There is no need for a "one-true-style" approach to coding. What is important is consistency. I like the way the authors say it: "While some disagreement is inevitable with a book of this sort, we hope that the expert will not condemn this book if only a few rules conflict with his or her opinions...If you disagree with a rule, read the discussion. If you still disagree with the rule, do not use it. Every programmer has personal preferences...Rules should be applied consistently, but not slavishly. A good example is the structured programming approach. Strict structured programming...prohibits the use of goto. Strict adherence to structured programming results, in general, in good code. In some cases, however, very ugly code can be generated by blind adherence to some of its rules. (For example, not allowing more than one exit point per function can create complicated situations). Consistency is very important. Blind obedience is not." While the claim on the back cover, that this is "the first book offering clear and direct rules for good C programming style," is stretching things a bit (I think Tom Plum beat them to it in the eighties with C Programming Guidelines as well as every other Plum-Hall publication), it is true that not many books on C programming style have been written. The authors identify their book as being in the spirit of Strunk and White's The Elements of Style, and The Elements of Programming Style by Kernighan and Plauger. (It wants to be sequel to the latter.) Since both of those books are about one-fifth the size of this book, they have produced a work of a different genre (should I say style?). But what they lose in length they make up in completeness. There is a place for The Elements of C Programming Style on the C practitioner's bookshelf. Title: The Elements of C Programming Style Author: Jay Ranade and Alan Nash Publisher: McGraw-Hill, 11 West 19th Street, New York, NY 10011, 1-800-822-8158, FAX 212-337-4092 Price: $29.95 ISBN: 0-07-051278-7 Editor's Forum Two years ago, I used this space to say goodbye to the Managing Editor of The C Users Journal. Howard Hyten moved on to new pursuits in Austin, Texas. As I predicted, we went through a few rough spots. (MEs do all the real work of putting together a magazine, in case you didn't know, not us Senior Editors.) And as I predicted, we muddled through. CUJ has a life of its own, aided and abetted by a dedicated staff at R&D Publications. There was, indeed, life after Howard. His eventual replacement as Managing Editor was Diane Thomas. Once she figured out my areas of (in)competence, she wised up and took over the magazine. She learned when to ask my advice and when to ignore it, what work to feed me and what to protect from me, what I did well enough and what needed close checking. In short, she found her own best solution to a perennial problem--how to put out a monthly magazine when the Senior Editor and essentially all the writers appear mostly as disembodied streams of ASCII text shipped across the Internet. You've probably guessed by now that Diane, in turn, is stepping down as Managing Editor of The C Users Journal. Fortunately for all of us, she is not disappearing from the scene completely. She is simply changing her work mix, devoting more time to raising family and less to mothering CUJ contributors. Much as I will miss having her do half my job, I respect her priorities (having been in a similar position myself). And I celebrate her freedom to make such a choice (having had similar freedom myself). So I take this opportunity to say thanks to Diane Thomas. I thank her on your behalf as well as my own. I have enjoyed working with her and I appreciate the flock of good issues she has shepherded out the door. I'm just glad we don't have to say goodbye. P.J. Plauger pjp@plauger.com P.S. R&D Publications has been actively interviewing for a replacement Managing Editor, but I don't know who will win that dubious distinction. We'll let you know. New Products Industry-Related News & Announcements Microsoft Releases Visual Programming Tools Set Microsoft Corporation has also released other products supporting developers: Microsoft Delta, a visual version control system; the Microsoft Developer Network (MSDN), a source for technical information on Windows and NT; and the Visual Control Pack, a collection of custom controls for Visual C++ (or Visual Basic). Microsoft Delta supports team-oriented software development by recording the history of a development project, storing and retrieving previous versions of files and releases, and preventing unrecorded changes to source files. Microsoft Developer Network is a membership program that provides a single, comprehensive source for technical information from Microsoft. Annual enrollment is $195 and includes four quarterly editions of the Developer Network CD; six issues of Developer Network News; discounts on Microsoft Press books; and a $25 credit toward CompuServe connect charges. Microsoft Visual Control Pack collects 19 custom controls, providing 3-D controls, and adding charting, communications, and multimedia capabilities. Contact Microsoft Corporation, One Microsoft Way, Redmond, WA, 98052-6399, (800) 227-4679, extension 11700; FAX (206) 936-7329. Microsoft Releases Visual C++ Microsoft has released the Microsoft Visual C++ development system for the Microsoft Windows operating system. Visual C++ provides an integrated visual development system for C++ programming, with a standard edition replacing QuickC and a professional edition succeeding C/C++ v7.0. The Visual C++ development system includes the following new features: Visual WorkBench, AppStudio, AppWizard, ClassWizard, and Microsoft Foundation CLass Library v2.0. Visual WorkBench is an integrated, Windows-hosted development environment. A toolbar provides one-click access to common operations. WorkBench provides access to the other tools (AppStudio, AppWizard, and ClassWizard) and provides an integrated Windows-hosted debugger. AppStudio provides a graphical approach to user interface design, resource editing, and Visual Basic control manipulation. Developers can create and edit the standard Windows-based resources, and they can install Visual Basic custom controls within the AppStudio environment, allowing use of third-party custom controls. AppStudio supports editing of multiple resources concurrently. AppWizard automates the first steps of using an application framework, creating skeleton application source code files. ClassWizard automates the process of connecting user interface elements (such as dialog buttons) created in AppStudio to the C++ code that handles them. The Microsoft Foundation Class v2.0 is an application framework for creating Windows-based applications, and is compatible with v1.0. New classes and capabilities added to v2.0 include printing and print preview, documents and views, toolbar and status bar classes, form and edit views, support for Visual Basic custom controls, scrolling and split-pane windows, OLE classes, dialog data exchange and validation (DDX/DDV), and context-sensitive help. The professional edition includes MS-DOS application targeting, CodeView for MS-DOS and Windows, Source Profiler for Windows and MS-DOS, and essential components of the Windows v3.1 SDK. The Visual C++ professional edition is $500, (standard edition is $200), with upgrade discounts available. Visual C++ requires MS-DOS v5.0+, MS-Windows v3.1, 4MB RAM (8MB recommended), and 30MB to 50MB of disk space. Contact Microsoft Corporation, One Microsoft Way, Redmond, WA, 98052-6399, (800)227-4679, extension 11700; FAX (206)936-7329. Other Vendors Release Visual C++ Support Blinkinc has released BLINKER v2.1, a free upgrade to the language-independent version of its dynamic overlay linker that supports Microsoft Visual C++, Microsoft C/C++ v7.0, Borland, Zortech, and Watcom C compilers, and other languages. Contact Blinkinc, 8001 West Broad St., Richmond, VA, 23294, (804)-747-6700. Knowledge Garden Inc. has released KPWIN++ v1.1, an integrated environment for Windows application development. The upgrade is free for existing KPWIN++ users and is compatible with Visual C++ and Borland C++ v3.1. KPWIN++ v1.1 includes a speed bar and an improved desktop. Contact Knowledge Garden Inc., 12-8 Technology Dr., Setauket, NY 11733, (516) 246-5400; FAX (516)246-5452. Dyad Software Corporation has released M++ v4.0 for Visual C++. M++ provides a math library and multidimensional array language for C++ programmers, including LINPACK code and EISPACK code. M++ is $295, $495 with full C++ source code. Modules are $195, including source code. Contact Dyad Software Corporation, 515 116th Ave. NE, Suite 120, Bellevue, WA 98004, (800)366-1573; FAX (206)637-9428. SLR Systems' OPTLIB Library Manager now supports Visual C++. OPTLIB is upward compatible with LIB, and also creates and services DOS, Windows, and OS/2 libraries. OPTLIB is $199. Contact SLR Systems, Inc., 1622 N Main St., Butler, PA 16001, (412))282-0864; FAX (412)282-7965. Data Techniques has released ImageMan v2.0, an image processing library for MS-Windows. ImageMan provides Visual C++ with support for image processing in formation including TIFF, PCX, DCX, GIF, Targa, Encapsulated Postscript, Windows Bitmap, and Windows Metafile. Version 2.0 adds support for digital cameras and scanners, as well as additional image processing features, such as high-speed rotation, wipes, and fades. ImageMan v2.0 is $395 for object code and $995 for source code. Contact Data Techniques, Inc., 1000 Business Center Dr., Suite 120, Savannah, GA 31405, (800)955-8015 or (912)651-8003. MMC AD Systems has released C Programmer's Toolbox v3.0. Version 3.0 supports Visual C++ with a collection of 30+ programming and documentation utilities, including CFlow, Clint, CXref, PMon, and others. Contact MMC AD Systems, P.O. Box 360845, Milpitas, CA 95036, (510) 770-0858; FAX (510)770-0116. Concurrent Sciences has announced support for Visual C++ in Soft-Scope III/CSiMON, a source-level, PC-hosted, remote-target debugger for embedded systems development. Soft-Scope III/CSiMON is $1,750. Contact Concurrent Sciences Inc., 530 S Asbury, P.O. Box 9666, Moscow, ID 83843, (208)882-0445; FAX (208)882-9774. Object/Designer, ImageSoft's extensible C++, C, and Pascal source code generator, has been extended with support for Visual C++. Object/Designer enables developers to generate code that directly supports derived classes. Object/Designer for Visual C++ is $499. Contact ImageSoft, 2 Haven Ave., Port Washington, NY 11050, (516)767-2233. Zinc Application Framework v3.5, an object-oriented, multiplatform development tool and interactive design tool, now supports Visual C++. The Zink Engine is $499, with at least one key required. Keys are available for DOS, MS-Windows, MS-Windows NT, OS/2, Pen-DOS, and OSF/Motif with prices ranging from $99 to $1,499. Contact Zinc Software, Inc., 405 South 100, East 2nd Floor, Pleasant Grove, UT 84062, (801) 785-8900; FAX (801)785-8996. Sax Software has released Sax Comm Objects, a Visual C++ compatible, C/C++ function and class library for building Serial Communications applications for MS-Windows. Sax Comm Objects includes a custom control for Visual C++. Sax Comm Objects is $299, upgrades are available for a limited time at $99. Contact Sax Software, Callaertstraat 23, B9100 Sint-Niklaas, Belgium, 32-37663264; FAX 32-3-7781405. Visual C++ compatibility announcements have overflowed the new products queue. Many other companies have sent in announcements, some of them are: Systems & Software, Inc. (CV/Tools), (714) 833-1700; Sequiter Software (Code-Base v5.0), (403) 437-2410; Coromandel (Integra Visual Database Builder), (718) 793-7963; Quadbase Systems, Inc. (Quadbase-SQL v2.0 for Windows), (408) 738-6989; Software Source (CB/ISAM Beta v2.0 for Windows), (510) 623-7854; FarPoint Technologies, Inc. (Visual Architect), (800) 645-5913; ProtoView Development Co. (The DataTable), (908) 329-8588; Pinnacle Publishing, Inc. (Graphics Server SDK), (206) 251-1900; PocketSoft, Inc. (RTLinkPlus v6.1 and VMData/Windows), (713) 460-5600; MicroHelp, Inc. (VBTools v2.5, 3-D Gizmos, HighEdit, and Muscle), (800) 922-3383; Media Architects, Inc. (ImageKnife/VBX vl.2), (503) 297-5010; Crystal Computer Services, Inc. (Crystal Reports v2.0 and Crystal Report Pro v2.0), (604) 681-3435; and National Instruments (NI-DAQ, NI-488.2, and NI-VXI), (800)433-3488. Columbia University Upgrades Kermit Columbia University has upgraded their Kermit communications software. C-Kermit Release 5A(188) is distributed as C source code. C-Kermit is portable and consistent across many platforms, including UNIX, VMS, OS/2, Commodore, and Atari. C-Kermit 5A(188) has added sliding windows (up to 31 window slots), and long packets (up to 9024 bytes). According to the University, by using maximum values, it is possible to have up to 280,000 bytes "in the pipe" before acknowledgment is required, bringing text file transfer efficiency into the 85 percent to 95 percent range before compression. The sliding window transport uses selective retransmission to minimize overhead on noisy connections, and the packet length adapts automatically to noise characteristics. Errors are detected by 16-bit CRC and other methods. Other C-Kermit 5A(188) features include: translation capabilities that have been extended to include East European languages, Cyrillic alphabet languages, and Japanese Kanji; a feature which allows application of language-specific rules when translating text into ASCII; keymapping, keystroke macros, command macros, and a script programming language; accessing, or acting as a Kermit file server with file management and host access functions; and documentation. C-Kermit 5A(188) supports serial connections, TCP/IP connections, and X.25 connections on Sun computers equipped with SunLink X.25. The price for C-Kermit 5A(188) depends upon the platform. Contact Columbia University, Kermit Distribution, 612 W. 115th St., New York, NY 10025, (212) 854-3703. Liant Releases C++/Views for the Macintosh Liant Software Corp. has released C++/Views for the Macintosh. C++/Views lets programmers develop applications portable across Windows, Presentation Manager, Motif, and Macintosh while avoiding the details of the APIs. Programmers can develop for one GUI and port the source code unchanged to another by recompiling. C++/Views for the Macintosh uses native GUI components to give applications a Macintosh "look and feel." As applications are ported to other environments, the "look and feel" changes to reflect the target GUI. C++/Views provides a library of C++ classes and an interactive class browser. Classes contained in C++/Views include: 100 object classes that form a library of software components to support windows, views, menus, dialogs, bitmaps, graphics, and printers; foundation classes that provide support for runtime type checking, construction, identification, and object persistence; data management classes (containers, collections, sets, and dictionaries); and higher level classes such as VTableView (for tables or simple spreadsheets). C++/Views includes C++/Browse, an interactive object class browsing utility similar to the Small-Talk browser. C++/Views is $3,000. Contact Liant Software Corp., 959 Concord St., Framingham, MA 01701-4613, (508)872-8700; FAX (508)626-2221. Segue Software Releases QA Partner Segue Software, Inc. has released QA Partner, a cross-platform, object-oriented GUI test automation system. QA Partner supports test automation of GUI applications and lets developers directly port regression suites, and test results on multiple GUI platforms. QA Partner is available for MS-Windows, Macintosh, and UNIX/Motif. Segue claims that QA Partner's "GUI-aware" design makes capture/playback and bitmap-oriented testing systems originally developed for character-mode applications obsolete. Single user licenses for QA Partner are $4,995 for the Motif version, and $1,495 for the Windows and Macintosh versions. Segue plans to release versions of QA Partner for OPEN LOOK, OS/2, and NT in 1993. Contact Segue Software, Inc., 1320 Centre St., Newton Centre, MA 02159, (617) 969-3771; FAX (617) 969-4326. WATCOM Announces ClientServer SQL for QNX WATCOM International Corp. has announced QNX support for the WATCOM SGL product line. WATCOM SQL Developer's Edition for QNX provides tools for developing database applications with WATCOM SQL, and includes a Network Server Edition of QNX, database administration utilities, and tools for C development of SQL applications using the QNX system C compiler. The package also includes the Application Creation Made Easy (ACME) front-end application development system. WATCOM SQL Network Server Edition for QNX provides software and licensing for one SQL database sewer and one client node. WATCOM SQL Client for QNX supports additional concurrent client nodes. The WATCOM SQL database server supports ANSI standard SQL and includes: bidirectional scrollable updatable cursors; referential, and entity integrity; query optimization; row-level locking; symmetric multithreading of server request; compression; and encryption. The Developer's Edition for QNX is $995, the Network Server Edition is $250, and the Client software is $150. Contact WATCOM International Corp., 415 Phillip St., Waterloo, Ontario, Canada, N2L 3X2, (800) 265-4555; FAX (519) 747-4971. Sheridan Releases 3D Widgets Sheridan Software Systems has released 3D Widgets, a programming tool for Microsoft's Visual C++. 3D Widgets supplies pre-built objects for use in dialogs, adding components to the Visual C++ AppStudio toolbox. 3D Widgets includes the following controls: ListBoxes supporting multi-column, multi-select, horizontal scrolling; hard or soft tabs; pictures for each list item; synchronized scrolling capability for multiple listboxes; command buttons with captions or pictures; plus variable forecolors and bevel width; ribbon buttons that can be used in logical groups to create tool bars; versatile panels for backgrounds or status bars with percentage display; and specialized file, directory, and drive listboxes. Other 3D Widget controls in the set include: Option Button, Frame, ComboBox, and CheckBox. Six of the Widgets are included in the Microsoft Visual Control Pack for Visual C++. 3D Widgets is $109. Contact Sheridan Software Systems, (516) 753-0985. Kuck & Associates Release C Preprocessor for IBM RS/6000 Kuck and Associates have Released KAP/C, an optimizing C preprocessor for IBM RS/6000 platforms. RS/6000 users can improve the performance of C programs in a variety of applications including robotics, simulation, and logic design. KAP/C's optimizations automatically restructure C floating-point and integer applications to execute faster on single processor RISC architectures, and improve the object code produced by the IBM XL C compiler. KAP/C performance improvements have been tested by Kuck & Associates with the SPEC benchmark suites, CFP92 and CINT93. Kuck & Associates state that KAP/C performs particularly well on three C benchmarks: alvinn, ear, and eqntott. The performance improvement users can expect from KAP/C depends on the nature of particular programs and the programming style and optimizations already coded in the program. On single processor workstation, KAP/C's optimizations are most likely to improve performance of C programs which: use significant computation time; use a large data set--the total size of the data must be larger that the cache of the machine, or a portion of each array that is used is larger than the cache; have significant data reuse within a loop nest, for example, matrix multiplication; and have loop nests that use many local arrays. Additionally, KAP/C performs pointer and I/O analysis to identify optimization opportunities. KAP/C for IBM is $595 per user for a floating license. Contact Kuck and Associates, 1906 Fox Dr., Champaign, IL 61820, (217) 356-2288; FAX (217) 356-5199. XVT Releases XVT-Design++ v1.0, XVT ToolKit v3.0 and Design Platform Support v2.0 XVT Software Inc. has released XVT-Design++ v1.0, an interactive GUI layout, design and prototyping tool for C++. The Designer can lay out interfaces interactively and specify connections between elements of the interface. Interfaces can be tested without compiling the program. XVT-Design++ v1.0 includes a C++ code fragment editor and an application framework. C++ source can be added to the graphical interface item it refers to by using the C++ code fragment editor. The XVT++ application framework provides a skeleton for portable applications and general utility objects. A C++ class is provided for each definable element. XVT-Design++ is licensed on a developer-seat basis with no additional user licensing or royalties. The price ranges from $1,395 to $2,095 depending on the platform. XVT Software has also released XVT Portability Toolkit v3.0 and XVT-Design v2.0, adding support for HP 800, AViiON, IRIS 4D RISC-based workstations, and SCO Open Desktop v2.0. Contact XVTSoftware, Inc., 4900 Pearl East Circle, P.O. Box 18750, Boulder, CO 80308, (303) 443-4223; FAX (303)443-0969. Gpf Systems Releases GpfTools Gpf Systems, Inc., has released GpfTools, a design management tool for its OS/2 v2.0 GUI development packages. GpfTools supports graphical and textual design browsing. GpfTools also allows designs to be merged and Gpf object dictionaries to be created. Large designs can be viewed graphically at any level of object hierarchy. In addition, GpfTools uses the OS/2 View utility to facilitate design browsing with hyper-text links between objects, actions, and help panels. User can merge designs and share objects by dragging an iconic representation to a separate view of a new or existing application GUI. GpfTools works with 16 bit and 32 bit version of Gpf. Gpf-Tools is $195 for a single copy. Contact Gpf Systems, Inc., P.O. Box 414, 30 Falls Rd, Moodus, CT 06469-0414, (800)831-0017; FAX (203)873-3302. Eyring Releases BCS v1.1 Eyring Corporation has released v1. 1 of PDOS/MS-DOS Backplane Communication Services (BCS). BCS lets developers use an 80x86 PC to develop real-time PDOS applications targeted for Motorola 680x0 CPUs. BCS combines a multitasking, source-level debugging environment with an MS-DOS interface. BCS lets the host MS-DOS CPU and target PDOS CPU reside in the same VME backplane, or they can be connected through a VMEbus adapter. Communications between the PC host and the target occur over the VME backplane. Host system applications can communicate through the backplane with applications on the target system. BCS is part of the PDOS Full Spectrum development kit which supports the Microtec ANSI C compiler and runtime library of POSIX and PDOS extensions. Contact Eyring Corp., 1455 W 820 North St., Provo, UT 84601, (801) 375-2434; FAX (801) 374-8339. MainSoft Ports MS-Windows Applications MainSoft (MAchine INdependent SOFTware Corporation) has introduced MainWin, a tool for porting MS-Windows applications to UNIX. Using MainWin's compatibility library and toolkit, developers can convert Windows applications written in portable ANSI C to UNIX systems supporting X Window and POSIX. MainWin uses the Windows API and the application is natively implemented on Xlib. Developers do not have to rewrite source code and the application runs under UNIX at the same level as the Windows version runs under MS-Windows. With MainWin, a converted application appears and behaves like the original Windows application. MainWin, as a native port to UNIX, avoids the tradeoffs in performance, maintainablility, and upgradeability of emulators and other layered software approaches. MainWin Software Development kits are dedicated to specific UNIX architectures, with support for Solaris v2.1, UNIXWare vl.0, AIX v3.2 for RS 6000, and HP-UX v8.0. MainWin supports Windows v3.0, with an upgrade to Windows v3.1 due later this year. Contact Machine Independent Software Corp., 185 BerrySt., Suite 5411, San Francisco, CA 94107, (800) 624-6946; FAX (415) 896-0709. Cray Releases Data Storage Management Products Cray Research has released Cray Data Migration Facility (DMF) and Cray/REELlibrarian, software products for data storage management. DMF software is a hierarchical storage manager that tracks file system space, automatically moving inactive or infrequently used files from online disk storage to off-line tape storage devices. DMF users can retrieve data files transparently, retrieving files stored offline as if they were stored online. Cray/REELlibrarian is a volume management and cataloging system that administers large-volume tape libraries. Cray/REELlibrarian users can access data on tape volumes by file name only. Cray/REELlibrarian can track millions of volumes, managing security and access control over multiple storage sites and media types. DMF and Cray/REELlibrarian are priced according to the Cray Research platform, beginning at $35,000 for DMF and $33,000 for Cray/REELlibrarian. Contact Cray Research, Inc., (612) 683-7198. Accelerated Technology Releases Nucleus PLUS and Nucleus NET Accelerated Technology Inc. has released two products, Nucleus PLUS and Nucleus NET. Nucleus PLUS, a real-time kernel, supports the components found in Nucleus RTX, but adds dynamic creation and deletion of objects: pipes, variable-length queues, counting semaphores, timers, upgraded variable memory management, signals, and the Advanced Interrupt Management Mechanism (AIMM). AIMM spawns high priority task threads out of interrupts to reduce interrupt latencies and let interrupt management be more synchronous to the tasking environment. Nucleus PLUS comes with ANSI C standard source code. Nucleus NET is an embedded TCP/IP protocol suite for use with Nucleus RTX and Nucleus PLUS. The suite includes TCP, UDP, IP, ICMP, ARP, RARP, and BOOTP protocols. Prices for Nucleus PLUS range from $5,000 to $9,995, depending on the platform. Nucleus NET TCP/IP is $9,500. Contact Accelerated Technology Inc., P.O. Box 850245, Mobile, AL 36685, (800) 468-6853; FAX (205) 661-5788. Microsoft Visual C++ Includes Phar Lap DOS-Extender Technology Microsoft's Visual C++ compiler incorporates Phar Lap Software, Inc.'s DOS-Extender technology, both within the compiler itself, by using TNT DOS-Extender, and as a tool for Visual C++ developers, by including 286/DOS-Extender Lite. To run under DOS and Windows, Visual C++ incorporates Phar Lap's TNT DOS-Extender. By using TNT DOS-Extender, Microsoft wrote its compiler for future compatibility with Windows NT, while enabling it to run under DOS as a 32 bit Extended-DOS application. 286/DOS-Extender Lite lets developers write C and C++ programs that access up to 2MB of memory, and run under DOS, DESQview, and Windows. 286/DOS-Extender Lite is included with C++, and comes with on-disk documentation and example programs, 286/DOS-Extender Lite is $495. Contact Phar Lap Software, Inc., 60 Aberdeen Ave., Cambridge, MA 02138, (617) 661-1510, FAX (617) 876-2972. Phar Lap Upgrades 386/DOS-Extender Phar Lap has released 386/DOS Extender v5.0, adding support for use of the Microsoft Windows NT 32-bit C/C++ compiler for 32-bit Extended-DOS development. The extender supports the 32 bit Microsoft compiler under DOS, giving developers the functionality of the 16-bit Microsoft C/C++ DOS runtime libraries, including graphics, using the standard 16-bit calls. The 386/DOS-Extender v5.0 Software Development Kit (SDK) also includes Phar Lap's 386/SRCBug, a 32-bit source code debugger for protected mode 386/DOS-Extender programs. 386/DOS-Extender supports the XMS, VCPI, and DPMI specification, so Extended-DOS programs will run under DESQview, OS/2 v2.0, and all modes of Windows. 386/DOS-Extender v5.0 is $495. Contact Phar Lap Software, Inc., 60 Aberdeen Ave., Cambridge, MA 02138, (617)661-1510, FAX (617) 876-2972. Rational Systems Upgrades DOS/16M Rational Systems, Inc., has released DOS/16M v5.0, an upgrade to its 16-bit DOS extender. DOS/16M v5.0 supports Microsoft's C/C++ v7.0, Visual C++, and Borland C/C++ v3.1. DOS/16M v5.0 includes the ability to build DLL-based segmented executables. At runtime, when the application calls code located in the DLL, DOS/16M transfers the required code into memory and continues execution. DOS/16M supports CodeView and Turbo Debugger. DOS/16M supports both real and protected modes. DOS/16M v5.0 conforms to memory-management standards, including DPMI, VCPI, and XMS. Prices for DOS/16M v5.0 start at $5,000. Contact Rational Systems, Inc., 220 N. Main St., Natick, MA 01760, (508) 653-6006; FAX (508)655-2753. We Have Mail Dear C Users Journal, Your article in the February 1993 issue on pointers by Christopher Skelly had several good points to it. The most useful was the author's careful distinction on page 98 regarding the mistake often made by teachers of C that "an array name is a pointer" which as the author points out "is in fact quite false..." His example, of the use of the sizeof operator, evaluating to the size of the entire array instead of the size of a pointer, is a good one. I also find his "Ladder of Indirection" a very simple and useful visualization concept. However, there are several serious errors of distinction that your author makes philosophically which should be considered important by any human being to whom precision of thought is important to their life and their work. All programmers would want to be included in this category. The first error is on page 94 in the heading "Key fact #2", wherein Mr. Skelly states that "a pointer always "knows" the type of thing it addresses." His use of the quotation marks around the work "knows" does not eliminate the error that neither a computer nor the code used to run it "knows" anything. Perhaps you'll find this distinction a bit silly or trivial. But consider living in a world where machines are elevated to the status of the human and the human is relegated to the status of the animal. Mr. Skelly moves us just a bit closer. Consider the second paragraph on page 94, "Each pointer has a built in sense of the type of the object stored at the address which the pointer contains." The word "sense" applies to a human ability. The proper word here would be description. Also consider the following paragraph containing the sentence, "Even the special case pointer to void points at a specific type and such a pointer has its own set of resources and limitations." The correct word instead of resources could be "attributes", or possibly "functions". Consider the author's worst error in the use of the word lives on page 98 toward the top of the second column in the sentence, "The [] operator, usually thought of as being related to arrays, is also a dereferencing operator. p[n] lives on the plane below p. Mr. Skelly, life is a self perpetuating and regenerative activity engaged in by an organism. Can your p[n] make this claim? A computer is a machine, a tool without volitional consciousness--without the choice to think or not to think. It must be turned on by the human operator. The application of these types of expressions and descriptions to inanimate objects or to coded central processor instructions is a key to the author's psychology as well as to that of many lonely programmers. Can not some of the best brains in the world, the programmers, consider the importance of a human centered philosophy. After all, they are not Tron, they are human. I wonder what Mr. Skelly thinks the differences are, if any, between a human and a computer. Perhaps sex? These errors are diverting to the student programmer in what could have been a much shorter, simpler and informative article in one part instead of two. Mr. Skelly should get to the solutions sooner including especially, a detailed explanation for his teaser, ++*--*pppp[0]. The entire first section up to "Key Fact #1" could have been eliminated, thereby improving the article substantially. In other words cut the B.S. and get to the precise facts. Most writers are paid by the word however. The final blow against humanity in this potentially useful article was struck by Mr. Skelly (wherein he delineates his philosophy very clearly) toward the beginning of the article on page 94 wherein he states that, "...I would rather be trout fishing than practically anything else." Mr. Skelly, I would rather be controlling my environment through the use of my computers, electronics, chemistry, machinery and any other method I can devise through the use of my living human brain to get the trout to swim right onto my plate, than anything else. Even if I had the surgical ability to tie flies, I doubt I would want to re-live the primitive life of savages like "John the Apostle Fisherman of Biblical times" in a set of wading boots. I compute for a living yes, but I also enjoy the steadily increasing control I obtain through my computer of my environment more than "practically" anything else. Most Sincerely, Hutchinson Persons Engineer I don't mean this to sound rude, but I know that Chris Skelly is a lot of fun at parties. After reading your letter, I'm not too sure about you. Translation: lighten up a little. I'm not an animist any more than Chris is. Still, at three o'clock in the morning, it's kind of fun to look on life as a kind of Warner Brothers cartoon played for keeps. I believe that the best teachers can package this kind of edgy near insanity for the enjoyment and edification of their students. The Reverend Charles Dodgson (aka Lewis Carroll) had that gift in spades. Chris Skelly has it. I aspire to it too without for a moment abandoning the notion of the universe as a marvelously contrived clockwork. One man's B.S. is another man's tutorial. One man's reversion to savagery is another man's recreation (re-creation). I'm glad the world has people like you steadily improving the technology. I'm glad it also has people like Skelly explaining it and enjoying it. And I wish Dennis Ritchie hadn't confused arrays and pointers for all time. -- pjp Editor's name game (PJP): Are you (The CUJ) the League of Women Voters? Are you the ACM? Or are you a Users Journal? What kind of users? Surely, C users, it's written on the cover! But what should I do, if I want to read some thing about C++? Standing in a book and news store reading all the titles? Mainly, I would look after a cover marking C++, but PJP not? No, he knows that inside CUJ there are C++ articles. But from what impression you derive that the normal reader has X-ray eyes detecting C++ inside? Enter and shop: all articles are marked well. Read an advertisment, always it's C a-n-d C++, and they are proud if they reached this C a-n-d C++. Imagine the book and periodical store of a company. The clerk with the task to make a magazine about C++ available, C++, n-o-t C. Could he present a magazine with only C on the cover to his clients? Try to complete your cover, otherwise you will have a veritable EDITOR'S NAME GAME and risk to fall short in C++ recognition on the CUJ cover! Sincerely, an old C and newer C++ reader of CUJ Hans G.W. Muller Yup. We're continually working on better ways to show the world what's inside our magazine. -- pjp Dear CUJ: Matt Weisfeld's article on "Solving Linear Equations Using C" (CUJ, Feb. 1993) contained a small slip in the problem's setup. The graph described by Figure 1 is not the solution space for the given constraint equations x1 = 4, x2 = 6 and 3x1 + 4x2 = 24. Nonetheless, if we switch the coefficients in the third equation, thus obtaining 4x1 + 3x2 = 24, the graph becomes correct. The correct wording, then, should have been: "Each ton of diet needs four pounds of the secret ingredient, while each ton of regular needs three pounds." Sincerely, Sergio Gonik Computational Mechanics Company, Inc. 7701 North Lamar, Sutie 200 Austin, TX 78752 Oops. -- pjp Dear CUJ: Having just finished reading the February 1993 issue, I would like to comment on two articles: 1) I found Frederick Hegeman's article "Sorting Networks" very interesting, and immediately proceeded to run some tests of my own. I found that the Microsoft 7.0 qsort function does indeed show the problems he mentioned for small numbers of comparisons; however, a slightly better-tuned qsort avoids this problem, making (on average) roughly the log(n!)/log(2) comparisons he mentions as optimal worst-case performance. I would also point out that MergeSort comes very close to the limit, even for small n. Compare worst-case behavior for MergeSort to log(n!)/log(2) and to the worst-case behavior for Bose-Nelson nets: n log(n!)/log(2) MergeSort Bose-Nelson 3 3 3 3 4 5 5 5 5 7 8 9 6 10 11 12 7 13 14 16 10 22 25 32 16 45 49 65 32 118 129 211 50 215 237 487 100 525 573 1511 1000 8530 8977 57158 10000 118459 123617 2426723 Note also that this is worst case behavior. MergeSort can do better (though not much so); Bose-Nelson networks will always require the same number of compares. MergeSort's only drawback is that it requires a scratch space as large as the input array. (Code for an easy-to-understand MergeSort, more efficient MergeSort, improved qsort, and test code used to produce the above table is enclosed. I doubt you'll want to print it, for space reasons if nothing else, but you might want to put it on your monthly code disk.) MergeSort seems to stay within about 10 percent of optimum. It would be interesting, though probably not very useful, to know how much more closely the optimum value can be approached. As an aside, I should note that Microsoft's qsort goes to n2 behavior if the array is almost, but not quite, sorted, or if the array is in descending order. Why this is left so is beyond me, since a simple change of pivot value could fix it. 2) Two small quibbles with David Burki's article on "Date Conversions:" First, it drops out the leap day February 29, 4800 BC, making dates before then off by one day. Secondly, the code does treat the year before 1 AD as 0, and the year before that as 1 BC. This is in accord with how astronomers see things (it's the reasonable way to do it), but contrary to how historians view things. They treat the year before 1 AD as 1 BC. Thus, the solar eclipse of 28 Aug 1203 BC (as astronomers reckon it) happened on 28 Aug 1204 BC (as historians reckon it). The point is made slightly moot by the fact that dates before 1582 AD are almost always Julian anyway. (Code to do the date conversions without encountering these problems is enclosed. Same comment about printing it as before.) As long as I'm here... Ever thought about putting the C Users Group code on CD-ROM? Bill J. Gray Ridge Rd, Box 1607 Bowdoinham, ME 04008 We can always benefit from more lore on sorting. And I always enjoy more calendar trivia. -- pjp Berney Williams, New Products Manager, replies: CUG Vols. 101-360, including the full text of all three volumes of the directory, are available from Walnut Creek CD-ROM, 1-800-786-9907. Dear Mr. Plauger, Since I started using Gimpel's PCLINT, I've become more aware of typing issues in my C code. I am curious about the "proper" use of the standard library function write. I use Microsoft C v6 under MS-DOS where an int is 16 bits. The prototype int write(int fid, void *buf, unsigned cnt); returns the number of bytes written if the operation is successful and a -1 otherwise. I certainly accept that a returned value of-1 implies that I can not distinguish between an error and a successful write of 65,535 bytes. But if I want to write, say, 48,000 bytes, how is write supposed to return successful value? This problem would have existed on the good old 16-bit PDP-11's, the machine C was originally designed for. More generally, shouldn't (in a perfect world) write be prototyped to return unsigned and the error value be defined as ~0? Is the fact that write accepts an unsigned count but returns an int count a semantic flaw in the standard C definition of this function? I've been programming in C for 18 years and have an appreciation of difficulties the ANSI committee must have had putting a lasso around the definition of C. It's just not obvious to me why write couldn't have been prototyped differently to avoid this particular problem. Sheldon Hoffman Reflective Computing 917 Alanson Dr. St. Louis, MO. 63132 (314) 993-6132 You've discovered an ancient type weakness in the UNIX system calls. Actually, *write* originally took an *int* count argument. Looks like Microsoft tried to half reform it. The C standards committee didn't even try -- we left *write* out of the Standard C library. (Other functions have similar problems, however.) In practice, the problem is not too severe if you follow the pattern if (write(fd, buf, n) != n) On twos-complement machines, this does what you want in all cases except a write error on a write of 65,535 bytes (not a likely occurrence if that's essentially all the bytes in your data space!). Your "perfect world" fix for write is about the best you can do, I believe. -- pjp Dear Mr. Plauger: I have been a subscriber to The C Users Journal for a couple of years now. Before that, I read the magazine sporadically. I'm generally pleased, since I find that I learn something new from almost every issue. On occasion, I have watched silently as computational and mathematical atrocities were committed within your pages. However, I can sit silently no longer. The article by P. J. LaBrocca [Mixed Numbers in C] in your April 1993 issue requires a response. LaBrocca asserts that "[t]o reduce fractions to lowest terms, you need to know the prime factors of the numerator and denominator." [p.71] LaBrocca expands on this assertion by stating that "taking out gcds does not guarantee lowest terms." [p.73] The problem with these statements is that they are completely and absolutely false. If g is the gcd of two integers a and b, then it is necessarily the case that the integers a/g and b/g have no prime factors in common--that is the essence of the definition of the gcd. It follows that the fraction a/b, when written in lowest terms, must have the form (a/g)/(b/g). Since LaBrocca obviously believes the falsehood that I quoted above, the code presented in the article is incredibly inefficient. It trades increased space and increased time for increased complexity in the algorithm. All in all, that seems like a poor trade. To be more specific, the data structure that LaBrocca uses to store the ratio of two four-byte integers occupies at least 330 bytes, since it contains room for 40 prime factors each for the numerator and denominator. No wonder that the article must point out that "stack overflow is likely!" [p.83] The code is also inefficient in the choice of the algorithm. In order to reduce fractions to lowest terms, the author uses trial division to find the prime factors. Factorization is hard--that's why public key cryptography works. Moreover, trial division is a lousy way to factor numbers. The author is again forced to reveal the inadequacies in the code, saying that "[n]umbers with [large] prime factors ... produce all kinds of annoying errors." [p.83] Of course they do. You are solving the wrong problem, using the wrong methods. Perhaps the worst aspect of LaBrocca's approach is that the correct solution has been known for 2500 years. Euclid's algorithm for computing gcds is a model of elegance and efficiency. If you combine the algorithm with back-substitution, then you can produce not only the gcd of a and b, but the reductions a/g and b/g with a mininal amount of work. (It is also interesting to note that you can determine the worst case performance of Euclid's algorithm, since you know the worst case: consecutive Fibonacci numbers. It's easy to see from this that the behavior is O(log(n)) in the size of the inputs.) Since I've already shifted into critical mode, let me complain about one more aspect of the design. LaBrocca has confused the Model with the View. (Smalltalk programmers will know exactly what I mean.) Mixed numbers are, at best, a View of the underlying Model for rational numbers. Every arithmetic operation requires LaBrocca to convert the mixed number representation into an "improper" fractional representation. It seems to me that it would be better to store and use the fractional representation internally, and only convert to mixed numbers for input or output. (This depends, of course, on whether you expect to spend more time using numbers for input/output or for computation. Usually, computation wins. With the MixCalc application that was presented in the article, it's probably an even trade.) Kevin R. Coombes Department of Mathematics University of Maryland College Park, MD 20742 email: krc@math.umd.edu I knew the approach was suboptimal in several ways, but we can't throw out every article that doesn't take the best possible approach to solving a problem. The botch about the GCD algorithm is my fault, since I edited this piece and I should know better. -- pjp Dear Sir: I have been wondering about the distribution of the ISO/ANSI C Standard. I know that paper copies can be purchased, but is there an electronic version available? Physically, I'm sure it exists, you made reference to using it in The Standard C Library, but is it available to others? For that matter, what is the legal status of the standard? Who actualy owns it, or is it held in some form of public trust? Thanks much Kevin Nickerson nickerson@bix.com ANSI and ISO are still wrestling with the logistics of electronic distribution of standards documents. Don't expect an answer soon. Yes, the machine-readable text exists, but it is only available "for the purposes of standard formation." Both ANSI and ISO jealously guard their right to make money selling copies of standards. ANSI, at least, has asserted copyright ownership of the standards it develops. That ownership can be, and has been, challenged, but I for one have better things to do with my time and money than to take on such a battle. I obtained formal permission from ISO to reproduce large portions of the C Standard in my book, and I'm grateful for their cooperation. -- pjp Dear Mr. Plauger, Apologies for the rather late comment, but it's always some time before CUJ turns up at the local newsstand. I'm wondering what is the reason for making time_t an unsigned long holding seconds since 1/1/1900 0:00. What comes first to my mind would be a structure similar to struct tm, sans the tm_?day and tm_isdst fields, probably. That would also, I guess, make coding the time conversion functions easier, since you would not have to worry that much about possible overflows. Also, a larger range of dates could be represented using such a time_t. I can imagine that time_t was chosen the way it is in your implementation because a UNIX system usually returns seconds since 1970 when you ask the current time of it. Still, the question remains: What was the reason for making UNIX behave that way? Yours sincerely Soenke Behrens uunet!informatik. tumuenchen.de!behrenss The C Standard requires that *time_t* be an arithmetic type, and that the *tm_year* field of *struct tm* count from 1900. I could have chosen a floating-point type for *time_t*, but the tradition is to use an integer type. An *unsigned long* is big enough for most purposes, anyway. I believe the UNIX folk chose a 32-bit integer representation because that was the largest integer type you could manipulate at all comfortably on the PDP-11. (At that, you had to execute two or three instructions to do most moves, adds, or compares.) They chose the starting date as the start of the decade in which UNIX was born, just a year or so before it started catching on. -- pjp Clipping Polygons with Sutherland-Hodgman's Algorithm Thomas R‚v‚sz Thomas R‚v‚sz has an MSc in Engineering Physics from the Royal Institute of Technology, Stockholm, Sweden and an MBA from INSEAD, Fontainebleau, France. He currently works with business development of software services at Digital Equipment Services AB in Sweden. His computer-related interests include object-oriented technology and computer graphics. Introduction In computer graphics, clipping is the calculation process where invisible and visible parts of a picture are separated from each other to allow for outputting only the visible parts. Clipping is used to limit output to a certain area on the screen, such as in windowing systems where a picture must not be drawn outside its host window's boundaries (Figure 1). A general clipping system must be able to handle all possible picture components; points, lines, curves, text, etc. The clipping process can often be simplified by breaking down the different picture elements into straight-line segments, which are run through a line clipping algorithm. For instance, clipping a circle would generate chains of small linear segments, forming one or more circular arcs. In the clipping process, the input graphic entity circle has been transformed into the graphic entity line. However, for some graphic entities it is advantageous to be able to preserve the entity through the clipping process. This is particularly true of polygons--closed outlines bounded by straight edges--which are used to define areas of colours or patterns. An area that is clipped by a rectange is still an area, albeit smaller. Applying a line clipping algorithm to the perimeter of an area would generate picture B in Figure 2, while the desired result has a closed outline as in picture C in Figure 2. Hence, we must use an algorithm that clips areas rather than line segments. The algorithm differs from a simple line clipping algorithm in that it must close the area outline with appropriate sections of the clipping area edge. To complicate things further, a concave polygon (one that has at least one interior angle greater than 180 degrees) might be clipped into several smaller polygons. Sutherland-Hodgman's Algorithm A solution to these problems is Sutherland-Hodgman's algorithm, as described by Newman and Sproull in their excellent Principles of Interactive Computer Graphics (McGraw-Hill, 1979). The algorithm clips a polygon against each clipping edge in turn, thereby generating intermediate polygons as in Figure 3. Figure 3 shows what may happen when a concave polygon is clipped. The output consists of two polygons connected by a degenerate edge. However, for calculation and display purposes the output can still be treated as a single polygon. Degenerate edges do not interfere with the display of areas, since they outline a zero area. While the input polygons can be of any shape, the clipping area has to be convex. The algorithm relies on the fact that a point that lies on the invisible side of an edge is always outside the clipping area. This is not true for concave clipping areas. Except for this restriction, neither the number of clipping edges nor their shape is limited. It is perfectly possible to clip polygons against a clipping area outlined by, say, three circular arcs. The algorithm for clipping against a single edge consists of two parts. The first part is applied to every vertex of the polygon, while the second closes the polygon when all the vertices have been processed. At various points the algorithm outputs a vertex, which may be added to a list representing the output polygon. This polygon is then clipped against the next clipping edge and so on. Listing 1 shows part 1 of the clipping algorithm, which is applied to every vertex. Listing 2 shows part 2, which closes the output polygon. One would expect the intermediate polygons to require large amounts of storage, but this is not the case. Only two coordinate pairs need to be stored for each clipping edge. Newman and Sproull say: "The algorithm generates the vertices of the output polygon in sequential order and then feeds each of these vertices in the same order to the next step in the process. Instead of storing each output vertex until the step is complete, the vertex can be passed directly to the next step by invoking the same algorithm in a recursive manner." What makes things complicated is that the clipping process for each edge is a state machine, since the processing depends on whether the received vertex is the first vertex or not. The clipping processes are not necessarily in the same state. For example, several vertices may be clipped against the first edge and discarded until a first visible vertex is output and clipped against the next edge. This implies that each recursive step of the algorithm must not only be supplied with data about which clipping edge to use, but also with the state of the clipping process against that edge. Implementing this algorithm in C is feasible, but C++ is by far the superior tool. By representing each clipping edge with a C++ object that contains all the necessary information about the edge, including the processing state, the proposed recursion algorithm becomes a processing pipeline (Figure 4). Implementation The C++ implementation of the algorithm consists of two classes: ClipEdge and LinearEdge. While ClipEdge represents a clipping edge in general, LinearEdge is derived from ClipEdge and implements a linear clipping edge. Class ClipEdge ClipEdge contains the function add, which is used for assembling ClipEdge objects to a processing pipeline. The main processing functions are vertex and close, which implement the two parts of the algorithm. vertex takes two parameters, the vertex point itself and a reference to an array where generated output is to be stored. close needs only the first and the last polygon vertex for its processing, which are already stored in the object, and therefore takes only the output array as parameter. vertex and close send the generated output vertices to the function output, which passes them on to the next ClipEdge object in the pipeline. Only at the end of the chain are vertices written to the output array. In their processing, vertex and close rely on the virtual functions visible and clip. visible tests whether a point is on the visible side of an edge or not and clip clips a straight line against that edge. visible and clip cannot be defined in ClipEdge since they depend on information that is not available in ClipEdge, namely the edge position and shape. Instead, these functions must be defined in a derived class. This requirement has been enforced by declaring the functions as pure virtual. Class objects that contain pure virtual functions cannot be explicitly constructed, but must be created through a derived class. Clipping edge intersection is detected by XORing the visibility of the current vertex with the visibility of the previous vertex. If one of the vertices is visible and the other invisible, the line between them must intersect the clipping edge. vertex_received is a state flag that indicates whether the ClipEdge object has received a vertex before or not. Class LinearEdge Class LinearEdge implements the linear clipping edge. The edge can have any slope and need not be parallel to the coordinate axe. Since most of the work is done by ClipEdge, the LinearEdge class only has to supply the functions visible and clip. The visibility is checked by simply deeming all points on the right side of the edge invisible. This definition requires a directed line, a vector, which is achieved by using the first constructor parameter as the line starting point and the other as the line end point. (See the sidebar, Checking the Visibility of a Point.) The algorithm assumes that the coordinate system used is right-handed. In right-oriented coordinate systems the X-axis is always "to the right" of the Y-axis. That is, the X-axis points to the right and the Y-axis points upward, or the X-axis points downward and the Y-axis points to the right, etc. Left-handed coordinate systems are not very common in printed drawings and diagrams, but are sometimes used to address points (pixels) on computer displays. An example is when the origin is located in the upper left corner of the screen with the X-axis pointing to the right and the Y-axis pointing downward. The line intersection is calculated by using the parameterized definition of the line: x = x1 + t*(x2 - x1) -x < t < +x y = y1 + t*(y2 - y1) where (x1, y1) and (x2, y2) are two points on the line, instead of the more common y = k*x + c .x < k, c < +x where k is the line slope and c the intersection with the Y-axis. The latter definition requires special treatment of vertical lines, while the parameterized definition is direction independent. This is an advantage from an algorithmic point of view. (See the sidebar, Calculating the Intersection between Two Lines.) The Test Program The first section in the test program sets up the clipping pipeline. In this case, the clip area is a tilted quadrant with the corner points (150, 75), (225, 150), (150, 225) and (75, 150). Note how the LinearEdge objects are initialized. The interior of the clipping area is to the left of all edges. The definition of the input polygon is self-explanatory. It is followed by the creation of the array that will hold the vertices of the output polygon. With linear edges, the number of output vertices will always be smaller than two times the number of input vertices plus the number of clip area corners. The clipping is done by feeding the first ClipEdge object with the vertices of the input polygon. When all vertices have been processed, the output polygon is closed and the result can be enjoyed by viewing the output array, preferably by using the polygon drawing primitive of a graphics library. Figure 5 shows the polygon outline before and after clipping. Summary Most graphics libraries support clipping, but only against rectangular areas aligned with the coordinate axes. This program offers a more flexible approach to windowing and clipping. It is also gives a first insight into some mathematical tools that can be put to work in graphics applications. Since ClipEdge is an abstract base class, the implementation can be extended easily without changing the existing code. Note that the implementation already supports the use of convex polygons of any size as clipping areas. It is just a matter of inserting the desired number of LinearEdge objects in the processing pipeline. An extension of the implementation could supply classes for handling non-linear edges, such as circular or elliptical ones. Or, classes could be derived from LinearEdge to handle vertical and horizontal edges only. This would improve performance, since the multiplication and division operations in visible and clip could be replaced by simple comparisons. Thanks to the object-oriented features of C++, the implementation becomes really compact and flexible. It would be hard to do it as elegant in another language. Checking the Visibility of a Point Technically, the position of a point, relative to an arbitrary straight line, is computed with a vector cross-product. This concept should be familiar to those who have studied vector algebra. The formula is not intuitively easy to understand, but can be illustrated with the following example: (xs, ys) point on the line; used as the vector starting point (xe, ye) point on the line; used as the vector end point (xt, yt) test point Rotate the vector 90 degrees counter-clockwise, the new vector end point (xr, yr) is given by: xr = xs - (ye - ys) yr = ys + (xe - xs) Project the test point onto the new vector, using the vector inner product (which many people are familiar with): dxt = xt - xs dyt = yt - ys dxv = xr - xs = ys - ye dyv = yr - ys = xe - xs D = dxt*dxv + dyt*dyv If D is positive, the test point is on the positive side of the vector's starting point (xs, ys), which implies that it is also to the left of the original line (assuming that a right-handed coordinate system is used). Figure 6 illustrates the algorithm with the two test points (x1, y1) and (x2, y2). Calculating the Intersection between Two Lines The common algebraic formula for defining two lines gives: y = k1*x + c1 y = k2*x + c2 At their intersection point (if any), x and y of both lines are identical. Solving the two equations yields: x = (c2 - c1)/(k1 - k2) y = k1*(c2 - c1)/(k1 - k2) + c1 The denominator (k1 - k2) is zero when the slope parameters k1 and k2 are equal. This singularity is expected, since parallel lines do not have an intersection point. Another singularity occurs when one of the lines is vertical--it has an infinite slope. Such cases must be treated separately, which is a serious drawback with this algorithm. Another way to define two lines is in terms of two fixed points ( (x1,y1) , (x2,y2) or (x3,y3) , (x4,y4) in this case) and a parameter (t1 or t2). x = (x2 - x1)*t1 + x1 x < t1 < +x y = (y2 - y1)*t1 + y1 x < t1 < +x x = (x4 - x3)*t2 + x3 -x < t2 < +x y = (y4 - y3)*t2 + y3 -x < t2 < +x At the intersection point, x and y of both lines are again identical: (x2 - x1)*t1 + x1 = (x4 - x3)*t2 + x3 (y2 - y1)*t1 + y1 = (y4 - y3)*t2 + y3 Solving these equations yields: t2 = [(y3 - y1)(x2 - x1) - (x3 - x1)(y2 - y1)] / [(x4 - x3)(y2 - y1) - (y4 - y3)(x2 - x1)] The denominator becomes zero when (x4 - x3) (y2 - y1) equals (y4 - y3) (x2 - x1). Vertical and horizontal lines will create zero terms in the denominator, but the whole expression will evaluate to zero only when the lines are parallel. The intersection point can then be computed: x = (x4 - x3)*t2 + x3 y = (y4 - y3)*t2 + y3 Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Listing 1 Part 1 of the clipping algorithm, applied to every vertex enter with vertex CurrentPoint IF first point THEN store CurrentPoint as FirstPoint for later use ELSEIF the line from LastPoint to CurrentPoint intersects edge THEN output the intersection point ENDIF IF CurrentPoint is on visible side of edge THEN output CurrentPoint ENDIF store CurrentPoint as LastPoint for later use * End of File */ Listing 2 Part 2 of the clipping algorithm, closes the output polygon enter to close IF the line between LastPoint and FirstPoint intersects edge THEN output the intersection point ENDIF /* End Of File */ Listing 3 Implementation of the ClipEdge and LinearEdge Classes /***************************************************** GENERAL Name : pgnclip.cpp Author : Thomas Rvsz PURPOSE This file implements Sutherland-Hodgman's polygon clipping algorithm. DOCUMENTS Name : --- DEPENDENCIES Compilers dependency : C++ Machine dependency : None Environment dependency : None HISTORY 930303 1.0 Thomas Rvsz Created *****************************************************/ #include #include /****************************** * * Simple data types * ******************************/ enum bool {false, true}; // boolean typedef signed long coord; // coordinate class point // point { public: point () : x (0), y (0) {} point (coord x, coord y) : x (x), y (y) {} int operator == (point&); int operator != (point&); coord x, y; }; inline point::operator == (point& p) { return (x == p.x && y == p.y); } inline point::operator != (point& p) { return !(*this == p); } inline ostream& operator << (ostream& o, point& p) { return o << '(' << p.x << ',' << p.y << ')'; } /********************** * * Class ClipEdge * **********************/ class ClipEdge { public: ClipEdge () {vertex_received = false; next = 0;} ~ClipEdge () {if (next) {delete (next);}} void add (ClipEdge *); void vertex (point, point*&); void close (point*&); protected: virtual bool visible (point) const = 0; virtual point clip (point, point) const = 0; void output (point, point*&) const; private: ClipEdge *next; bool vertex_received; point first, previous; bool first_visible, previous_visible; } inline voif ClipEdge::add (ClipEdge *edge) { edge->next = next; next = edge; }; void ClipEdge::vertex (point current, point*& outpoint) { bool current_visible = visible (current); if (!vertex_received) { vertex_received = true; first = current; first_visible = current_visible; } else if (previous_visible ^ current_visible) { point clipped = clip (previous, current); output (clipped, outpoint); } if (current_visible) { output (current, outpoint); } previous= current; previous_visible = current_visible; } void ClipEdge::close (point*& outpoint) { if (vertex_received) { if (previous_visible ^ first_visible) { point clipped = clip (previous, first); output (clipped, outpoint); } if (next) { next->close (outpoint); } vertex_received = false; } } void ClipEdge::output (point curent, point*& outpoint) const { if (next) { next->vertex (current, outpoint); } else { *outpoint++ = current; } } /************************** * * Class LinearEdge * **************************/ class LinearEdge : public ClipEdge { public: LinearEdge (point, point); protected: virtual bool visible (point) const; virtual point clip (point, point) const; bool isOnLeft (point) const; const point start, end; coord dx, dy; private: int sign (double value) const; cord round (double value) const; }; LinearEdge::LinearEdge (point start, point end): start (start, end (end) { dx = end.x - start.x; dy = end.y - start y; } inline bool LinearEdge::visible (point p) const { return isOnLeft (p); } inline int LinearEdge::sign (double value) const { return (value >= 0 ? 1 : -1); } inline coord LinearEdge::round (doble value) const { return (coord) (value + sign (value)/2.0); } inline bool LinearEdge::isOnLeft (point p) const { return (bool) (dx * (p.y - start.y) - dy * (p.x - start.x) >= 0); } point LinearEdge::clip (point p1, point p2) const { coord dx1 = p2.x - p1.x; coord dy1 = p2.y - p1.y; coord denominator = dx*dyl - dy*dxl; assert (denominator != 0); double t = (doule) (dxl*(start.y - p1.y) - dyl*(start.x - p1.x))/ denominator; return point (start.x + round (t*dx), start.y + round (t*dy)); } /**************************** * * The test function * ****************************/ void main () { // Set up the ClipEdge pipeline point p1 ( 75, 150); point p2 (150, 75); point p3 (225, 150); point p4 (150, 225); ClipEdge *clip = new LinearEdge (p1, p2); clip->add (new LinearEdge (p2, p3)); clip->add (new LinearEdge (p3, p4)); clip->add (new LinearEdge (p4, p1)); // Define the input polygon const int no_of_points = 8; points indata [no_of_points] = { point ( 50, 50), point (200, 50), point (200, 200), point (150, 200), point (150, 100), point (100, 100), point (100, 200), point ( 50, 200) }; // Allocate memory for the clipped polygon // (2*#input points + #clip edges) point *outpoint = new point [2*no_of_points + 4]; point *base = outpoint; // Clip the polygon for (int i = 0; i < no_of_points; i++) { clip->vertex (indata [i], outpoint); } clip->close (outpoint); // Display the result while (base < outpoint) { cout << *base++ << endl; } cout << endl; } // End of File Detecting Text Regions Using Cellular Automata David Weber David Weber has been writing software for the last 22 years and managing projects for a decade. He is currently an independent algorithm designer hiding out in rural Colorado. Cellular automata are a powerful tool. You will find them used in a wide range of disciplines, including physics, simulation modeling, statistical mechanics, and aerodynamics. Any problem expressed as local effects between neighboring cells is a candidate for cellular automata. In this article I present an algorithm for finding text regions on a bitmapped image of a page using cellular automata. This algorithm takes advantage of the fact that Western languages are essentially horizontal. Nothing in the algorithm invalidates it for the requirements of a vertical language, such as Chinese. You just need to make a few modifications. I will note these changes where they are applicable. The Algorithm I adjusted the rules of the original cellular automata, the game of Life by J.H. Conway (see the box "How Cellular Automata Works"), to find text regions. Because I want rectangular regions, I ignore the cell neighbors along the diagonals. Instead of an octant of neighbors, I have a quadrant. I use those above, below, left and right of the cell. This helps square out the regions. It also enhances speed by reducing the number of calculations. Second, I throw out the rule that causes death by overcrowding. Overcrowded, filled-in regions are what I want, so I make no penalty for them. Third, I encourage fertility. I make it easy to birth new cells and fill the interstices. Finally, I don't worry about the edges. Edge effects in this variation are self cancelling. They do not change the outcome. The CUJ online source code includes the full code for a demonstration exercise. Like most real projects, the bulk of the software deals with the user interface. zone.h (Listing 1) and zone.c (Listing 2) contain the actual guts of the machine. I took two liberties in the example code. I turned off the vertical-line cutter (see the box called "How OCR Works") so people would not wonder why pictures were being chopped. I also sampled the image while I scaled it for display. Usually, you would read the entire image into memory and sample it from there. The memory image is then available for future operations like filtering or OCR. Since images of pages are quite large, this means storing the page in EMS/XMS memory or using a 286/386 extender. I wanted the demonstration program to run on anything from CGA to SVGA and not be constrained by memory requirements. So I sampled the page on the fly. You may see where judicious use of pointers, instead of array indexing, will lead to speed. I saw some of them, too, but leaned toward clarity of expression rather than raw results. In a commercial implementation, I would sacrifice at the altar of performance, but keep a clear exposition of the algorithm in my bottom drawer. Detecting Text Regions Region detection, the first phase of the OCR process, presents several problems. Text page format, in Western languages, consists of regions of horizontal lines separated by vertical white space called column breaks. You need to find these column breaks and use them to differentiate regions. Text varies widely in size, typically from six to 24 points. To complicate matters, many pages include a random spattering of pictures, logos, and graphics across the page, conditions that can seriously hamper OCR. Forms introduce more problems for the region detector. Sometimes lines and boxes separate regions of text. The lines are convenient for the human eye, drawing attention to restricted regions. The computer finds them a hindrance. They are just noise in the way of the characters. There are five steps to detecting text regions: Sample the page Cut vertical lines out of the sample Block the sample using cellular automata Extract rectangular zones from the blocks Sequence the zones in normal reading order Page Sampling Proper sampling is the key to effective OCR. Mess up the sample and everything else falls apart. Sample too frequently and the regions spread out beyond the capacity of cellular automata to coalesce them. Sample infrequently and everything collapses into one big region. I developed sample_page from the empirical testing of a range of typical pages. The exact sizes of sample_page's parameters are important but not critical. The algorithm is flexible enough to adapt to a variety of inputs without breaking down. Looking at the sampling intervals tells you two things. First, the size of the parameters depends on the DPI (dots per inch) density of the original page. For demonstration purposes, I hardcoded midrange values for a 200 DPI page (see Listing 1). For a 300 DPI page, you would scale up these parameters. Similarly, for a 100 DPI page, you would decrease them. If the source page has a non-square density, like a 200 x 100 DPI FAX, the sampling values you choose should be proportionally out of square. Second, sampling rates are not identical in both directions. The horizontal axis uses a greater sampling frequency (10 times) than the vertical axis (eight times), because Western text is primarily horizontal. In this application, I emphasize horizontal phenomena. If the target language was Chinese, the vertical sampling would be more frequent than the horizontal. For more efficient page sampling, I double the sampling rate in the Y direction and OR pairs together, instead of just increasing the Y sample frequency (see Listing 2). This technique prevents text from going unnoticed while doing a low frequency scan down the page. By choosing bit testing or byte testing, depending on the sample rate, I can minimize the chance of missing a piece of text, and maximize the width of my sampling glance without overlapping sequential samples. If the sample rate is below eight, I look at bits. If it is above eight, I look at bytes. Lastly, I store each sample in the buffer as a byte, not as a bit. This wastes some space, but enhances speed. It also allows me to use the extra bits as generation flags when iterating over the page. A typical letter-size page scanned at 200 x 200 DPI has a raw image size of about half a megabyte. The sampled image, on the other hand, fits in a 46KB buffer. It is possible to do cellular automation over the original image at the original size but it is abysmally slow. The sampled image gives the same results as the full page but with much greater speed. Removing Vertical Lines With a sampled page in the buffer, the next step removes vertical lines. cut_vertical_lines removes boxes enclosing areas of text. A box has both horizontal and vertical lines, but only vertical lines need removing. Once the vertical lines disappear, the outline breaks, exposing the text to cellular combination. Trying to cut the horizontal lines of a box is tricky. After sampling, small text (six-point) looks much like a line. Any attempt to cut the horizontal lines will probably remove some text along with the lines. Of course, Chinese is opposite. It requires cutting horizontal lines and leaving vertical lines alone. Before removing the lines you must decide on the minimum length of a vertical line suitable for cutting. Choose a line too short and you cut up large text fonts. Make it too long and you miss small boxes. The value I chose is arbitrary. It seems to function well over a spectrum of forms. The optional step of removing vertical lines has its hazards. A picture with lots of dark space will look much like a cluster of vertical lines. Therefore, the cutter will chop it up. If you want to extract graphics from the page, pull them first before cutting the vertical lines. For OCR purposes, it doesn't matter since the recognizer discards pictures anyway. Blocking Text Regions with Cellular Automata After sampling the page and possibly cutting out vertical lines, you need to block text regions with cellular automata. block_zones is the heart of the algorithm I am presenting. To achieve accurate region detection you make a single pass over the sample area with a cellular automaton filter, then traverse it with a coalescing filter that fills the gaps. The cellular automaton pass filters out dirt and enhances text blocks. Multiple cellular iterations will improve the image, but you pay with time. Like image convolutions, cellular automata make many visits to each pixel. In this algorithm we encounter each pixel five times, either as a neighbor or as a candidate cell. People with faster processors or more patience might want to experiment with multiple iterations. Rather than having two toggle buffers for cellular automation, I use a single buffer. Bit 7 and bit 0 distinguish between current and future states of life and death. After processing the sample, I take another pass to bury the pixels slated to die and deliver all pixels destined for birth. After cellular automation comes coalescence. A user input parameter, coarseness, ranging from 0 to 5, establishes how pixel clusters coalesce into regions. A coarseness of zero produces many small regions while a coarseness of five creates a few large regions. Coalescence works by looking at neighbors some distance from a pixel. For example, a coarseness of 3 closes horizontal and vertical gaps of three or fewer pixels. The coalescence process takes two passes. Horizontal coalescence goes first, then vertical. Why not save time and combine these into a single pass? Again, our need for horizontal preference requires separate passes. Chinese, once more, inverts the order, having the vertical pass first and the horizontal second. Extracting Rectangular Zones Extracting rectangular zones from the blocked out regions is the next step of the algorithm. extract_zone starts in the upper left corner of the page and proceeds downwards in raster scan fashion. When it hits a black region, it walks the region's perimeter, and saves the minimum and maximum X/Y values of the perimeter. This defines a rectangular zone. If the enclosed region is significantly large, the program saves the region on a zone list. extract_zone discards any small fry. A region, once extracted, should not be reconsidered. To prevent this redundancy, the program whites out the region after extraction. It repeats this process down the page, gathering the zones onto a list. Overlapping zones can cause problems. You do not want to perform OCR on the same area of the page more than once. There are several ways to solve this. This application uses a somewhat simplistic approach. If two or more zones overlap, overlap_zones combines them and makes one larger zone. The new X/Y minima for the combined zone come from the lesser of the source zones' minima. The maximum, likewise, comes from the greater of the maxima. If you define a zone as a polygon instead of a rectangle, you can combine two rectangular zones into a larger polygonal region. These polygons will more truly reflect the layout of the page, compared with trying to force it into rectangles. However, most page formats follow rectangular grid lines. The complexity of polygonal region boundaries may produce an insignificant reward. Sequencing the Zones The last step in resolving text regions is sequencing the zones. When you visually scan a page in an English language magazine the flow of the page follows one of two patterns. The text either moves in columns down the page and then across. This is termed column major ordering. Or it moves first across the page and then later down. This is row major order.sequence_zones takes a user- provided type parameter that chooses either column- or row- major order. The sequencer reorganizes the zones on the list to reflect their order on the page. Its operation is quite simple. It just compares the X/Y ordering of the zones and swaps any that are out of place. Sequencing is heavily language-dependent. I have mentioned Chinese, which naturally requires a different sequencer. It affects other languages also. Hebrew reads right to left. The horizontal sequence runs opposite from English while the vertical sequence remains the same. The Hebrew sequencer needs to flip the horizontal numbering. Hebrew presents even more problems for OCR systems. Although it reads right to left, it can contain embedded English words or phrases that scan left to right. This means the scanning engine and page reconstructor depend on linguistic context. This can lead to lots of late night coding entertainment. Extensions of the Algorithm As it stands, the algorithm requires two user input parameters, coarseness and sequence ordering. How can I automate these? Placing the algorithm in a feedback loop will remove the coarseness parameter. If the results have many small regions, the coarseness is too low. Rerun the function with a higher coarseness. If there is only one large region, retry with a lower coarseness value. Removing the sequence ordering parameter is difficult. It requires knowledge of text content. Everyone has seen a poorly laid out magazine page where it took a second to find the continuation of a column. Think how much fancy processing you are doing when your eye and brain link to the next column. Completely automating column linkage requires a higher level of knowledge about the page. Dot matrix has always been a hurdle for OCR. The separated dots cause havoc for the character isolator. Filtering with cellular automata fills in the spaces between the dots, simplifying the problem. It is a good preprocessor for dot matrix. The techniques discussed here apply to monochrome images. How do you handle color? This is a subtle obstacle. It requires remapping the color space to two values, foreground and background. You then extract the foreground elements from the background. You might have to vary the definition of foreground and background colors across different parts of the page. Conclusion Cellular automata are more than just a source of entertainment. They are a powerful tool, useful for simplifying difficult tasks. And simplification is the essence of intelligent software design. When deciding whether or not to use cellular automata, keep these few rules in mind. Can the problem be divided into cellular space? Does the problem involve local effects, not action at a distance? Can cellular evolution take the problem from a complex state to a simpler one? If these rules apply, give cellular automation a try. How OCR Works Optical character recognition (OCR) gets much press. Unfortunately, anyone who has spent any time running typical documents through a commercial OCR package quickly realizes how far the technology is from acceptability. This predicament has spawned many ingenious algorithms, ranging from matrix matching through machine-learned fragment analysis, contour analysis, and neural networks. The problem, however, is much bigger than these solutions suggest. There are four phases to OCR. First, a region detector cuts the page into regions, filters dirt, and ignores white space. Second, an isolator isolates shapes inside a region then decides whether they are candidates for character recognition or just ambiguous graphic garbage. Third, a recognizer operates on this detached shape and generates a list of best character guesses. Finally, a page reconstructor takes a raw list of characters and their locations on the page, then merges them into a reasonable facsimile of the original page. This last phase usually incorporates a dictionary comparator that permutes the guesses of the recognizer into reasonable words. Character recognition, phase 3, is the sexy phase of the procedure and, as such, gets all the attention. But the other phases are just as essential for effective character recognition. Although it sounds sequential, the process is neither discrete nor linear. For example, the isolator may pull a blob from the region and pass it to the recognizer. Upon which, the recognizer says, "I don't know this at all," and passes it back to the isolator. The isolator takes a second glance, thinks it sees a ligature joining multiple characters, chops them apart and sends them back to the recognizer. Any effective solution must use this kind of feed-back to correct discrimination errors between phases. How Cellular Automata Works The first example of cellular automata was the game of Life by J.H. Conway. This game inspired all later cellular automata research. The game earned the name Life because it vaguely mimics population dynamics. But don't dig too deep into the metaphor. In reality, most uses of cellular automata borrow just a few salient points from the original game, then create an entirely different context. In particular, the most important ideas behind cellular automata are: the division of space into cells, the state of the cell depending only on the state of its neighbors, and the iteration of these cells over time. For example, an atmospheric cellular simulation might divide a three-dimensional volume into cubes. Each cube has several degrees of freedom involving temperature and mass exchange through the cube walls. Each cube has 6 immediate neighbors that directly affect the state of the cube. By setting up complex rules of cellular interaction based on physical laws, it is possible to study simulations of cloud formation and propagation. One valuable advantage to cellular automata is their localization and simultaneity. This means they work great on parallel processing systems. How Life Works Take a sheet of standard graph paper and a pencil, and fill a number of squares. Darken squares based on some spatial model, such as text characters on a page. Take a second piece of graph paper and lay it near the first. For each square on the first graph, look at the neighbors surrounding the square. Count them. What happens next will depend on this count. There are four possible outcomes. If the number of neighbors around a filled square is below some magic value, the square starves to death. If the number of neighbors around a filled square is above some level, the square dies of overcrowding. If the number of neighbors lies between starvation and crowding, the filled square continues to exist. If the square is empty and the number of neighbors is just right, the square becomes pregnant. On the second sheet of graph paper, mark the results of applying these rules to the squares on the first sheet. Either a full square dies and becomes empty, an empty square births and is filled, or a square continues like it was. After visiting the squares of the first graph, counting neighbors and using the transformation rules, you have a new population of squares on the second graph. You can take this new population as a starting point for yet another graph and repeat the loop, ad infinitum. Each new generation will reflect the original distribution of squares and the cumulative effect of the transformation rules. Why two sheets of paper? Why not just mark it in place on one sheet? Because the transformation must be instantaneous and parallel. If you mark changes and count on the same sheet, the marked squares will influence the partially-completed transformation. This creates a different result from that obtained by marking the changes on a second sheet. The life in each square needs to evolve simultaneously with its neighbors. Because of this, most cellular-automata techniques use a pair of toggle buffers. One holds the current generation and another holds the future state under construction. Once you finish constructing the future state, you display it. Then you clear out the initial buffer and proceed anew, building the next iteration. Alternatively, you can use only one buffer but have some form of bit flagging to tell the difference between current and future states. Another point to consider is the edge. If you glance at the corners and edges you will count fewer than eight neighbors for each pixel. How do you handle this? In the original Conway game the best solution is a closed universe. Bend the left side of the graph around to the right side and tape them. You now have a tube of paper without a left or right edge. Similarly, bend the top of the tube down to the bottom and tape them. You have a closed torus with no edges. The next most popular solution to the edge problem is an invisible pseudo edge, one square wide, which acts as a sentinel boundary encircling the graph. It provides the proper number of neighbors for cells on the graph but never appears on the display. Cellular propagation, into and beyond this boundary, falls off the edge of the world. Listing 1 zone.h /*************************************************************** * file: ZONE.H * purpose: definitions, types and prototypes for zone.c **************************************************************/ /* definitions */ /* sampling */ #define SAMPLE_200_X 8 /* x sample frequency for a 200 dpi image */ #define SAMPLE_200_Y 10 /* y sample frequency for a 200 dpi image */ #define MAX_MALLOC 0xff00 /* maximum malloc size */ /* vertical line cutting */ #define CUT_VERTICAL_LINES 0 /* turn vertical line cutting on/off */ #define VERTICAL_LINE_SIZE 16 /* minimum vertical line length @ 200 dpi */ /* blocking */ #define WHITE 0x00 /* white color in sample */ #define BLACK 0xff /* black color in sample */ #define PREGNANT 0x01 /* ready to be black */ #define SICK 0xfe /* ready to be white */ #define ALIVE 0x80 /* still alive */ #define LIFE_SCORE 2 /* neighbors for birth */ #define DEATH_SCORE 0 /* neighbors for death */ /* sequencing */ typedef struct { int x1,y1,x2,y2; } ZONE; #define MIN_X_ZONE 4 /* minimum allowable x extent for a zone */ #define MIN_Y_ZONE 2 /* minimum allowable y extent for a zone */ typedef enum { GOING_UP,GOING_RIGHT,GOING_DOWN,GOING_LEFT } HEADING; #define MAX_ZONES 99 /* maximum number of zones */ #define COLUMN_MAJOR 1 /* order zones with columns dominate */ #define ROW_MAJOR 2 /* order zones with rows dominate */ /* prototypes */ int zone(int coarseness,int order); /* End of File */ Listing 2 zone.c *************************************************************** * file: ZONE.C * purpose: text region detection via cellular automata *************************************************************/ #include #include #include #include #include "zone.h" /* local data */ static ZONE zone_list[MAX_ZONES]; /* this could be dynamically allocated */ static int zone_count; /* local prototypes */ static unsigned char *sample_page(int *dx,int *dy,int *samplex,int *sampley); static void cut_vertical_lines(unsigned char *image,int dx,int dy); static void block_zones(unsigned char *image,int dx,int/, dy,int coarseness); static void sequence_zones(unsigned char *image,int dx,int dy,int order); static int extract_zone(unsigned char *image,int dx,int dy,int x,int y,ZONE *zone_ptr); static void overlap_zones(ZONE *zone_array,int *array_size); /*********************************************** * function: int zone(int coarseness) * The process steps are: * 1) Sample the page * 2) Cut vertical lines from the page * 3) Block out zones via cellular automata * 4) Extract the zones * 5) Sequence zones * parameters: coarseness value 0-5, order (COLUMN or ROW) * returns: 1 if OK or 0 if error, see errno *********************************************/ int zone(int coarseness,int order) { unsigned char *image; int dx,dy; int samplex,sampley; int i; if (coarseness < 0 «« coarseness > 5) { /* test coarseness parameter */ errno = EINVAL; return 0; } /* get a scaled copy of the page */ image = sample_page(&dx,&dy,&samplex,&sampley); if (image == NULL) /* memory? */ return 0; #if CUT_VERTICAL_LINES cut_vertical_lines(image,dx,dy); /* remove boxes around text */ #endif block_zones(image,dx,dy,coarseness); /* block out zones */ sequence_zones(image,dx,dy,order); for (i = 0 ; i < zone_count ; i++) { /* translate to full page */ zone_list[i].x1 *= samplex; zone_list[i].y1 *= sampley; zone_list[i].x2 *= samplex; zone_list[i].y2 *= sampley; } free(image); /* clean up */ return 1; } /**************** LOCAL FUNCTIONS ****************/ /************************************************ * function: static unsigned char *sample_page(int *dx,int *dy,int *samplex,int * sampley) * Sample the page. Normally, the entire page is stored in memory. Since the * memory requirements are typically a megabyte, the page, in DOS machines, * is somewhere in extended memory. So that this demo can work on machines * lacking extended memory, I sampled the page when I scaled it for display. * See display_sample() below. However, I have #ifed around the code that is * normally used to sample the page from the memory image. You need to provide * two functions, as well as the extended memory handler. They are void * memory_info(DISPLAY_BOX *); which gets the x/y extents of the image, and * (unsigned char * memory_ptr(int y); which returns a pointer to a scan line * in the memory image. Sample_page() creates a standardized, reduced image * suitable for cellular automation. The sample has each byte, not bit, * representing a pixel area. The byte is 0xff if black or 0x00 if white. The * sampling procedure is important for region finding. If possible it should be * a function of the density of the original image. If the image isn't square, * for example a 200x100 fax file, then the x and y sampling should be adjusted * accordingly. Since I don'thave dpi information here, I am scaling it to a * typical, 200dpi, value after it was adjusted for screen display. * Note: Bit 7 is leftmost and 0 is rightmost; 1 bits are black, 0 are white. * parameters: pointers to storage for the byte width and height of the sampled * image and the sample bit distance in x and y * returns: pointer to sampled page or NULL if no memory **************************************************/ static unsigned char *sample_page(int *dx,int *dy,int *samplex,int *sampley) { static unsigned char bit_mask[] = {0x80,0x40,0x20,0x10,0x08,0x04,0x02,0x01}; unsigned char *image,*line_ptr,*line_ptr2,*buff_ptr; DISPLAY_BOX file; unsigned int x,y,width,height; memory_info(&file); /* need to provide this, gets file dimensions */ /* from memory image of file */ *samplex = SAMPLE_200_X; /* sample sizes */ *sampley = SAMPLE_200_Y; *dx = file.width / *samplex; /* extent of sample */ *dy = file.height / *sampley; while (((long)*dx * (long)*dy) > MAX_MALLOC) { /* adjust sampling to fit memory restrictions */ (*samplex)++; (*sampley)++; *dx = file.width / *samplex; *dy = file.height / *sampley; } if ((image = malloc(*dx * *dy)) == NULL) /* allocate sample buffer */ return NULL; memset(image,WHITE,*dx * *dy); /* set to white */ width =*dx * *samplex; height = *dy * *sampley; if (*samplex >= 8) { /* byte sampling */ for (y = 0, buff_ptr = image ; y < height ; y += *sampley) { /* for each y sample */ /* need to provide memory_ptr which gets a pointer * to a scan line from the memory image of the file */ line_ptr = memory_ptr(y); line_ptr2 = memory_ptr(y + *sampley/2); /* double sample in y */ for (x = 0 ; x < width ; x += *samplex, buff_ptr++) if (*(line_ptr+(x>>3)) « *(line_ptr2+(x>>3))) *buff_ptr = BLACK; /* if byte has black, set sample */ } } else { /* bit sampling */ for (y = 0, buff_ptr = image ; y < height ; y += *sampley) { /* for each y sample */ /* need to provide memory_ptr which gets a pointer * to a scan line from the memory image of the file */ line_ptr = memory_ptr(y); line_ptr2 = memory_ptr(y + *sampley/2); /* double sample in y */ for (x = 0 ; x < width ; x += *samplex, buff_ptr++) if ((*(line_ptr+(x>>3)) « *(line_ptr2+(x>>3))) & bit_mask[x&7]) *buff_ptr = BLACK; /* if bit is black, set sample */ } } return image; } #if CUT_VERTICAL_LINES /*********************************************** * function: static void cut_vertical_lines(unsigned char *image,int dx,int dy) * Remove vertical lines from sample. The purpose of this function is to * unbox text. Removing the vertical box lines accomplishes this. Trying * to remove the horizontal lines is dangerous because you might also remove * the text. * parameters: pointer to sampled image buffer and x/y extents of buffer * returns: nothing *************************************************/ static void cut_vertical_lines(unsigned char *image,int dx,int dy) { int x,y,count,y1; unsigned char *ptr,*qtr; for (x = 0 ; x < dx ; x++) /* scan image left to right */ { ptr = image+x; count = 0; for (y = 0 ; y < dy ; y++, ptr += dx) { /* scan up and down counting line pixels */ if (*ptr) count++; else count = 0; if (count >= VERTICAL_LINE_SIZE) { /* we have a veritcal line */ for (y1=y, qtr=ptr ; *ptr!=WHITE && y>=0 ; y--, ptr-=dx) *ptr = WHITE; /* white out moving up */ for (y=y1+1, ptr=qtr+dx ; *ptr!=WHITE && y= LIFE_SCORE) *pixel = PREGNANT; else if (*pixel == BLACK && score <= DEATH_SCORE) *pixel = SICK; } } for (pixel = image ; pixel < image + dy * dx ; pixel++) { /* birth and bury */ if (*pixel == PREGNANT) *pixel = BLACK; else if (*pixel == SICK) *pixel = WHITE; } if (coarseness <= 0) /* no need to coalesce */ return; /* coalesce regions, based on coarseness, i.e. coarseness == 2 * will close all horizontal/vertical gaps of 2 pixels */ for (y=0, line=image+1 ; y < dy ; y++, line+=dx) { /* coalesce horizontally */ for (x=1, pixel=line ; x < dx-coarseness ; x++, pixel++) { if (*pixel == WHITE && *(pixel-1) == BLACK) { for (i = 1 ; i <= coarseness ; i++) { if (*(pixel+i)) { for (i = 0 ; i < coarseness ; i++, pixel++) *pixel = BLACK; pixel-; x += coarseness-1; break; } } } } } for (x=0, line=image+dx ; x < dx ; x++, line++) { /* coalesce vertically */ for (y=1, pixel=line ; y < dy-coarseness ; y++, pixel+=dx) { if (*pixel == WHITE && *(pixel-dx) == BLACK) { for (i=1, j=dx ; i <= coarseness ; i++, j+=dx) { if (*(pixel+j)) { for (i = 0 ; i < coarseness ; i++, pixel+=dx) *pixel = BLACK; pixel -= dx; y += (coarseness-1); break; } } } } } } /*************************************************** * function: static void sequence_zones(unsigned char *image,int dx,int dy,int order) * Extract zones from buffer, place in zone_list, update zone_count and * sequence zones so they are in either COLUMN_MAJOR or ROW_MAJOR reading order. * parameters: pointer to image buffer and x/y extents of image in buffer * order, COLUMN_MAJOR or ROW_MAJOR * returns: nothing ***********************************************************/ static void sequence_zones(unsigned char *image,int dx,int dy,int order) { int x,y,i,j,index,fudge_x,fudge_y; unsigned char *ptr; ZONE temp; for (y=0, zone_count=0 ; y < dy && zone_count < MAX_ZONES ; y+=MIN_Y_ZONE) { /* extract zones from block images in y order */ ptr = image + y * dx; for (x = 0 ; x < dx ; x += MIN_X_ZONE) if (*(ptr+x)) /* found point */ { if (zone_count >= MAX_ZONES) break; while (x > 0 && *(ptr+x-1)) /* back up to left side */ x--; if (extract_zone (image,dx,dy,x,y,zone_list+zone_count)) zone_count++; /* get zone */ } } if (zone_count == 0) { /* if no zones, make the entire page one big zone */ zone_list[0].x1 = zone_list[0].y1-0; zone_list[0].x2 = dx-1; zone-list[0].y2 = dy-1; zone_count = 1; return; } overlap_zones(zone_list,&zone_count); /* remove overlap */ if (order == COLUMN_MAJOR) { for (i = 0; i < zone_count-1 ; i++) { /* sort on x1 */ for (j = i+1 ; j < zone_count ; j++) { if (zone_list[j].x1 < zone_list[i].x1) { temp = zone_list[i]; zone_list[i] = zone_list[j]; zone_list[j] = temp; } } } for (i = 0; i < zone_count-1 ; i++) { /* order zones left to right, up to down */ x = zone_list[i].x2; y = zone_list[i].y1; fudge_x = zone_list[i].x1+dx/20; /* 5% slippage on alignment */ for (j = i+1, index = -1 ; j < zone_count ; j++) { /* find any zones above and within x extent of zone_list[i] */ if (zone_list[j].x1 > x) break; if (zone_list[j].y1 < y && zone_list[j].x1 <= fudge_x) { x = zone_list[j].x2; y = zone_list[j].y1; index = j; } } if (index != -1) { temp = zone_list[i]; zone_list[i] = zone_list[index]; zone_list[index] = temp; } } } else /* ROW_MAJOR */ { /* already sorted in y1 order */ for (i = 0 ; i < zone count-1 ; i++) { /* order zones up to down, left to right */ y = zone_list[i].y2; x = zone_list[i].x1; fudge_y = zone_list[i].y1+dy/20; /* 5% slippage on alignment */ for (j = i+1, index = -1 ; j < zone_count ; j++) { /* find any zones left of and within y extent of zone_list[i] */ if (zone_list[j].y1 > y) break; if (zone_list[j].x1 < x && zone_list[j].y1 <= fudge_y) { y = zone_list[j].y2; x = zone_list[j].x1; index = j; } } if (index != -1) { temp = zone_list[i]; zone_list[i] = zone_list[index]; zone_list[index] = temp; } } } } /************************************************ * function: static int extract_zone(unsigned char *image,int dx,int dy,int * x,int y,ZONE *zone_ptr) * Extracts a rectangular region from the buffer starting at a point. It does * this by walking the perimeter, recording the min and max x/y dimensions. * If the region is big enough to be significant, it is saved as a zone and * then whited out so it won't be reconsidered. * parameters: pointer to image buffer and dx/dy extents of that buffer. * x & y point to start extracting the zone. * pointer to storage for zone * returns: 1 if zone OK or 0 if not ************************************************/ static int extract_zone(unsigned char *image,int dx,int dy,int x,int y,ZONE *zone_ptr) { int ix,iy,minx,miny,maxx,maxy; /* perimeter variables & min/max values */ HEADING dir; /* current direction */ unsigned char *previous,*next,*here; /* buffer pointers */ minx = maxx = ix = x; /* preset min/max x/y and perimeter vars */ miny = maxy = iy = y; dir = GOING_UP; /* starting direction */ do /* walk perimeter, recording min/max of rectangular region */ { if (ix < minx) /* update min/max */ minx = ix; if (ix > maxx) maxx = ix; if (iy < miny) miny = iy; if (iy > maxy) maxy = iy; here = image + iy * dx + ix; /* where are we? */ next = here + dx; previous = here - dx; switch (dir) /* based on current direction, */ { /* look around for next direction */ case GOING_UP: if (ix > 0 && *(here-1)) { ix-; dir = GOING_LEFT; break; } if (iy > 0 && *previous) { iy; break; } if (ix < dx-1 && *(here+1)) { ix++; dir = GOING_RIGHT; break; } if (iy < dy-1 && *next) { iy++; dir = GOING_DOWN; break; } break; case GOING_RIGHT: if (iy > 0 && *previous) { iy--; dir = GOING_UP; break; } if (ix < dx-1 && *(here+1)) { ix++; break; } if (iy < dy-1 && *next) { iy++; dir = GOING_DOWN; break; } if (ix > 0 && *(here-1)) { ix--; dir = GOING_LEFT; break; } break; case GOING_DOWN: if (ix < dx-1 && *(here+1)) { ix++; dir = GOING_RIGHT; break; } if (iy < dy-1 && *next) { iy++; break; } if (ix > 0 && *(here-1)) { ix--; dir = GOING_LEFT; break; } if (iy > 0 && *previous) { iy--; dir = GOING_UP; break; } break; case GOING_LEFT: if (iy < dy-1 && *next) { iy++; dir =- GOING_DOWN; break; } if (ix > 0 && *(here-1)) { ix--; break; } if (iy > 0 && *previous) } iy--; dir = GOING_UP; break; } if {ix < dx-1 && *(here+1)) { ix++; dir = GOING_RIGHT; break; } break; } } while (ix != «« iy != y); /* until we return to the start */ for (iy=miny, here=image+miny*dx+minx ; iy <= maxy ; iy++, here+=dx) memset(here,WHITE,maxx-minx+1); /* whiteout the region */ if ((maxx-minx+1 < MIN_X_ZONE) «« (maxy-miny+1 < MIN_Y_ZONE)) return 0; /* big enough? */ if {minx > 0) /* expand dimensions by one pixel */ minx--; if (maxx < dx-1) maxx++; if (miny > 0) miny--; if (maxy < dy-1) maxy++; zone_ptr->x1 = minx; /* save zone */ zone_ptr->y1 = miny; zone_ptr->x2 = maxx; zone_ptr->y2 = maxy; return 1; } /************************************************ * function: static void overlap_zones{ZONE *zone_array,int *array_size) * find zones that overlap and combine them * parameters: pointer to array of zones and pointer to count of zones * returns: nothing ************************************************/ static void overlap_zones(ZONE *zone_array,int *array_size) { in* i,j,ax1,ay1,ax2,ay2,bx1,by1,bx2,by2; for (i = 0 ; i < *array_size-1 ; i++) { /* compare each zone against the rest */ ax1 = (zone_array+i)->x1; ay1 = (zone_array+i)->y1; ax2 = (zone_array+i)->x2; ay2 = (zone_array+i)->y2; for (j = i+1 ; j < *array_size ; j++) { bx1 = (zone_array+j)->x1; by1 = (zone_array+j)->y1; bx2 = (zone_array+j)->x2; by2 = (zone_array+j)->y2; if ((bx1>=ax2) «« (ax1>=bx2) «« (by1>=ay2) «« (ay1>=by2)) continue; /* no overlap */ bx1 = min(bx1,ax1); /* combine zones */ by1 = min(by1,ay1); bx2 = max(bx2,ax2); by2 = max(by2,ay2); (zone_array+i)->x1 = bx1; (zone_array+i)->y1 = by1; (zone_array+i)->x2 = bx2; (zone_array+i)->y2 = by2; (*array_size)--; /* new zone count */ memmove((char *)(zone_array+j),(char *)(zone_array+j+1), sizeof(ZONE)*(*array_size-j)); /* shift array to remove zone */ i = -1; /* start all over */ break; } } } /* End of File */ C Image Processing, Part 11 Manipulating Shapes Dwayne Phillips The author works as a computer and electronics engineer with the U.S. Department of Defense. He has a PhD in Electrical and Computer Engineering from Louisiana State University. His interest include computer vision, artificial intelligence, software engineering and programming languages. Introduction to the Series of Articles This is the eleventh in a series of articles on images and image processing. Previous articles discussed reading, writing, displaying, and printing images (TIFF format), histograms, edge detection, spatial frequency filtering, sundry image operations, and image segmentation. (See box called "The C Image Processing Series...so far" on page 91.) This article will discuss manipulating shapes. The last two articles discussed image segmentation. Segmentation took an image and produced a binary output showing the objects of interest. This article will describe several techniques for taking those objects and improving their appearance. Working with Shapes You process images, such as segmentation results, to improve their appearance (Phillips February 1993, June 1993). Photograph 1 shows an aerial image, and Photograph 2 a segmentation of it. Photograph 3 shows a house, and Photograph 4 a segmentation of it. These are good segmentations, but they have problems. Segmentation results have "holes" in them. The roof in Photograph 4 should be solid, but has holes. Larger holes can even break objects. The opposite can also be true, as segmentation can join separate objects. Segmentation results often have little or no meaning. The solid objects resemble blobs and are hard to interpret--especially if you want to use a computer to interpret them. The answer to these problems is morphological filtering, or manipulating shapes. Useful techniques include: erosion and dilation opening and closing outlining thinning and skeletonization These techniques work on binary images where the object equals a value and the background is zero. Figure 1 shows a binary image with the background equal to 0 and the object equal to 200. All of the figures in the article will use the same format. Erosion and Dilation The erosion and dilation operations make objects smaller and larger. These operations are valuable in themselves and are the foundation for the opening and closing operations. Erosion, discussed briefly in "Image Segmentation Using Edges and Gray Shades" (Phillips June 1993), makes an object smaller by removing or eroding away the pixels on its edges. Figure 2 shows the result of eroding the rectangle in Figure 1. Dilation makes an object larger by adding pixels around its edges. Figure 3 shows the result of dilating the rectangle in Figure 1. I set any zero pixel that was next to a 200 pixel (shown as asterisks). There are two general techniques for erosion and dilation. The technique introduced in the last article (Phillips June 1993) employs a threshold (Russ 1992). Another technique (Myler and Weeks 1993) uses masks to erode and dilate in desired directions. The threshold technique looks at the neighbors of a pixel and changes its state if the number of differing neighbors exceeds a threshold. Listing 1 shows the erosion and dilation routines that use this method. The loops in the erosion routine examine every pixel in the_image equal to value. They count the number of zero neighbors and set the pixel in question to zero if the count exceeds the threshold parameter. Figure 2 used a threshold parameter of 3. Compare this to Figure 4 (threshold = 2). The loops in the dilation routine do the opposite. They count the value pixels next to a zero pixel. If the count exceeds the threshold parameter, set the zero pixel to value. The dilation in Figure 3 used threshold = 0 while Figure 5 used threshold = 2. The masking technique (Myler and Weeks 1993) lays an nxn (3x3, 5x5, etc.) array of 1s and 0s on top of an input image and erodes or dilates the input. With masks you can control the direction of erosion or dilation. Figure 6 shows four 3x3 masks (you could use 5x5, 7x7, etc. masks). The first two masks modify the input image in the vertical or horizontal directions while the second two perform in both directions. Figure 7 shows the results of dilating the object of Figure 1 using the four masks of Figure 6. The procedure is: 1. Place the 3x3 mask on the object so that the center of the 3x3 array lies on the edge of the object. 2. Place a 200 everywhere a 1 from the mask lies. The object in the first part of Figure 7 has been dilated, or stretched, vertically. The second result is a horizontal dilation. The third and fourth show dilation in both directions. These last two differ in dilating the corners of the object. Mask erosion is the same, but opposite. You lay the 3x3 mask on the image so that the center of the array is on top of a zero. If any of the 1s in the mask overlap a 200 in the image, set the 200 to zero. Vertical mask erosion removes the top and bottom rows from an object. Horizontal mask erosion removes the left and right columns, and the other masks remove pixels from all edges. Listing 2 shows the routines for mask erosion and dilation. mask_dilation copies the correct directional mask specified by the mask_type parameter and goes into the looping code. The loop moves through the input image and lays the 3x3 mask on top of every pixel in the image. The inner loops examine those places where the 3x3 mask equals 1. If the_image is greater than 1 (non-zero) at that place, set max to the input image value. After exiting the loop, set the out_image to max. This changes zero pixels to value and enlarges or dilates an object in the_image. The mask_erosion routine performs the opposite function. Its inner loops look at those places where the 3x3 mask is 1 and try to find pixels in the_image that are less than min (that are zero). If there are any zeros in this part of the_image, set out_image to zero. This removes value pixels, makes them zeros, and erodes an object in the_image. Photograph 5 illustrates directional dilation. The left section shows the segmentation of the house image. The center section shows dilating with a vertical mask, and the right section shows dilating with a horizontal mask. Opening and Closing opening and closing help separate and join objects. They are powerful operators that are simple combinations of erosion and dilation. opening spaces objects that are too close together, detaches objects that are touching and should not be, and enlarges holes inside objects. The first part of Figure 8 shows two objects joined by a "thread." The second part shows how opening eliminated the thread and separated the two objects. The rest of the figure shows how opening enlarges a desired hole in an object. Opening involves one or more erosions followed by one dilation. Eroding the object of Figure 8 twice erases the thread. A dilation enlarges the two objects back to their original size, but does not recreate the thread. The left side of Photograph 6 is a segmentation of the house image (Phillips June 1993). The right side is the result of opening (three erosions followed by one dilation). Although excessive, it shows how opening spaces the major objects. closing joins broken objects and fills in unwanted holes in objects. The first part of Figure 9 shows two objects that should be joined to make a line. The second part shows how closing removes the break in the line. The last two parts of Figure 9 show how closing fills in unwanted holes in objects. Closing involves one or more dilations followed by one erosion. Dilating the top part of Figure 9 twice enlarges the two objects until they join. An erosion thins them back to their original width. Dilating the third part of Figure 9 twice makes the box bigger and eliminates the hole. Eroding shrinks the box back to its initial size. Listing 3 shows the routines that perform opening and closing. They call the mask erosion and dilation routines, but calling the threshold erosion and dilation routines would work just as well (homework for the reader). opening calls mask_ dilation one or more times and mask_erosion once. closing calls mask_erosion one or more times and mask_dilation once. These are simple, yet powerful routines. Special Opening and Closing The opening and closing operators work well, but sometimes produce undesired side effects. closing merges objects, but sometimes merges objects that it shouldn't. Figure 10 shows such a case. Photograph 7 shows the result of closing applied to Photograph 2. closing closed the holes in the objects, but also joined distinct objects. This distorted the segmentation results. opening enlarges holes in objects, but can break an object. Figure 11 shows a case where opening broke an object and eliminated half of it. The answer is special opening and closing routines that avoid these problems. Figure 12 shows the desired result of such special routines that open and close objects, but do not join or break them. The first difficulty to overcome is what I call the 2-wide problem. In opening, you erode an object more than once, and an object that is an even number of pixels wide can disappear. The first part of Figure 13 shows a 2-wide object. The second part shows the object after one erosion, and the third part shows it after two erosions. The object will disappear entirely after several more erosions. A solution to the 2-wide problem is my own variation of the grass fire wavefront technique (Levine 1985). My technique scans across the image from left to right looking for a 0 to value transition. When it finds one, it examines the value pixel to determine if removing it will break an object. If removal does not break the object, it sets the pixel to 0 and continues scanning. Next, it scans the image from right to left and does the same operation. Then it scans from top to bottom, and finally from bottom to top. The different scans will not erode away an object that is 2-wide. The key to special opening is not breaking the object. One solution places the pixel in question in the center of a 3x3 array. Find every value pixel next to the center pixel. Do all of those pixels have value neighbors other than the center pixel? If yes, then erode or remove the center pixel. If no, then removing the center pixel will break the object. The top part of Figure 14 shows cases where removing the center pixel will break the object. The bottom part shows cases where removing the center pixel will not break the object. Here, every 200 has a 200 neighbor other than the center pixel. A similar problem in special closing is not joining two separate objects when you dilate or set a pixel. One solution is to place the pixel in question in the center of a 3x3 array. You grow objects in this array and check if the center pixel has neighbors whose values differ (Phillips February 1993). If their values differ, do not set the center pixel because this will join different objects. The top part of Figure 15 shows 3x3 arrays and the results of growing objects. The center pixel has neighbors that are parts of different objects, so you do not set the center pixel. The bottom part of Figure 15 shows another set of 3x3 arrays. Here, the non-zero neighbors of the center pixel all have the same value, and we can set the center pixel. The source code to implement special opening and closing, shown in Listing 4, is basic but long. The special_opening routine calls thinning (instead of erosion) one or more times before calling dilation once. thinning works around the 2-wide problem while performing basic threshold erosion. thinning has four sections--one each for scan (left to right, right to left, top to bottom, and bottom to top) recounted earlier. Whenever thinning finds a pixel to remove, it calls can_thin to prevent breaking an object. can_thin checks the non-zero neighbors of the center pixel. If every non-zero pixel has a non-zero neighbor besides the center pixel, can_thin returns a 1, else it returns a zero. The special_closing routine calls dilate_not_join one or more times before calling erosion once. dilate_not_join uses the basic threshold technique for dilation and calls can_dilate to prevent joining two separate objects. can_dilate grows objects in a 3x3 array and checks if the center pixel has neighbors with different values. If it does, the neighbors belong to different objects, so it returns a zero. can_dilate grows objects like the routines in the previous two C Image Processing installments (Phillips February 1993, June 1993). can_dilate calls little_label_and_check which resembles routines described in those two articles. Photograph 8 shows the result of special closing. Compare this with Photograph 2 and Photograph 7. Photograph 2, the original segmentation, is full of holes. Photograph 7 closed these holes, but joined objects and ruined the segmentation result. Photograph 8 closes the holes and keeps the segmentation result correct by not joining the objects. Photograph 9 and Photograph 10 show how to put everything together to improve segmentation results. Photograph 9 shows the outcome of eroding the segmentation result of Photograph 4. Applying special closing to Photograph 9 produces Photograph 10. Compare Photograph 4 and Photograph 10. Photograph 10 has all the major objects cleanly separated without holes. Outlining Outlining is a type of edge detection. It only works for zero-value images, but produces better results than regular edge detectors. Photograph 11 shows the exterior outline of the objects in Photograph 4. Outlining helps you understand an object. Figure 16 and Figure 17 show the interior and exterior outlines of objects. Outlining zero-value images is quick and easy if you have erosion and dilation at your disposal. To outline the interior of an object, you erode the object and subtract the eroded image from the original. To outline the exterior of an object, you dilate the object and subtract the original image from the dilated image. Exterior outlining is easiest to understand. Dilating an object makes it one layer of pixels larger. Subtracting the input from this dilated, larger object yields the outline. Listing 5 shows the source code for the interior and exterior outline operators. The functions call the mask_erosion a n d mask_dilation routines. They could have called the threshold erosion and dilation routines (homework for the reader). The interior_outline routine erodes the input image and subtracts the eroded image from the original. The exterior_outline routine dilates the input image and subtracts the input image from the dilated image. Thinning and Skeletonization Thinning is a data reduction process that erodes an object until it is one pixel wide, producing a skeleton of the object. It is easier to recognize objects such as letters or silhouettes by looking at their bare bones. Figure 18 shows how thinning a rectangle produces a line of pixels. There are two basic techniques for producing the skeleton of an object: basic thinning medial axis transforms Thinning erodes an object over and over again (without breaking it) until it is one pixel wide. Listing 4 contains the thinning routine. I used it as part of the special_opening routine to erode objects without breaking them. In that context, I set the once_only parameter of thinning to one, so that it would erode an image only one time. Setting once_only to zero causes thinning to keep eroding until the objects in the image are all one pixel wide. This basic thinning technique works well, but you cannot recreate the original object from the result of thinning. If you want to do that, you must use the medial axis transform. The medial axis transform finds the points in an object that form lines down its center, that is, its medial axis. It is easier to understand the medial axis transform if you first understand the Euclidean distance measure (don't you love these big terms that really mean very simple things?). The Euclidean distance measure is the shortest distance from a pixel in an object to the edge of the object. Figure 19 shows a square, its Euclidian distance measure (distance to the edge), and its medial axis transform. The medial axis transform consists of all points in an object that are minimally distant to more than one edge of the object. Every pixel in the bottom of Figure 19 was the shortest distance to two edges of the object. The advantage of the medial axis transform is you can recreate the original object from the transform (more homework). Figure 20 shows a rectangle (from Figure 18) and its medial axis transform. Photograph 12 shows a block letter A, and going clockwise, the result of exterior outline, medial axis transform, and thinning. Listing 6 shows the source code to implement the Euclidean distance measure and the medial axis transform. edm calculates the Euclidean distance measure. It loops through the image and calls distance_8 to do most of the work. distance_8 has eight sections to calculate eight distances from any value pixel to the nearest zero pixel. distance_8 returns the shortest distance it found. The functions mat and mat_d calculate the medial axis transform in a similar manner. mat loops through the image and calls mat_d to do the work. mat_d calculates the eight distances and records the two shortest distances. If these two are equal, the pixel in question is minimally distant to two edges, is part of the medial axis transform, and causes mat_d to return value. Integrating Shape Operations into the C Image Processing System Listing 7 shows the new additions to the main routine of the C Image Processing System. Case 19 handles the 14 shape operations discussed herein. The show_menu routine underwent a big change from past installments. We now have 19 options, so I used a two column display for them. The final routine, get_shape_options, interacts with the user to obtain all the options needed to use the techniques. Listing 8 shows an application program that allows you to apply the shape operations to entire image files. This program created the images shown in the photographs of this article. Conclusions This installment in the series discussed shape operations or morphological filters. These techniques help you improve the appearance of segmentation results. They are also useful for other situations. As with all the image processing operators in this series, you must experiment. Try the techniques and tools in different combinations until you find what works for the image or class of images at hand. References Levine, Martin D. 1985 Vision in Man and Machine. New York, New York: McGraw-Hill. Myler, Harley R. and Arthur R. Weeks. 1993. Computer Imaging Recipes in C. Englewood Cliffs, New Jersey: Prentice Hall Publishing. Phillips, Dwayne. February 1993. "Image Processing, Part 9: Histogram-Based Image Segmentation," The C Users Journal. Lawrence, KS: R&D Publications. Phillips, Dwayne. June 1993. "Image Processing, Part 10: Segmentation Using Edges and Gray Shades," The C Users Journal. Lawrence, KS: R&D Publications. Russ, John C. 1992. The Image Processing Handbook. Boca Raton, Florida: CRC Press. The C Image Processing System ...so far Part # Title CUJ Issue Part 1 Reading the Tagged Image File Format March 1991 Part 2 Displaying Images and Printing Numbers May 1991 Part 3 Displaying and Printing Images June 1991 Using Halftoning Part 4 Histograms and Histogram Equalization August 1991 Part 5 Writing Images to Files November 1991 and Basic Edge Detection Part 6 Advanced Edge Detection January 1992 Part 7 Spatial Frequency Filtering October 1992 Part 8 Image Operations November 1992 Part 9 Histogram-Based Image Segmentation February 1993 Part 10 Image Segmentation June 1993 Using Region Growing and Edges Figure 1 A binary image with the background equal to 0 and the object equal to 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 2 The result of eroding the rectangle in Figure 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 3 The result of dilating the rectangle in Figure 1. 0 0 0 0 0 0 0 0 0 0 0 *** *** *** *** *** *** *** *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** *** *** *** *** *** *** *** 0 0 0 0 0 0 0 0 0 0 0 Figure 4 The result of eroding the rectangle in Figure 1 using a threshold of 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 5 The result of dilating rectangle in Figure 1 using a threshold of 2 0 0 0 0 0 0 0 0 0 0 0 0 0 *** *** *** *** 0 0 0 0 0 200 200 200 200 200 200 0 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 0 200 200 200 200 200 200 0 0 0 0 0 *** *** *** *** 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 6 Four 3x3 masks vertical mask horizontal mask 0 1 0 0 0 0 0 1 0 1 1 1 0 1 0 0 0 0 horizontal mask and vertical mask 0 1 0 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 Figure 7 The results of dilating the rectangle in Figure 1 using the masks in Figure 6 Vertical Dilation Only 0 0 0 0 0 0 0 0 0 0 0 0 *** *** *** *** *** *** 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 *** *** *** *** *** *** 0 0 0 0 0 0 0 0 0 0 0 0 Horizontal Dilation Only 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Dilation in Both Directions 0 0 0 0 0 0 0 0 0 0 0 0 *** *** *** *** *** *** 0 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 0 *** *** *** *** *** *** 0 0 0 0 0 0 0 0 0 0 0 0 Dilation in Both Directions 0 0 0 0 0 0 0 0 0 0 0 *** *** *** *** *** *** *** *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 0 *** 200 200 200 200 200 200 *** 0 Figure 8 Two objects joined by a thread, separated by opening, and a hole enlarged by opening Two objects joined by a thread 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 Opening separates them 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 0 0 200 200 200 200 An object with a single small hole in it 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 Opening enlarges the hole 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 200 200 200 200 200 200 200 0 0 0 200 200 200 200 200 200 200 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 Figure 9 Two objects that should be joined, how closing removes the break and fills unwanted holes A broken line 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 0 200 200 200 200 200 200 200 200 200 0 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Closing joins the broken line 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 An object with a hole in it 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Closing fills the hole 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 10 An unwanted merging of two objects Two separate objects 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Closing joins the objects (unwanted) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 11 opening broke this object and eliminated half of it. Object with a hole in it 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Opening removes the hole but breaks the object 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 12 Result of special routines that open and close objects but do not join or break them Special closing does not join objects 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 200 200 0 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 special opening does not break object 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 0 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 13 Result of opening of a 2-wide object A 2-wide object 0 0 200 200 0 0 0 0 200 200 0 0 The object after one erosion 0 0 0 200 0 0 0 0 200 200 0 0 The object after two erosions 0 0 0 0 0 0 0 0 200 200 0 0 Figure 14 Cases that do and do not require a special opening routine Cases where you cannot erode center pixel 0 200 0 200 0 200 200 200 0 0 200 0 0 200 0 0 200 0 0 200 0 0 0 0 0 0 200 Cases where you can erode the center pixel 200 200 200 200 0 0 0 200 0 0 200 0 200 200 0 200 200 0 0 0 0 0 0 0 0 0 0 Figure 15 Cases that do and do not require a special closing routine Cases where you cannot dilate the center pixel 200 0 200 200 0 0 0 200 0 200 0 200 0 0 0 200 0 0 200 0 200 200 0 0 0 0 200 1 0 2 1 0 0 0 1 0 1 0 2 0 0 0 1 0 0 1 0 2 1 0 0 0 0 2 Cases where you can dilate the center pixel 200 200 200 0 200 0 0 200 200 200 0 0 200 0 0 200 0 200 200 200 200 0 0 0 0 0 200 1 1 1 0 1 0 0 1 1 1 0 0 1 0 0 1 0 1 1 1 1 0 0 0 0 0 1 Figure 16 The interior outline of an object 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 0 0 0 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 Figure 17 The exterior outline of an object 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 0 0 0 0 0 0 200 0 0 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 Figure 18 Thinning a rectangle until it is one pixel wide 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 19 A square, its Euclidian distance measure, and its medial axis transform 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 1 2 2 2 2 2 2 2 2 2 2 2 1 0 0 0 0 1 2 3 3 3 3 3 3 3 3 3 2 1 0 0 0 0 1 2 3 4 4 4 4 4 4 4 3 2 1 0 0 0 0 1 2 3 4 5 5 5 5 5 4 3 2 1 0 0 0 0 1 2 3 4 5 6 6 6 5 4 3 2 1 0 0 0 0 1 2 3 4 5 6 7 6 5 4 3 2 1 0 0 0 0 1 2 3 4 5 6 6 6 5 4 3 2 1 0 0 0 0 1 2 3 4 5 5 5 5 5 4 3 2 1 0 0 0 0 1 2 3 4 4 4 4 4 4 4 3 2 1 0 0 0 0 1 2 3 3 3 3 3 3 3 3 3 2 1 0 0 0 0 1 2 2 2 2 2 2 2 2 2 2 2 1 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Figure 20 A rectangle and its medial axis transform 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 200 200 200 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Photo 1 Aerial Image Photo 2 Segmentation of Aerial Image Photo 3 House Image Photo 4 Segmentation of House Image Photo 5 Examples of Masked Vertical and Horizontal Dilations Photo 6 A Segmentation and the Result of Opening Photo 7 Closing of Segmentation in Photograph 2 Photo 8 Special Closing of Segmentation of Photograph 2 Photo 9 Erosion of Segmentation in Photograph 4 Photo 10 Special Closing of Photograph 9 Photo 11 Outline of Segmentation in Photograph 4 Photo 12 (Clockwise from upper left) Block 'A', its outline, Medial Axis Transform, and Thinning Listing 1 Erosion and dilation routines using the threshold technique /***************************************************** * erosion(... * * This function performs the erosion operation. if * a value pixel has more than the threshold number * of 0 neighbors, you erode it by setting it to 0. ******************************************************/ erosion(in_name, out_name, the_image, out_image, il, ie, ll, value, threshold) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf{"n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header}; round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /*************************** * Loop over image array ***************************/ for(i-0; i threshold) out_image[i][j] = 0; } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ fix_edges(out_image, 3); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends erosion */ /******************************************************* * dilation(... * * This function performs the dilation operation. If * a 0 pixel has more than threshold number of value * neighbors, you dilate it by setting it to value. *******************************************************/ dilation(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; if(does_not_exist(outname)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /************************ * Loop over image array ************************/ printf("\n"); for(i=1; i threshold) out_image[i][j] = value; } /* ends if the_image == 0 */ } /* ends loop over j */ } /* ends loop over i */ fix_edges(out_image, 3); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends dilation */ /* End of File */ Listing 2 Routines for mask erosion and dilation short edmask1[3][3] = {{0, 1, 0}, {0, 1, 0}, {0, 1, 0}}; short edmask2[3][3] = {{0, 0, 0}, {1, 1, 1}, {0, 0, 0}}; short edmask3[3][3] = {{0, 1, 0}, {1, 1, 1}, {0, 1, 0}}; short edmask4[3][3] = {{1, 1, 1}, {1, 1, 1}, {1, 1, 1}}; /************************************************ * mask_dilation(... * * This function performs the dilation operation * using the erosion-dilation 3x3 masks given * above. It works on 0-value images. *************************************************/ mask_dilation(in_name, out_name, the_image, out_image, il: ie, ll, le, value, mask_type) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], mask_type, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; short mask[3][3], max; /*************************************** * Copy the 3x3 erosion-dilation mask * specified by the mask_type. ***************************************/ switch (mask_type){ case 1; copy_3_x_3(mask, edmask1); break; case 2: copy_3_x_3(mask, edmask2); break; case 3: copy_3_x_3(mask, edmask3); break; case 4: copy_3_x_3(mask, edmask4); break; default: printf("\nInvalid mask type, using mask 4"); copy_3_x_3(mask, edmask4); break; } if (does_not_exist (out_name)){ printf("\n\n output file does not exist %s", out name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /************************** * Loop over image array ***************************/ printf("\n"); for(i=1; i max) max = the_image[i+a][j+b]; } /* ends if mask == 1 */ } /* ends loop over b */ } /* ends loop over a */ out_image[i][j] = max; } /* ends loop over j */ } /* ends loop over i */ fix_edges(out_image, 3); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends mask_dilation */ /**************************************************** * mask_erosion(... * * This function performs the erosion operation * using the erosion-dilation 3x3 masks given * above. It works on 0-value images. *****************************************************/ mask_erosion(in_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], mask_type, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; short mask[3][3],min; /**************************************** * Copy the 3x3 erosion-dilation mask * specified by the mask_type. *****************************************/ switch (mask_type){ case 1: copy_3_x_3(mask, edmask1); break; case 2: copy_3_x_3 (mask, edmask2); break; case 3: copy_3_x_3(mask, edmask3); break; case 4: copy_3_x_3(mask, edmask4); break; default: printf("\nInvalid mask type, using mask 4"); copy_3_x_3(mask, edmask4); break; } if (does_not_exist (out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); /***************************** * Loop over image array *****************************/ printf("\n"); for(i=1; i 1){ count = 1; while(count < number){ count++; mask_erosion(out_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); } /* ends while */ } /* ends if number > 1 */ mask_dilation(out_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); write_array_into_tiff_image (out_name, out_image, il, ie, ll, le); } /* ends opening */ /********************************************************* * closing(... * * Closing is dilation followed by erosion. This routine * will use the mask erosion and dilation. You could * use the other types and you could mix the two types. * * The number parameter specifies how dilations * to perform before doing one erosion. *********************************************************/ closing(in_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type, number) char in_name[], out_name[]; int il, ie, ll, le, number; short the_image[ROWS][COLS], out_image[ROWS][COLS], mask_type, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct_image_header; short mask[3][3], max; /************************************* * Copy the 3x3 erosion-dilation mask * specified by the mask_type. **************************************/ switch (mask_type){ case 1: copy_3_x_3(mask, edmask1); break; case 2: copy_3_x_3(mask, edmask2); break; case 3: copy_3_x_3(mask, edmask3); break; case 4: copy_3_x_3(mask, edmask4); break; default: printf("\nInvalid mask type, using mask 4"); copy_3_x_3(mask, edmask4); break; } if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); mask_dilation(in_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); if(number > 1){ count = 1; while(count < number){ count++; mask_dilation(out_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); } /* ends while */ } /* ends if number > 1 */ mask_erosion(out_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends closing */ /* End of File */ Listing 4 Routines that will erode without breaking an object and dilate without joining objects /******************************************* * special_opening(... * * Opening is erosion followed by dilation. * This routine will use the thinning * erosion routine. This will not allow * an object to erode to nothing. * * The number parameter specifies how * erosions to perform before doing one * dilation. *******************************************/ special_opening(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold, number) char in_name[], out_name[]; int il, ie, ll, le, number; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); thinning(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold, 1); if(number > 1){ count = 1; while(count < number){ count++; thinning(out_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold, 1); } /* ends while */ } /* ends if number > 1 */ dilation(out_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends special_opening */ /******************************************* * thinning(... * * Use a variation of the grass fire * wave front approach. * * Raster scan the image left to right * and examine and thin the left edge pixels * (a 0 to value transition). Process them * normally and "save" the result. Next, * raster scan the image right to left and * save. Raster scan top to bottom and save. * Raster scan bottom to top and save. * * That is one complete pass. * * Keep track of pixels thinned for a * pass and quit when you make a complete * pass without thinning any pixels. *******************************************/ thinning(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold, once_only) char in_name[], out_name[]; int il, ie, ll, le, once_only; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, big_count, count, i, j, k, not_finished; int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); for(i=0; i threshold){ if(can_thin(the_image, i, j, value)){ out_image[i][j] = 0; big_count++; } /* ends if can_thin */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ /************************************** * Copy the output back to the input. ***************************************/ for(i=0; i threshold){ if(can_thin(the_image, i, j, value)){ out_image[i][j] = 0; big_count++; } /* ends if can_thin */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ /************************************** * Copy the output back to the input. **************************************/ for(i=0; i threshold){ if(can_thin(the_image, i, j, value)){ out_image[i][j] = 0; big_count++; } /* ends if can_thin */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over i */ } /* ends loop over j */ /*********************************** * Copy the output back to the input. ***********************************/ for(i=0; i threshold){ if(can_thin(the_image, i, j, value)){ out_image[i][j] = 0; big_count++; } /* ends if can_thin */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over i */ } /* ends loop over j */ /************************************ * Copy the output back to the input. ************************************/ for(i=0; i= 0) && ((a+c) <= 2) && ((b+d) >= 0) && ((b+d) <= 2)){ if(temp[a+c][b+d] == value){ no_neighbor = 0; } /* ends if temp == value */ } /* ends if part of temp array */ } /* ends loop over d */ } /* ends loop over c */ temp[a][b] = value; /************************************** * If the non-zero pixel did not * have any non-zero neighbors, * no_neighbor still equals 1, * and we cannot thin, so return * zero. ***************************************/ if(no_neighbor){ return(zero); } } /* ends if temp[a][b] == value */ } /* ends loop over b */ } /* ends loop over a */ return(one); } /* ends can_thin */ /******************************************* * special_closing(... * * Closing is dilation followed by erosion. * This routine will use the dilate_not_join * dilation routine. This will not allow * two separate objects to join. * * The number parameter specifies how * dilations to perform before doing one * erosion. *******************************************/ special_closing(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold, number) char in_name[], out_name[]; int il, ie, ll, le, number; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); dilate_not_join(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold); if(number > 1){ count = 1; while(count < number){ count++; dilate_not_join(out_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold); } /* ends while */ } /* ends if number > 1 */ erosion(out_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold); write_array_into_tiff_image(out_name, out_image, il, ie, ll, le); } /* ends special_closing */ /******************************************* * dilate_not_join(... * * Use a variation of the grass fire * wave front approach. * * Raster scan the image left to right * and examine and dilate the left edge pixels * (a value to 0 transition). Process them * normally and "save" the result. Next, * raster scan the image right to left and * save. Raster scan top to bottom and save. * Raster scan bottom to top and save. * * That is one complete pass. *******************************************/ dilate_not_join(in_name, out_name, the_image, out_image, il, ie, ll, le, value, threshold) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], threshold, value; { int a, b, count, i, j, k; int length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); for(i=0; i threshold){ if(can_dilate(the_image,i,j,value)){ out_image[i][j] = value; } /* ends if can_dilate */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ /************************************** * Copy the output back to the input. **************************************/ for(i=0; i threshold){ if(can_dilate(the_image,i,j,value)){ out_image[i][j] = value; } /* ends if can_dilate */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over j */ } /* ends loop over i */ /************************************** * Copy the output back to the input. **************************************/ for(i=0; i threshold){ if(can_dilate(the_image,i,j,value)){ out_image[i][j] = value; } /* ends if can_dilate */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over i */ } /* ends loop over j */ /************************************** * Copy the output back to the input. **************************************/ for(i=0; i threshold){ if(can_dilate(the_image,i,j,value)){ out_image[i][j] = value; } /* ends if can_dilate */ } /* ends if count > threshold */ } /* ends if the_image == value */ } /* ends loop over i */ } /* ends loop over j */ /************************************** * Copy the output back to the input. ***************************************/ for(i=0; i= 0 && c <= 2 && d >= 0 && d <= 2) if(temp[c][d] == value){ /* PUSH */ *stack_pointer = *stack_pointer + 1; stack[*stack_pointer][0] = c; stack[*stack_pointer][1] = d; *stack_empty = 0; } /* ends if temp == value */ } /* ends loop over d */ } /* ends loop over c */ } /* ends little_label_and_check */ /* End of File */ Listing 5 Routines for outlining objects /***************************************************** * interior_outline(... * * This function produces the outline of any "holes" * inside an object. The method is: * output = erosion of input * final output = input - output *******************************************************/ interior_outline(in_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], mask_type, value; { int a, b, count, i, j, k; short mask[3][3], max; int length, width; struct tiff_header_struct image_header; /************************************** * Copy the 3x3 erosion-dilation mask * specified by the mask type. ***************************************/ switch(mask_type){ case 1: copy_3_x_3(mask, edmask1); break; case 2: copy_3_x_3(mask, edmask2); break; case 3: copy_3_x_3(mask, edmask3); break; case 4: copy_3_x_3(mask, edmask4); break; default: printf("\nInvalid mask type, using mask 4"); copy_3_x_3(mask, edmask4); break; } if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); mask_erosion(in_name, out_name, the_image, out_image, il, ie, ll, le, value, mask_type); for(i=0; i= 0){ if(the_image[i][j] == value) dist1++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ result = dist1; /* straight down */ measuring = 1; i = a; j = b; while(measuring){ i++; if(i <= ROWS-1){ if(the_image[i][j] == value) dist2++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist2 <= result) result = dist2; /* straight left */ measuring = 1; i = a; j = b; while(measuring){ j--; if(j >= 0){ if(the_image[i][j] == value) dist3++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist3 <= result) result = dist3; /* straight right */ measuring = 1; i = a; j = b; while(measuring){ j++; if(j <= COLS-1){ if(the_image[i][j] == value) dist4++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist4 <= result) result = dist4; /* left and up */ measuring = 1; i = a; j = b; while(measuring){ j--; i--; if(j >= 0 && i>=0){ if(the_image[i][j] == value) dist5++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist5 = (dist5*14)/10; if(dist5 <= result) result = dist5; /* right and up */ measuring = 1; i = a; j = b; while(measuring){ j++; i--; if(j <= COLS-1 && i>=0){ if(the_image[i][j] == value) dist6++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist6 = (dist6*14)/10; if(dist6 <= result) result = dist6; /* right and down */ measuring = 1; i = a; j = b; while(measuring){ j++; i++; if(j <=COLS-1 && i<=ROWS-1){ if(the_image[i][j] == value) dist7++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist7 = (dist7*14)/10; if(dist7 <= result) result = dist7; /* left and down */ measuring = 1; i = a; j = b; while(measuring){ j--; i++; if(j >=0 && i<=ROWS-1){ if(the_image[i][j] == value) dist8++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist8 = (dist8*14)/10; if(dist8 <= result) result = dist8; return(result); } /* ends distance_8 */ /******************************************* * mat(.. * * This function finds the medial axis * transform for objects in an image. * The mat are those points that are * minimally distant to more than one * boundary point. *******************************************/ mat(in_name, out_name, the_image, out_image, il, ie, ll, le, value) char in_name[], out_name[]; int il, ie, ll, le; short the_image[ROWS][COLS], out_image[ROWS][COLS], value; { int a, b, count, i, j, k, length, width; struct tiff_header_struct image_header; if(does_not_exist(out_name)){ printf("\n\n output file does not exist %s", out_name); read_tiff_header(in_name, &image_header); round_off_image_size(&image_header, &length, &width); image_header.image_length = length*ROWS; image_header.image_width = width*COLS; create_allocate_tiff_file(out_name, &image_header, out_image); } /* ends if does_not_exist */ read_tiff_image(in_name, the_image, il, ie, ll, le); for(i=0; i= 0){ if(the_image[i][j] == value) dist1++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ result = dist1; min1 = dist1; /* straight down */ measuring = 1; i = a; j = b; while(measuring){ i++; if(i <= ROWS-1){ if(the_image[i][j] == value) dist2++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist2 <= result) result = dist2; if(dist2 < min1){ min2 = min1; min1 = dist2; } else if(dist2 < min2) min2 = dist2; /* straight left */ measuring = 1; i = a; j = b; while(measuring){ j--; if(j >= 0){ if(the_image[i][j] == value) dist3++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist3 <= result) result = dist3; if(dist3 < min1){ min2 = min1; min1 = dist3; } else if(dist3 < min2) min2 = dist3; /* straight right */ measuring = 1; i = a; j = b; while(measuring){ j++; if(j <= COLS-1){ if(the_image[i][j] == value) dist4++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ if(dist4 <= result) result = dist4; if(dist4 < min1){ min2 = min1; min1 = dist4; } else if(dist4 < min2) min2 = dist4; /* left and up */ measuring = 1; i = a; j = b; while(measuring){ j--; i--; if(j >= 0 && i>=0){ if(the_image[i][j] == value) dist5++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist5 = ((dist5*14)+7)/10; if(dist5 <= result) result = dist5; if(dist5 < min1){ min2 = min1; min1 = dist5; } else if(dist5 < min2) min2 = dist5; /* right and up */ measuring = 1; i = a; j = b; while(measuring){ j++; i--; if(j <=COLS-1 && i>=0){ if(the_image[i][j] == value) dist6++; else measuring = 0; } else measuring =0; } /* ends while measuring */ dist6 = ((dist6*14)+7)/10; if(dist6 <= result) result = dist6; if(dist6 < min1){ min2 = min1; min1 = dist6; } else if(dist6 < min2) min2 = dist6; /* right and down */ measuring = 1; i = a; j = b; while(measuring){ j++; i++; if(j <= COLS-1 && i<=ROWS-1){ if(the_image[i][j] == value) dist7++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist7 = ((dist7*14)+7)/10; if(dist7 <= result) result = dist7; if(dist7 < min1){ min2 = min1; min1 = dist7; } else if(dist7 < min2) min2 = dist7; /* left and down */ measuring = 1; i = a; j = b; while(measuring){ j--; i++; if(j >=0 && i<=ROWS-1){ if(the_image[i][j] == value) dist8++; else measuring = 0; } else measuring = 0; } /* ends while measuring */ dist8 = ((dist8*14)+7)/10; if(dist8 <= result) result = dist8; if(dist8 < min1){ min2 = min1; min1 = dist8; } else if(dist8 < min2) min2 = dist8; if(min1 == min2) result = value; else result = 0; if(min1 == 0) result = 0; return(result); } /* ends mat_d */ /* End of File */ Listing 7 New routines for the C Image Processing System to implement shape operations and upgrade the show_menu routine case 19: /* shape operations */ printf("\nCIPS> Enter input image name\n"); get_image_name(name); printf("\nCIPS> Enter output image name\n"); get_image_name (name2); get_parameters(&il, &ie, &ll, &le); get shape_options(low_high, &value, &threshold, &number); if(strncmp("thi", low_high, 3) == 0){ thinning(name, name2, the_image, out_image, il, ie, ll, le, value, threshold, 0); } if(strncmp("dnj", low_high, 3) ==0){ dilate_not_join(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("ero", low_high, 3) == 0){ erosion(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("dil", low_high, 3) == 0){ dilation(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("mer", low_high, 3) == 0){ mask_erosion(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("mdi", low_high, 3) == 0){ mask_dilation(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("int", low_high, 3) == 0){ interior_outline(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("ext", low_high, 3) == 0){ exterior_outline(name, name2, the_image, out_image, il, ie, ll, le, value, threshold); } if(strncmp("ope", low_high, 3) ==0){ opening(name, name2, the_image, out_image, il, ie, ll, le, value, threshold, number); } if(strncmp("clo", low_high, 3) == 0){ closing(name, name2, the_image, out_image, il, ie, ll, le, value, threshold, number); } if(strncmp("spo", low_high, 3) == 0){ special_opening(name, name2, the image, out_image, il, ie, ll, le, value, threshold, number); } if(strncmp("spc", low_high, 3) == 0){ special_closing(name, name2, the_image, out_image, il, ie, ll, le, value, threshold, number); } if(strncmp("edm", low_high, 3) == 0){ edm(name, name2, the_image, out_image, il, ie, ll, le, value); } if(strncmp("mat", low_high, 3) == 0){ edm(name, name2, the image, out_image, il, ie, ll, le, value); } break; case 30: /* exit system */ not finished = 0; break; . . . /************************************************* * show_menu(.. * * This function displays the CIPS main menu. **************************************************/ show_menu() { printf("" "\n\n\t\t\tWelcome to CIPS" "\n\t\tThe C Image Processing System" "\n\t\tDwayne Phillips 1990-1993" "\nThese are your choices:" "\n1. Display image header 11. Image add & subtract" "\n2. Show image numbers 12. Image cut & paste" "\n3. Print image numbers 13. Image rotate & flip" "\n4. Display (EGA & VGA) 14. Image scaling" "\n5. Display or print using halftoning 15. Create blank image" "\n6. Print using dithering 16. Image thresholding" "\n7. Print or display histogram 17. Image segmentation" "\n8. Edge Detection 18. Edge & gray segmentation" "\n9. Edge Enhancement 19. Shape operations" "\n10. Image filtering" "\n" "\n30. Exit system\n" "\nEnter choice __\b\b"); } /* ends show_menu */ . . . /**************************************************** * get_shape_options(... * * This function interacts with the user to obtain * the parameters for calling the shape routines. ****************************************************/ get_shape_options(type, value, threshold, number) char type[]; int *number, *threshold; short *value; { int not_finished = 1, response; while(not_finished){ printf("\nThe shape options are:"); printf("\n\t1. Type is %s", type); printf("\n\t recall: EROsion DILation Mask-ERosion" "\n\t Mask_DIlation INTerior-outline" "\n\t EXTerior-outline THInning" "\n\t Dilate-Not-Join OPEning" "\n\t CLOsing SPecial-Opening" "\n\t SPecial-Closing" "\n\t Euclidean-Distance-Measure" "\n\t Medial-Axis-Transform"); printf("\n\t2. value is %d", *value); printf("\n\t3. threshold or mask type is %d", *threshold); printf("\n\t4. number of iterations is %d", *number); printf("\n\t (used only in opening and closing)"); printf("\n\nEnter choice (0 = no change) _\b"); get_integer(&response); if(response == 0){ not_finished = 0; } if(response == 1){ printf("\nEnter type of operation"); printf("\n\t recall: EROsion DILation Mask-ERosion" "\n\t Mask_DIlation INTerior-outline" "\n\t EXTerior-outline THinning" "\n\t Dilate-Not-Join OPEning" "\n\t CLOsing SPecial-Opening" "\n\t SPecial-Closing" "\n\t Euclidean-Distance-Measure" "\n\t Medial-Axis-Transform"); printf("\n\t:"); gets(type); } if(response == 2){ printf("\nEnter value: __\b\b\b"); get_integer(value); } if(response == 3){ printf("\nEnter threshold or mask type: __"); printf("\b\b\b"); get_integer(threshold); } if(response == 4){ printf("\nEnter number of iterations: __"); printf("\b\b\b"); get_integer(number); } } /* ends while not_finished */ } /* ends get_shape_options */ /* End of File */ Listing 8 A sample application that allows you to apply the shape operations to entire image files /********************************************* * file d:\cips\mainsk.c * * Functions: This file contains * main * * Purpose: * This file contains the main calling * routine that calls the erosion, dilation, * outline, and skeleton functions. * * External Calls: * gin.c - get_image_name * numcvrt.c - get_integer * int_convert * tiff.c - read_tiff_header * ed.c - erosion * dilation * mask_erosion * mask_dilation * interior_outline * exterior_outline * opening * closing * skeleton.c - thinning * skeleton * dilate_not_join * special_opening * special_closing * edm * mat * * Modifications: * 7 March 1993 - created ***********************************************/ #include "cips.h" short the_image[ROWS][COLS]; short out_image[ROWS][COLS]; main(argc, argv) int argc; char *argv[]; { char name[80], name2[80], name3[80], type[80]; int count, i, j, il, ie, ll, le, length, lw, mask_type, number, threshold, width; short value; struct tiff_header_struct image_header; my_clear_text_screen(); /********************************************* * Interpret the command line parameters. **********************************************/ if(argc < 5){ printf( "\n\nNot enough parameters:" "\n" "\n usage: mainsk in-file out-file type value" "[threshold-or-mask-type][number]" "\n" "\n recall type: EROsion DILation Mask-ERosion" "\n Mask_DIlation INTerior-outline" "\n EXTerior-outline THInning" "\n Dilate-Not-Join OPEning" "\n CLOsing SPecial-Opening" "\n SPecial-Closing" "\n Euclidean-Distance-Measure" "\n Medial-Axis-Transform" "\n" "\n [number] needed for opening" "\n and closing" "\n"); exit(0); } strcpy(name, argv[1]); strcpy(name2, argv[2]); strcpy(type, argv[3]); value = atoi(argv[4]); threshold = atoi(argv[5]); mask_type = atoi(argv[5]); number = atoi(argv[6]); il = 1; ie = 1; ll = ROWS+1; le = COLS+1; /********************************************* * Read the input image header and setup * the looping counters. *********************************************/ read_tiff_header(name, &image_header); length = (90 + image_header.image_length)/ROWS; width = (90 + image_header.image_width)/COLS; count = 1; lw = length*width; printf("\nlength=%d width-%d", length, width); /********************************************* * Loop over the input images and * apply the desired function. **********************************************/ for(i=0; i