Debugging and system testing are vital tasks during the software-development cycle, with the ultimate goal to identify and solve software-related and hardware-related defects on a target device. Debugging generally refers to the process of stepping through the code and analyzing debug messages during code execution in order to diagnose root causes of errors. It can also be an efficient tool to study the implementation of system components and applications in general. System testing, on the other hand, is a quality-assurance activity to validate the system in its final configuration in terms of typical usage scenarios, performance, reliability, security, and any other relevant aspects. The overall purpose of system testing is to discover product defects and faults, such as memory leaks, deadlocks, or hardware conflicts, whereas debugging is a means to get to the bottom of these problems and eliminate them. For many developers of small-footprint and consumer devices, locating and eliminating system defects is the hardest part of software development, with a measurable impact on productivity. This chapter covers the debugging and testing tools available in Microsoft® Visual Studio® 2005 with Platform Builder for Microsoft Windows® Embedded CE 6.0 R2 and in the Windows Embedded CE Test Kit (CETK) to help you automate and accelerate these processes so that you can release your systems faster and with fewer bugs. The better you master these tools, the more time you can spend writing code instead of fixing code.
Exam objectives in this chapter:
■ Identifying requirements for debugging a run-time image
■ Using debugger features to analyze code execution
■ Understanding debug zones to manage the output of debug messages
■ Utilizing the CETK tool to run default and user-defined tests
■ Debugging the boot loader and operating system (OS)
To complete the lessons in this chapter, you must have the following:
■ At least some basic knowledge about Windows Embedded CE software development and debugging concepts.
■ A basic understanding of the driver architectures supported in Windows Embedded CE.
■ Familiarity with OS design and system configuration concepts.
■ A development computer with Microsoft Visual Studio 2005 Service Pack 1 and Platform Builder for Windows Embedded CE 6.0 R2 installed.
Software-related errors range from simple typos, uninitialized variables, and infinite loops to more complex and profound issues, such as critical race conditions and other thread synchronization problems. Fortunately, the vast majority of errors are easy to fix once they are located. The most cost-effective way to find these errors is through code analysis. You can use a variety of tools on Windows Embedded CE devices to debug the operating system and step through drivers and applications. A good understanding of these debugging tools will help you accelerate your code analysis so that you can fix software errors as efficiently as possible.
After this lesson, you will be able to:
■ Identify important debugging tools for Windows Embedded CE.
■ Control debug messages through debug zones in drivers and applications.
■ Use the target control shell to identify memory issues.
Estimated lesson time: 90 minutes.
The primary tool to debug and control a Windows Embedded CE target device is by using Platform Builder on the development workstation, as illustrated in Figure 4-1. The Platform Builder integrated development environment (IDE) includes a variety of tools for this purpose, such as a system debugger, CE target control shell (CESH), and debug message (DbgMsg) feature, that you can use to step through code after reaching a breakpoint or to display information on memory, variables, and processes. Moreover, the Platform Builder IDE includes a collection of remote tools, such as Heap Walker, Process Viewer, and Kernel Tracker, to analyze the state of the target device at run time.
Figure 4-1 CE debugging and target control architecture
In order to communicate with the target device, Platform Builder relies on the Core Connectivity (CoreCon) infrastructure and debugging components deployed on the target device as part of the run-time image. The CoreCon infrastructure provides OS Access (OsAxS), target control, and DbgMsg services to Platform Builder on one side, and interfaces with the target device through the Kernel Independent Transport Layer (KITL) and the bootstrap service on the other side. On the target device itself, the debugging and target control architecture relies on KITL and the boot loader for communication purposes. If the run-time image includes debugging components, such as the kernel debugger stub (KdStub), hardware debugger stub (HdStub), and the OsAxS library, you can use Platform Builder to obtain kernel run-time information and perform just-in-time (JIT) debugging. Platform Builder also supports hardware- assisted debugging through Extended Debugging Interface (eXDI), so that you can debug target device routines prior to loading the kernel.
The Kernel Debugger is the CE software-debugging engine to debug kernel components and CE applications. On the development workstation, you work directly in Platform Builder, such as to insert or remove breakpoints in the source code and run the application, yet you must include support for KITL and debugging libraries (KdStub and OsAxS) in the run-time image so that Platform Builder can capture debugging information and control the target device. Lesson 2, "Configuring the Run-Time Image to Enable Debugging," later in this chapter provides detailed information about system configurations for kernel debugging.
The following target-side components are essential for kernel debugging:
■ KdStub Captures exceptions and breakpoints, retrieves kernel information, and performs kernel operations.
■ OsAxS Retrieves information about the state of the operating system, such as information about memory allocations, active processes and threads, proxies, and loaded dynamic-link libraries (DLLs).
By using the Kernel Debugger, you can control the entire run-time image as well as individual applications. However, KdStub is a kernel component that receives first-chance and second-chance exceptions, as explained in Chapter 3, "Performing System Programming." If you stop the Kernel Debugger during a session without stopping the target-side KdStub module first and an exception occurs, the run-time image stops responding until you reconnect the debugger, because the Kernel Debugger must handle the exception so that the target device can continue to run.
In Platform Builder, when you attach to a KITL-enabled and KdStub-enabled target device, you can examine debug information in the Output window of Microsoft Visual Studio 2005, which Platform Builder obtains from the target device by using the DbgMsg service in the CoreCon infrastructure. Debug messages provide detailed information about the running processes, signal potentially critical issues, such as invalid input, and give hints about the location of a defect in the code that you can then study further by setting a breakpoint and stepping through the code in Kernel Debugger. One of the kernel debugger stub's features is support for dynamic management of debug messages, so you can configure the debugging verbosity without source code modifications. Among other things, you can exclude Timestamps, Process IDs, or Thread IDs, if you display the Debug Message Options window that you can reach through the Target menu in Visual Studio. You can also send the debug output to a file for analysis in a separate tool. On the target device, all debug messages are sent directly to the default output stream handled through the NKDbgPrintf function.
When both Kernel Debugger and KITL are enabled, the debug messages are displayed in the Output window of Visual Studio. If KITL is not available, the debug information is transferred from the target device to the development computer over a serial port configured and used by the OEM adaptation layer (OAL).
To generate debug information, Windows Embedded CE provides several debugging macros that generally fall into two categories, debug macros and retail macros. Debug macros output information only if the code is compiled in the debug build configuration (environment variable WINCEDEBUG=debug), while retail macros generate information in both debug and retail build configurations (WINCEDEBUG=retail) unless you build the run-time image in ship configuration (WINCESHIP=1). In ship configuration, all debugging macros are disabled.
Table 4-1 summarizes the debugging macros that you can insert in your code to generate debug information.
Table 4-1 Windows Embedded CE macros to output debugging messages
| Macro | Description |
|---|---|
| DEBUGMSG | Conditionally prints a printf-style debug message to the default output stream (that is, the Output window in Visual Studio or a specified file) if the run-time image is compiled in debug build configuration. |
| RETAILMSG | Conditionally prints a printf-style debug message to the default output stream (that is, the Output window in Visual Studio or a specified file) if the run-time image is compiled in debug or release build configuration, yet not in ship build configuration. |
| ERRORMSG | Conditionally prints additional printf-style debug information to the default output stream (that is, the Output window in Visual Studio or a specified file) if the run-time image is compiled in debug or release build configuration, yet not in ship build configuration. This error information includes the name of the source code file and the line number, which can help to quickly locate the line of code that generated the message. |
| ASSERTMSG | Conditionally prints a printf-style debug message to the default output stream (that is, the Output window in Visual Studio or a specified file) and then breaks into the debugger, if the run-time image is compiled in debug configuration. In fact, ASSERTMSG calls DEBUGMSG followed by DBGCHK. |
| DEBUGLED | Conditionally passes a WORD value to the WriteDebugLED function, if the run-time image is compiled in debug build configuration. This macro is useful on devices that provide only light-emitting diodes (LEDs) to indicate the system status and requires an implementation of the OEMWriteDebugLED function in the OAL. |
| RETAILLED | Conditionally passes a WORD value to the WriteDebugLED function, if the run-time image is compiled in debug or release build configuration. This macro is useful on devices that provide only LEDs to indicate the system status and requires an implementation of the OEMWriteDebugLED function in the OAL. |
Debug messages are particularly useful tools to analyze multi-threaded processes, especially thread synchronization and other timing issues that are difficult to detect by stepping through the code. However, the number of debug messages generated on a target device can be overwhelming if you heavily use debugging macros in your code. To control the amount of information generated, debugging macros enable you to specify a conditional expression. For example, the following code outputs an error message if the dwCurrentIteration value is greater than the maximum possible value.
ERRORMSG(dwCurrentIteration > dwMaxIteration,
(TEXT("Iteration error: the counter reached %u, when max allowed is %u\r\n"),
dwCurrentIteration, dwMaxIteration));
In the example above, ERRORMSG outputs debugging information whenever dwCurrentIteration is greater than dwMaxIteration. You can also control debugging messages by using debug zones in the conditional statement. This is particularly useful if you want to use the DEBUGMSG macro to examine code execution in a module (that is, an executable file or a DLL) at varying levels without changing and recompiling the source code each time. First, you must enable debug zones in your executable file or DLL, and register a global DBGPARAM variable with the Debug Message service to specify which zones are active. You can then specify the current default zone programmatically or through registry settings on the development workstation or the target device. It is also possible to control debug zones dynamically for a module in Platform Builder via CE Debug Zones on the Target menu or in the Target Control window.
You can bypass debug zones in drivers and applications if you pass a Boolean variable to the DEBUGMSG and RETAILMSG macros that you can set to TRUE or FALSE when you rebuild the run-time image.
To use debug zones, you must define a global DBGPARAM variable with three fields that specify the module name, the names of the debug zones you want to register, and a field for the current zone mask, as summarized in Table 4-2.
Table 4-2 DBGPARAM elements
| Field | Description | Example |
|---|---|---|
| lpszName | Defines the name of the module with a maximum length of 32 characters. | |
| rglpszZones | Defines an array of 16 names for the debug zones. Each name can be up to 32 characters long. Platform Builder displays this information to the user when selecting the active zones in the module. | |
| ulZoneMask | The current zone mask used in the DEBUGZONE macro to determine the currently selected debug zone. | |
Windows Embedded CE supports a total of 16 named debug zones, yet not all have to be defined if the module doesn’t require them. Each module uses a separate set of zone names that should clearly reflect the purpose of each implemented zone.
The Dbgapi.h header file defines the DBGPARAM structure and debugging macros. Because these macros use a predefined DBGPARAM variable named dpCurSettings, it is important that you use the same name in your source code as well, as illustrated in the following code snippet.
#include
// A macro to increase the readability of zone mask definitions
#define DEBUGMASK(n) (0x00000001<
// Definition of zone masks supported in this module
#define MASK_INIT DEBUGMASK(0)
#define MASK_DEINIT DEBUGMASK(1)
#define MASK_ON DEBUGMASK(2)
#define MASK_FAILURE DEBUGMASK(13)
#define MASK_WARNING DEBUGMASK(14)
#define MASK_ERROR DEBUGMASK(15)
// Definition dpCurSettings variable with the initial debug zone state
// set to Failure, Warning, and Error.
DBGPARAM dpCurSettings = {
TEXT("ModuleName"), // Specify the actual module name for clarity!
{
TEXT("Init"), TEXT("Deinit"), TEXT("On"), TEXT("Undefined"),
TEXT("Undefined"), TEXT("Undefined"), TEXT("Undefined"),
TEXT("Undefined"), TEXT("Undefined"), TEXT("Undefined"),
TEXT("Undefined"), TEXT("Undefined"), TEXT("Undefined"),
TEXT("Failure"), TEXT("Warning"), TEXT("Error")
},
MASK_INIT | MASK_ON | MASK_ERROR
};
// Main entry point into DLL
BOOL APIENTRY DllMain( HANDLE hModule,
DWORD ul_reason_for_call, LPVOID lpReserved) {
if (ul_reason_for_call == DLL_PROCESS_ATTACH) {
// Register with the Debug Message service each time
// the DLL is loaded into the address space of a process.
DEBUGREGISTER((HMODULE)hModule);
}
return TRUE;
}
The sample code above registers six debug zones for the module that you can now use in conjunction with conditional statements in debugging macros. The following line of code shows one possible way to do this:
DEBUGMSG(dpCurSettings.ulZoneMask & (0x00000001<<(15)),
(TEXT("Error Information\r\n")));
If the debug zone is currently set to MASK_ERROR, the conditional expression evaluates to TRUE and DEBUGMSG sends the information to the debug output stream. However, to improve the readability of your code, you should use the DEBUGZONE macro defined in Dbgapi.h, as illustrated in the following code snippet, to define flags for your zones. Among other things, this approach simplifies the combination of debug zones through logical AND and OR operations.
#include
// Definition of zone flags: TRUE or FALSE according to selected debug zone.
#define ZONE_INIT DEBUGZONE(0)
#define ZONE_DEINIT DEBUGZONE(1)
#define ZONE_ON DEBUGZONE(2)
#define ZONE_FAILURE DEBUGZONE(13)
#define ZONE_WARNING DEBUGZONE(14)
#define ZONE_ERROR DEBUGZONE(15)
DEBUGMSG(ZONE_FAILURE, (TEXT("Failure debug zone enabled.\r\n")));
DEBUGMSG(ZONE_FAILURE && ZONE_ WARNING,
(TEXT("Failure and Warning debug zones enabled.\r\n")));
DEBUGMSG(ZONE_FAILURE || ZONE_ ERROR,
(TEXT("Failure or Error debug zone enabled.\r\n")));
The DBGPARAM field ulZoneMask is the key to setting the current debug zone for a module. You can accomplish this programmatically in the module by changing the ulZoneMask value of the global dpCurSettings variable directly. Another option is to change the ulZoneMask value in the debugger at a breakpoint within the Watch window. You can also control the debug zone through another application by calling the SetDbgZone function. Another option available at run time is to use the Debug Zones dialog box, illustrated in Figure 4-2, which you can display in Visual Studio with Platform Builder via the CE Debug Zones command on the Target menu.
Figure 4-2 Setting debug zones in Platform Builder
The Name list shows the modules running on the target device that support debug zones. If the selected module is registered with the Debug Message service, you can find the list of 16 zones displayed under Debug Zones. The names correspond to the selected module's dpCurSettings definition. You can select or deselect zones to enable or disable them. By default, the zones defined in the dpCurSettings variable are enabled and checked in the Debug Zones list. For modules not registered with the Debug Message service, the Debug Zone list is deactivated and unavailable.
Windows Embedded CE enables the zones you specify in the dpCurSettings variable when you start the application or load the DLL into a process. At this point, it is not yet possible to change the debug zone unless you set a breakpoint and change the ulZoneMask value in the Watch window. However, CE supports a more convenient method through registry settings. To facilitate loading a module with different active debug zones, you can create a REG_DWORD value with a name that corresponds to the module name specified in the lpszName field of the dpCurSettings variable and set it to the combined values of the debug zones you want to activate. You can configure this value on the development workstation or the target device. It is generally preferable to configure this value on the development workstation because changing target device registry entries requires you to rebuild the run-time image, whereas a modification of the registry entries on the development workstation only requires you to restart the affected modules.
Table 4-3 illustrates the configuration for a sample module called ModuleName. Make sure you replace this placeholder name with the actual name of your executable file or DLL.
Table 4-3 Startup registry parameter examples
| Location | Development Workstation | Target Device |
|---|---|---|
| Registry Key | HKEY_CURRENT_USER\Pegasus\Zones | HKEY_LOCAL_MACHINE\DebugZones |
| Entry Name | ModuleName | ModuleName |
| Type | REG_DWORD | REG_DWORD |
| Value | 0x00000001-0x7FFFFFFF | 0x00000001-0x7FFFFFFF |
| Comments | The Debug Message system uses the target-side value for a module only if the development workstation is unavailable or if the development-side registry does not contain a value for the module. |
Windows Embedded CE uses the lower 16 bits of the REG_DWORD value to define named debug zones for application debugging purposes. The remaining bits are available to define unnamed debug zones, with the exception of the highest bit, which is reserved for the kernel. Therefore, you should not set a module's debug zone value to 0xFFFFFFFF. The maximum value is 0x7FFFFFFF, which enables all named and unnamed debug zones.
The name Pegasus refers to the code name of the first Windows CE version that Microsoft released for handheld personal computers and other consumer electronics in 1996.
When working with debug messages, keep in mind that heavy use of debug messages slows down code execution. Perhaps even more important, the system serializes the debug output operations, which can provide an unintentional thread synchronization mechanism. For example, multiple threads running unsynchronized in release builds might cause issues not noticeable in debug builds.
When working with debug messages and debug zones, consider the following best practices:
■ Use Conditional statements Use debug macros with conditional statements based on debug zones. Do not use DEBUGMSG(TRUE). Also avoid using retail macros without conditional statements, such as RETAILMSG(TRUE), although some model device driver (MDD)/platform dependent driver (PDD) drivers must use this technique.
■ Exclude debugging code from release builds If you only use debug zones in debug builds, include the global variable dpCurSettings and zone mask definitions in #ifdef DEBUG #endif guards and restrict the use of debug zones to debug macros (such as DEBUGMSG).
■ Use retail macros in release builds If you also want to use debug zones in release builds, include the global variable dpCurSettings and zone mask definitions in #ifndef SHIP_BUILD #endif guards and replace the call to DEBUGREGISTER with a call to RETAILREGISTERZONES.
■ Clearly identify the module name If possible, set the dpCurSettings.lpszName value to the module's file name.
■ Limit verbosity by default Set the default zones for your drivers to ZONE_ERROR and ZONE_WARNING only. When bringing up a new platform, enable ZONE_INIT.
■ Restrict the error debug zone to unrecoverable issues Use ZONE_ERROR only when your module or significant functionality fails due to incorrect configuration or other issues. Use ZONE_WARNING for recoverable issues.
■ Eliminate all errors and warnings if possible Your module should be able to load without any ZONE_ERROR or ZONE_WARNING messages.
The Target Control service provides access to a command shell for the debugger to transfer files to the target device and debug applications. This target control shell, displayed in Figure 4-3, is accessible from within Visual Studio with Platform Builder via the Target Control option on the Target menu. However, it is important to keep in mind that the target control shell is only available if the Platform Builder instance is attached to a device through KITL.
Figure 4-3 The target control shell
Among other things, the target control shell enables you to perform the following debugging actions:
■ Break into the Kernel Debugger (break command).
■ Send a memory dump to the debug output (dd command) or to a file (df command).
■ Analyze memory usage for the kernel (mi kernel command) or the entire system (mi full command).
■ List processes (gi proc command), threads (gi thrd command), and thread priorities (tp command), as well as the modules loaded on the system (gi mod command).
■ Launch processes (s command) and end processes (kp command).
■ Dump the processes heap (hp command).
■ Enable or disable the system profiler (prof command).
For a complete list of target control commands, see the section "Target Control Debugging Commands" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN® Web site at http://msdn2.microsoft.com/en-us/library/aa936032.aspx.
In addition to the regular debugger commands, the Target Control service provides the debugger with a debugger commands extension (CEDebugX) to increase the efficiency of kernel and application debugging. This extension provides additional features to detect memory leaks and deadlocks and diagnose the overall health of the system. The additional commands are accessible through the target control shell and start with an exclamation point (!).
To use CEDebugX, you need to break into the Kernel Debugger by using the break command in the target control shell or the Break All command on the Target menu in Visual Studio. Among other things, you can then enter a !diagnose all command to identify the potential reason for a failure, such as heap corruption, deadlocks, or memory starvation. On a healthy system, the CEDebugX should detect no any issues, as illustrated in Figure 4-4.
Figure 4-4 Diagnosing a run-time image with CEDebugX
The !diagnose all command runs the following diagnostics:
■ Heap Diagnoses all the heap objects of all processes on the system to identify potential content corruption.
■ Exception Diagnoses an exception that occurs on the system and is able to provide details on the exception, such as process and thread ID, and PC address at the exception time.
■ Memory Diagnoses the system memory to identify potential memory corruptions and low memory conditions.
■ Deadlock Diagnoses the thread states and system objects (see Chapter 3 for more details on thread synchronization). It can provide a list of system objects and thread IDs that generated the deadlock.
■ Starvation Diagnoses threads and system objects to identify potential thread starvation. Starvation occurs when a thread is never scheduled on the system by the scheduler because the scheduler is busy with higher-priority threads.
The target control shell and CEDebugX commands enable you to perform a thorough analysis of a running system or a CE dump file (if you select the CE Dump File Reader as the debugger to perform postmortem debugging), yet you are not restricted to the command-line interface. Platform Builder includes several graphical tools with a dedicated purpose to increase your debugging efficiency. You can access these advanced debugger tools in Visual Studio via the Debug menu when you open the Windows submenu.
The Platform Builder IDE includes the following advanced debugger tools:
■ Breakpoints Lists the breakpoints enabled on the system and provides access to the breakpoint properties.
■ Watch Provides read and write access to local and global variables.
■ Autos Provides access to variables similar to the Watch window, except that the debugger creates this list of variables dynamically, while the Watch window lists all manually added variables whether they are accessible or not. The Autos window is useful if you want to check the parameter values passed to a function.
■ Call Stack Accessible only when the system is in a break state (code execution has halted on a breakpoint). This window provides a list of all processes enabled on the system and a list of hosted threads.
■ Threads Provides a list of the threads running in the processes on the system. This information is dynamically retrieved and can be updated at any time.
■ Modules Lists the modules loaded and unloaded on the system and provides the memory address where those modules are loaded. This feature is useful for identifying whether a driver DLL is actually loaded or not.
■ Processes Similar to the Threads window, this window provides a list of the processes running on the system. Among other things, you can terminate processes if required.
■ Memory Provides direct access to device memory. You can use memory addresses or variable names to locate the desired memory content.
■ Disassembly Reveals the assembly code of the current code line executed on the system.
■ Registers Provides access to the CPU register values when running a specific line of code.
■ Advanced Memory Can be used to search the device memory, move portions of memory to different sections, and fill memory ranges by using content patterns.
■ List Nearest Symbols Determines a specific memory address for the nearest symbols available in the binaries. It also provides the complete path to the file containing the symbol. This tool is useful to locate the name of a function that generated an exception.
The Memory and Advanced Memory tools can modify memory content. Using these tools incorrectly can cause system failures and damage the operating system on the target device.
Another useful tool to identify potential application compatibility and stability issues and necessary source code-level fixes is the Application Verifier tool, included in the CETK. This tool can attach to an application or a DLL to diagnose problems that are otherwise difficult to track on standalone devices. The Application Verifier tool does not require a device connection to a development workstation and can be launched at system startup to check and validate drivers and system applications. You can also start this tool from the CETK user interface or manually on the target device. If you want to use the Application Verifier tool outside of the CETK, you should use the Getappverif_cetk.bat file to copy all the required files into the release directory.
For detailed information about the Application Verifier tool, including how to use shim extension DLLs to run custom test code or change the behavior of functions during application testing, see the section "Application Verifier Tool" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa934321.aspx.
Windows Embedded CE includes an extensible event-tracking system that you can include in a run-time image to diagnose performance problems. The CeLog event-tracking system logs a set of predefined kernel and coredll events related to mutexes, events, memory allocation, and other kernel objects. The extensible architecture of the CeLog event-tracking system also enables you to implement custom filters to track user-defined events. For platforms connected to a development workstation through KITL, the CeLog event-tracking system can selectively log events based on zones specified in the ZoneCE registry entry, as summarized in Table 4-4.
Table 4-4 CeLog registry parameters for event logging zones
| Location | HKEY_LOCAL_MACHINE\System\CELog |
|---|---|
| Registry Entry | ZoneCE |
| Entry Type | REG_DWORD |
| Value | <Zone IDs> |
| Description | By default, all zones are logged. For a list of all possible zone ID values, see the section "CELog Zones" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa909194.aspx. |
By using the CeLog event-tracking system, you can collect data, which CeLog stores in a buffer in RAM on the target device. Performance tools, such as Remote Kernel Tracker and Readlog, can then process the collected data. It is also possible to flush the data periodically to a file by using the CELogFlush tool.
You should not include the CeLog event-tracking system in final builds to avoid performance and memory penalties due to CeLog activities, and to reduce the attack surface through which a malicious user could try to compromise the system.
The Remote Kernel Tracker tool enables you to monitor system activities on a target device based on processes and threads. This tool can display information from the target device in real time through KITL, yet it is also possible to use Remote Kernel Tracker offline based on CeLog data files. You can find more information about the Remote Kernel Tracker tool in Chapter 3, "Performing System Programming."
Figure 4-5 shows Kernel Tracker on a target device collecting information about thread activities.
Figure 4-5 Thread information in Kernel Tracker
To create CeLog data files, use the CeLogFlush tool to save the CeLog event data buffered in RAM into a .clg file. This file can be located in the RAM file system, persistent storage, or the release file system on a development workstation. To minimize data loss due to buffer overruns, you can specify a larger RAM buffer and increase the frequency at which CeLog flushes the buffer. You can optimize the performance if you keep the file open to avoid repeated file open and close operations and store the file in the RAM file system instead of a slower persistent storage medium.
For detailed information about the CeLogFlush tool, including how to configure this tool through registry settings, see the section "CeLogFlush Registry Settings" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa935267.aspx.
In addition to the graphical Remote Kernel Tracker application, you can process CELog data files by using the Readlog tool, located in the %_WINCEROOT%\Public\Common\Oak\Bin\i386 folder. Readlog is a command-line tool to process and display information not exposed in Remote Kernel Tracker, such as debug messages and boot events. It is often useful to analyze system activities in Remote Kernel Tracker first and then focus on an identified process or thread with the Readlog tool. The raw data that the CeLogFlush tool writes into the .clg file is ordered by zones to facilitate locating and extracting specific information. You can also filter the data and extend filtering capabilities based on extension DLLs to process custom data captured through the custom events collector.
One of the most useful Readlog scenarios is to replace thread start addresses (the functions passed to the CreateThread call) in CeLog data files with the names of the actual thread functions to facilitate system analysis in Remote Kernel Tracker. To accomplish this task, you must start Readlog with the -fixthreads parameter (readlog -fixthreads). Readlog looks up the symbol .map files in the release directory to identify the thread functions based on the start addresses and generates new logs with the corresponding references.
Figure 4-6 shows CeLog data in Remote Kernel Tracker, captured through the CeLog event-tracking system, flushed to a .clg file with the CeLogFlush tool, and prepared for a more user-friendly display of the information by using the Readlog application with the -fixthreads parameter.
Figure 4-6 A CeLog data file prepared with readlog -fixthreads and opened in Remote Kernel Tracker
The CeLog event-tracking system can take advantage of the kernel profiler to look up thread function names based on start addresses when capturing CreateThread events, if you explicitly enable the kernel profiler and add profiling symbols to the run-time image by rebuilding the image with the IMGPROFILER environment variable set. However, CeLog can only look up the profiling symbols built into the run-time image. Symbols of applications developed based on a Software Development Kit (SDK) are generally unavailable to the CeLog event-tracking system.
Debugging an operating system and applications requires familiarity with both the CE system and the debugging tools included in Platform Builder and CETK. The most important debugging tools are the system debugger, debug message feature, and CE target control shell. The system debugger enables you to set breakpoints and step through kernel and application code, while the debug message feature provides the option to analyze system components and applications without interrupting code execution. A variety of debug and retail macros are available to output debugging information from target devices with or without a display component. Because systems and applications can potentially generate a large number of debug messages, you should use debug zones to control the output of debugging information. The key advantage of debug zones is that you can change the debug information verbosity dynamically without having to rebuild the run-time image. The target control shell, on the other hand, enables you to send commands to the target device, such as a break command followed by a !diagnose all command to break into the debugger and perform a CEDebugX check on the overall system health, including memory leaks, exceptions, and deadlocks.
Apart from these core debugging tools, you can use typical CE configuration and troubleshooting tools, such as the Application Verifier tool, to identify potential application compatibility and stability issues, and Remote Kernel Tracker to analyze processes, threads, and system performance. Remote Kernel Tracker relies on the CeLog event-tracking system, which typically maintains logged data in memory on the target device; you can also flush this data to a file by using the CeLogFlush tool. If symbol files are available for the modules that you want to analyze, you can use the Readlog tool to replace the thread start addresses with the actual function names and generate new CeLog data files for more convenient offline analysis in Remote Kernel Tracker.
The debugging features of Windows Embedded CE rely on development workstation components and the target device, and require specific settings and hardware support. Without a connection between the development workstation and the target device, debug information and other requests cannot be exchanged. If this communication link breaks — for example, because you stop the debugger on the development workstation without first unloading the target-side debugging stub — the run-time image might stop responding to user input while waiting for the debugger to resume code execution after an exception occurred.
After this lesson, you will be able to:
■ Enable the Kernel Debugger for a run-time image.
■ Identify the KITL requirements.
■ Use the Kernel Debugger in a debugging context.
Estimated lesson time: 20 minutes.
As discussed in Lesson 1, the development environment for Windows Embedded CE 6.0 includes a Kernel Debugger that enables developers to step through and interact with code running on a CE target device. This debugger requires you to set kernel options and a communication layer between the target device and the development computer.
To enable an OS design for debugging, you must unset the environment variables IMGNODEBUGGER and IMGNOKITL so that Platform Builder includes the KdStub library and enables the KITL communication layer in the Board Support Package (BSP) when building the run-time image. Platform Builder provides a convenient method to accomplish this task. In Visual Studio, right-click the OS design project and select Properties to display the OS Design property pages dialog box, switch to the Build Options pane, and then select the Enable Kernel Debugger and Enable KITL check boxes. Chapter 1, "Customizing the Operating System Design," discusses the OS Design property pages dialog box in more detail.
Having enabled KdStub and KITL for a run-time image, you can select a debugger to analyze the system on the target device in the communication parameters for your target device. To configure these parameters, display the Target Device Connectivity Options dialog box in Visual Studio by opening the Target menu and selecting Connectivity Options, as explained in Chapter 2, "Building and Deploying a RunTime Image."
By default, no debugger is selected in the connectivity options. You have the following debugger choices:
■ KdStub This is the software debugger for the kernel and applications to debug system components, drivers, and applications running on a target device. KdStub requires KITL to communicate with Platform Builder.
■ CE Dump File Reader Platform Builder provides you with an option to capture dump files, which you can then open by using the CE dump-file reader. Dump files enable you to study the state of a system at a particular point in time and are useful as references.
■ Sample Device Emulator eXDI 2 Driver KdStub cannot debug routines that the system runs prior to loading the kernel, nor can it debug interrupt service routines (ISRs), because this debugging library relies on software breakpoints. For hardware-assisted debugging, Platform Builder includes a sample eXDI driver that you can use in conjunction with a joint test action group (JTAG) probe. The JTAG probe enables you to set hardware breakpoints handled by the processor.
For detailed information about hardware-assisted debugging, see the section "Hardware-assisted Debugging" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa935824.aspx.
As illustrated in Figure 4-1 at the beginning of this chapter, KITL is an essential communication layer between the development computer and the target device and must be enabled for Kernel Debugger support. As the name implies, KITL is completely hardware independent and works over network connections, serial cables, Universal Serial Bus (USB), or any other supported communication mechanism, such as Direct Memory Access (DMA). The only requirement is that both sides (development computer and target device) support and use the same interface. The most common and fastest KITL interface for the device emulator is DMA, as illustrated in Figure 4-7. For target devices with a supported Ethernet chip, it is typically best to use the network interface.
Figure 4-7 Configuring the KITL communication interface
KITL supports the following two methods of operation:
■ Active mode By default, Platform Builder configures KITL to connect to the development computer during the start process. This setting is most useful for kernel and application debugging during the software-development cycle.
■ Passive mode By clearing the check box Enable KITL on Device Boot, you can configure KITL for passive mode, meaning Windows Embedded CE initializes the KITL interface, but KITL does not establish a connection during the startup process. If an exception occurs, KITL makes an attempt to establish a connection to the development computer so that you can perform JIT debugging. Passive mode is most useful when working with mobile devices that do not have a physical connection to the development computer at startup.
The Enable KITL on Device Boot setting is a boot argument (BootArgs) that Platform Builder configures for the boot loader. For more information about boot loaders and their advantages during the BSP development process, see the section "Boot Loaders" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa917791.aspx.
It is important to keep in mind that development-side and target-side debugger components run independently of each other. For example, it is possible to run the Kernel Debugger in Visual Studio 2005 with Platform Builder without having an active target device. If you open the Debug menu and click Start or press the F5 key, the Kernel Debugger starts and informs you in the Output window that it is waiting for a connection to the target device. On the other hand, if you start a debugging- enabled run-time image without an active KITL connection to a debugger and an exception occurs, the run-time image appears to hang because the system halts, waiting for control requests from the debugger, as mentioned earlier in this chapter. For this reason, the debugger typically starts automatically when you attach to a debugging-enabled target device. Instead of pressing F5 to start a debugging session, use Attach Device on the Target menu.
The debugging features of Platform Builder provide most of the functionality also found in other debuggers for Windows desktop applications. You can set breakpoints, step through the code line-by-line, and use the Watch window to view and change variable values and object properties, as illustrated in Figure 4-8. Keep in mind, however, that the ability to use breakpoints depends on the existence of the KdStub library in the run-time image.
Figure 4-8 Debugging a Hello World application
To set a breakpoint, use the Toggle Breakpoint option on the Debug menu in Visual Studio. Alternatively, you can press F9 to set a breakpoint at the current line or click the left margin area of the code line. According to your selection, Platform Builder indicates the breakpoint with a red dot or a red circle, depending on whether the debugger can instantiate the breakpoint or not. The red circle indicates an un-instantiated breakpoint. Un-instantiated breakpoints occur if the Visual Studio instance is not linked to the target code, the breakpoint is set but has not yet been loaded, the debugger is not enabled, or if the debugger is running but code execution has not yet halted. If you set a breakpoint while the debugger is running, then the device must break into the debugger first before the debugger can instantiate the breakpoint.
You have the following options to manage breakpoints in Visual Studio with Platform Builder:
■ Source code, Call Stack, and Disassembly windows You can set, remove, enable, or disable a breakpoint by either pressing F9 and selecting Toggle
Breakpoint from the Debug menu or selecting Insert/Remove Breakpoint from the context menu.
■ New Breakpoint dialog box You can display this dialog box via the submenus available under New Breakpoint on the Debug menu. The New Breakpoint dialog box enables you to set breakpoints by location and conditions. The debugger stops at a conditional breakpoint only if the specified condition evaluates to TRUE, such as when a loop counter or other variable has a specific value.
■ Breakpoints window You can display the Breakpoints window by clicking Breakpoints under the Windows submenu on the Debug menu, or by pressing Alt+F9. The Breakpoints window lists all set breakpoints and enables you to configure breakpoint properties. For example, instead of using the New Breakpoint dialog box, which requires you to specify location information manually, you can set the desired breakpoint directly in the source code and then display the properties of this breakpoint in the Breakpoints window to define conditional parameters.
Use breakpoints sparingly. Setting too many breakpoints and constantly selecting Resume impacts debugging efficiency and makes it hard to focus on one aspect of the system at a time. Consider disabling and re-enabling breakpoints as necessary.
When configuring the properties of a breakpoint in the New Breakpoint dialog box or the Breakpoints window, you may notice a Hardware button, which you can use to configure the breakpoint as a hardware breakpoint or software breakpoint. You cannot use software breakpoints in OAL code or interrupt handlers, because the breakpoint must not completely halt the execution of the system. Among other system processes, the KITL connection must remain active, because it is the only way to communicate with the debugger on the development workstation. KITL interfaces with the OAL and uses the kernel's interrupt-based communication mechanisms. If you set a breakpoint in an interrupt handler function, then the system will not be able to communicate any longer when the breakpoint is reached because interrupt handling is a single-threaded and non-interruptible function.
If you must debug interrupt handlers, you can use debug messages or hardware breakpoints. However, hardware breakpoints require an eXDI-compliant debugger (such as JTAG Probe) to register the interrupt in the processor's debug register. Typically, only one hardware debugger can be enabled on a processor at a time, although JTAG can manage multiple debuggers. You cannot use the KdStub library for hardware-assisted debugging.
To configure a hardware breakpoint, follow these steps:
1. Open the Breakpoint window by opening the Debug menu and then clicking Breakpoint.
2. Select the breakpoint in the breakpoint list and then right-click it.
3. Click Breakpoint Properties to display the Breakpoint Properties dialog box and then click the Hardware button.
4. Select the Hardware radio button and then click OK twice to close all dialog boxes.
Figure 4-9 Setting a hardware breakpoint
Enabling the debugger is a straightforward configuration process in the Platform Builder IDE if you include KITL and debugger libraries in the run-time image. You can then display the Target Device Connectivity Options dialog box and select an appropriate transport and debugger. Typically, the transport is DMA or Ethernet, but you can also use a USB or serial cable to connect the development workstation to the target device.
Platform Builder provides most of the debugging features also found in other debuggers for Windows desktop applications. You can set breakpoints, step through the code line-by-line, and use the Watch window to view and change variable values and object properties. Platform Builder also supports conditional breakpoints to halt code execution according to specified criteria. The debugger of choice for software debugging is KdStub, although you can also use an eXDI driver with Platform Builder for hardware-assisted debugging based on a JTAG probe or other hardware debugger. Hardware-assisted debugging enables you to analyze system routines that run prior to loading the kernel, OAL components, and interrupt handler functions where you cannot use software breakpoints.
Automated software testing is a key to improving product quality while lowering development and support costs. This is particularly important if you create a custom BSP for a target device, added new device drivers, and implemented custom OAL code. Before releasing a new series of the system to production, it is vital to perform functional testing, unit testing, stress testing, and other types of testing to validate each part of the system and ensure that the target device operates reliably under normal conditions. It is generally much more expensive to fix defects after a new product reaches the market than to create testing tools and scripts that simulate users operating the target device and fix any defects while the system is still under development. System testing should not be an afterthought. To perform system testing efficiently throughout the software development cycle, you can use the CETK.
After this lesson, you will be able to:
■ Describe typical usage scenarios for CETK test tools.
■ Create user-defined CETK tests.
■ Run CETK tests on a target device.
Estimated lesson time: 30 minutes.
The CETK is a separate test application included with Platform Builder for Windows Embedded CE to validate the stability of applications and device drivers based on a series of automated tests organized in a CE test catalog. The CETK includes numerous default tests for several driver categories of peripheral devices and you can also create custom tests for your specific needs.
For a complete list of default tests included in the CETK, see the section "CETK Tests" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa917791.aspx.
As illustrated in Figure 4-10, the CETK application is a client/server solution with components running on the development computer and on the target device. The development computer runs the workstation server application (CETest.exe) while the target device runs the client-side application (Clientside.exe), test engine (Tux.exe), and test results logger (Kato.exe). Among other things, this architecture enables you to run concurrent tests on multiple different devices from the same development workstation. Workstation server and client-side applications can communicate through KITL, ActiveSync® or a Windows Sockets (Winsock) connection.
Figure 4-10 The CETK client/server architecture
The CETK application includes the following components:
■ Development workstation server CETest.exe provides the graphical user interface (GUI) to run and manage CETK tests. This application also enables you to configure server settings and connection parameters, as well as connect to a target device. Having established a device connection, the workstation server can automatically download and start the client-side application, submits test requests, and compile test results based on captured logs in real-time for display.
■ Client-side application Clientside.exe interfaces with the workstation server application to control the test engine and return test results to the server application. If Clientside.exe is unavailable on the target device, the workstation server cannot establish a communication stream to the target device.
■ Test engine CETK tests are implemented in DLLs that Tux.exe loads and runs on the target device. Typically, you start the test engine remotely through workstation server and client-side application, yet it is also possible to start Tux.exe locally, in stand-alone fashion with no workstation server requirement.
■ Test results logger Kato.exe tracks the results of the CETK tests in log files. Tux DLLs can use this logger to provide additional information about whether a test succeeded or failed and have their output routed to multiple user-defined output devices. Because all CETK tests use the same logger and format, it is possible to use a default file parser or implement a custom log file parser for automatic result processing according to specific requirements.
A managed version of the CETK is available to validate native and managed code. For details about the managed version, see the section "Tux.Net Test Harness" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa934705.aspx.
You can run CETK tests in a variety of ways according to the connectivity options supported on the target device. You can use KITL, Microsoft ActiveSync, or a TCP/IP connection to connect to the target device, download the target-side CETK components, run the desired tests, and view the results in the graphical user interface on the development workstation. On the other hand, if your target device does not support these connectivity options, you must run the tests locally with appropriate command-line options.
To work with the workstation server application, click Windows Embedded CE 6.0 Test Kit in the Windows Embedded CE 6.0 program group on your development computer, open the Connection menu and select the Start Client command. You can then configure the transport by clicking the Settings button. If the target device is switched on and connected to your development workstation, click Connect, select the desired target device, and then click OK to establish the communication channel and deploy the required binaries. The CETK application is now ready to run tests on the target device.
As illustrated in Figure 4-11, the CETK application automatically detects the device drivers available on the target and provides a convenient method to run the tests. One way is to click the device name under Start/Stop Test on the Tests menu, which causes CETK to test all detected components. Another way is to right-click the Test Catalog node and select the Start Tests command. You can also expand the individual containers, right-click an individual device test, and click Quick Start to test only a single component. The workstation server application also provides access to Application Verifier, CPU Monitor, Resource Consume, and Windows Embedded CE Stress tool when you right-click the device node and open the Tools submenu.
Figure 4-11 The graphical user interface of the CETK application
Apart from running all tests at once or quick tests individually, you can create test suites that include a custom series of tests that you want to perform repeatedly throughout the software development cycle. To create a new test suite, use the Test Suite Editor, available in the workstation server application on the Tests menu. The Test Suite Editor is a graphical tool to select the tests that belong to a suite conveniently. You can export test suite definitions in the form of Test Kit Suite (.tks) files and import these files on additional development computers to ensure that all workstation server applications perform the same set of tests. These .tks files can also provide the basis for test definition archives.
The graphical user interface also enables you to customize the command lines that the workstation server application sends to the test engine (Tux.exe) to perform the tests. To modify the parameters of a test, right-click the test in the Test Catalog and select the Edit Command Line option. For example, the Storage Device Block Driver Benchmark Test analyzes the performance of a storage device by reading and writing data to every sector on the device. This implies that all existing data on the storage device will be destroyed. To protect you from accidental data loss, the Storage Device Block Driver Benchmark Test is skipped by default. To run the Storage Device Block Driver Benchmark Test successfully, you must edit the command line and explicitly add a special parameter called -zorch.
The supported command-line parameters depend on each individual CETK test implementation. Tests might support or require a variety of configuration parameters, such as an index number to identify the device driver to test, or additional information that must be provided to run the test.
For a complete list of default CETK tests with links to additional information, such as command-line parameters, see the section "CETK Tests" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/ms893193.aspx.
If you have included the Windows Embedded CE Test Kit catalog item in your runtime image, downloaded the CETK components with the workstation server application, or exported the components from your development workstation to the target device by using the File Viewer remote tool, you can start Clientside.exe on the target device and establish a connection to a workstation server manually. If your target device does not provide the Run dialog box for this purpose, open the Target menu in the Platform Builder IDE, select Run Programs, select Clientside.exe, and then select Run.
Clientside.exe supports the following command-line parameters that you can specify to connect to a specific workstation server application, detect installed drivers, and run tests automatically:
Clientside.exe [/i= IP Address> | /n=<Server Name>] [/p=<Server Port Number>] [/a] [/s] [/d] [/x]
It is important to note that you can define these parameters also in a Wcetk.txt file or in the HKEY_LOCAL_MACHINE/Software/Microsoft/CETT registry key on the target device so that you can start Clientside.exe without command-line parameters.
In this case, Clientside.exe searches for Wcetk.txt in the root directory, then in the Windows directory on the target device, and then in the release directory on the development workstation. If Wcetk.txt does not exist in any of these locations, it checks the CETT registry key. Table 4-5 summarizes the Clientside.exe parameters.
Table 4-5 Clientside.exe start parameters
| Command Line | Wcetk.txt | CETT Registry Key | Description |
|---|---|---|---|
| /n | SERVERNAME | ServerName (REG_SZ) | Specifies the host server name. Cannot be used together with /i and requires Domain Name System (DNS) for name resolution. |
| /i | SERVERIP | ServerIP (REG_SZ) | Specifies the host IP address. Cannot be used together with /n. |
| /p | PORTNUMBER | PortNumber (REG_DWORD) | Specifies the server port number that can be configured from the workstation server interface. |
| /a | AUTORUN | Autorun (REG_SZ) | When set to one (1), the device automatically starts the test after the connection is established. |
| /s | DEFAULTSUITE | DefaultSuite (REG_SZ) | Specifies the name of the default test suite to run. |
| /x | AUTOEXIT | Autoexit (REG_SZ) | When set to one (1), the application automatically exits when the tests are completed. |
| /d | DRIVERDETECT | DriverDetect (REG_SZ) | When set to zero (0), the detection of devices drivers is disabled. |
Clientside.exe connects to CETest.exe on the developer workstation, yet it is also possible to run CETK tests without a connection, which is particularly useful for devices that provide no connectivity possibilities. If you include the Windows Embedded CE Test Kit catalog item in the run-time image, you can start the test engine (Tux.exe) directly, which implicitly starts the Kato logging engine (Kato.exe) to track the test results in log files. For example, to perform mouse tests (mousetest.dll) and track the results in a file called test_results.log, you can use the following command line:
Tux.exe -o -d mousetest -f test_results.log
For a complete list of Tux.exe command-line parameters, see the section "Tux Command-Line Parameters" in the Windows Embedded CE 6.0 Documentation, available on the Microsoft MSDN Web site at http://msdn2.microsoft.com/en-us/library/aa934656.aspx.
The CETK includes a large number of tests, yet the default tests cannot cover all testing requirements, especially if you added your own custom device drivers to a BSP. To provide you with an option to implement user-defined tests for your custom drivers, the CETK relies on the Tux framework. Platform Builder includes a WCE TUX DLL template to create a skeleton Tux module with a few mouse clicks. When implementing the logic to exercise your driver, you might find it useful to check out the source code of existing test implementations. The CETK includes source code, which you can install as part of the Windows Embedded CE Shared Source in the Setup wizard for Windows Embedded CE. The default location is %_WINCEROOT%\Private\Test.
To create a custom test library that is compliant with the Tux framework, start the Windows Embedded CE Subproject Wizard by adding a subproject to the OS design of your run-time image and select the WCE TUX DLL template. This causes the Tux wizard to create a skeleton that you can customize according to your driver requirements.
You must edit the following files in the subproject to customize the skeleton Tux module:
■ Header file Ft.h Defines the TUX Function Table (TFT), including a function table header and function table entries. The function table entries associate test IDs with the functions that contain the test logic.
■ Source code file Test.cpp Contains the test functions. The skeleton Tux module includes a single TestProc function that you can use as reference to add custom tests to the Tux DLL. You can replace the sample code to load and exercise your custom driver, log activities through Kato, and return an appropriate status code back to the Tux test engine when the tests are completed.
The skeleton Tux module is fully functional, so you can compile the solution and build the run-time image even without code modifications. To run the new test function on a target device, you must configure a user-defined test in the CETK workstation server application. For this purpose, CETK includes a User-Defined Test Wizard, which you can start by clicking the User Defined command on the Tests menu. Figure 4-12 shows the User-Defined Test Wizard with configuration parameters to run a skeleton Tux module.
Figure 4-12 Configuring a custom test in the User-Defined Test Wizard
Because Tux tests rely on code and logic implemented in Tux DLLs, it might be necessary to debug the test code. One issue worth mentioning is that you can set breakpoints in your test routines, but when code execution halts on those breakpoints, you lose the connection between the client-side application (Clientside.exe) and the workstation server application (CETest.exe). Consider using debug messages instead of breakpoints. If you must use breakpoints for thorough debugging, run Tux.exe directly on the target device in standalone mode, as mentioned earlier in this lesson. You can display the required command line in the workstation server application when you right-click the test and select Edit Command Line.
CETK tests should use Kato to log test results, as demonstrated in the skeleton Tux module:
g_pKato->Log(LOG_COMMENT, TEXT("This test is not yet implemented."));
The workstation server application retrieves these logs automatically through Clientside.exe and stores them on the development workstation. You can also access these log files through other tools. For example, if you are using CETK in stand-alone fashion, you can import the log files to the development workstation by using the File Viewer remote tool.
The CETK includes a general CETK parser (Cetkpar.exe) located in the C:\Program Files\Microsoft Platform Builder\6.00\Cepb\Wcetk folder for convenient viewing of imported log files, as shown in Figure 4-13. Typically, you start this parser by right- clicking a completed test in the workstation server application and selecting View Results, yet you can also start Cetkpar.exe directly. Some tests, particularly performance tests based on PerfLog.dll, can also be parsed into comma-separated values (CSV) format and opened in a spreadsheet to summarize the performance data. The CETK includes a PerfToCsv parser tool for this purpose, and you can develop custom parsers for special analysis scenarios. Kato log files use a plain text format.
Figure 4-13 Analyzing CETK test results
The Windows Embedded CE Test Kit is an extensible tool that enables you to test drivers and applications on a target device in connected mode and in standalone mode. Running the CETK tools in standalone mode is useful if the target device does not support connectivity over KITL, ActiveSync, or TCP/IP. Most typically, developers use the CETK to test device drivers added to the BSP of a target device.
The CETK relies on the Tux test engine, which provides a common framework for all test DLLs. The Tux DLLs contain the actual testing logic and run on the target device to load and exercise the driver. Tux DLLs also interface with Kato to track test results in log files, which you can access directly in the CETK test application or process in separate tools, such as custom parsers and spreadsheets.
The general task of a boot loader is to load the kernel into memory and then call the OS startup routine after powering up the device. On Windows Embedded CE specifically, the boot loader is part of the BSP and in charge of initializing the core hardware platform, downloading the run-time image, and starting the kernel. Even if you do not plan to ship a boot loader in your final product and directly bootstrap the run-time image, you might find a boot loader helpful during the development cycle. Among other things, a boot loader can help to simplify run-time image deployment complexities. Downloading the run-time image over Ethernet connections, serial cable, DMA, or USB connections from a development computer is a convenience feature that can help to save development time. Based on the source code included with Platform Builder for Windows Embedded CE 6.0, you can also develop a custom boot loader to support new hardware or features. For example, you can use a boot loader to copy the run-time image from RAM into flash memory and eliminate the need for a separate flash memory programmer or Institute of Electrical and Electronic Engineers (IEEE) 1149.1-compliant test access port and boundary-scanning technology. However, debugging and testing a boot loader is a complex undertaking because you are working with code that runs before the kernel loads.
After this lesson, you will be able to:
■ Describe the CE boot loader architecture.
■ List common debugging techniques for boot loaders.
Estimated lesson time: 15 minutes.
The underlying idea of a boot loader is to bootstrap a small program with pre-boot routines in linear, nonvolatile, CPU-accessible memory. Having placed the initial boot loader image on the target device at the memory address where the CPU begins to retrieve code through a built-in monitor program provided by the board manufacturer or a JTAG probe, the boot loader runs whenever you power up or reset the system. Typical boot loader tasks performed at this stage include initializing the Central Processing Unit (CPU), the memory controller, system clock, Universal Asynchronous Receiver/Transmitters (UARTs), Ethernet controllers, and possibly other hardware devices, downloading the run-time image and copying it into RAM according to the binary image builder (BIB) layout, and jumping to the StartUp function. The last record of the run-time image contains this function's start address. The StartUp function then continues the boot process by calling the kernel initialization routines.
Although the various boot loader implementations differ in their complexity and the tasks they perform, there are common characteristics that Windows Embedded CE covers through static libraries to facilitate boot loader development, as illustrated in Figure 4-14. The resulting boot loader architecture influences how you can debug boot loader code. Chapter 5, "Customizing a Board Support Package," discusses boot loader development in more detail.
Figure 4-14 Windows Embedded CE boot loader architecture
The Windows Embedded CE boot loader architecture is based on the following code portions and libraries:
■ BLCOMMON Implements the basic boot loader framework for copying the boot loader from flash memory to RAM for faster execution, decoding image file contents, verifying checksums, and keeping track of load progress. BLCOMMON calls well-defined OEM functions throughout the process to handle hardware-specific customizations.
■ OEM code This is the code that OEMs must implement for their hardware platforms to support the BLCOMMON library.
■ Eboot Provides Dynamic Host Configuration Protocol (DHCP), Trivial File Transfer Protocol (TFTP), and User Datagram Protocol (UDP) services to download run-time images over Ethernet connections.
■ Bootpart Provides storage partitioning routines so that the boot loader can create a binary ROM image file system (BinFS) partition and a second partition with another file system on the same storage device. Bootpart can also create a boot partition to store boot parameters.
■ Network drivers Encapsulate the basic initialization and access primitives for a variety of common network controller devices. The interface for the libraries is generic so that both the boot loader and the OS can use the interface. The boot loader uses the interface for downloading run-time images and the OS uses the interface to implement a KITL connection to Platform Builder.
The boot loader design typically consists of at least two distinctive parts. The first part is written in assembly language and initializes the system before jumping to a second part written in C. If you are using a BLCOMMON-based architecture as illustrated in Figure 4-14, you might not have to debug assembly code. If your device is equipped with a UART, you can use the RETAILMSG macro in C code to send data over a serial output interface to the user for display.
Depending on whether you must debug assembly or C code, the following different debugging techniques are available:
■ Assembly code Common debugging techniques for the initial startup code rely on LEDs, such as a debugging board with seven-segment LEDs and UARTs for a serial communication interface, because it is relatively straightforward to access General Purpose Input/Output (GPIO) registers and modify the state of an input/output line.
■ C Code Debugging is much easier at the C-code level because you can access advanced communication interfaces and debugging macros.
■ Assembly and C code If a hardware debugger (JTAG probe) is available, you can use Platform Builder in conjunction with an eXDI driver to debug the boot loader.
To pass the certification exam, make sure you know the different techniques to debug the boot loader, kernel, device drivers, and applications.
Debugging the boot loader is a complex task that requires a good understanding of the hardware platform. If a hardware debugger is available, you can use Platform Builder in conjunction with an eXDI driver for hardware-assisted debugging. Otherwise, consider using an LED board for debugging assembly code and C-style macros to output debug messages over a serial communication interface in C code.
In this lab, you debug a console application added as a subproject to an OS design based on the Device Emulator BSP. To enable debugging, you include KdStub and KITL in the run-time image and configure corresponding target-device connectivity options. You then modify the source code of the console application to implement support for debug zones, specify initially active debug zones in the Pegasus registry key, and attach to the target device with the Kernel Debugger to examine the debug messages in the Output window of Visual Studio. Subsequently, you use the CETK to test the mouse driver included in the run-time image. To create the initial OS design in Visual Studio, follow the procedures outlined in Lab 1, "Creating, Configuring, and Building an OS Design."
To help you successfully master the procedures presented in this Lab, see the document "Detailed Step-by-Step Instructions for Lab 4" in the companion material for this book.
1. Open the OS design project created in Lab 1 in Visual Studio, right-click the OSDesign name and select Properties to edit the OS design properties, select Configuration Properties and then Build Options, and then select the Enable KITL check box for the run-time image.
2. In the OS Design property pages dialog box, also enable the Kernel Debugger feature, apply the changes, and then close the dialog box.
3. Verify that you are currently working in debug build configuration to build an image that contains the KITL and Kernel Debugger components activated in the previous steps.
4. Build the OS design by selecting Rebuild Current BSP and Subprojects under Advanced Build Commands on the Build menu (perform a Clean Sysgen if you encounter errors during subsequent steps).
5. Open the Target menu and click Connectivity Options to display the Target Device Connectivity Options dialog box. Configure the following settings, as shown in Table 4-6 and then click OK.
Table 4-6 Device connectivity settings
| Configuration Parameter | Setting |
|---|---|
| Download | Device Emulator (DMA) |
| Transport | Device Emulator (DMA) |
| Debugger | KdStub |
6. Add a subproject to the OS design and select the WCE Console Application template. Name the project TestDbgZones and select the option A Typical Hello World Application in the CE Subproject Wizard.
7. Add a new header file called DbgZone.h to the subproject and define the following zones:
#include
#define DEBUGMASK(n) (0x00000001<
#define MASK_INIT DEBUGMASK(0)
#define MASK_DEINIT DEBUGMASK(1)
#define MASK_ON DEBUGMASK(2)
#define MASK_ZONE3 DEBUGMASK(3)
#define MASK_ZONE4 DEBUGMASK(4)
#define MASK_ZONE5 DEBUGMASK(5)
#define MASK_ZONE6 DEBUGMASK(6)
#define MASK_ZONE7 DEBUGMASK(7)
#define MASK_ZONE8 DEBUGMASK(8)
#define MASK_ZONE9 DEBUGMASK(9)
#define MASK_ZONE10 DEBUGMASK(10)
#define MASK_ZONE11 DEBUGMASK(11)
#define MASK_ZONE12 DEBUGMASK(12)
#define MASK_FAILURE DEBUGMASK(13)
#define MASK_WARNING DEBUGMASK(14)
#define MASK_ERROR DEBUGMASK(15)
#define ZONE_INIT DEBUGZONE(0)
#define ZONE_DEINIT DEBUGZONE(1)
#define ZONE_ON DEBUGZONE(2)
#define ZONE_3 DEBUGZONE(3)
#define ZONE_4 DEBUGZONE(4)
#define ZONE_5 DEBUGZONE(5)
#define ZONE_6 DEBUGZONE(6)
#define ZONE_7 DEBUGZONE(7)
#define ZONE_8 DEBUGZONE(8)
#define ZONE_9 DEBUGZONE(9)
#define ZONE_10 DEBUGZONE(10)
#define ZONE_11 DEBUGZONE(11)
#define ZONE_12 DEBUGZONE(12)
#define ZONE_FAILURE DEBUGZONE(13)
#define ZONE_WARNING DEBUGZONE(14)
#define ZONE_ERROR DEBUGZONE(15)
8. Add an include statement for the DbgZone.h header file to the TestDbgZones.c file:
#include "DbgZone.h"
9. Define the dpCurSettings variable for the debug zones above the _tmain function, as follows:
DBGPARAM dpCurSettings = {
TEXT("TestDbgZone"), {
TEXT("Init"), TEXT("Deinit"), TEXT("On"), TEXT("n/a"),
TEXT("n/a"), TEXT("n/a"), TEXT("n/a"), TEXT("n/a"),
TEXT("n/a"), TEXT("n/a"), TEXT("n/a"), TEXT("n/a"),
TEXT("n/a"), TEXT("Failure"), TEXT("Warning"), TEXT("Error")
},
MASK_INIT | MASK_ON | MASK_ERROR
};
10. Register the debug zones of the module in the first line of the _tmain function:
DEBUGREGISTER(NULL);
11. Use the RETAILMSG and DEBUGMSG macros to display debug messages and associate them with debug zones, as follows:
DEBUGMSG(ZONE_INIT,
(TEXT("Message : ZONE_INIT")));
RETAILMSG(ZONE_FAILURE || ZONE_WARNING,
(TEXT("Message : ZONE_FAILURE || ZONE_WARNING")));
DEBUGMSG(ZONE_DEINIT && ZONE_ON,
(TEXT("Message : ZONE_DEINIT && ZONE_ON")));
12. Build the application, attach to the target device, and then start the application by using the Target Control window.
13. Note that only the first debug message is displayed in the debug Output window:
4294890680 PID:3c50002 TID:3c60002 Message : ZONE_INIT
14. Open the registry editor (Regedit.exe) on your development computer to activate the remaining debug zones, by default.
15. Open the HKEY_CURRENT_USER\Pegasus\Zones key and create a REG_DWORD value called TestDbgZone (according to the name of the module defined in the dpCurSettings variable).
16. Set the value to 0xFFFF to enable all 16 named zones, which correspond to the lower 16 bits in this 32 bit DWORD value (see Figure 4-15).
17. In Visual Studio, start the application again, and notice the following output:
4294911331 PID:2270006 TID:2280006 Message : ZONE_INIT
4294911336 PID:2270006 TID:2280006 Message : ZONE_FAILURE || ZONE_WARNING
4294911336 PID:2270006 TID:2280006 Message : ZONE_DEINIT && ZONE_ON
18. Change the TestDbgZone value in the registry to enable and disable different debug zones and verify the results in the Output window of Visual Studio.
Figure 4-15 HKEY_CURRENT_USER\Pegasus\Zones: "TestDbgZone"=dword:FFFF
You cannot control the debug zones for the TestDbgZone module in Platform Builder because the application process exits before you can open and modify the active zone for this module. You can only manage debug zones for loaded modules in Platform Builder, such as for graphical applications and DLLs.
1. Open the Windows CE Test Kit application from the Start menu on your development computer (open the Windows Embedded CE 6.0 menu and click Windows Embedded CE Test Kit).
2. In the Windows Embedded CE Test Kit window, open the Connection menu and click Start Client to establish a connection to the target device.
3. Click Connect and select the device in the Connection Manager window.
4. Verify that the workstation server application connects successfully to the device, deploys the required CETK binaries, detects available device drivers, and displays a list of all components in a hierarchical tree, as shown in Figure 4-16.
5. Right-click the Windows CE Test Catalog node and click Deselect All Tests.
6. Open each node in the list and select the Mouse Test check box.
7. Open the Test menu and then clock on Start/Stop Test to perform a mouse test.
8. On the target device perform the required mouse actions.
9. Complete the test and then can access the test report by right-clicking the test entry and selecting View Results.
10. Examine the results in the CETK parser and notice successful, skipped, and failed test procedures.
Figure 4-16 Device categories in the Windows Embedded CE Test Kit window
Platform Builder for Windows Embedded CE ships with a comprehensive set of debugging and testing tools to diagnose and eliminate root causes of errors, and validate the system in its final configuration prior to its release to production. The debugging tools integrate with Visual Studio and communicate over KITL connections with the target device. Alternatively, you can create a memory dump and use the CE Dump File Reader to debug the system in offline mode, which is particularly useful for postmortem debugging. The debugging environment is also extensible by means of eXDI drivers to perform hardware-assisted debugging beyond the capabilities of the standard Kernel Debugger.
The Kernel Debugger is a hybrid debugger for kernel components and applications. Debugging starts automatically if you attach to a target device with KdStub and KITL enabled. You can use the Target Control window to start applications for debugging and perform advanced system tests based on CEDebugX commands. However, it is important to keep in mind that you cannot set breakpoints in interrupt handlers or OAL modules because at these levels, the kernel operates in single-thread mode and stops communicating with the development workstation if code execution halts. To debug interrupt handlers, use a hardware debugger or debug messages. The debug messages feature supports debug zones to control the information output without having the rebuild the run-time image. You can also use debug messages to debug the C-code portion of a boot loader, yet for the assembly code portion you must use a hardware debugger or an LED panel.
KITL is also a requirement if you want to centralize system testing based on the CETK Test application, although it is also possible to run CETK tests in standalone mode. If you are developing a custom BSP for a target device, you can use the CETK to perform automated or semi-automated component tests based on custom Tux DLLs. Platform Builder includes a WCE TUX DLL template to create a skeleton Tux module that you can extend to meet your specific testing needs. You can integrate the custom Tux DLL in the CETK test application and run tests individually or as part of a larger test suite. Because all CETK tests use the same logging engine and log file format, you can use the same parser tool to analyze the results of default and user-defined tests.
Do you know what these key terms mean? You can check your answers by looking up the terms in the glossary at the end of the book.
■ Debug Zones
■ KITL
■ Hardware debugger
■ dpCurSettings
■ DebugX
■ Target Control
■ Tux
■ Kato
To help you successfully master the exam objectives presented in this chapter, complete the following tasks.
Add a subproject to the OS design for a console application that generates memory leaks by allocating memory blocks and never freeing them. Using the tools discussed in this chapter, isolate the issue and fix it.
Add a subproject to the OS design for a WCE TUX DLL. Build the Tux DLL and register it in the Windows Embedded CE Test Kit application. Run a CETK test and verify the test results. Set breakpoints in your Tux DLL and debug the code by running a CETK test in standalone mode.