Writing Mobile Code Essential Software Engineering for Building Mobile Applications [Electronic resources]

Memory Management and Garbage Collection

When running through all of its normal execution paths, your code will cause type definitions to get loaded into memory, code to get compiled and executed and objects to get allocated and discarded. The .NET Compact Framework was designed to provide the best balance between startup performance, steady-state performance, and the need to continue executing code even in highly memory-constrained environments. To do this, the class loader, JIT engine, memory allocator, and garbage collector work together.

As your code is running, it will likely be creating new objects and discarding these objects periodically. To handle the need for object allocation and cleanup efficiently, the .NET Compact Framework uses garbage collection. Most modern managed-code environments use some form of garbage collection.

A garbage collector fundamentally is designed to do two things: (1) reclaim memory that is no longer being used; and (2) compact the pieces of memory being used so that the largest blocks of continuous memory are available for new allocations. This solves a common problem called memory fragmentation. Memory fragmentation is similar in concept to disk fragmentation. If no reorganization of storage takes place periodically, after a time the storage area becomes disorganized with small free and occupied storage spaces scattered; these are fragments. Think of this as entropy applied to disk and memory storage; active work is required to reverse these entopic effects. The result of disk fragmentation is slow access times because lots of hopping around the disk is required to read a file. Fragmentation in memory is even worse because object allocation requires a continuous block of memory of sufficient size for the object's data. After a large number of allocations and releases of different-sized chunks of memory, the free pieces of memory are too small and scattered to allow the larger allocation sizes required by objects. Compacting all the memory that is currently in use but scattered can create large blocks of free memory that can efficiently be used for new object allocations.

Reclamation of memory is a relatively straightforward process. Execution is temporarily suspended and the set of live objects (objects that can be reached directly and indirectly by your code) is traced down and recursively marked. The rest of the objects in memory, because they are no longer reachable by live code, remain unmarked and so can be identified as garbage objects and can be reclaimed. This kind of garbage collection is known as "mark and sweep." Typically this operation is fairly quick.

The ability to compact the objects in memory is an advanced benefit of managed-code execution. Unlike native code, all known references to your objects are known to the execution engine. This allows objects to be moved around in memory when it becomes useful to do so. The manager of the memory (the execution engine) keeps track of which objects are where and can move them around when it needs to. In this way, a set of managed objects scattered around in memory can be compacted.

The .NET Compact Framework, with its JIT compiler, memory manager, and garbage collector, also has one additional trick up its sleeve: the capability to garbage collect JITed code. Normally this is not a desirable thing to do, but under special circumstances it can prove very valuable. If the execution engine has gone to the effort of JIT compiling the managed code to native instructions in order to achieve maximum execution performance, throwing out the JITed code seems wasteful because it will need to re-JIT the code the next time it needs to execute. It is, however, beneficial to be able to do this under two circumstances:

When the application has JITed and run a lot of code that will not need to be executed again any time soon. A common case for this is when an application runs in different stages. Code that executes in the beginning of the application to set things up may never need to be run again. If memory is needed, it makes sense to reclaim the memory that holds this code.

When the set of live objects in memory is so large that the application execution will fail if more memory cannot be found for additional object allocations that need to occur during the normal execution of the application's algorithms. In this case, the execution engine must be willing to throw out and periodically recompile code the application needs just so the application can keep running. If the alternative is the application halting execution because there is no memory left, throwing out JITed code is the only solution, even if it means wasting time recompiling code later.

Memory Management and Garbage Collection Walkthrough

It is useful to understand how the execution engine interacts with your application's code and objects during execution. The following set of schematic diagrams walk through the memory management that occurs during different application execution stages.

Things to observe in the schematic above include the following:

The size of any item in the schematic is meant to give an idea of relative memory usage. Larger classes and types use more memory than smaller types because they contain more "stuff." Objects of different types use different amounts of memory. Dead objects use up memory until they are garbage collected. Un-JITed methods take up very little space.

All of the definitions for the types and classes your application is using are loaded into memory. Different types and classes take up different amounts of memory.

A class' methods that have been called at least once have been JIT compiled. Example: ClassXYZ has Code1, Code2, Code3, and Code7 JIT compiled.

Classes' methods that have not been called yet are not yet compiled and therefore do not take up much memory. Example: ClassXYZ has two methods, Code4 and Code5 that exist in pre-JITed state. If they are called, they will be JIT compiled and memory will be allocated to store the compiled code.

Objects are instances of types and classes and take up memory.

"Live objects" are objects that can be reached by your code either directly through global and static variables, variables on the stack, or through these variables.

"Dead objects" are objects that can no longer be reached by your code but have not yet been reclaimed by the execution engine. These are represented as Obj3, Obj6, Obj9, and Obj11 in Figure 3.2. Until they are cleaned up, they take up memory just like live objects and JITed code.

As your application goes about creating and discarding objects and other heap-allocated types, eventually it will hit a point where no additional objects can be created without cleaning up the dead objects. At this point, the execution engine will force a garbage collection. Figure 3.3 shows the application's memory state right before garbage collection.

When a garbage collection occurs, the live objects can also be compacted. It is often possible to free up a good deal of continuous memory for the creation of new objects by removing the dead objects from memory and compacting live objects. Figure 3.4 shows the application's memory state right after garbage collection and compaction.

In normal steady-state execution, objects are periodically created and discarded. The execution engine does a garbage collection and memory compaction when needed to free up memory required for new objects. Figure 3.5 shows a typical application's memory state with a mix of live and dead objects, JITed and un-JITed code, and some spare memory to allocate new objects in.

In well-performing applications, there exists ample memory for the creation and discarding of the objects, and garbage collection does not need to occur very often. The amount of space recovered when garbage collection occurs is sufficient to let the application continue to create and discard all the objects it needs for a good while. Figure 3.6 shows a typical application's memory state right after a periodic garbage collection.

In some cases, the large number of live objects being used makes it impossible for a simple "mark and sweep" garbage collection of discarded objects to free up enough memory to allow the execution engine to create the new objects demanded by the application's running code. If memory cannot be found for required object allocations, the application execution must be terminated due to "out-of-memory" problems. Figure 3.7 shows an application that has reached this point.

To address this situation, the .NET Compact Framework is capable of releasing a large amount of the currently JITed code. All code that is not executing on the stack (or stacks in the case of multiple threads) can be released. Doing this allows memory to be reclaimed that can be used to meet the application's demands for new objects or the new JITing of new code in the case of a method that has never been run before but has now been called. Figure 3.8 shows the application's memory state after previously JITed code has been thrown out and garbage collected.

The discarding of JITed code is a drastic step because some of this code will need to be re-JITed, but it can be a very effective strategy if a significant amount of code that was previously JITed is no longer required for execution. This is often the case when a significant amount of "startup" code was run to set things up or if the application is partitioned into logical blocks that do not all need to execute at the same time. Figure 3.9 shows the application's memory state shortly after tossing out all the JITed code it can. Methods are re-JITed as they are called and new objects can be allocated as needed.

Severe performance problems will occur if the application continues to allocate more and more objects and not discard them. This will cause a condition where the only thing that can be kept in memory is all of the live objects and the code that is immediately executing on the application's stack (or "stacks" if there are multiple threads). In this case common code is continually JITed and discarded because only a small portion of the system's memory is available to hold the JITed code. At this point the system is in constant churn; this is commonly referred to as "thrashing" because the application and run-time are inefficiently struggling just to keep incrementally running the next piece of code. As the application asymptotically approaches this state application performance will decline drastically. This is a state to be avoided! Figure 3.10 shows an application caught in such as state.