Slaying the Virtual Memory Monster (Large Object Heap Fragmentation Edition)

If you’re interested in the equivalence of Stephen King’s "It" in the world of software programming, have a look at ReedR’s Slaying the Virtual Memory Monster post. It’s really scary.

This week I’m starting to face a similar monster and that is Large Object HeapManaged Address Space Fragmentation, as aptly described in Andrew Hunter’s The Dangers of the Large Object Heap.

However, just like "It" lurking in sewers and dark waters, my monster hides behind the 3rd-party curtain, in the disguise of the Microsoft Report Viewer Redistributable controls.

 

SYMPTOM:

After displaying a couple of large server reports, where "large" means many Custom Report Items, i.e. bitmaps, rendering fails with "Exception of type System.OutOfMemoryException was thrown."

Investigation shows the exception occurs on the client:

0:017> !CLRStack -p
OS Thread Id: 0x1bc (17)
ESP       EIP     
0f71ea54 7c90e514 [HelperMethodFrame: 0f71ea54] 
0f71eac0 792ec8d0 System.IO.MemoryStream.set_Capacity(Int32)
    PARAMETERS:
        this = 0x11b50308
        value = 0x10384000

0f71ead4 792ec868 System.IO.MemoryStream.EnsureCapacity(Int32)
    PARAMETERS:
        this = <no data>
        value = <no data>

0f71eae0 792ec7b6 System.IO.MemoryStream.Write(Byte[], Int32, Int32)
    PARAMETERS:
        this = 0x11b50308
        buffer = 0x11b6d45c
        offset = 0x00000000
        count = 0x0000fe53

0f71eb10 09c4a8d3 Microsoft.Reporting.WinForms.ServerReport.ServerUrlRequest(Boolean, System.String, System.IO.Stream, System.String ByRef, System.String ByRef)
    PARAMETERS:
        this = <no data>
        isAbortable = <no data>
        url = <no data>
        outputStream = 0x11b50308
        mimeType = <no data>
        fileNameExtension = <no data>

0f71eb54 09c4a72a Microsoft.Reporting.WinForms.ServerReport.InternalRender(Boolean, System.String, System.String, System.Collections.Specialized.NameValueCollection, System.IO.Stream, System.String ByRef, System.String ByRef)
    PARAMETERS:
        this = 0x0d7a956c
        isAbortable = <no data>
        format = <no data>
        deviceInfo = <no data>
        urlAccessParameters = <no data>
        reportStream = <no data>
        mimeType = <no data>
        fileNameExtension = <no data>

 

CAUSE:

The process’ address space is fragmented to the point that a large object (byte[272121856]) cannot be allocated because the longest contiguous block is just little over 120 MB.

0:017> !address -summary

-------------------- Usage SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots) Pct(Busy)   Usage
   37942000 (  910600) : 43.42%    80.68%    : RegionUsageIsVAD
   3b1b8000 (  968416) : 46.18%    00.00%    : RegionUsageFree
    ab10000 (  175168) : 08.35%    15.52%    : RegionUsageImage
    1300000 (   19456) : 00.93%    01.72%    : RegionUsageStack
      13000 (      76) : 00.00%    00.01%    : RegionUsageTeb
    16d0000 (   23360) : 01.11%    02.07%    : RegionUsageHeap
          0 (       0) : 00.00%    00.00%    : RegionUsagePageHeap
       1000 (       4) : 00.00%    00.00%    : RegionUsagePeb
       1000 (       4) : 00.00%    00.00%    : RegionUsageProcessParametrs
       1000 (       4) : 00.00%    00.00%    : RegionUsageEnvironmentBlock
       Tot: 7fff0000 (2097088 KB) Busy: 44e38000 (1128672 KB)

-------------------- Type SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots)  Usage
   3b1b8000 (  968416) : 46.18%   : <free>
    cda4000 (  210576) : 10.04%   : MEM_IMAGE
    13e0000 (   20352) : 00.97%   : MEM_MAPPED
   36cb4000 (  897744) : 42.81%   : MEM_PRIVATE

-------------------- State SUMMARY --------------------------
    TotSize (      KB)   Pct(Tots)  Usage
   3cf34000 (  998608) : 47.62%   : MEM_COMMIT
   3b1b8000 (  968416) : 46.18%   : MEM_FREE
    7f04000 (  130064) : 06.20%   : MEM_RESERVE

Largest free region: Base 45fd0000 - Size 07e20000 (129152 KB)

 

RESOLUTION:

I’m not in control of the many byte arrays used to read the various streams that make up a rendered report. However, I shall endeavor a thorough investigation into all the chunks that stain my memory blanket to see how I would possibly be able to contain the same.

To be continued …

Advertisements
This entry was posted in Coding Horror. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s