Development C# Architecture

Object Pooling and Memory Management in High-Performance .NET

The .NET garbage collector is very capable, right up until a workload exposes its limits. In data-intensive applications, careless allocation patterns generate GC pauses that erode throughput predictability. The fix isn't to eliminate allocations entirely (that tends to produce brittle, unreadable code), but to eliminate the ones that serve no purpose.

Over years of building production .NET systems that process thousands of records per second, I have settled on a set of memory management patterns that consistently deliver results: object pooling, Span<T>, ArrayPool<T>, Large Object Heap awareness, and continuous memory monitoring. This post walks through each of them with concrete code and the reasoning behind the choices.

Performance Targets

The first step in any performance effort is defining what "fast enough" actually means. Without explicit targets, optimization becomes an exercise in diminishing returns with no clear stopping point.

public static class PerformanceTargets
{
    public const int MinRecordsPerSecond = 5000;
    public const int MaxProcessingLatencyMs = 100;
    public const long MaxMemoryUsageBytes = 2L * 1024 * 1024 * 1024; // 2 GB
    public const double MaxGCTimePercent = 5.0;
}

Encoding these targets as constants makes performance regressions testable in the CI pipeline:

[TestMethod]
[TestCategory("Performance")]
public async Task ProcessBatch_MeetsMinimumThroughput()
{
    var stopwatch = Stopwatch.StartNew();
    var result = await _processor.ProcessBatchAsync(_testRecords);
    stopwatch.Stop();

    double recordsPerSecond = _testRecords.Count / stopwatch.Elapsed.TotalSeconds;
    Assert.IsTrue(recordsPerSecond > PerformanceTargets.MinRecordsPerSecond,
        $"Throughput {recordsPerSecond:F0} r/s below target {PerformanceTargets.MinRecordsPerSecond}");
}

When the test goes green, that's the signal to stop tuning. When it goes red, it points straight at what regressed and by how much.

Object Pooling

The highest-impact optimization in allocation-heavy code is pooling objects that are created and discarded in tight loops. Rather than letting the garbage collector clean up after every iteration, the caller rents an object, uses it, and returns it for the next caller.

StringBuilder Pooling

StringBuilder is the textbook case. In a loop that formats thousands of records, allocating a fresh builder each time creates measurable GC pressure:

// Before: a new StringBuilder is allocated per iteration
foreach (var record in records)
{
    var sb = new StringBuilder();  // GC pressure
    sb.Append(record.Name);
    sb.Append(',');
    sb.Append(record.Value);
    output.Add(sb.ToString());
}

// After: a pooled StringBuilder is rented and returned each iteration
private static readonly ObjectPool<StringBuilder> _sbPool =
    new DefaultObjectPoolProvider().CreateStringBuilderPool();

foreach (var record in records)
{
    var sb = _sbPool.Get();
    try
    {
        sb.Append(record.Name);
        sb.Append(',');
        sb.Append(record.Value);
        output.Add(sb.ToString());
    }
    finally
    {
        _sbPool.Return(sb);  // Cleared and returned to the pool
    }
}

The Microsoft.Extensions.ObjectPool library provides CreateStringBuilderPool(), which automatically clears the builder's contents on return. For custom domain objects, implement IPooledObjectPolicy<T> to define the reset logic.

ArrayPool for Temporary Buffers

Any code that allocates temporary byte arrays (file I/O, serialization, network reads) benefits from ArrayPool<T>:

// Before: allocates a fresh array on every call
public byte[] ReadChunk(Stream stream, int size)
{
    var buffer = new byte[size];  // Allocated, used once, collected
    stream.Read(buffer, 0, size);
    return buffer;
}

// After: rents a buffer from the shared pool
public void ProcessChunk(Stream stream, int size)
{
    byte[] buffer = ArrayPool<byte>.Shared.Rent(size);
    try
    {
        int bytesRead = stream.Read(buffer, 0, size);
        ProcessData(buffer.AsSpan(0, bytesRead));
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(buffer, clearArray: true);
    }
}

One subtlety matters here. Rent may return a buffer larger than requested, so always track the actual number of bytes read separately rather than assuming the buffer length matches the requested size. The clearArray: true parameter zeros the buffer on return, which matters when processing security-sensitive data.

The 80KB Buffer

For file operations, I consistently use an 80KB buffer. It's large enough for efficient sequential I/O, yet small enough to stay below the Large Object Heap threshold of 85,000 bytes:

private const int FileBufferSize = 81_920; // 80 KB -- just under the LOH threshold

public async Task ProcessFileAsync(string path)
{
    byte[] buffer = ArrayPool<byte>.Shared.Rent(FileBufferSize);
    try
    {
        await using var stream = new FileStream(path, FileMode.Open,
            FileAccess.Read, FileShare.Read, FileBufferSize,
            FileOptions.SequentialScan | FileOptions.Asynchronous);

        int bytesRead;
        while ((bytesRead = await stream.ReadAsync(buffer)) > 0)
        {
            ProcessChunk(buffer.AsSpan(0, bytesRead));
        }
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(buffer);
    }
}

This buffer size avoids LOH allocation while still being large enough that the operating system can issue efficient read-ahead operations.

Span<T>: Zero-Copy Slicing

Span<T> works with slices of arrays and strings without allocating new objects on the heap. The canonical example is CSV parsing, where String.Split allocates a new string for every field:

// Before: String.Split allocates a new string per field
string[] fields = line.Split(',');  // N allocations

// After: Span-based parsing with zero heap allocations
public static void ParseCsvLine(ReadOnlySpan<char> line, Span<Range> fields)
{
    int fieldIndex = 0;
    int start = 0;

    for (int i = 0; i < line.Length; i++)
    {
        if (line[i] == ',')
        {
            fields[fieldIndex++] = start..i;
            start = i + 1;
        }
    }
    fields[fieldIndex] = start..line.Length;
}

Each Range is just a pair of integers with no heap allocation and no GC involvement. The caller accesses individual fields via line[fields[0]], which returns a ReadOnlySpan<char> that's a view into the original data.

Because Span<T> is a stack-only type, it can't be used across await boundaries. For asynchronous code, use ReadOnlyMemory<T> instead, which provides the same slicing semantics with heap safety:

public async Task ProcessAsync(ReadOnlyMemory<byte> data)
{
    // Memory<T> is heap-safe; Span<T> is stack-only
    await ProcessHeaderAsync(data.Slice(0, HeaderSize));
    await ProcessBodyAsync(data.Slice(HeaderSize));
}

The Large Object Heap Problem

Objects larger than 85,000 bytes are allocated on the Large Object Heap, which is only collected during Generation 2 garbage collection, the most expensive kind. Worse, the LOH isn't compacted by default, so fragmentation can cause memory bloat even when the total number of live objects is small.

A few rules of thumb keep LOH allocation in check:

// Capture GC generation counts before and after a workload
int gen0Before = GC.CollectionCount(0);
int gen1Before = GC.CollectionCount(1);
int gen2Before = GC.CollectionCount(2);

// ... run workload ...

int gen0After = GC.CollectionCount(0);
int gen2After = GC.CollectionCount(2);

// Gen 2 collections during a batch are a strong signal of LOH pressure
if (gen2After > gen2Before)
{
    _logger.LogWarning("Gen2 GC during batch processing -- check for LOH allocations");
}

Continuous Memory Monitoring

In long-running services, memory leaks are slow-motion disasters that only become visible when the process is approaching its memory limit. A simple timer-based monitor catches growth trends early:

public class MemoryMonitor : IDisposable
{
    private readonly Timer _timer;
    private long _lastMemoryUsage;

    public MemoryMonitor(TimeSpan interval)
    {
        _lastMemoryUsage = GC.GetTotalMemory(false);
        _timer = new Timer(CheckMemory, null, interval, interval);
    }

    private void CheckMemory(object? state)
    {
        long current = GC.GetTotalMemory(false);

        if (current > _lastMemoryUsage * 1.5)
        {
            // 50% growth since the last check warrants investigation
            _logger.LogWarning(
                "Memory spike: {Previous}MB -> {Current}MB",
                _lastMemoryUsage / (1024 * 1024),
                current / (1024 * 1024));
        }

        _lastMemoryUsage = current;
    }

    public void Dispose() => _timer.Dispose();
}

The 50% threshold is a starting point. Tighten it for services with stable, predictable workloads, and loosen it for batch processors whose memory consumption naturally fluctuates. The important thing is having any monitoring at all, so that a leak surfaces as a warning in the logs rather than as an out-of-memory crash in production.

Batch Processing for Predictable Memory Usage

Processing records one at a time suffers from poor cache locality. Processing an entire dataset at once risks exhausting available memory. Batching provides a middle ground that keeps memory usage bounded while maintaining reasonable throughput:

public async IAsyncEnumerable<ProcessResult> ProcessInBatchesAsync(
    IAsyncEnumerable<Record> records,
    int batchSize = 1000)
{
    var batch = new List<Record>(batchSize);

    await foreach (var record in records)
    {
        batch.Add(record);

        if (batch.Count >= batchSize)
        {
            yield return await ProcessBatchAsync(batch);
            batch.Clear();  // Reuse the list -- no reallocation
        }
    }

    if (batch.Count > 0)
    {
        yield return await ProcessBatchAsync(batch);
    }
}

Several details matter here. Pre-allocating the list with new List<Record>(batchSize) avoids the repeated resizing that occurs when the list grows dynamically. Calling batch.Clear() retains the internal array, so the next batch reuses the same memory. And returning results via IAsyncEnumerable lets the caller consume them incrementally rather than waiting for the entire dataset to finish processing.

Channel-Based Producer-Consumer Pipelines

For concurrent processing scenarios that require backpressure, System.Threading.Channels is the modern replacement for BlockingCollection<T>:

public async Task ProcessWithBackpressureAsync(
    IAsyncEnumerable<Record> source,
    int maxBuffer = 100)
{
    var channel = Channel.CreateBounded<Record>(new BoundedChannelOptions(maxBuffer)
    {
        FullMode = BoundedChannelFullMode.Wait,
        SingleReader = true,
        SingleWriter = true
    });

    // Producer
    var producer = Task.Run(async () =>
    {
        await foreach (var record in source)
        {
            await channel.Writer.WriteAsync(record);
        }
        channel.Writer.Complete();
    });

    // Consumer
    await foreach (var record in channel.Reader.ReadAllAsync())
    {
        await ProcessRecordAsync(record);
    }

    await producer;
}

The bounded channel with FullMode.Wait handles backpressure automatically. If the consumer falls behind, the producer pauses at WriteAsync until space becomes available. No manual synchronization, no risk of unbounded memory growth from a fast producer overwhelming a slow consumer.

Combining the Patterns

Here is a data processing method that brings several of these techniques together:

public async Task<ProcessResult> ProcessLargeDatasetAsync(
    IAsyncEnumerable<Record> records)
{
    var sb = _sbPool.Get();
    byte[] buffer = ArrayPool<byte>.Shared.Rent(FileBufferSize);
    long totalProcessed = 0;

    try
    {
        await foreach (var record in records)
        {
            sb.Clear();
            FormatRecord(record, sb);  // Reuse the pooled StringBuilder

            ReadOnlySpan<char> formatted = sb.ToString();
            // ... process formatted data ...

            totalProcessed++;

            if (totalProcessed % 50_000 == 0)
            {
                long memory = GC.GetTotalMemory(false);
                _logger.LogInformation(
                    "Processed {Count} records, memory: {MB}MB",
                    totalProcessed, memory / (1024 * 1024));
            }
        }

        return new ProcessResult(totalProcessed);
    }
    finally
    {
        _sbPool.Return(sb);
        ArrayPool<byte>.Shared.Return(buffer);
    }
}

A pooled StringBuilder, a rented byte buffer, periodic memory logging, and try/finally blocks that guarantee resources are returned. None of this is exotic. It's disciplined resource management applied consistently.

Summary

Technique When to Use Impact
Object pooling Tight loops creating and discarding objects Eliminates GC pressure
ArrayPool<T> Temporary byte or array buffers Avoids LOH allocations
Span<T> Parsing, slicing, zero-copy operations Zero heap allocation
Memory<T> Same as Span but across await boundaries Zero allocation (async-safe)
80KB buffer limit File I/O buffer sizing Avoids LOH
Memory monitoring Long-running services Catches leaks early
Batch processing High-volume data pipelines Predictable memory usage
Channels Producer-consumer workflows Built-in backpressure

The diagnosis is straightforward. Unusually frequent Gen 0 collections point to an allocation problem. Frequent Gen 2 collections point to a LOH problem. Memory that grows without bound points to a leak. The patterns above address all three, and the best starting point is always profiling to identify which one is actually in play before reaching for any of them.

References