Recommend Using Collections.Pooled for .NET Performance Optimization

Introduction

Performance optimization is about how to handle the same number of requests while occupying fewer resources. These resources are usually CPU or memory, but also include OS I/O handles, network traffic, disk usage, etc. However, most of the time, we are simply trying to reduce CPU and memory usage.

Previously shared content had some limitations, making it difficult to directly apply. Today, I want to share a simple method – just replacing a few collection types can achieve performance improvements and reduce memory usage.

I will introduce a library called Collections.Pooled. As the name suggests, it reduces memory usage and GC pressure by pooling memory. We will directly look at its performance and also examine its source code to understand why it brings these performance improvements.

Collections.Pooled

Project link: https://github.com/jtmueller/Collections.Pooled

This library is based on the classes in System.Collections.Generic, which have been modified to leverage the new System.Span<T> and System.Buffers.ArrayPool<T> libraries to reduce memory allocation, improve performance, and enable greater interoperability with modern APIs.

Collections.Pooled supports .NET Standard 2.0 (.NET Framework 4.6.1+) and includes optimized builds targeting .NET Core 2.1+. A comprehensive set of unit tests and benchmarks have been ported from corefx.

Total tests: 27501. Passed: 27501. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 9.9019 seconds

How to Use

Installing this library via NuGet is straightforward. NuGet Version.

Install-Package Collections.Pooled
dotnet add package Collections.Pooled
paket add Collections.Pooled

In the Collections.Pooled library, pooled versions of commonly used collection types are implemented. The comparison with the native .NET types is as follows:

When using, simply replace the .NET native version with the Collections.Pooled version, as shown in the code below:

using Collections.Pooled;

// Usage is the same
var list = new List<int>();
var pooledList = new PooledList<int>();

var dictionary = new Dictionary<int,int>();
var pooledDictionary = new PooledDictionary<int,int>();

// Including PooledSet, PooledQueue, PooledStack, usage is the same

var pooledList1 = Enumerable.Range(0,100).ToPooledList();
var pooledDictionary1 = Enumerable.Range(0,100).ToPooledDictionary(i => i, i => i);

However, note that Pooled types implement the IDisposable interface. They return the used memory to the pool via the Dispose() method, so you need to call the Dispose() method after using the Pooled collection object. Alternatively, you can directly use the using var keyword.

using Collections.Pooled;

// Using 'using var' will automatically dispose the pooled object when it goes out of scope
using var pooledList = new PooledList<int>();
Console.WriteLine(pooledList.Count);

// Using a 'using' block; it gets disposed when the block ends
using (var pooledDictionary = new PooledDictionary<int, int>())
{
    Console.WriteLine(pooledDictionary.Count);
}

// Manually calling Dispose method
var pooledStack = new PooledStack<int>();
Console.WriteLine(pooledStack.Count);
pooledList.Dispose();

Note: It is best to dispose of the collection objects inside Collections.Pooled. However, if you forget, GC will eventually collect them, but they won't be returned to the pool, thus failing to save memory.

Since it reuses memory space, when the memory space is returned to the pool, the elements inside the collection need to be handled. It provides an enum called ClearMode for this purpose, defined as follows:

namespace Collections.Pooled
{
    /// <summary>
    /// This enum controls how data is handled when the internal array is returned to the ArrayPool.
    /// Before using any option other than the default, please carefully understand the effect of each option.
    /// </summary>
    public enum ClearMode
    {
        /// <summary>
        /// <para><code>Auto</code> has different behavior depending on the target framework.</para>
        /// <para>.NET Core 2.1: Reference types and value types containing references are cleared when the internal array is returned to the pool. Value types that do not contain references are not cleared when returned.</para>
        /// <para>.NET Standard 2.0: All user types are cleared before returning to the pool in case they contain reference types. For .NET Standard, Auto and Always have the same behavior.</para>
        /// </summary>
        Auto = 0,

        /// <summary>
        /// <para><code>Always</code> means user types are always cleared before returning to the pool.</para>
        /// </summary>
        Always = 1,

        /// <summary>
        /// <para><code>Never</code> means the pooled collection will never clear user types before returning them to the pool.</para>
        /// </summary>
        Never = 2
    }
}

Under default conditions, using Auto is sufficient. If there are special performance requirements and the risks are understood, you can use Never.

For reference types and value types that contain references, we must clear the array references when returning the memory space to the pool. If not cleared, GC cannot release that memory space (because the pool still holds references to the elements). For pure value types, clearing is unnecessary. In the article Using Struct Instead of Class, I described the storage differences between reference types and structs (value types). Pure value types have no object header, so GC intervention is not needed for deallocation.

Performance Comparison

I did not create separate benchmarks; I used the benchmark results from the open source project. Many projects show 0 memory usage because they use pooled memory with no extra allocations.

PooledList<T>

In the benchmark, 2048 elements are added to the collection in a loop. The native .NET List<T> takes 110us (according to the actual benchmark results, the millisecond in the figure should be a typo) and 263KB memory, while PooledList<T> takes only 36us and 0KB memory.

PooledDictionary<TKey, TValue>

In the benchmark, 100,000 elements are added to the dictionary in a loop. The native Dictionary<TKey, TValue> takes 11ms and 13MB memory, while PooledDictionary<TKey, TValue> takes only 7ms and 0MB memory.

PooledSet<T>

In the benchmark, 100,000 elements are added to the hash set in a loop. The native HashSet<T> takes 5348ms and 2MB, while PooledSet<T> takes only 4723ms and 0MB memory.

PooledStack<T>

In the benchmark, 100,000 elements are pushed onto the stack in a loop. The native Stack<T> takes 1079ms and 2MB, while PooledStack<T> takes only 633ms and 0MB memory.

PooledQueue<T>

In the benchmark, 100,000 elements are enqueued in a loop. The native Queue<T> takes 681ms and 1MB, while PooledQueue<T> takes only 408ms and 0MB memory.

Scenario Without Manual Dispose

As mentioned earlier, Pooled collection types should be disposed, but it's not a big problem if you forget, because GC will collect them.

private static readonly string[] List = Enumerable
    .Range(0, 10000).Select(c => c.ToString()).ToArray();

[Benchmark(Baseline = true)]
public int UseList()
{
    var list = new List<string>(1024);
    for (var index = 0; index < List.Length; index++)
    {
        var item = List[index];
        list.Add(item);
    }
    return list.Count;
}

[Benchmark]
public int UsePooled()
{
    using var list = new PooledList<string>(1024);
    for (var index = 0; index < List.Length; index++)
    {
        var item = List[index];
        list.Add(item);
    }
    return list.Count;
}

[Benchmark]
public int UsePooledWithOutUsing()
{
    var list = new PooledList<string>(1024);
    for (var index = 0; index < List.Length; index++)
    {
        var item = List[index];
        list.Add(item);
    }
    return list.Count;
}

Benchmark results:

From the above benchmark results, we can conclude:

• Promptly disposing the Pooled collection type triggers almost no GC and memory allocation – from the chart, it allocated only 56 bytes.
• Even without disposing the Pooled collection type, it still reuses memory from the pool during resize operations, and skips the memory allocation initialization step, making it faster.
• The slowest option is using the normal collection type; each resize operation requires allocating new memory space, and GC also needs to reclaim the previous memory space.

Principle Analysis

If you have read my previous blog posts You Should Set Initial Capacity for Collections and A Brief Analysis of C# Dictionary Implementation, you know that for high-performance random access, the underlying data structures of these basic collection types are arrays. Taking List<T> as an example:

• A new array is created to store the added elements.
• If the array space is insufficient, a resize operation is triggered, allocating twice the space.

The constructor code is as follows, showing that a generic array is directly created:

public List(int capacity)
{
    if (capacity < 0)
        ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.capacity, ExceptionResource.ArgumentOutOfRange_NeedNonNegNum);

    if (capacity == 0)
        _items = s_emptyArray;
    else
        _items = new T[capacity];
}

So, to pool memory, we simply replace the places where the new keyword is used in the library with pooled allocations. Here I want to introduce a type from .NET BCL called ArrayPool. It provides a pool of reusable generic array instances, reducing pressure on GC and improving performance when arrays are frequently created and destroyed.

The Pooled types internally use ArrayPool to share the resource pool. From their constructors, we can see that by default they use ArrayPool<T>.Shared to allocate array objects. Of course, you can create your own ArrayPool for them to use.

// Default uses ArrayPool<T>.Shared pool
public PooledList(int capacity, ClearMode clearMode, bool sizeToCapacity) : this(capacity, clearMode, ArrayPool<T>.Shared, sizeToCapacity) { }

// Uses ArrayPool for array allocation
public PooledList(int capacity, ClearMode clearMode, ArrayPool<T> customPool, bool sizeToCapacity)
{
    if (capacity < 0)
        ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.capacity, ExceptionResource.ArgumentOutOfRange_NeedNonNegNum);
    _pool = customPool ?? ArrayPool<T>.Shared;
    _clearOnFree = ShouldClear(clearMode);
    if (capacity == 0)
    {
        _items = s_emptyArray;
    }
    else
    {
        _items = _pool.Rent(capacity);
    }

    if (sizeToCapacity)
    {
        _size = capacity;
        if (clearMode != ClearMode.Never)
        {
            Array.Clear(_items, 0, _size);
        }
    }
}

During capacity adjustment (resizing), the old array is returned to the pool, and the new array is also obtained from the pool.

public int Capacity
{
    get => _items.Length;
    set
    {
        if (value < _size)
        {
            ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.value, ExceptionResource.ArgumentOutOfRange_SmallCapacity);
        }

        if (value != _items.Length)
        {
            if (value > 0)
            {
                // Rent from the pool
                var newItems = _pool.Rent(value);
                if (_size > 0)
                {
                    Array.Copy(_items, newItems, _size);
                }
                // Return old array to the pool
                ReturnArray();
                _items = newItems;
            }
            else
            {
                ReturnArray();
                _size = 0;
            }
        }
    }
}

private void ReturnArray()
{
    if (_items.Length == 0)
        return;
    try
    {
        // Return to pool
        _pool.Return(_items, clearArray: _clearOnFree);
    }
    catch (ArgumentException)
    {
        // ArrayPool may throw exceptions, we swallow them
    }
    _items = s_emptyArray;
}

Additionally, the author used Span to optimize APIs such as Add, Insert, etc., providing better random access performance. They also added TryXXX series APIs for easier usage. For instance, List<T> has over 170 modifications compared to PooledList<T>.

Conclusion

In our actual production usage, we can completely replace the native collection types with the pooled versions provided by Collections.Pooled. This greatly helps in reducing memory usage and P95 latency.

Moreover, even if you forget to dispose of them, the performance is not much worse than using native collection types. However, the best practice is to dispose of them promptly.