C# 7 Series, Part 10: Span and universal memory management

  • Article

Part 1: Value Tuples
Part 2: Async Main
Part 3: Default Literals
Part 4: Discards
Part 5: Private Protected
Part 6: Read-only structs
Part 7: Ref Returns
Part 8: “in” Parameters
Part 9: ref structs
Part 10: (This post) Span<T> and universal memory management

Background

.NET is a managed platform, that means the memory access and management is safe and automatic. All types are fully managed by .NET, it allocates memory either on the execution stacks, or managed heaps.

In the event of interop or low-level development, you may want the access to the native objects and system memory, here is why the interop part comes, there are types that can marshal into the native world, invoke native APIs, convert managed/native types and define a native structure from the managed code.

Problem 1: Memory access patterns

In .NET world, there are three types of memory you may be interested:

  • Managed heap memory, such as an array;
  • Stack memory, such as objects created by stackalloc;
  • Native memory, such as a native pointer reference.

Each type of memory access may need to use language features that are designed for it:

  • To access heap memory, use the fixed (pinned) pointer on supported types (like string), or use other appropriate .NET types that have access to it, such as an array or a buffer;
  • To access stack memory, use pointers with stackalloc;
  • To access unmanaged system memory, use pointers with Marshal APIs.

You see, different access pattern needs different code, no single built-in type for all contiguous memory access.

Problem 2: Performance

In many applications, the most CPU consuming operations are string operations. If you run a profiler session against your application, you may find the fact that 95% of the CPU time is used to call string and related functions.

Trim, IsNullOrWhiteSpace, and SubString may be the most frequently used string APIs, and they are also very heavy:

  • Trim() or SubString() returns a new string object that is part of the original string, this is unnecessary if there is a way to slice and return a portion of the original string to save one copy.
  • IsNullOrWhiteSpace() takes a string object that needs a memory copy (because string is immutable.)
  • Specifically,  string concatenation is expensive, it takes n string objects, makes n copy, generate n - 1 temporary string objects, and return a final string object, the n – 1 copies can be eliminated if there is a way to get direct access to the return string memory and perform sequential writes.

Span<T>

System.Span<T> is a stack-only type (ref struct) that wraps all memory access patterns, it is the type for universal contiguous memory access. You can think the implementation of the Span<T> contains a dummy reference and a length, accepting all 3 memory access types.

You can create a Span<T> using its constructor overloads or implicit operators from array, stackalloc’d pointers and unmanaged pointers.

 // Use implicit operator Span<char>(char[]).
Span<char> span1 = new char[] { 's', 'p', 'a', 'n' };

// Use stackalloc.
Span<byte> span2 = stackalloc byte[50];

// Use constructor.
IntPtr array = new IntPtr();
Span<int> span3 = new Span<int>(array.ToPointer(), 1);

Once you have a Span<T> object, you can set value with a specified index, or return a portion of the span:

 // Create an instance.
Span<char> span = new char[] { 's', 'p', 'a', 'n' };
// Access the reference of the first element.
ref char first = ref span[0];
// Assign the reference with a new value.
first = 'S';
// You get "Span".
Console.WriteLine(span.ToArray());

 // Return a new span with start index = 1 and end index = span.Length - 1.
// You get "pan".
Span<char> span2 = span.Slice(1);
Console.WriteLine(span2.ToArray());

You can then use the Slice() method to write a high performance Trim() method:

 private static void Main(string[] args)
{
    string test = "   Hello, World! ";
    Console.WriteLine(Trim(test.ToCharArray()).ToArray());
}

private static Span<char> Trim(Span<char> source)
{
    if (source.IsEmpty)
    {
        return source;
    }

    int start = 0, end = source.Length - 1;
    char startChar = source[start], endChar = source[end];

    while ((start < end) && (startChar == ' ' || endChar == ' '))
    {
        if (startChar == ' ')
        {
            start++;
        }

        if (endChar == ' ')
        {
            end—;
        }

        startChar = source[start];
        endChar = source[end];
    }

    return source.Slice(start, end - start + 1);
}

The above code does not copy over strings, nor generate new strings, it returns a portion of the original string by calling the Slice() method.

Because Span<T> is a ref struct, all ref struct restrictions apply. i.e. you cannot use Span<T> in fields, properties, iterator and async methods.

Memory<T>

System.Memory<T> is a wrapper of System.Span<T>, make it accessible in iterator and async methods. Use the Span property on the Memory<T> to access the underlying memory, this is extremely helpful in the asynchronous scenarios like File Streams and network communications (HttpClient etc..)

The following code shows simple usage of this type.

 private static async Task Main(string[] args)
{
    Memory<byte> memory = new Memory<byte>(new byte[50]);
    int count = await ReadFromUrlAsync("https://www.microsoft.com", memory).ConfigureAwait(false);
    Console.WriteLine("Bytes written: {0}", count);
}

private static async ValueTask<int> ReadFromUrlAsync(string url, Memory<byte> memory)
{
    using (HttpClient client = new HttpClient())
    {
        Stream stream = await client.GetStreamAsync(new Uri(url)).ConfigureAwait(false);
        return await stream.ReadAsync(memory).ConfigureAwait(false);
    }
}

The Framework Class Library/Core Framework (FCL/CoreFx) will add APIs based on the span-like types for Streams, strings and more in .NET Core 2.1.

ReadOnlySpan<T> and ReadOnlyMemory<T>

System.ReadOnlySpan<T> is the read-only version of the System.Span<T> struct where the indexer returns a readonly ref object instead of ref object. You get read-only memory access when using System.ReadOnlySpan<T>readonly ref struct.

This is useful for string type, because string is immutable, it is treated as read-only span.

We can rewrite the above code to implement the Trim() method using ReadOnlySpan<T>:

 private static void Main(string[] args)
{
    // Implicit operator ReadOnlySpan(string).
    ReadOnlySpan<char> test = "   Hello, World! ";
    Console.WriteLine(Trim(test).ToArray());
}

private static ReadOnlySpan<char> Trim(ReadOnlySpan<char> source)
{
    if (source.IsEmpty)
    {
        return source;
    }

    int start = 0, end = source.Length - 1;
    char startChar = source[start], endChar = source[end];

    while ((start < end) && (startChar == ' ' || endChar == ' '))
    {
        if (startChar == ' ')
     {
            start++;
        }

        if (endChar == ' ')
        {
            end—;
        }

        startChar = source[start];
        endChar = source[end];
    }

    return source.Slice(start, end - start + 1);
}

As you can see, Nothing is changed in the method body; I just changed the parameter type from Span<T> to ReadOnlySpan<T>, and used the implicit operator to convert a string literal to ReadOnlySpan<char>.

System.ReadOnlyMemory<T> is the read-only version of System.Memory<T> struct where the Span property is a ReadOnlySpan<T>. When using this type, you get read-only access to the memory and you can use it with an iterator method or async method.

Memory Extensions

The System.MemoryExtensions class contains extension methods for different types that manipulates with span types, here is a list of commonly used extension methods, many of them are the equivalent implementations for existing APIs using the span types.

  • AsSpan, AsMemory: Convert arrays into Span<T> or Memory<T> or their read-only counterparts.
  • BinarySearch, IndexOf, LastIndexOf: Search elements and indexes.
  • IsWhiteSpace, Trim, TrimStart, TrimEnd, ToUpper, ToUpperInvariant, ToLower, ToLowerInvariant: Span<char> operations similar to string.

Memory Marshal

In some case, you probably want to have lower level access to the memory types and system buffers, and convert between spans and read-only spans. The System.Runtime.InteropServices.MemoryMarshal static class provides such functionalities to allow you control these access scenarios. The following code shows to title case a string using the span types, this is high performant because there is no temporary string allocations.

 private static void Main(string[] args)
{
    string source = "span like types are awesome!";
    // source.ToMemory() converts source from string to ReadOnlyMemory<char>,
    // and MemoryMarshal.AsMemory converts ReadOnlyMemory<char> to Memory<char>
    // so you can modify the elements.
    TitleCase(MemoryMarshal.AsMemory(source.AsMemory()));
    // You get "Span like types are awesome!";
    Console.WriteLine(source);
}

private static void TitleCase(Memory<char> memory)
{
    if (memory.IsEmpty)
    {
        return;
    }

    ref char first = ref memory.Span[0];
    if (first >= 'a' && first <= 'z')
    {
        first = (char)(first - 32);
    }
}

Conclusion

Span<T> and Memory<T> enables a uniform way to access contiguous memory, regardless how the memory is allocated. It is very helpful for native development scenarios, as well as high performance scenarios. Especially, you will gain significant performance improvements while using span types to work with strings. It is a very nice feature innovated in C# 7.2.

NOTE: To use this feature, you will need to use Visual Studio 2017.5 and language version 7.2 or latest.