Nullable and notnull Serialisation Properties

The C# 8 feature Nullable has been well received by myself and others into our workflows and has improved code bases immensely. However, there is one niggly workaround that I’m not a fan of, and that is = default!;

problem definition

// This class is used to serialise/deserialise a payload from a server
public class MyContractDTO
{
    public string Value { get; set; } = default!;
}

My codebase is now littered with this code hack to get it to compile because nullable says it can’t guarantee that this property is not null.

Let’s take a closer look at what we are saying.


// With the Nullable feature enabled, I need to ensure Value is not null:
// - either in the constructor from a notnull argument
// - or giving it a default non-null value.
public string Value { get; set; }

// Initialise the property to its default value (which for reference types like string is null)
public string Value { get; set; } = default;

// The '!' says to treat the value as if it is notnull, the developer knows better
public string Value { get; set; } = default!;

What does this achieve? Well, the compiler ignores the fact that it could be null, and we initialise it to be null. This is a contradiction if ever I saw one, and I don’t like it.

Normal solutions

There are two solution paths to take here:

Acknowledge that it might be null and make it nullable.
Initialise in the constructor.

These work great with normal codebases and in some cases, one is better than the other. If you know it certainly can be null, use the first option. If you know you never expect or want it to be null, use the second.

Bonus points once you have the constructor is to make the type immutable. For all domain and application logic, This is what I have done successfully so far and will continue doing.

But serialisation

Here is the problem. Serialisation.

When we have a type that is going to be used to deserialise transport models we again have the same choices as above, but a few more points to consider.

The JSON may or may not include the field
The JSON may have the field set to null
The Deserialiser might not support constructors
The Deserialiser doesn’t know about Nullable and can’t ensure the safety is upheld.

Again we can make the call to acknowledge that it might be null and make it nullable. But there are drawbacks here. If you do this, you have to add all the error-handling for dealing with nullable checks. And if you control both server and client in this situation, then you might be writing and testing code for something you never plan to, nor may never need to ever support. Being null is an exceptional/fatal situation you don’t want to have to constantly guard against.

Which leads us back to the de-facto solution currently being advocated and used:

public string Value { get; set; } = default!;

I’m still not happy. So why don’t we work on that?

The JSON may or may not include the field or it might be null

Luckily for us, this concern is fairly easy to address. Say we are using Newtonsoft.Json and want this extra piece of reassurance.

[JsonProperty(Required = Required.Always)]
public string Value { get; set; } = default!;

The Required attribute annotation is designed for exactly this situation. When we use Newtonsoft.Json as our deserialiser, we can get a JsonSerializationException for free.

This ensures we:

have minimal code doing the null checking
don’t have to guard every access to a nullable property
Treat null or missing as fatal errors as part of an existing serialisation error handling process (which we should always have anyway).

This still doesn’t stop any other piece of code from creating an invalid object state, though. But this may be the easiest solution to add those missing guarantees alongside using default!.

Use Constructors

Newtonsoft.Json helps once again by supporting constructors. Make sure all the mandatory non-nullable properties are in the constructor. And as long as there is no default constructor (which when doing nullable right you can’t anyway) and the constructor parameters have names matching the properties, this just works as expected.

public class MyContractDTO
{
    public MyContractDTO(string value)
    {
        if(value is null) throw new ArgumentNullException(nameof(value));
        Value = value;
    }
    
    public string Value { get; set; };
}

You do however have to write your null-guard into the constructor to ensure it fails with an appropriate error message. Without this, null might still sneak through, even if you annotate the property.

Once more, Newtonsoft.Json with constructors also means support for read-only (immutable) objects.

What about not using Newtonsoft.Json?

There is a new kid on the block - System.Text.Json. However, this is one area where it doesn’t shine so bright compared to Newtonsoft.Json.

Pretty much none of the above works. Constructors are not supported. Required annotation is not supported. Along with many other things.

More on what does and does not translate across from Newtonsoft.Json is documented here.

Instead, you will need to write your own converter and manually deserialise your object with your own explicit null checks (which could be done inside the constructor).

For example, given our simple type above, we might do the following:

[System.Text.Json.Serialization.JsonConverter(typeof(MyContractDTOConverter))]
public class MyContractDTO
{
    public MyContractDTO(string value)
    {
        if (value is null) throw new ArgumentNullException(nameof(value));
        Value = value;
    }
    
    public string Value { get; }
}

// Made using the examples given at https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to#required-properties
// Your decisions may vary.
// e.g. this only supports {"value": "..."} format, no extra properties allowed.
// For a more complex object, you would probably be more flexible.
// Better guides here: https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-converters-how-to
public class MyContractDTOConverter : JsonConverter<MyContractDTO>
{
    private readonly JsonEncodedText ValueName = JsonEncodedText.Encode("value");

    public override Implementation Read(
        ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options)
    {
        if (reader.TokenType != JsonTokenType.StartObject)
        {
            throw new JsonException();
        };

        string? value = default;

        reader.Read();
        // One property must exist
        if (reader.TokenType != JsonTokenType.PropertyName)
        {
            throw new JsonException();
        }

        // That property must have the right name
        if (reader.ValueTextEquals(ValueName.EncodedUtf8Bytes))
        {
            value = ReadProperty(ref reader, options);
        }
        else
        {
            throw new JsonException();
        }

        reader.Read();
        // There must be no other properties
        if (reader.TokenType != JsonTokenType.EndObject)
        {
            throw new JsonException();
        }

        return new MyContractDTO(value);
    }

    private string ReadProperty(ref Utf8JsonReader reader, JsonSerializerOptions options)
    {
        Debug.Assert(reader.TokenType == JsonTokenType.PropertyName);

        reader.Read();

        return reader.GetString();
    }

    private void WriteProperty(Utf8JsonWriter writer, JsonEncodedText name, string stringValue, JsonSerializerOptions options)
    {
        writer.WritePropertyName(name);
        writer.WriteStringValue(stringValue);
    }

    public override void Write(
        Utf8JsonWriter writer,
        Implementation implementation,
        JsonSerializerOptions options)
    {
        writer.WriteStartObject();
        WriteProperty(writer, ValueName, implementation.Value, options);
        writer.WriteEndObject();
    }
}

At which point you now have a bunch more code and logic to look after, but can achieve the same/similar results to what Newtonsoft.Json could do. Depending on your situation, maintaining this code may be more effort than declaring it nullable and maintaining checks around that in consuming code. Up to you.

Decisions, Decisions

So what would I recommend?

Firstly, don’t just use default! on your serialised types.

For maximum effect, make your classes have constructors for notnull values, and maybe even make your properties Immutable, if that makes sense. Have your constructors guard against nulls so that your compile-time assurances have runtime verifications. Especially is this is a client library you provide to others.

If you are using Newtonsoft.Json you should at least apply the [JsonProperty(Required = Required.Always)] to all your notnull properties. But add the constructors as well because you can. If you can stick with Newtonsoft.Json, your life will be very easy.

If you are using (or have to use) System.Text.Json, write custom converters for your types so that you can have those constructors mentioned above. Make sure you keep them flexible enough to ignore any extra properties you might add in the future to avoid breaking backwards compatibility.

This experiment has a companion GitHub repo of tests (a mixture of proof they work, and proof they fail tests) available here.

Happy Null-Hunting.