Decode an array with a corrupted element

Let's say you want to load and display a user's (User) list in your app. Each User required first name and last name; the middle name is optional.

struct User: Codable {
    let firstName: String
    let middleName: String?
    let lastName: String
}

In a perfect world, you would get a JSON that match the model.

[
    {
        "firstName": "John",
        "lastName": "Doe"
    },
    {
        "firstName": "Luffy",
        "middleName": "D.",
        "lastName": "Monkey"
    }
]

We can decode this JSON without any problems. We will get an array of two users.

let json = """...the JSON string above..."""

do {
    let users = try JSONDecoder().decode([User].self, from: json.data(using: .utf8)!)
    XCTAssertEqual(users.count, 2)
} catch {
    print(error)
}

Default Behavior

The world is not perfect. Sometimes you might get an unexpected format JSON.

[
    {
        "firstName": "John",
        "lastName": "Doe"
    },
    {
        "firstName": "Luffy",
        "middleName": "D.",
        "lastName": "Monkey"
    },
    {
        "firstName": "Alice" // <1>
    }
]

<1> Missing lastName which is a required field.

When you try to decode the above JSON, you will get DecodingError.keyNotFound because the last object doens't have a lastName.

DecodingError
  ▿ keyNotFound : 2 elements
    - .0 : CodingKeys(stringValue: "lastName", intValue: nil)
    ▿ .1 : Context
      ▿ codingPath : 1 element
        ▿ 0 : _JSONKey(stringValue: "Index 2", intValue: 2)
          - stringValue : "Index 2"
          ▿ intValue : Optional<Int>
            - some : 2
      - debugDescription : "No value associated with key CodingKeys(stringValue: \"lastName\", intValue: nil) (\"lastName\")."
      - underlyingError : nil

The default behavior of the decoder treats array decoding as an atomic operation. JSON array will decode successfully once every item in an array is decoded. That's mean a single malformed or corrupted object in an array would result in the whole array decoding fail. Like our previous example, a missing of the last name in the last object causing the whole array to fail.

I think this is the right behavior; API specification should work as a source of truth. Codable already supports Optional (?) value, if any fields can be nil, it should document in the specification, so we can create a proper Struct for the object.

We would declare our User object like this, if lastName is nullable.

struct User: Codable {
    let firstName: String
    let middleName: String?
    let lastName: String?
}

If a JSON doesn't follow the specification, the operation should fail as fast as possible, so we can raise the issue for a backend to fix the problem.

You can easily support sarunw.com by checking out this sponsor.

AI Grammar: Correct grammar, spell check, check punctuation, and parphrase.

Try it for Free

Problem

This default behavior makes perfect sense if you have full control over an API. If you don't, this behavior might be a bit harsh. There might be a time when you can't do anything with an API, e.g., third party API, legacy API that backend folks refuse to change, API with legacy corrupted data. When the time comes, you might not want a single corrupted item to prevent you from displaying the rest of the valid ones.

Solution

To mitigate the problem, we will create a wrapper object OptionalObject<T>, which work quite the same way as Swift Optional.

public struct OptionalObject<Base: Decodable>: Decodable {
    public let value: Base?

    public init(from decoder: Decoder) throws {
        do {
            let container = try decoder.singleValueContainer()
            self.value = try container.decode(Base.self)
        } catch {
            self.value = nil
        }
    }
}

Try to decode a malformed JSON again, but use OptionalObject<User> this time.

do {
    let optionalUsers = try JSONDecoder().decode([OptionalObject<User>].self, from: json.data(using: .utf8)!)
} catch {
    print(error)
}

The operation will succeed, but what we get is an array of OptionalObject<User>, not a User. We can fix this with the help of compactMap(_:).

compactMap(_:)
Returns an array containing the non-nil results of calling the given transformation with each element of this sequence.

do {
    let optionalUsers = try JSONDecoder().decode([OptionalObject<User>].self, from: json.data(using: .utf8)!)
    let users = optionalUsers.compactMap { $0.value }
} catch {
    print(error)
}

Now users would be an array of two valid user objects.

Array in another object

If your problem array is a property in another object, let's say you have a company (Company) that contains employees ([User]).

struct Company: Codable {
    let employees: [User]
}

Decode a malformed JSON would result in the same error.

let json =
"""
{
"employees":
    [
        {
            "firstName": "John",
            "lastName": "Doe"
        },
        {
            "firstName": "Luffy",
            "middleName": "D.",
            "lastName": "Monkey"
        },
        {
            "firstName": "Alice"
        }
    ]
}
"""

do {
    let company = try JSONDecoder().decode(Company.self, from: json.data(using: .utf8)!)
} catch {
    print(error)
}

DecodingError
  ▿ keyNotFound : 2 elements
    - .0 : CodingKeys(stringValue: "lastName", intValue: nil)
    ▿ .1 : Context
      ▿ codingPath : 2 elements
        - 0 : CodingKeys(stringValue: "employees", intValue: nil)
        ▿ 1 : _JSONKey(stringValue: "Index 2", intValue: 2)
          - stringValue : "Index 2"
          ▿ intValue : Optional<Int>
            - some : 2
      - debugDescription : "No value associated with key CodingKeys(stringValue: \"lastName\", intValue: nil) (\"lastName\")."
      - underlyingError : nil

The solution is quite the same as we discussed before, but this time we have to put the decoding logic in the initializer.

struct Company: Codable {
    let employees: [User]

    public init(from decoder: Decoder) throws {
        let values = try decoder.container(keyedBy: CodingKeys.self)

        let nullableEmployees = try values.decode([OptionalObject<User>].self, forKey: .employees) // <1>
        self.employees = nullableEmployees.compactMap { $0.value } // <2>
    }
}

<1> Use OptionalObject<User> instead of User.
<2> Filter out nil value and assign users back to employees.

Just like that and now you have a working company object.

You can easily support sarunw.com by checking out this sponsor.

AI Grammar: Correct grammar, spell check, check punctuation, and parphrase.

Try it for Free

Related Resources

• Enum & custom type from primitive JSON type
• compactMap(_:)