Moya: Abstract Response Deserialization in Moya?

[manas-chaudhari]

Alamofire layer: Generic Input -> Network call -> Generic Output

Generic = XML/Json or any format

With Moya, we are abstracting the serialization of request parameters. Thus, it looks like:
Api specific params -> Generic Input -> Network call -> Generic Output

Deserialization of output is still outside Moya. An example from Artsy's code:

// This belongs to AppViewController.swift
let auctionEndpoint: ArtsyAPI = ArtsyAPI.auctionInfo(auctionID: auctionID)

        return provider.request(auctionEndpoint)
            .filterSuccessfulStatusCodes()
            .mapJSON()
            .mapTo(object: Sale.self)
            .logError()
            .retry()
            .throttle(1, scheduler: MainScheduler.instance)

Thus, AppViewController is now aware that the request uses JSON & it is deciding the type Sale in which the JSON should be parsed into.

Ideally, this logic should belong to the network abstraction layer i.e. inside Moya.
So basically, I want request(auctionEndpoint) function to return an Observable<Sale>.

What would be a good way to achieve this?

[scottrhoyt]

This would be hard to put in the Moya framework because you'd have to come up with a way to inject a lot of application-domain-specific information into Moya without bloating the API. You'd also lock the consumer into a particular JSON deserialization library which they'd need to adopt in their models.

That being said here are two ways to accomplish this right now:

Subclass/extend MoyaProvider

I override RxMoyaProviders request method to handle my object deserialization as well. The signature looks something like this:

request<T: ModelProtocol>(_ token: TargetType, object: T.Type) -> Obeservable<T>
request<T: ModelProtocol>(_ token: TargetType, array: T.Type) -> Obeservable<[T]>

In this example ModelProtocol is a protocol that all my models conform to and extends the JSON deserialization protocol of your choice (e.g. Unboxable or Mappable). You could omit passing the type and instead overload just on the return type. RxSwift does plenty of this. Either way you'll have to supply the type info implicitly or explicitly at the call site, and I have a more pleasant experience with autocompletion the sooner and more explicitly I do that.

Create a Networking layer

This is similar to the above but instead of putting that logic in RxMoyaProvider or MoyaProvider, you put it in a layer that wraps the provider. The nice thing here is that you can supply type information in the function signatures (e.g. getUsers() -> Observable<[User]>). But you are adding another layer of abstraction and at this point the primary benefit that Moya would provide is just organization of API information.

Both of these options are also nice because it gives you a spot to put domain specific error mapping and other information. For example, I'm currently using Moya to interact with an app that returns responses with the signature:

{
  "code": "<success or error code>",
  "message": "<message>",
  "data": "<dictionary or array of data>"
}

So at this middle layer I can also map to domain specific errors and embed the information that I need to look at the data key to map my models.

Hope that all helps. Let me know if you have any other questions.

[manas-chaudhari]

Thanks a lot.

Perhaps deserialization is difficult to integrate because of the enum structure. It seems doable if we use protocols with associated types for Request & Response.

For example:

protocol Api {
    associatedtype Result
    associatedtype Request

    static func parameters(request: Request) -> [String: Any]?
    static func parse(data: Data) -> Result?
}

extension Api {
    static func call(request: Request) -> Result? {
        let params = parameters(request: request)
        // Invoke api get result
        return parse(data: Data())
    }
}

extension Api where Result: ModelProtocol {
    internal static func parse(data: Data) -> Result? {
        return Result.init(data: data)
    }
}

With this, Api can be declared using a struct

struct AuctionInfo: Api {
    typealias Request = Int
    typealias Result = Auction
}

Are there any plans to add support for deserialization?

[scottrhoyt]

Using associatedTypes will allow you to inject model type information into the protocol. The problem is that then you can only return one model type for each conformance to Api. This forces you to organize your API client code by return type which might not be the ideal organization--particularly since ModelProtocol and [ModelProtocol] returns would need separate Apis.

While Moya was designed with enums in mind for use as TargetTypes, you can use any data structure to conform to TargetType. The nice thing then is that if this makes sense in your situation, you can use structs to conform and add a protocol on top of TargetType with your associated types. Then you can subclass or extend MoyaProvider to add the deserialization behavior.

I can't speak for all @Moya/contributors, but IMO I don't see model deserialization as being added in the near future. Mainly because doing so would force consumers into a particular convention for their model code and there is no single best practice that has emerged for JSON deserialization--all the libraries have their pros and cons. However, Moya Community Extensions offer a good way to get this functionality while still being flexible on how you want to do JSON deserialization.

[manas-chaudhari]

Structs conforming to TargetType, should work in my case.

If I go ahead with this, will I need to create a different provider instance for every type of api call OR could there be a better approach?

[AndrewSB]

@manas-chaudhari yeah, you would have to create a different provider instance for each kind of model that your API calls return. Maybe instead of having associatedTypes on the protocol, you could pass the types in. Your call function would have to then be generic, and your parse function would take the type as an argument.

static func call<T>(request: Request) -> T? {
        let params = parameters(request: request)
        // Invoke api get result
        return parse(data: Data(), type: T.self)
}

internal static func parse<T>(data: Data, type: T) -> Result? {
        return T.init(data: data)
}

and you'd probably have to also create a protocol where you can specify that T can be initialized with some Data, and specify that conformance in your generic, Mappable, or Unboxable, as @scottrhoyt said.

FWIW It sounds like you want an extremely simple API layer, where a consumer doesn't have the opportunity to mess anything up by putting the wrong type in; somewhat similar to the API layer one might ship in an SDK.
If that is what your goal is, I'd recommend going with @scottrhoyt's Networking layer. The Networking layer would be the closest thing to what Facebook, or Stripe's SDK vend.
I recently built an SDK, and did the same thing. I had Moya included as a private implementation detail, and I wrote functions for each API call that looked something like

static func login(email: String, password: String) -> Observable<User> {
    return Email.validate(email)
        .flatMap { email in networkingProvider.request(.login(email: email.value, password: password)) }
        .mapJSON()
        .do(onNext: saveAuthTokenToKeychain)
        .map(User.init)
}

[AndrewSB]

@scottrhoyt https://github.com/Moya/Moya/issues/823#issuecomment-264653488 was incredibly detailed. Thank you 🙏

Should we create a documentation page for this questions like this issue? Your comment is 90% of the content I think we'd need for that doc page

[manas-chaudhari]

Thanks @AndrewSB, I will mostly go ahead with creating a Networking layer.

However, I strongly feel that a network abstraction library is incomplete without any support for deserialization. We can take inspiration from other platforms. For example, Retrofit for Java/Android really makes networking seamless.

[AndrewSB]

Hmm, I see where you're coming from...

I dont think we'd add it to Moya right now, since it would definitely force us to include (or depend upon) a JSON parsing library, and force all consumers of Moya to use the same.

What about defining another protocol that conforms to TargetType?

protocol DecodableTargetType: Moya.TargetType {
    var modelType: SomeJSONDecodableProtocolConformance.Type { get }
}

// then you'd be able to take your model
struct User: SomeJSONDecodableProtocolConformance {}

// and then convert your `TargetType` enum conformance to `DecodableTargetType`, and add
enum MyAPI: DecodableTargetType {
    ...
    var modelType: SomeJSONDecodableProtocolConformance.Type {
        switch self {
             case .me: return User.self
        }
    }
    ...

by doing that, you could have all the functionality you're looking for from Retrofit in Moya.

Your API consumer can be oblivious to the fact that the response is in JSON by defining a parse function in your networking stack that mapJSONs, and then calls your JSON decode function with the modelType of the target; making it look like

provider.request(.me).decode() // returns Observable<User>

[manas-chaudhari]

Regarding dependency with JSON parsing, wouldn't it be possible to create a new Parser protocol in Moya for parsing data to any object?
Consumers can then create these parsers using any JSON/XML parsing library. These parsers could be registered when creating MoyaProvider. This way, deserialization can be implemented without Moya having to depend on any JSON library.

DecodableTargetType approach is great. It will definitely hide the JSON format from API consumer. But user will still need to cast it to the correct model type, right?

enum MyApi: DecodableTargetType {
    ...
    func decode() -> Observable<SomeJSONDecodableProtocolConformance> {
        ...
    }
}

provider.request(.me).decode() // returns Observable<SomeJSONDecodableProtocolConformance>
        .mapTo(User.self)      // <- Can this be eliminated?

Am I missing something here?

[AndrewSB]

I hope it can be eliminated 😄 But I'm not sure how to. I think we might need to something type-erasurey here. I'm not totally sure, but the goal is to be able to express

enum MyAPI: DecodableTargetType {
  ...
   var modelType: SomeJSONDecodableProtocolConformance {
     switch self {
       case .upload: return String.self
     }
    }

    func decoded(response: Response) -> Observable<Self.modelType> { // This line doesn't compile 😭
      // parse things and return the correct modelType
    }

  ...
}

Right now, without any type magic, its possible to return Observable< SomeJSONDecodableProtocolConformance >. It makes sense that you should be able to return Observable<String> (or Observable<any other modelType>) because I know how to express that in logic, but I'm not sure how to express it in the Swift type system.
@Moya/contributors: Does anyone more experienced with such problems care to chime in?

I feel as if the answer would be to make each case in the MyAPI generic over type T: SomeJSONDecodableProtocolConformance, (or give each case an associatedValue of SomeJSONDecodableProtocolConformance). But I'm not sure to do that

[manas-chaudhari]

I think it cannot be eliminated without using associatedType. If we change ModelType to an associatedType, the decode function can return the desired Observable<ModelType>.

protocol DecodableTargetType: Moya.TargetType {
    associatedType ModelType: SomeJSONDecodableProtocolConformance
}

enum MyAPI: DecodableTargetType {
  ...
    typealias ModelType = String

    func decoded(response: Response) -> Observable<ModelType> { // This works
      // parse things and return the correct modelType
    }

  ...
}

However, this takes us back to the initial discussion with @scottrhoyt. One DecodableTargetType conformance can return only specific type of Model. We could go ahead by creating a struct for every network api.

But then, the next problem is that a different MoyaProvider instance is required for different api calls. This is undesirable as any setup (like custom endpoint closure, etc.) would be required for every api invocation.

Can we build a MoyaProvider variant which does not have a generic <Target> parameter?

From what I understood, the Target generic parameter is used to have static checks in request(target: Target). We should be able to move the class level generic parameter to the request function. This would allow using the same MoyaProvider for different types that conform to TargetType.

[justinmakaila]

I think the best bet is to provide a custom wrapper around your provider, similar to the Ello app.

My current implementation, which is built around a GraphQL server:

struct MyProvider {
    public static var sharedProvider: ReactiveCocoaMoyaProvider<MyAPI>  = WellthProvider.DefaultProvider()

    public static func request(_ target: MyAPI, queue: DispatchQueue? = nil, progress: @escaping Progress = { _ in  }) -> SignalProducer<JSON, Error> {
        return sharedProvider
            .requestWithProgress(token: target)
            .mapProgressResponse(progress)
            .mapSwiftyJSON()
    }

    public static func graphQL(_ operation: GraphQLType, variables: [String: Any] = [:]) -> SignalProducer<GraphQLResponse, Error> {
        return request(.graphOperation(operation: operation, variables: variables))
            .mapGraphQLResponse()
    }
}

At this point, all you have to do is wrap the Moya API, and provide the proper extensions to process the request.

Ultimately, I agree with @scottrhoyt that deserialization is beyond the scope of the library. It should be the consumer of the API that maps the response to specific results.

You can achieve this with protocols and dependency injection:

protocol MySpecializedProvider {
   func request(target: MyAPITarget) -> SignalProducer<MyModel, RequestError>
}

class MyViewController {
  let networkProvider: MySpecializedProvider

  func refresh() {
    let endpoint = MyAPITarget.doSomething(parameter)
    networkProvider.request(endpoint).startWithResult { (result: Result<MyModel, RequestError>) in
      switch result {
        case .success: /// Handle successful request
        case let .failure(error): /// Handle error
      }
    }
  }
}

[AndrewSB]

@manas-chaudhari: I think using an approach like Ello would be best for your use case under the current Moya.

Your point on

build[ing] a MoyaProvider variant which does not have a generic <Target> parameter

does sound interesting. If its possible for us to make Moya better by implementing that, we should look into it. I've created an issue for you to expand on that idea (https://github.com/Moya/Moya/issues/830).

If you think your initial question was answered, and you're satisfied with the Networking layer, go ahead and close this issue, and we can continue discussion on #830

[manas-chaudhari]

@justinmakaila @AndrewSB Thank you for your inputs. I will be going ahead with creating a wrapper over MoyaProvider.