Realm-cocoa: Investigate the feasibility of consolidating the Swift and Objective-C APIs

[jpsim]

This could potentially let us avoid have two frameworks like we do now: Realm.framework & RealmSwift.framework.

But we may not be able to get most of our current Swift API like this. But it's worth looking into how far we can get.

[JadenGeller]

Even more useful, we can rename functions with the swift_name clang attribute.

#define SWIFT_NAME(x) __attribute__((swift_name(#x)))
+ (instancetype)defaultRealm SWIFT_NAME(init());

[bdash]

AKA NS_SWIFT_NAME.

[JadenGeller]

Swift 3.0 made swift_name a whole lot more powerful: https://github.com/apple/swift-evolution/blob/master/proposals/0044-import-as-member.md

[JadenGeller]

Note that our enums would not be fully compatible between Objective-C and Swift. For example, the Objective-C code right now uses NSString values to indicate the type of notification while Swift uses enums with String raw values. We could either (a) use string constants in both, (b) use integer-backed enums in both, or (c) reimplement this function for Swift in an extension and mark the Objective-C version unavailable.

In the case of enums with associated values (such as swift fine-grain notifications), we either have to ditch them in favor of the Objective-C implementation or reimplement this function in a Swift extension.

[JadenGeller]

Another issue is that we cannot bridge Objective-C classes to Swift struct overlays. Though a protocol _ObjectiveCBridgeable exists, it isn't intended to be used by 3rd parties. Foundation uses this to provide value-semantic versions of some of its classes, like Date, but we couldn't (yet) use it for providing a value-semantic version of Configuration. We'd either have to (a) [hopefully temporarily] have totally separate APIs surrounding value-semantic classes or (b) settle for a less-Swifty, reference-semantic API in certain places.

[austinzheng]

That's a good point. Do we have any 'structy' data types besides the configuration object?

[JadenGeller]

@austinzheng Expect for a few enums, no, but we'll also need separate APIs for those enums that are algebraic data types.

[JadenGeller]

Is there a good reason that RLMNotificationBlock uses a String instead of an NS_ENUM in the Objective-C API? This would bridge to Swift, though you lose the runtime name with both APIs since it'd have to use an integer raw value. It's not clear to me that this is important though.

We'd definitely need separate types for the fine-grain notification enum, or a redesigned API.

[tgoyne]

The externed const string approach is idiomatic for notification names in obj-c. They have the advantage of not implying that the set of possible notification is closed and make it easy to figure out what notification you're getting when you get one that you weren't expecting.

[JadenGeller]

I see, that makes sense! It seems that, if we don't want to treat notifications as a closed type, we shouldn't use an enum in Swift, no?

[austinzheng]

No, but I think Swift 3 now imports certain things (like notification constants) as static members of a wrapper struct, so they aren't bare strings anymore. (Such a type can gain new members in an extension, so it's probably preferable to an enum for our use case.)

[tgoyne]

The collection change notification types are logically a closed set and want to be an enum, but the Realm notifications shouldn't be as it could actually make sense to add more and we wouldn't want it to be a breaking change to do so.

[JadenGeller]

@tgoyne In Realm Swift, Realm Notifications are currently implemented as a closed enum, so we might want to change that then.

[JadenGeller]

Potentially a deal breaker: RLMLinkingObjects<T> do not carry their generic information over to the runtime. In our Objective-C API, we work around this by requiring this class information to be provided in an overload. While we could do this in Swift, it would be a significant API regression, so we'd definitely want separate types here.

Maybe it's possible to create a generic Swift subclass of RLMLinkingObjects that provides this capability without much duplicated code?

[austinzheng]

It might be possible; at least there's nothing stopping someone from creating a Swift generic class that inherits from an Objective-C generic class.

Would you just mark the Objective-C initializers as unavailable in Swift, and provide Swift generic initializers instead in your subclass?

[JadenGeller]

I think so, though I haven't yet gotten around to testing it.

[JadenGeller]

@austinzheng Good catch!

typedef NSString * NSCalendarIdentifier NS_EXTENSIBLE_STRING_ENUM;

The above typedef is used to convert all these stringy cases into an enum in Swift!

[JadenGeller]

There's currently no support for annotating default argument values in Objective-C code for import into Swift. We can work around this by providing overloads, but this is unideal: 2^n overloads are required for n default arguments. We usually only have 1-2 default arguments in practice though, so this should be a problem. It's probably worth either filing a radar or posting on Swift evolution about this, though I'm unsure people will be behind this given that Objective-C doesn't support default arguments (maybe Xcode 9?).

We could also obviously simply drop defaulted arguments from out API, but this is pretty clearly a big loss, so I think overloads would be a better solution.

[JadenGeller]

It's probably worthwhile noting that these sorts of importer omissions don't matter much: worse case, we could mark nearly every function as NS_REFINED_FOR_SWIFT to provide a Swiftier API (though I'm unsure if this extra layer of indirection would be optimized away on non-final classes…). What really needs to be investigated still is whether we can support parsing models without separate RLMObject and Object subclasses of RLMBaseObject.

[JadenGeller]

@austinzheng Subclassing RLMLinkingObjects is problematic because we've marked init() unavailable, and every Swift class must have an initializer (even if private). One workaround is to introduce an initWith__: method on RLMLinkingObjects and override this on the LinkingObjects subclass. I guess we would then tell users to mark the class as @NSManaged (rather than @dynamic) so they would not be required to give it a default value. Does that seem reasonable?

[JadenGeller]

@NSManaged is only valid on Objective-C representable objects, so we'll have to be more clever.

[JadenGeller]

We might have issues documenting methods in a way that applies to both Swift and Objective-C callers. For example, addOrUpdateObject: mentions "Use -[RLMObject createOrUpdateInRealm:withValue:] to copy values to a different Realm." Even if that same method is to be used (it isn't), the syntax should be totally different. We might need to find ways to provide separate documentation for Swift and Objective-C for the same function, otherwise we'll need to provide Swift wrappers just for documentation purposes.

[JadenGeller]

Looks like Objective-C generics are now imported into Swift, which _might_ make it possible to consolidate RLMArray and List. I'm not certain it will be possible though—working on a prototype!

EDIT: Though we might need to require an init(type:) initializer, but I think this is reasonable.

EDIT: In fact, the user would have to write let arr = RLMArray<Foo>(type: Foo.self), AND there would be no compile-time checks that the type parameter and the init parameter are the same. We can't even provide an init overload that infers the type parameter, so if they left it out it'd be RLMObject. We could however provide a static function RLMArray.with(type: Foo.self) that infers the parameter (since it would use the type parameter in the return type).

It may be worthwhile to draft a Swift Evolution proposal regarding exposing the type parameter value in a Swift extension init since it should always be known there. Also, it should be possible to require the specification of the type parameter (rather than defaulting to AnyObject with revenant constraints when not provided).

[JadenGeller]

Our other option is to subclass RLMArray in Swift, but this would prevent such Swift models being used from Objective-C.

[JadenGeller]

I think it's a good time to sum up the feasibility of this, so here goes.

Goal

Unify the Realm Objective-C and Realm Swift API into a single Realm Cocoa API that feels native in both Objective-C and Swift.

Benefits Gained

• Reduce user confusion with a single Cocoa API
  
  
  
  • Will eliminate user confusion regarding which module to import
  
  
• Allow interop between Objective-C and Swift code without sacrificing "Swiftiness"
• Make maintenance easier without an extra wrapper layer

Steps

• [ ] Annotate the Objective-C API with NS_SWIFT_NAME macro that describe how things ought to be named in Swift.
• [x] [Properly follow Cocoa's NSError conventions](https://github.com/realm/realm-cocoa/issues/3816) so functions are correctly imported as throws in Swift.
• [ ] Annotate Objective-C specific methods with NS_SWIFT_UNAVAILABLE or NS_REFINED_FOR_SWIFT to hide them from Swift.
• [ ] Implement Swift specific functions and protocol conformances in Swift extensions.
• [ ] Unify divergent Realm types, specifically generic collection types, for compatibility between Swift and Objective-C.
  
  
  
  • [ ] Blocked by: Better Swift interoperability with Objective-C generics. (Explained below.)
  
  

Challenges

Most steps are fairly straightforward. Single the clang importer for Swift 3 automatically renames many methods following standard Cocoa conventions, the existing Objective-C API already feels much Swiftier when used in Swift 3. With minimal effort, we can make it feel very good.

The biggest hurdle seems to be architecting RLMResults and RLMLinkingObjects in a way that works well in both languages. Currently, this is what it looks like to use these classes in a Swift 3 model using Realm Objective-C:

class Person: RLMObject {
    dynamic var pets = RLMArray<Cat>(objectClassName: Cat.className())
}

class Cat: RLMObject {
    dynamic var owners: RLMLinkingObjects<Person>?

    override class func linkingObjectsProperties() -> [String : RLMPropertyDescriptor] {
        return ["owners": RLMPropertyDescriptor(with: Person.self, propertyName: "pets")]
    }
}

Unfortunately, this requires using a stringily typed objectClassName initializer that does _not_ ensure the generic parameter matches the class type. Further, the user has to implement a linkingObjectsProperties override to provide more information about the linking object. Compare this to the current Swift-native models:

class Person: Object {
    let pets = List<Cat>()
}

class Cat: Object {
    let owners = LinkingObjects(fromType: Person.self, property: "pets")
}

It's not _yet_ possible to implement such an API for RLMArray and RLMLinkingObjects without having a separate Swift wrapper. Specifically, Objective-C generics _do not_ store their generic type at runtime, so we're unable to write code such as let pets = List<Cat>() since the generic information is lost. Though Swift 3 brings the ability for Objective-C generics to expose their runtime generic information, they must first store this information themselves. Ideally, we'd be able to write an init that captures the inferred type, but this isn't yet possible.

extension RLMArray {
    // error: Extension of a generic Objective-C class cannot access the class's generic parameters at runtime
    @nonobjc final override convenience init() {
        self.init(elementType: RLMObjectType.self)
    }
}

This has been described as out of scope for Swift 3 and not a starter bug by members of the Swift compiler team, so it might not be something we can address. Still might be worth looking into though since it pretty much the only thing (that I've identified) that'd block us implementing an identical API.

Other Considerations

Eliminating the Swift wrapper on top of the Objective-C API would give us a single "Realm Cocoa" module users could import and user, with interoperability, in their mixed or pure Swift or Objective-C codebase. Eliminating the Realm Swift wrapper does not preclude a "pure Swift" API in the future. This may be something we are interested in (especially with Swift on Linux!), but it is dependent on Swift improving reflection support (or us using codegen) and on Swift providing dynamism independent of the Objective-C runtime (or us follow the Java API w/ separate managed objects). If we do want to eventually support "pure Swift", we'll probably also still want to support some Swift+Objective-C API for interop.

Conclusion

It might not be the right time to eliminate the Swift wrapper. We probably want to wait until Swift can support the Objective-C generic features we desire so we can get runtime information about the type of the object. If we decide we'd like to move forward without this consolidation, the Swift API will suffer a deprecation in which the user will have to specify redundant information.

class Person: Object {
    let pets = List<Cat>(ofType: Cat.self)
}

That said, given the Clang importer improvements introduced recently, it is very feasible to eventually consolidate these modules, and I'd argue that this should be our eventual goal. We can get 90% the way with minimal effort, and I think we'll want an interoperable Objective-C+Swift module available regardless of an introduction of a "pure Swift" Realm API.

[JadenGeller]

One last thing worth noting: There's a deferred proposal for allowing 3rd party frameworks to define custom bridging behavior between Objective-C and Swift APIs. This might be something that's relevant to investigate once (presumably) implemented in Swift 4.0.