Realm-cocoa: The static framework is huge (is 422MB)
The static Realm framework is 422MB, much too big for GitHub (100MB limit) and it's hard to imagine there's enough code to merit a binary of this size. Can it be thinned?
jscalo
All 4 comments
Why is Realm's Binary Bundle so big?
By supporting a lot of platforms across different architectures the size of our bundle has grown over time a lot in size. There are several conscious decisions of the team which contribute to the size as much as to a better development experience for our users as we think.
Because of so much great stuff!
How much code does Realm even consist of?
The binaries are a product of compiling the SDK including all its dependencies and bundling it up.
What are those dependencies?
- Realm Objective-C - The Objective-C part of the Cocoa SDK itself, containing all the runtime symbols.
- Realm Core - The custom open-source database managing access to files. This has highly optimized algorithms to make Realm as fast as it is and vendors its own file system abstraction layer.
- Realm Object Store - An object-oriented C++ layer on top of Realm Core, allowing to build database bindings across a wide-range of supported platforms.
- Realm Mobile Platform's secret sauce - The client synchronization functionalities are compiled into the binaries as well.
Where is the FAT file taking all the space?
Let's explore it a bit by tapping into our release bundle.
~/Downloads > du -hs realm-objc-2.1.1
516M realm-objc-2.1.1
~/Downloads > cd realm-objc-2.1.1
~/Downloads/realm-objc-2.1.1 > du -hs **
16K LICENSE.txt
8.0K Swift
4.0K docs.webloc
1.7M examples
455M ios
6.3M osx
108K plugin
26M tvos
26M watchos
~/Downloads/realm-objc-2.1.1 > cd ios
~/Downloads/realm-objc-2.1.1/ios > du -hs **
53M dynamic
402M static
~/Downloads/realm-objc-2.1.1 > cd static
~/Downloads/realm-objc-2.1.1/ios/static > du -hs **
402M Realm.framework
That confirms that it's indeed the static framework which is the largest binary of our bundle, which we in fact only distribute for iOS for backwards-compatibility to iOS 7.
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > du -hs *
188K Headers
4.0K Info.plist
4.0K Modules
68K PrivateHeaders
402M Realm
If we go deeper we'll see that's indeed the static binary of the framework, which is occupying all the space. There's nothing else of much weight in there, just headers, the Info.plist and the module.modulemap.
Let's take a look why this is soo big.
file Realm
Realm: Mach-O universal binary with 4 architectures
Realm (for architecture i386): current ar archive random library
Realm (for architecture armv7): current ar archive random library
Realm (for architecture x86_64): current ar archive random library
Realm (for architecture arm64): current ar archive random library
So this is a so-called FAT binary with slices for 4 different architectures. Technically it's an archive of archives. Why do we need them all?
There is armv7. This is required to support actual iOS devices ranging back to the iPhone 4, the iPad 2, the iPad mini 1 and even the iPod touch of the 5th generation. These all have in common that they have a 32bit processor architecture but still are able to run iOS 7, which is the lowest iOS version which we still support today.
arm64 is the architecture used if your app runs on a newer device (after late 2013, excluding the iPhone 5c) and all its dependencies were compiled for 64bit architectures.
Last but not least we've i386 and x86_64, these are required for running apps in the simulator in 32bit and 64bit mode.
So let's extract all these architecture slices and see what's going on in these MachO binaries and how the size distributes across the architectures.
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > ARCHS=$(lipo -info Realm | cut -d':' -f3 | xargs | tr ' ' "\n")
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > for arch in $ARCHS; do; lipo -thin $arch Realm -o "Realm.$arch"; done
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > du -hs Realm.*
138M Realm.arm64
140M Realm.armv7
61M Realm.i386
64M Realm.x86_64
This confirms something we've seen in the large before. It seems to require more than twice as much of code to run Realm on an actual iOS device.
You Get Bitcode! You Get Bitcode! Everyone Gets Bitcode!
Why are they so much bigger? The answer is bitcode. Ironically it is supposed to actually help Apple to ship a smaller app bundle to your users.
But let's validate this thesis. So first of all let's check whether there is actually bitcode in these libraries. Using otool with the option -l, we can get the load commands used in the library. The load command used by bitcode is __LLVM. It is an enriched intermediate representation of a compiled binary implemented in the compiler backend.
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > otool -l Realm.arm64 | grep __LLVM
segname __LLVM
segname __LLVM
segname __LLVM
segname __LLVM
…
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > otool -l Realm.arm64 | grep __LLVM | wc -l
274
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > otool -l Realm.armv7 | grep __LLVM | wc -l
274
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > otool -l Realm.i386 | grep __LLVM | wc -l
148
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > otool -l Realm.x86_64 | grep __LLVM | wc -l
148
So it's in there, in all libraries on all architectures. But how much more space comes through that? Let's figure it out by stripping it away.
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > xcrun bitcode_strip -r Realm -o Realm.nobitcode
~/Downloads/realm-objc-2.1.1/ios/static/Realm.framework > du -hs Realm Realm.nobitcode
402M Realm
40M Realm.nobitcode
Down by 362MB to just 10% of the size. 😳
What Options Do You Have?
Now we've assessed the situation and have learned that even if it's hard to imagine there's quite a lot code, bitcode to be precise. Given that the size just blows up by that, we unfortunately have to distribute a binary of this size, keeping in mind that at the same time, it is thinned when shipping your app to your users.
This doesn't help though with your initial problem of the GitHub file size limit. So what options do you have?
- Use the dynamic framework if you don't need iOS 7 backwards-compatibility.
- Use CocoaPods to integrate Realm. This way you don't have to commit the framework.
- Use GitHub's LFS to commit the framework. See also GitHub's Pricing info regarding LFS storage and bandwidth usage. 💸
- Remove bitcode by using
bitcode_strip:
xcrun bitcode_strip -r ios/static/Realm.framework/Realm -o ios/static/Realm.framework/Realm
But be careful: if you want to provide bitcode yourself, all apps, extensions and frameworks in the app bundle need to include bitcode as well.
mrackwitz
on 18 Dec 2016
Hahahah that was the best comment I've ever read on GitHub @mrackwitz! Thanks for writing that! :)
@jscalo I hope that answered your question and provided a suitable solution you can use for your project. Please let us know if you have any additional questions. :)
Thanks! Happy holidays everyone!
TimOliver
on 24 Dec 2016
Ha, yes that was quite the response. To be clear though— this isn't some sort of tech support request (I can and did work around the issue) and it will continue to be an issue for users. If bitcode is the main culprit then I'd suggest making another edition of the static framework available that doesn't include bitcode. (My anecdotal info suggests that bitcode use isn't in the majority yet. Does anyone have data on this?)
jscalo
on 24 Dec 2016
You can strip the bitcode out of binaries fairly easily if you'd like (using xcrun bitcode_strip): https://pspdfkit.com/guides/ios/current/faq/bitcode/#toc_stripping-bitcode
If we included variants with and without bitcode for iOS, that would even further complicate our variety of binary formats that we produce, leading to more confusion for beginner developers, and larger release archives. We currently have binaries for (static, dynamic, swift 2.2, swift 2.3, swift 3.0, swift 3.0.1, swift 3.0.2), all of which could have both variants with/without bitcode.
jpsim
on 24 Dec 2016
Related issues
ishidakei
·
3Comments
dmorrow
·
3Comments
ciminuv
·
3Comments
duribreux
·
3Comments
Bogidon
·
3Comments
Most helpful comment
Why is Realm's Binary Bundle so big?
By supporting a lot of platforms across different architectures the size of our bundle has grown over time a lot in size. There are several conscious decisions of the team which contribute to the size as much as to a better development experience for our users as we think.
Because of so much great stuff!
How much code does Realm even consist of?
The binaries are a product of compiling the SDK including all its dependencies and bundling it up.
What are those dependencies?
Where is the FAT file taking all the space?
Let's explore it a bit by tapping into our release bundle.
That confirms that it's indeed the static framework which is the largest binary of our bundle, which we in fact only distribute for iOS for backwards-compatibility to iOS 7.
If we go deeper we'll see that's indeed the static binary of the framework, which is occupying all the space. There's nothing else of much weight in there, just headers, the
Info.plistand themodule.modulemap.Let's take a look why this is soo big.
So this is a so-called FAT binary with slices for 4 different architectures. Technically it's an archive of archives. Why do we need them all?
There is
armv7. This is required to support actual iOS devices ranging back to the iPhone 4, the iPad 2, the iPad mini 1 and even the iPod touch of the 5th generation. These all have in common that they have a 32bit processor architecture but still are able to run iOS 7, which is the lowest iOS version which we still support today.arm64is the architecture used if your app runs on a newer device (after late 2013, excluding the iPhone 5c) and all its dependencies were compiled for 64bit architectures.Last but not least we've
i386andx86_64, these are required for running apps in the simulator in 32bit and 64bit mode.So let's extract all these architecture slices and see what's going on in these MachO binaries and how the size distributes across the architectures.
This confirms something we've seen in the large before. It seems to require more than twice as much of code to run Realm on an actual iOS device.
You Get Bitcode! You Get Bitcode! Everyone Gets Bitcode!
Why are they so much bigger? The answer is bitcode. Ironically it is supposed to actually help Apple to ship a smaller app bundle to your users.
But let's validate this thesis. So first of all let's check whether there is actually bitcode in these libraries. Using
otoolwith the option-l, we can get the load commands used in the library. The load command used by bitcode is__LLVM. It is an enriched intermediate representation of a compiled binary implemented in the compiler backend.So it's in there, in all libraries on all architectures. But how much more space comes through that? Let's figure it out by stripping it away.
Down by 362MB to just 10% of the size. 😳
What Options Do You Have?
Now we've assessed the situation and have learned that even if it's hard to imagine there's quite a lot code, bitcode to be precise. Given that the size just blows up by that, we unfortunately have to distribute a binary of this size, keeping in mind that at the same time, it is thinned when shipping your app to your users.
This doesn't help though with your initial problem of the GitHub file size limit. So what options do you have?
bitcode_strip:xcrun bitcode_strip -r ios/static/Realm.framework/Realm -o ios/static/Realm.framework/RealmBut be careful: if you want to provide bitcode yourself, all apps, extensions and frameworks in the app bundle need to include bitcode as well.