Alamofire: How to debug crash in SessionDelegate.urlSession(URLSession, dataTask : URLSessionDataTask, didReceive : Data)?

[blixt]

I'm seeing a fairly common EXC_BREAKPOINT crash in Crashlytics caused by a delegate handler in Alamofire. See below for the stack trace. The app in question is iOS 10 only, and the crash seems to happen on all devices and 10.x versions.

Crashed: NSOperationQueue 0x170034f60 :: NSOperation 0x1708519a0 (QOS: DEFAULT)
0  libswiftFoundation.dylib       0x100dc2e4c static Data._unconditionallyBridgeFromObjectiveC(NSData?) -> Data (__hidden#3516_:1698)
1  Alamofire                      0x10061388c @objc SessionDelegate.urlSession(URLSession, dataTask : URLSessionDataTask, didReceive : Data) -> () + 184
2  CFNetwork                      0x185672b1c __67-[NSURLSession delegate_dataTask:didReceiveData:completionHandler:]_block_invoke.242 + 36
3  Foundation                     0x185993fb0 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 16
4  Foundation                     0x1858d8aa8 -[NSBlockOperation main] + 96
5  Foundation                     0x1858c90a4 -[__NSOperationInternal _start:] + 620
6  Foundation                     0x18599635c __NSOQSchedule_f + 228
7  libdispatch.dylib              0x183d2e1bc _dispatch_client_callout + 16
8  libdispatch.dylib              0x183d3c3dc _dispatch_queue_serial_drain + 928
9  libdispatch.dylib              0x183d319a4 _dispatch_queue_invoke + 652
10 libdispatch.dylib              0x183d3e34c _dispatch_root_queue_drain + 572
11 libdispatch.dylib              0x183d3e0ac _dispatch_worker_thread3 + 124
12 libsystem_pthread.dylib        0x183f372a0 _pthread_wqthread + 1288
13 libsystem_pthread.dylib        0x183f36d8c start_wqthread + 4

Due to the crash being EXC_BREAKPOINT I'm assuming it's a failing assertion somewhere and may actually be happening in the Swift/Obj-C compatibility layer before the actual delegate code is even called. But I'm not sure how to debug this and was hoping someone on here has an idea why this would happen.

I use Alamofire for standard HTTPS requests to a JSON backend. I have one standard session manager and one background session manager. I'm setting the backgroundCompletionHandler in the app delegate as well.

[jshier]

A brief investigation indicates that this crash may be caused by a nil value for the Data passed into the delegate method. Then, when it tries to bridge the NSData? to Data, it crashes because of the nil value. Just speculation right now. @blixt Do you happen to know what requests are causing this? Or is it all requests for your app? Perhaps you're getting nil data back from the server somehow?

[blixt]

Since this crashes independently from any of my code I'm having a very hard time tracking whether this is happening for random requests, or specific ones. That said, it shouldn't be possible for any request to return "nil" data since even a request that returns nothing has a valid data payload of 0 bytes.

I've got a separate background URLSession that I'm managing myself (with a unique session identifier) and four Alamofire sessions (two of which are regular sessions, and the other two are background sessions with unique identifiers). Are there known conflicts with using multiple sessions with Alamofire?

[jshier]

No, this is a bridging issue from the Objective-C code of URLSession and Swift. As you can see, it's apparently trying to bridge an NSData? to Data, which will never work if the value is nil. Either they have an underlying bug that's returning nil when it shouldn't, or they need to update the bridging of this API. Please file a bug with Apple (and post the Radar number) and / or Swift (and post the SR link) so we can get them to look at it. In the meantime we'll see if there's anything we can do to prevent this.

[blixt]

Alright, I've filed radar 32631704. Oh and I forgot to mention that the project is built with Swift 3.

[ibarisic05]

+1 on this

[gneil90]

Experienced the same today. +1

[bhaveshbc]

found any solution i also got the same problem

[blixt]

I've taken all precautions I can imagine in my code now and this crash is still occurring.

Our project also has other background URLSessions with URLSessionDelegate assigned and does not crash in those, so is it possible this crash is related to how Alamofire does magic with respondsToSelector etc?

[jshier]

No, as you can see in the crash log, it's due to bridging from Objective-C to Swift with an unexpected nil value. I spoke with a URLSession engineer at WWDC and he said it was likely an underlying bug where URLSession was returning nil unexpectedly.

@bhaveshbc @gneil90 @ibarisic05 To that end, please file bugs with Apple and post the numbers here so that we can file a bug for Alamofire itself and cite all of your bugs. That way we have the best chance possible to get Apple to fix it.

[blixt]

@jshier I understand the crash itself is due to a bug in the bridging code, but it's also a bug that only happens in the specific code path caused by Alamofire. Since other URLSession configurations doing roughly the same thing are working in Swift 3, it's likely that Alamofire can work its way around the crash. Is this something you would consider investigating?

[jshier]

@blixt I'm not sure what gave you the impression that we didn't investigate this, but we have. The crash log is clear and the response from the Apple engineer was as well. Given the irregular nature of this bug, it's not really surprising that you're only seeing it for certain requests. Without a reproducible test case, it's difficult for us to investigate this beyond the crash reports from others. However, we are continuing to look at it, which is why I've left the issue open.

If anyone else is seeing this crash, posting a crash log and attempting to create a reduced test case would help a lot with our investigation as well as reporting the bug to Apple.

[blixt]

@jshier Okay thank you for looking into this.

For context, in the past 7 days this crash happened to ~8% of our active users (52% of them were using iOS 10.3.2, some older 10.x versions, some iOS 11), according to Crashlytics. Interestingly, it happened on average only ~1.2 times per unique user that experienced the crash.

Most of the reports have a user account id associated which (due to our app flow) probably means it's not associated with first-time install/launch. Several of the crash reports have the main thread initializing the app which suggests this may happen specifically for tasks that made progress in the background.

Hope this data is helpful.

[jshier]

It is, and it would be helpful to Apple as well, so be sure to add it to your bug report. Do you have any iOS 9 crashes (do you support it?)? Also, it's good that this shows up on iOS 11, as Apple may actually fix it then.

[blixt]

The app is not on iOS 9 so I can't provide any statistics there. I included most of this in my bug report but I've added the above to the report as additional context.

[jshier]

@blixt For any of your sessions, do you change the number of simultaneous connections at all? Any idea how big the responses might be? I'm trying to setup a test app that executes many requests in parallel, so I'd like to match your behavior.

[blixt]

@jshier Configuration-wise we don't change anything, but the number of concurrent connections may go up and down as different API calls and uploads are performed. In rare cases where the user has been disconnected there might be pending retryable requests which could spike the connection count (presumably the underlying URLSession will limit it though).

I would expect the connection count to stay around 2 for most of the requests, but while spiking, assuming the URLSession doesn't take advantage of pipelining or HTTP/2 (I haven't investigated if it is switching to HTTP/2 for our API endpoints), the count could probably reach 10-20 or so. I've not been able to correlate the crash with a burst of requests, though.

The calls made through Alamofire should all have request payloads and response sizes <100 kB (JSON API), while our own URLSession code will be uploading potentially hundreds of megabytes.

[jshier]

@blixt Thanks. Do the crashes seem limited to any particular devices or other unique configuration?

[blixt]

@jshier Unfortunately, I haven't been able to find any correlations based on configuration. It's happening on all iOS 10-enabled devices, including iPads and iPod Touch.

Available disk space does not appear to be a factor either. Free RAM is generally around 3-15% so could be an indicator, but it could also be the norm that most RAM is in use on iOS devices. I've seen several crashes with 25+% free RAM so it's definitely not a clear cause.

[trusk89]

Is this an issue since 4.5.0? Because I've been thinking of upgrading, but maybe I should wait for a bit.

[blixt]

This report is for Alamofire 4.4.0. I'll update to 4.5.0 for our next release to see if it makes any difference.

[trusk89]

So i'm also on 4.4.0 so it doesn't make any difference. Thanks @blixt!

[gneil90]

@blixt @jshier

My app crashed in the below delegate method, I changed my data to Optional parameter. Compiler shows me a warning "parameter has different optionality than expected by protocol", but this workaround worked for my case and I do not experience crashes anymore.
Note: And, yes, this method is still being called by delegate.

extension MySessionDelegate: URLSessionDataDelegate {
    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data?) {      
      if (data == nil) {
      } else {
      }
    }
}

[jshier]

@gneil90
I'm... surprised that works. I thought I had tried it before but got a compiler error. Now if only there was a way to disable that warning...

[Matze2010]

Experiencing the same problem
@gneil90s solution helps

I can reproduce the problem. For me this happens, after the app gets terminated while an session is running. When the app gets restarted and a new session is initiated the app crashes. For me it looks like it is trying to handle the old session somehow, or at least leftovers from the old session. Maybe this can be helpful

[sbhmajd]

How to apply @gneil90s solution with Alamofire? @jshier please help?

[jshier]

@Matze2010 @sbhmajd I hope you've both reported this issue to Apple, as that's really the only way it can get fixed.

I'm hesitant to make this change in Alamofire, as it would mean a permanent compiler warning for all users. As well, they may change the compiler at any time to disallow this sort of soft matching for delegate methods.

[sbhmajd]

@jshier,, thanks for response. I will leave it for Apple atm xD.

[quietpixel]

Do you have any updates on this issue? I am seeing it today after updating a project from Xcode 9.4.1 to 10.0.

[jshier]

@quietpixel There have been no new developments regarding this issue. It should be a rather rare crash from the underlying URLSession that Alamofire uses. Reporting the issue to Apple to duplicate the previously reported issues should help get it fixed on their end.

[quietpixel]

Thank you @jshier. I will report it to Apple again.

I hoped the XCode 10.1 update that came out today would fix it, unfortunately, the crash is still happening. It happens to me repeatably when doing a POST.

[jshier]

@quietpixel I forgot to mention, if you have a repeatable test case, it'd be great if you could attach it, as we haven't been able to reproduce this issue ourselves.

[2h4u]

@Matze2010 @sbhmajd I hope you've both reported this issue to Apple, as that's really the only way it can get fixed.

I'm hesitant to make this change in Alamofire, as it would mean a permanent compiler warning for all users. As well, they may change the compiler at any time to disallow this sort of soft matching for delegate methods.

I'd rather have 100 warnings than a single crash...

[jshier]

@2h4u An unconditional warning really isn't an acceptable workaround for the thousands of other Alamofire users who've never seen this issue. However, some sort of conditional workaround may be possible, at the cost of a permanent warning. Unfortunately Swift doesn't support the easy sort of conditional compilation check we'd need so it's more complicated than I'd like, but it should be doable. We'll investigate.

[jshier]

Also, @2h4u, if you're seeing this crash, are you seeing it on iOS 12 at all?

[Matze2010]

I also donˋt see the need for this to be fixed. From my observation this bug is only relevant in the development phase of an app. As mentioned before I think it has to do with unprocessed network data when an app is unexpectedly quit. This should only happen during development, as you have situations then, that may force an app to quit unexpectedly (crash, manually terminated via Xcode). If such an termination happens during an ongoing network transfer, this bug appears.
In production you should not have unexpected terminations of your app, as there should be no (more) crashes and, if at all, the app will be terminated gracefully by the OS with the possibility to do the cleanup of ongoing network traffic.

[2h4u]

@jshier: It is a rather rare crash (we have more than 10k daily users and get 5-10 crashes a day), so I can imagine that not too many care about it (we are also experiencing this crash for a long time but until now I couldn't find time to investigate it). So I don't know if that fraction of users who experience this crash is as small as you think.

Yes, we are also experiencing this issue on iOS 12 as you can see in this crash report:
http://crashes.to/s/93255275bb4

@Matze2010: I never experienced this issue during development but as you can see in the crash report above, we are experiencing it in production. So maybe your theory isn't correct.

[Matze2010]

@Matze2010: I never experienced this issue during development but as you can see in the crash report above, we are experiencing it in production. So maybe your theory isn't correct.

As you say, it is just a theory. I never experienced it in production, though.

[2h4u]

@Matze2010: I never experienced this issue during development but as you can see in the crash report above, we are experiencing it in production. So maybe your theory isn't correct.

As you say, it is just a theory. I never experienced it in production, though.

Thats interesting, can you reliable reproduce the crash in development?

[vonox7]

@2h4u An unconditional warning really isn't an acceptable workaround for the thousands of other Alamofire users who've never seen this issue.

@jshier And what about the millions of users that use the apps that use Alamofire? I guess we all should make sure that the actual users (users who use our apps) have a good experience.

[jshier]

@2h4u What version of Swift are you using? According to the Swift Forums, this shouldn't be an issue for Swift versions >= 4.

[2h4u]

@jshier Okay thats strange because I'm using Swift 4.0

[jshier]

@2h4u Are you sure you're seeing the same issue then? What's your crashing call stack?

[2h4u]

Ah okay you are right, it isn't the same crash:
Crashed: com.apple.root.background-qos 0 libsystem_malloc.dylib 0x1835f4da8 nanov2_allocate_from_block$VARIANT$armv81 + 528 1 libsystem_malloc.dylib 0x1835f4040 nanov2_allocate$VARIANT$armv81 + 140 2 libsystem_malloc.dylib 0x1835f3f64 nanov2_malloc$VARIANT$armv81 + 60 3 libsystem_malloc.dylib 0x1836019a0 malloc_zone_malloc + 156 4 libsystem_malloc.dylib 0x1836023b0 malloc + 32 5 libswiftCore.dylib 0x102e81f80 swift_slowAlloc + 32 6 libswiftCore.dylib 0x102e82000 _swift_allocObject_(swift::TargetHeapMetadata<swift::InProcess> const*, unsigned long, unsigned long) + 28 7 libswiftDispatch.dylib 0x1030a73a4 OS_dispatch_queue._syncHelper<A>(fn:execute:rescue:) + 28888 8 libswiftDispatch.dylib 0x1030a7a74 OS_dispatch_queue.sync<A>(execute:) + 30632 9 Alamofire 0x1018b31ac SessionManager.request(_:) (<compiler-generated>)

Looks like it crashes by trying to allocate some memory..

[jshier]

The original crash in this issue should be fixed for anyone using Swift 4 and above, so I'm closing this issue. Thanks to everyone who helped investigate and reported the issue to Apple!