Manually symbolicate crash reports

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
– Brian Kernighan

Nobody expected their app to crash. We try to think for every possible scenario that the code can go wrong and handle them before we release an app, but sometimes bugs slip through. That's why we have a safety net for this. I hope you have some crash reporting in place, e.g., Fabric (Firebase to be^[1]) as your net to catch all these bugs so you can fix them later.

Sometimes even the safety net is failing, and you have to find a way to get your crash file and read a gibberish inside yourself.

...
16  CoreFoundation                	0x00000001a2f1901c 0x1a2e75000 + 671772
17  CoreFoundation                	0x00000001a2f188bc 0x1a2e75000 + 669884
18  GraphicsServices              	0x00000001acd84328 0x1acd81000 + 13096
19  UIKitCore                     	0x00000001a6fae6d4 0x1a65be000 + 10421972
20  Last Time                     	0x0000000104605ca4 0x1045fc000 + 40100
...

How can a net fail?

If you have some crash reporting service implemented, every time there is a crash occur, it will send a report to their server and symbolicate to a readable format for you.

The only time it failed (as I know of) is when the crash occurs on the app startup. That's because most crash reports are usually sent to the server the next time you start the app. If the app failed to start, there is a chance that you won't get the crash report, and you have to do all the hard work yourself to get that and symbolicate it.

You can easily support sarunw.com by checking out this sponsor.

Screenshot Studio: Create App Store screenshots in seconds not minutes.

Try it for Free

How to get crash reports

Once an app crash, a report will generate on the crash device. You have to ask your user to hand it to you.

Your users can retrieve crash reports from their device and send them to you via email by following these instructions.

1. Open Settings app
2. Go to Privacy
3. Select Analytics, then Analytics Data
4. Locate the log for the crashed app. The logs will be prefixed with the app name
5. Select the desired log. Then, tap the share action on the upper right to send it as mail

How to Symbolicating crash reports

After you get the crash report file, you have to symbolicating^[2] it. To do that, you need another file, a dSYM file. Without it, you can't do anything with a crash file you just got.

dSYM

First things first, get the dSYM file for the build. There are two ways to retrieve this.

Bitcode enabled

If your app is Bitcode enabled, you have to get it from App Store Connect.

1. Open the App Details page.
2. Click Activity.
3. From the list of All Builds, select a version that matched the crash.
4. Click the Download dSYM link.

Bitcode disabled

If your app doesn't enable Bitcode, you have to get it from your archive.

1. Open Xcode.
2. Click Window > Organizer
3. From the list of All Builds, select a version that matched the crash
4. Right-click and select "Show in Finder"
5. Right-click on .xcarchive file and select "Show Package Contents"
6. You will find your <app_name>.app.dSYM file under dSYMs folder

Let's Symbolicate

So right now, you have a dSYM file and a crash file. You might get a crash file with an extension of .crash or .ips, it doesn't matter, just open it up in your preferred text editor.

There are a few data that you need to look for in the file.

In the file, you will find something resembling a call stack that you see when the app crash in the Xcode, but instead of a file name and line number, you will see some random address string like this:

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001a2d98ebc 0x1a2d74000 + 151228
1   libsystem_pthread.dylib       	0x00000001a2cb8c1c 0x1a2cb2000 + 27676
2   libsystem_c.dylib             	0x00000001a2c08824 0x1a2b95000 + 473124
3   libc++abi.dylib               	0x00000001a2d617d4 0x1a2d60000 + 6100
4   libc++abi.dylib               	0x00000001a2d619c4 0x1a2d60000 + 6596
5   libobjc.A.dylib               	0x00000001a2cc9358 0x1a2cc3000 + 25432
6   libc++abi.dylib               	0x00000001a2d6e304 0x1a2d60000 + 58116
7   libc++abi.dylib               	0x00000001a2d6e29c 0x1a2d60000 + 58012
8   libdispatch.dylib             	0x00000001a2c6e198 0x1a2c12000 + 377240
9   libdispatch.dylib             	0x00000001a2c4844c 0x1a2c12000 + 222284
10  FrontBoardServices            	0x00000001a8083540 0x1a802d000 + 353600
11  FrontBoardServices            	0x00000001a808320c 0x1a802d000 + 352780
12  FrontBoardServices            	0x00000001a8083734 0x1a802d000 + 354100
13  CoreFoundation                	0x00000001a2f1e7e0 0x1a2e75000 + 694240
14  CoreFoundation                	0x00000001a2f1e738 0x1a2e75000 + 694072
15  CoreFoundation                	0x00000001a2f1ded0 0x1a2e75000 + 691920
16  CoreFoundation                	0x00000001a2f1901c 0x1a2e75000 + 671772
17  CoreFoundation                	0x00000001a2f188bc 0x1a2e75000 + 669884
18  GraphicsServices              	0x00000001acd84328 0x1acd81000 + 13096
19  UIKitCore                     	0x00000001a6fae6d4 0x1a65be000 + 10421972
20  Last Time                     	0x0000000104605ca4 0x1045fc000 + 40100
21  libdyld.dylib                 	0x00000001a2da3460 0x1a2da2000 + 5216

Looking for a binary image name (Most of the time, it is your app name. In this case, Last Time) and take note of an address(0x0000000104605ca4) and a load address (0x1045fc000).

After you get a load address, find that address under the Binary Images section (It likely to locate below the call stack).

...
Binary Images:
0x1045fc000 - 0x104c77fff Last Time arm64  <3debea750b0530e3ae605a6713606202> /var/containers/Bundle/Application/99665CC5-D1DD-4D87-9349-2769250820AA/Last Time.app/Last Time
0x10508c000 - 0x105097fff libobjc-trampolines.dylib arm64  <048eb53f47913e0a9314876c6577aa10> /usr/lib/libobjc-trampolines.dylib
0x10533c000 - 0x10539ffff dyld arm64  <571392a7e1e6369f8805c1a141f3c1c5> /usr/lib/dyld
0x1a2b4b000 - 0x1a2b61fff libsystem_trace.dylib arm64  <f7e5141b7c243e5aaa79065004ecbf30> /usr/lib/system/libsystem_trace.dylib
0x1a2b62000 - 0x1a2b93fff libxpc.dylib arm64  <217dc1a778213f1fa8373825d770ef05> /usr/lib/system/libxpc.dylib
0x1a2b94000 - 0x1a2b94fff libsystem_blocks.dylib arm64  <c06042b841f63e4994717b606330928a> /usr/lib/system/libsystem_blocks.dylib
...

You will find Binary Architecture sit next to your app name (arm64 in this case).

Now you got everything you need to symbolicate your report. Run this command in your terminal.

atos -arch <Binary Architecture> -o <Path to dSYM file>/Contents/Resources/DWARF/<binary image name> -l <load address> <address to symbolicate>

In our example it would be:

atos -arch arm64 -o dSYMs/Last\ Time.app.dSYM/Contents/Resources/DWARF/Last\ Time -l 0x1045fc000 0x0000000104605ca4

This is what the result looks like:

-[AppDelegate application:didFinishLaunchingWithOptions:] (in Last Time) (AppDelegate.m:85)

You can specify multiple addresses to symbolicate, separated by a space, which is helpful if you have more than one address of interested.

atos -arch <Binary Architecture> -o <Path to dSYM file>/Contents/Resources/DWARF/<binary image name> -l <load address> <address#1 to symbolicate> <address#2 to symbolicate> <address#3 to symbolicate>

Find the right dSYM file

The dSYM file that you download from App Store Connect comes in a zip format containing multiple dSYM files. Named with unique UUID. You have to use the right one based on your crash log file.

To find which dSYM to use, open your crash log file and search for slice_uuid. You will find that at the very beginning of the file.

You won't find something like this:

"slice_uuid":"89177b42-c33d-364d-874b-bf3d19923f9c"

Then use this against 89177b42-c33d-364d-874b-bf3d19923f9c.dSYM file.

Conclusion

There are many ways to read a crash report. Most of them are automated and involve Xcode to see the report, but it has many constraints, e.g., users have to opt-in to send the report, you have to build and archive on your local machine. I think the method above is the simplest and the most straightforward.

I hope nobody needs to use this tip, but someday you might need it. When the day comes, I hope you are prepared.

You can easily support sarunw.com by checking out this sponsor.

Screenshot Studio: Create App Store screenshots in seconds not minutes.

Try it for Free

Related Resources

Understanding and Analyzing Application Crash Reports