Relic Solution: iOS dSYM upload deep-dive

One of the most useful features of the New Relic mobile SDK is crash reporting. When a crash occurs you want to be able to see the stack trace and figure out what’s causing the crash. It’s not very helpful when all you see is machine-readable memory addresses. In this post, I’ll cover how to get your iOS dSYM file to New Relic and some common issues.

What is a dSYM?

Let’s get this out of the way first. When you create a build of an iOS or tvOS application, the names of methods and classes are stripped, leaving only machine-readable memory addresses. When the application crashes, the stack trace consists of this machine-readable code.

A dSYM file is an Xcode project file that stands for debug symbols. It contains the debugging symbols that allow for translation of the initial crash report to human-readable information. This process is known as symbolication.

How to create a dSYM?

There are several build settings to confirm to be sure your Project Target is creating a dSYM.

Debug Information Format : Dwarf with dSYM File
Deployment Postprocessing: Yes
Strip Linked Product: Yes
Strip Debug Symbols During Copy : Yes

Uploading dSYMs to New Relic

There’s more than one way to upload your dSYM to New Relic. The easiest option is to use the post build script which is included with the SDK. To set this up follow step 6 of our install and configuration doc.

Add a build script to your target’s Build Phases. Ensure the new build script is the very last script. Then paste the following, replacing PUT_NEW_RELIC_APP_TOKEN_HERE with your application token:

SCRIPT=`/usr/bin/find "${SRCROOT}" -name newrelic_postbuild.sh | head -n 1`
/bin/bash "${SCRIPT}" "PUT_NEW_RELIC_APP_TOKEN_HERE"

Now your dSYM files will be automatically uploaded to New Relic after building for production or to a physical test device.

Note: Starting in Agent version 5.7.0 the post build script will upload all dSYMs associated with the app to symbolicate embedded frameworks. Before this version only the one dSYM for the app was uploaded. Please update to the latest version to take advantage of this and other improvements.

What if I need to manually upload a dSYM?

There are a few options for manually uploading the dSYM file. From the browser, you can navigate to a specific crash detail page and use our drag and drop upload UI.

There are two options for uploading dSYMs from the terminal. The dSYMs can be zipped and sent using a curl command.

Note: Be sure to use /usr/bin/zip --recurse-paths. Using a different command to archive such as tar or bzip and failing to recurse paths will cause problems.

The other option is to use the Python script included with the agent files. In iOS agent versions 6.0.0 or higher, the agent includes a Python script that automatically processes and uploads symbols. You can call this script from the command line:

NewRelicAgent.framework/Resources/generateMap.py "DSYM_ARCHIVE_PATH" "YOUR_NEW_RELIC_APPLICATION_TOKEN"

The DSYM_ARCHIVE_PATH can be a single zip file containing one or more dSYMs or a folder containing multiple dSYMs or zip files.

How do I make sure I’ve got the right dSYM for a crash?

To find the build uuid for a crash in Crash Analysis refer to our Find Build UUIDs for unsymbolicated crashes doc.

Use the command dwarfdump --uuid on the dSYM file to show the build uuid.

You can also use this helpful script to search a folder of dSYMs for a specific build uuid.

What if my app is Bitcode enabled?

A little background on what’s happening with dSYMs for Bitcode enabled apps. If your app is Bitcode enabled, it will be fully compiled by Apple after submitting to the App Store as Bitcode is an intermediary compile step. The dSYM needs to be downloaded from iTunes Connect.

The file that is downloaded can have more than one dSYM - one for your application binary, and others for dynamically built frameworks used by your app. This zip file can be uploaded using the curl command, Python script, or navigate to a specific crash and use our drag and drop upload UI.

What if I have a lot of dSYMs to upload?

If you have several dSYMs to upload at once, the Python script mentioned above can be used. You can specify a folder containing multiple dSYM or zip files as the DSYM_ARCHIVE_PATH value.

Troubleshooting

I’m looking at a crash report and a banner says, “We were unable to locate your dSYM.” What’s up with that?

  • You see this message if no dSYM has been uploaded or the dSYM that was uploaded did not match the build uuid for the crash. Please see above for how to find the build uuid.

I uploaded a dSYM and see this banner “We have received your dSYM and are working on processing it” But it’s been over an hour?

  • The structure of the dSYM is not correct. This can happen if you don’t recurse the paths when using zip
  • The dSYM or zip file is corrupted. Try zipping and uploading the file again

My app code is symbolicated but one or more of my 3rd party libraries still show memory addresses.

  • Dynamic libraries which are compiled when your app is built have separate build settings. These build settings should also match the settings given for the main app.
  • Static libraries my not always include symbols when they were built. These symbols are only accessible if included when originally built. New Relic includes symbols for our library.
6 Likes

Hi @spotts - thanks very much for the detailed article on dSYM upload. I have been looking at the NewRelic documentation
https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-ios/configuration/upload-dsyms-bitcode-apps#python-manual-upload
It says the failure to dSYM upload will report back as an XCode build failure, however the newrelic_postbuild.sh script handles it as a silent failure, is it expected? Is there a reason why its a silent failure ?

Is there a specific section you’re looking at? The script will fail with a non-zero exit code in various places. This will cause XCode to fail the build.

@spotts - yes the below lines in the script, lines 168 to 171( I am using the script packaged in 6.10.0 version)

Convert to function and call in background

echo “New Relic: Uploading dSYMs to New Relic”

echo “New Relic: This script will fail silently if dsym fails to upload.”

echo “New Relic: For troubleshooting, see upload_dsym_results file in project root folder”

upload_symbols_to_new_relic > upload_dsym_results 2>&1 &

I’m not sure why the that echo line is in the script, I’ll report it to the engineering team.

For each failure area in the script there are non-zero exit codes which will cause XCode to fail the build.

@spotts - I am not very good at bash scripts, what I am experiencing is the line
upload_symbols_to_new_relic > upload_dsym_results 2>&1 &

any echo texts are wrote to a file upload_dsym_results and any exit scenarios are not captured by the Xcode run script. Thus Xcode treats the build as a success even in case of a non-zero exit.

@spotts Now I realise the upload part is executed on a background process and any exit scenarios or failure to upload to NewRelic will result in a silent failure.
Could you please suggest any workarounds to fail the Xcode build incase the NR dSYM upload fails?

The only way to guarantee failing the build if the upload failed would be to not run the function in the background. Otherwise even if you were to bubble up an error the XCode build may be done already so it still wouldn’t cause a build failure. Doing this could add a lot of time to your builds depending on size and number of dSYM files.

One option would be to open the log file if there is a failure. This would give you a visual way to know an error occurred.

        else
            echo "new relic: error \"${SERVER_RESPONSE}\" while uploading \"${DSYM_ARCHIVE_PATH}\" to \"${DSYM_ARCHIVE_PATH}\""
            open upload_dsym_results
        fi

Keep in mind that if you alter the postbuild script you will need to maintain a version of it outside of the SDK files and update the build phase run script to use the modified version.

1 Like

Thanks @spotts - Using the Python script generateMap.py would be the best fit for us in that case. We need to integrate it part of our CICD and need to fail build (in CI) if the dSYM upload to NewRelic fails.

1 Like

Glad to hear the python script will work for you :slight_smile:

2 Likes