Say you are a mobile developer tasked with optimizing the performance of your React Native application. You've heard that using EAS builds and OTA updates can improve the user experience, but you're not sure where to start or how to measure the impact of these changes. You're about to embark on a potentially time-consuming rewrite, and you want to know whether it's worth it.

The Situation

You're trying to improve the performance of your React Native application, and you've identified EAS builds and OTA updates as potential areas for optimization. However, you're not sure how to measure the current performance of your app, or how to determine whether the rewrite will have a significant impact. You've tried using various metrics, such as app launch time and battery usage, but you're not sure which ones are most relevant or how to collect accurate data.

Why it Happens

EAS builds and OTA updates can have a significant impact on the performance of a React Native application, but measuring this impact can be challenging. One of the main reasons is that there are many factors that can affect performance, such as network connectivity, device hardware, and user behavior. Additionally, the metrics that are typically used to measure performance, such as app launch time and battery usage, may not be directly related to the changes made during the rewrite. To make matters worse, the documentation for EAS builds and OTA updates often focuses on the features and functionality, rather than the performance implications.

The Fix, Step by Step

To measure the impact of EAS builds and OTA updates, you'll need to instrument your application to collect relevant metrics. Here are the steps to follow:

  1. Identify the key metrics: Determine which metrics are most relevant to your application's performance, such as app launch time, battery usage, and user engagement. Consider using metrics that are specific to your application, such as the time it takes to load a particular screen or the number of errors that occur during a typical user session.
  2. Choose an instrumentation tool: Select a tool that can collect the metrics you've identified, such as New Relic, Datadog, or Firebase Performance Monitoring. Consider the trade-offs between different tools, such as the level of detail provided, the ease of use, and the cost.
  3. Implement the instrumentation: Add the instrumentation tool to your application, following the documentation provided by the tool vendor. Make sure to instrument all of the relevant parts of your application, including any third-party libraries or services.
  4. Collect baseline data: Run your application and collect data on the key metrics before making any changes. This will provide a baseline for comparison after the rewrite.
  5. Make the changes: Implement EAS builds and OTA updates, following the documentation provided by React Native.
  6. Collect post-change data: Run your application and collect data on the key metrics after making the changes.
  7. Compare the results: Compare the pre- and post-change data to determine whether the rewrite had a significant impact on performance.

What it Costs, and When Not to Do It

Measuring the impact of EAS builds and OTA updates can be time-consuming and may require significant resources, especially if you're using a third-party instrumentation tool. However, the cost of not measuring the impact can be even higher, as you may end up making changes that don't have a significant effect on performance. Consider the following trade-offs:

  • Time and resources: Measuring the impact of EAS builds and OTA updates can take several days or weeks, depending on the complexity of your application and the metrics you're collecting.
  • Cost: Using a third-party instrumentation tool can cost several hundred dollars per month, depending on the vendor and the level of service.
  • Opportunity cost: If you're spending time and resources measuring the impact of EAS builds and OTA updates, you may not have the resources to pursue other optimization opportunities.

At Your Level

  • Starting out: Focus on identifying the key metrics and choosing an instrumentation tool. Consider starting with a simple tool like Firebase Performance Monitoring and gradually moving to more advanced tools as you gain experience.
  • Working engineer: Implement the instrumentation and collect baseline data. Consider using a tool like New Relic to collect detailed metrics on your application's performance.
  • Senior or staff: Analyze the results and make recommendations for further optimization. Consider using a tool like Datadog to collect metrics on your application's performance and make data-driven decisions.
  • Lead or director: Consider the trade-offs and prioritize optimization opportunities based on business goals and resource constraints. Consider using a tool like Google Cloud Monitoring to collect metrics on your application's performance and make data-driven decisions.

In the Interview

When interviewing a candidate for a mobile development position, you may ask questions like:

  • "How would you measure the impact of EAS builds and OTA updates on a React Native application?"
  • "What metrics would you use to evaluate the performance of a mobile application, and why?" A weak answer might focus on general principles or vague metrics, while a strong answer would provide specific examples and metrics, such as:
  • "I would use a tool like New Relic to collect metrics on app launch time, battery usage, and user engagement. I would also consider using metrics like time-to-interact and error rates to evaluate the performance of the application."
  • "I would prioritize optimization opportunities based on business goals and resource constraints, using metrics like revenue and user acquisition costs to guide my decisions." The follow-up question might be:
  • "Can you give an example of a time when you had to measure the impact of a performance optimization on a mobile application? What metrics did you use, and what were the results?"

Amit Chakraborty is a founding engineer and senior architect — React Native, AI/RAG systems and production architecture. Portfolio: www.amitchakraborty.dev · LinkedIn · GitHub. Open to senior and founding engineering roles, remote worldwide.