What exactly is product page optimization?
Product Page Optimization - App Store - Apple Developer
Product page optimization lets you try out alternate versions
Overview
In a nutshell, you will be able to do AB testing on iOS!
By comparing app icons, screenshots, and app preview videos displayed on the App Store, you can check which patterns led to the most app downloads. You can compare and test up to 4 patterns by combining up to 3 test patterns and the original product page.
You can also view the results in App Analytics and set the App Store to show the highest performing versions to everyone. You can manually stop the test or let it run for up to 90 days until you release a new version of your app.
Testable elements
- Icons
- Screenshots
- Preview videos
Review
New metadata in test patterns must be approved by App Review before the test can be published.
Screenshots and preview videos for testing can be submitted to App Review regardless of the new app version. To test icons, all patterns for icons must be included in the app binary and built using Xcode 13 or later.
Monitoring Indicators
- Number of impressions
- Conversion rate
- Improvement (relative difference between patterns)
- Confidence (whether there is a significant difference in performance between patterns)
Display of the test pattern
The test pattern will be displayed to a certain percentage of randomly selected people using iOS 15 and iPadOS 15 or later.
Anyone who sees a particular test pattern will be able to see the same test pattern across the App Store for the duration of the test. As with the original product page, test pattern metadata can appear in the Today, Games, Apps tabs, and search results.
If you're testing an icon, the icon that was displayed when downloaded will also be displayed on the user's device.
Reference: https://developer.apple.com/app-store/product-page-optimization/
Permissions required to configure product page optimization in App Store Connect
You can check the App Store Connect permissions from the URL below, but it seems that the information related to product page optimization has not been updated. I will post here as soon as there is an update!
By the way, when I tried it with App Manager privileges, I was able to create and submit tests.
Reference: https://developer.apple.com/jp/support/roles/#adp
Let's utilize it right away!
I summarized the flow that I actually tried on the screen of our company application "Dragon Paradise".
Start setting up the test
On App Store Connect, first select the app whose product page you want to optimize from My App, and execute the following.
- Click [Optimize product page].
- Click [Create test].
Set the reference name here. The reference name is the name used to identify the test in App Store Connect, App Analytics, and is not displayed on the store page.
Determine the number of test patterns and traffic ratio
Set the number of logos (test patterns) and the percentage of traffic. Depending on the number of test patterns, the percentage of traffic will fluctuate.
⚠️Caution: Traffic ratio is divided equally for each test pattern.
Equal division example: Number of test patterns 3, traffic percentage 60%
Original product page: 40%
Test pattern 1: 20%
Test pattern 2: 20%
Test pattern 3: 20%
- It is possible to make it the same ratio as the original product page. (e.g. 25% each)
- The percentage of traffic for each test pattern cannot be set to exceed the percentage of the original product page.
Choose a localization. Remove the ✅ except for the localization you want to test. This time, the test will be conducted in Japanese only.
Before setting the test pattern
- [Edit name]: You can change the name of the test pattern. Give the pattern a name that makes it easy to identify.
(Note: The pattern you set will not be displayed as an image in Analytics)
2. [Delete all]: You can clean the test pattern. It seems to be useful when setting a pattern that is completely different from the original product page.
- Screenshots set in other languages were automatically displayed when there was no test pattern. In that case, press [Edit] to clean the test pattern.
Setting the test pattern
- You can add new screenshots and preview videos from [Select file] (dropping is also possible)
- Settings are divided for each display size, and changes do not affect each other.
- If you make a mistake in the size, a warning will be displayed on the setting screen.
- Screenshots and preview videos can be deleted by placing the cursor on the image and pressing ⊝ on the upper left.
- It is also possible to register both vertical and horizontal screenshots.
Reference: screenshot specifications , preview video specifications
Save and submit for review
Although there is no button such as [Save], it seems that the test pattern set at any time is saved.
Even if you leave the setting screen, you can return to the setting screen from the top of [Product page optimization].
⚠️Caution: When I left the setting screen and came back, an error occurred when reflecting some screenshots and preview videos
When all test patterns have been uploaded, press [Start Test] on the upper right.
Note: Although it says [Start test], the test has not started yet.
Select Add for review. This will take you to the final confirmation screen before submitting to App Review. If there are no problems, press [Submit to App Review] in the upper right.
By the way… If the app version was submitted for review, I couldn't submit it for review. The pattern I created before passing the review became unusable after passing the review, so I had to create it again.
Examination and test start
You'll see your review status at the top of Product Page Optimization. Submitted test patterns will be reviewed within 24 hours.
This time, I submitted it around 7:00 pm on December 13th. The test started around 10:00 am the next day, so I was judged within 24 hours!
⚠️Caution: The test starts automatically when you pass the screening, and it seems that you can't control the timing of the start.
See: https://developer.apple.com/app-store-connect/submission-update/
Take a look at your analytics
From the day after the test starts, you can check each indicator from the top of [Product Page Optimization] or from Analytics.
Viewing analytics may require different permissions than the one for which you requested the test. Please check here for details.
Explanation of each item
- [Baseline] pull-down: You can decide which pattern to use as a reference. A numerical value compared with the pattern set as the baseline is displayed.
- [Improvement] pull-down: You can select which values to display in the graph.
- [...] pull-down: Test results can be downloaded as CSV.
- Explanation of “Unique Impressions”: The number of unique users using iOS 15 or later who displayed the test pattern on the App Store.
- Explanation of "Conversion rate": The estimated percentage of users who see this test pattern and download the App.
- "Improvement" commentary: The estimated relative difference between this test pattern and the baseline.
- Explanation of “Reliability”: The significant difference between this test pattern and the baseline.
You can read more about other metrics here!
I checked the test pattern on the actual machine
After the test started, I actually checked the test pattern on the App Store! It turns out that the test pattern is displayed correctly no matter how you set it. Search results when mixed vertical and horizontal screenshots are registered.
Details page when mixed vertical and horizontal screenshots are registered.
Only for patterns that mixed vertical and horizontal screenshots, the first screenshot was displayed horizontally in the search results, and all screenshots were displayed vertically on the detail page.
Before starting the test, I was worried that if the original product page was a horizontal screenshot, the test pattern would also need to be horizontal, but I was able to start the AB test on iOS without any problems!
Summary
Thank you for reading this far! Product page optimization was the hottest new feature for me!
Now that A/B testing is possible not only on Android but also on iOS, I think there is more room to improve store creatives.
This time, we mainly introduced the setting method, but we will continue to deliver articles as needed when we use Product page optimization and find out anything! 💪
Last but not least, we are a professional partner for attracting app customers. Both iOS and Android are available, so please feel free to contact us if you have any problems with app advertising or ASO measures!