A/B Testing for Developers

Use the Salesforce B2C Commerce APIs if you want to use more than promotions, sorting rules, and slots to create a unique A/B testing user experience. For each test segment, you can include effects in templates that change business logic, visual elements, or other user experiences based on test group participation. You can use the B2C Commerce API to determine participation in a specific group of a specific test, and then use this information in conditional logic to alter the experience for participants of the segment in various ways.

You can configure a test group that has automatically applied effects, custom effects or both.

Tip: Don't create a test segment that has no automatically applied effects (for example, sorting rules, slot configurations or promotions) or any custom effects via the scripting API, because it would be the same as the control group. As a best practice, avoid this because it provides no useful results.
Note: B2C Commerce has no way to validate that custom effects have been or will be applied for a test group.

Create a custom test experience for a test segments (see Creating an A/B Test) by setting the Enable/Disable Custom Scripting setting for that segment within the Business Manager A/B test module. By setting this, you are notifying B2C Commerce that the specific A/B test segment will include additional changes to the storefront that you will implement in script. For example, you might want to select a different price book by creating a pipeline that makes a price book available to participants in a specific test group.

Caching Impact

Assuming that every A/B test group can have a custom experience through a script would lead to caching performance problems because B2C Commerce must cache a different version of each page for participants in each A/B test segment, in case the script affects the segments differently. Instead, B2C Commerce assumes that only the segments where the Enable Custom Scripting setting has been set must be cached separately.

If you write a script and forget to indicate that a segment has a custom experience, the cached version of the page is based on the participating segment for the first storefront customer to access the page. Thereafter, all storefront customers get the page as if they were in that segment. If the first storefront customer isn't a participant in any segments in the test, then it effectively eliminates custom experiences for that page for all segments of that test.

CAUTION:
If you use this setting but don't actually use a script, you might impact performance.

Related Links

APIs for A/B Testing

Caching with A/B Testing