A/B Testing and Benchmarking Benchmarking compares recommendation strategies while keeping assignment and attribution consistent. Froomle-managed assignment is recommended because it places every branch on the same assignment and measurement foundation. Assignment ownership and rendering ownership are separate decisions. First decide who assigns the visitor to a branch, then decide which application component renders and tracks the content for that branch. Agree the number and exact names of benchmark groups with your Froomle account team before implementation. Use those names consistently in configuration, response-driven rendering, and reporting. Whether an outbound request or event contains user_group depends on assignment ownership as defined below. Scenario 1: Froomle-Managed Split (Recommended) Froomle assigns visitors to the configured groups. The assignment is stable for the device, including anonymous visitors who later log in. Option A: Full A/B Test (Model vs. Model) Froomle compares two or more recommendation strategies. The response contains the items selected for the visitor’s assigned strategy, and the application renders the returned result. Option B: Froomle vs. Internal Baseline Froomle compares its recommendation strategy with customer/control content. The response user_group is the branch authority: Resolved branch Returned items Expected rendering Customer/control Non-empty or empty Render customer/control content. Do not select a branch from item count alone. Treatment/Froomle Non-empty Render the returned Froomle items. Treatment/Froomle Empty Render the agreed empty or fallback state for the Froomle branch. For Froomle-managed assignment: Do not pass user_group to select the request branch. Preserve the returned request_id and resolved user_group through rendering. Attribute impressions and clicks to the response that produced the rendered branch, but do not echo the returned user_group in those events. Froomle assigns the version and group server-side. Scenario 2: Customer-Managed Split Use customer-managed assignment only when an external experimentation platform must decide the visitor’s branch. For each request: Resolve the visitor’s branch in the customer experimentation system. Pass the agreed user_group for the customer/control branch. Omit user_group for the treatment/Froomle branch so normal Froomle version resolution applies. Render content according to the resolved branch, not according to whether returned items happen to be empty. Track impressions and clicks for the rendered branch with the same attribution decision. Customer/control recommendation events carry the agreed control user_group. Treatment/Froomle events omit user_group and rely on the returned request_id for attribution. User Group Overview Concern Froomle-managed Customer-managed Who assigns the branch? Froomle Customer experimentation system Pass user_group in the request? No Only for customer/control; omit it for treatment/Froomle Rendering authority Returned user_group Customer decision, represented by the agreed request contract Customer/control events Omit user_group; preserve request attribution and let the Events API assign it Include the agreed control user_group Treatment/Froomle events Omit user_group; attribute through request_id Omit user_group and attribute through request_id This matrix is implementation guidance for manual/custom API calls and bare programmatic SDK rendering. For SDK-owned declarative DOM and React placements, the SDK applies it automatically: keep rendering through the supported SDK primitives and do not duplicate branch-attribution logic in application code. Event Requirements A benchmark is valid only when shown content is measured for every branch. Send an impression for every displayed recommendation or control item included in the benchmark. Send a click_on_recommendation when a displayed recommendation or control item is clicked. Keep item, placement, request, and branch attribution aligned with what the visitor actually saw. Attribution field Shared contract action_item Stable ID of the item that was shown or clicked. action_item_type Item type used by the synchronized catalog. list_name Placement/list being benchmarked. request_id ID returned by the recommendation response. Never invent or hard-code it. user_group Include only the externally assigned customer/control group in a customer-managed split. Omit it for customer-managed treatment/Froomle and for every Froomle-managed branch. When both branches follow this attribution contract, Froomle dashboards can report their performance side by side using the same measurement pipeline. Consent Benchmark assignment must not override consent rules. Anonymous requests remain anonymous, and tracking events are sent only at a consent level that permits them. Keep only the routing information needed to resolve the agreed benchmark branch. Implement the benchmark Method Implementation guide Frontend SDK Benchmarking and A/B Testing with the SDK Manual / custom integration Manual recommendation benchmarking and Manual event integration Related Topics Performance & Metrics Event model and tracking contract Recommendation request model