Empresa engajada
A/B testing question - how would you test a certain linkedin feature. In my case it was 'people you may know' feature. How would you initially think about it when you don't have any data to base the decision off of?
Sigiloso
The two important things to understand before finding the methodology to test a certain feature are: 1. (Why) What was the reason that the feature was introduced? If the feature is "People You May Know", it might have been introduced in order to urge people to connect more with the people they either know outside the platform but haven't connected on the platform or to connect with people whom they might have professional interests with such as having the same job function in a different company or an executive in a company where I need to pitch my solution. It is important to understand the objective both from the user perspective as well as the business perspective so that we have the relevant metrics for both the objectives tracked. From a business perspective i.e. for LinkedIn, the more the connections, the more are the chances of triggering network effect and the more are the chances of selling business solutions or premium services. More interactions between connections can lead to more data that would help segment the users better and help better targeting for ad services. The objectives might be endless but we must find the basic objectives behind the features to test the success. Other objectives might be fulfilled later by tweaking the features and identifying the impacts through Impact Mapping. 2. (How) How do you know that the objective has been achieved? Every objective must be quantitatively evaluated to check if it has been achieved. This is because we have performed a tradeoff against some other feature to introduce this feature and the ROI needs to be justified. From user perspective, it can be the number of connect requests sent or even rejected (as this is another CTA that will help you prioritize one section over another). For example, if no of rejects in the section of "People You May Know from Bengaluru" are higher than "People You May Know from -previous company-", it might be placed above the fold as compared to the former. From business perspective, it is either through heuristics or through user research or through data that "people with more than 2k connections are more likely to buy a premium service". So that would be the relevant metric to track. Now, once we have determined the objective and the metric to track, we can determine the steps that can be taken to test the feature. This feature is a generic feature that has been created irrespective of the user segment. The feature, rather, functionality of "People You May Know" can be extended to "People You May Know Who.....". Now, different user segments would display different affinities towards the part ahead of "Who". Some users might have better connections during their educational days, some during their professional, some during volunteering, some personal, some location specific etc. Since there is no decision to base off to start with, we would have to derive the relations out of multivariate testing. A/B Testing is just one aspect of multi-variate testing involving 2 variable modifications. Since we have multiple sections after who such as Location, Education, Profession, Previous Workplace, Job Function etc, these variables can be tested simultaneously and user segments can be created based on the first 2 or first 3 combinations or even the first section to start with. Tools like Optimizely can help perform such multivariate tests by sampling the user base and providing appropriate sections. However, this assumes that we have a homogeneous sample base whose segmentation we are not aware of i.e. we do not have a base to perform the evaluation. If we already have segments provided, then we can perform A/B Testing on the segments to determine the sections or order of sections suitable for the segment.