Meta Platforms, Inc.
Shadow tagging to evaluate content review policy changes

Abstract:

An online system reviews content for violation of one or more policies of the system. The system may evaluate various content policies to determine how usage of the policy affects user experience and what content is shown to each user. The system can, for example, run an A/B validation for certain policies, such as before launching a new policy. To enable the validation, when content is determined to be violating a policy, it is labeled with the specific policy violated as a shadow tag that is not visible to the user viewing the content. Then, the system may track user interactions with newsfeeds of content that include no policy violating content and newsfeeds that include some policy-violating content, and detect at a policy-by-policy level how each policy affects the newsfeed and user experience.