Explore Recommendations

𝕏 is designed to let you know what’s happening in the world. Explore catches you up on a range of subjects. The For You tab in Explore, makes it easier to find content that is recommended to you in other areas of the app. As you scroll through the For You tab in Explore, you’ll see content recommended from Trends, Events, Topics, and Videos.

Typically, you’ll see these sections under Explore:

  • For You: Trends, Events, Topics, and Videos that 𝕏 recommends to you.

  • Trending: What’s happening in the world. Trends displayed on the Trending tab represent what's trending within a specific geographic area, and are not personalized for every account. 

  • News, Sports, Entertainment tabs: Other top Trends that 𝕏 recommends to you and filters into categories. Availability of these tabs may vary based on your location.

How 𝕏 finds content to show you 

𝕏 finds content to show you in Explore by sourcing it from other areas of 𝕏 where content is recommended to you (e.g. trends, videos). For example, the Trends recommendations made to you in the For You tab in Explore are explained in more detail here: Learn more

How 𝕏 decides which content to show you 

In Explore, 𝕏 aggregates the content from its various recommender systems and then removes content that may provide you with a negative experience (i.e. excess content, content that you have indicated that you are “not interested” in, or trends that you have indicated are “harmful or spammy”). 𝕏 will then position trends, videos and topics in a way that appears cohesive and appeals to you. 

How you can control the content you see 

You can change the content that is shown to you in the For You tab in Explore by changing the Topics that you follow. Learn more. You can also influence content that is shown to you in Explore by reporting content shown on any surface (i.e. content that you are “not interested” in, or trends that you have said are “harmful or spammy”). 

𝕏 has also designed tools that help you control all content that you see across the platform and to protect you from content you consider harmful. Learn more

How you can see non-personalized content

You can always choose to see trends that are not personalized for you by selecting the Trending tab in the Explore setting. These trends identify popular topics among people in a specific geographic location. You always have the ability to see what’s trending in other locations by changing your location setting. Learn more

More information

For a more detailed view of how our Explore recommendation system works, please see an overview from our engineering team below.

System Overview

 

The Explore services will be broken up into 3 distinct service layers:

  • The client (iOS/Android/Web) will request data from the explore endpoint when the user taps the Explore tab.

  • Macaw-guide is the HTTP API that the requests will be routed to that will return responses in JSON format.

  • Guide-Mixer will then be called. It is the backend service that aggregates & positions content

  • Candidates for the Guide will be generated by individual CandidateSourceService(s), which are separate services which prove the content shown on the Explore page. Examples of such services are trends, events, videos, and news

Macaw-Guide

Macaw-Guide is a lightweight http api that serves the json endpoint. Macaw-Guide is also responsible for hydrating all the objects such as posts, Users, Cards, and Events. Hydration adds context to posts, Users, Trends, Events, etc sent back by guide mixer.

Adding context is synonymous with adding details such as media information, hashtags, card url, user language, etc.

This is done in the Macaw guide stage and as close to the client as possible so as to avoid transfer of huge payloads over the network in the previous stages where it is not needed. This information is mostly utilized to appropriately display the content such as media and hashtags on 𝕏.

Guide Mixer

The guide-mixer is responsible for aggregating contents from a variety of candidate sources (e.g. trends service, events service, topics service, etc) and then positioning them into a cohesive response. The position of #trends vs videos vs topics such as which gets to appear on top and which goes at the end of the page is determined by guide mixer

The position is determined by a rules based engine that will allow users to change the way the explore is constructed. The rules are coded in a config (yml file) which is ingested by guide mixer and parsed through to determine what appears on the tab and at which positions. 

Guide Mixer sources data to display and positions them using the following steps

MixerCandidateSource

Guide mixer calls various other services (MixerCandidateSource(s) such as events service, trends service) to get the required data (candidates) to display on the explore page. Usually the data is in the form of Ids (postIds, userIds, trendIds) which are later added context by Macaw Guide. Ids are a unique representation of the particular post, trend, user, etc. 

Guide mixer calls the services that are specified in the config

Mixer-Formatter

The yml file will contain information about which candidate sources to call and which MixerPostProcessor to use on the candidate sources.

The Mixer-Formatter will parse through the yml file and call the services in the modules section of the file based on their corresponding path in the candidate_sources section. The path is how the guide mixer determines where the service lives and how to call it. 

The count determines how many ids from each service are required to show on the explore page. 

The order specified in the yml file is how the MixerCandidateSource(s) will be called and ordered in the response of Guide Mixer

 

GuideMixerFlow

This is the logic class that will actually fetch candidates based on the config , then filter and rank the results.

The filters remove excess candidates from the provided candidates of MixerCandidateSource based on the count provided in the formatter.

The ranker fetches the user feedback from the Explore FeedBack Settings. User feedback is explicit feedback from the user which is recorded when a user clicks on the hide options on the page . The feedback ( SHOW_LESS, UNDO_SHOW_LESS) is then combined into a single score. Based on the score the candidates are dropped from the response if the score is less than 0. 

Mixer-Postprocessor

The postprocessor is a piece of logic that lives in the “rank” step of the above FetchRankFlow. It deals with any other logic we may want to apply on the results passed from the formatter. Note that the postprocessor is a single object that ALL results will pass through. This logic can be consistently applied to the responses.

Post processor can be used for but not limited to the following operations 

  1. Promote ads to the top of the page

  2. Change the order of the modules (Trends, topic, videos) 

  3. Insert a certain module in between another module (Videos for you between the trends on for you page)