Sharing User Activity Data at Google

Every breath you take
And every move you make
Every bond you break
Every step you take
I’ll be watching you
~ The Police – Every Breath You Take

A new Google Patent tells us that the popularity and convenience of digital devices and the widespread use of Internet communications have allowed much information to be sent and received by users.

This patent focuses on user activity data, and how someone might make a request for it under a Google patent. Please let me know how you may feel about it (remember, I didn’t write the patent.) Thanks.

Related Content:

• Technical SEO Agency
• Ecommerce SEO Agency
• Shopify SEO Services
• Franchise SEO Agency
• Enterprise SEO Services

For example, user content such as text, digital photographs, audio recordings, etc., can get posted to various Internet sites, such as web pages and social networking services, for users to view or otherwise experience. Users can send content at any time to network servers for viewing by others from personal and portable devices. Users can browse social networking sites and other sites to see content posted by various friends and social connections.

Sharing User Activity Data With Other Users

The present patent is about sharing user activity data with other users. It includes evaluating the actions of a second user to detect that the second user gets interested in activities performed by a first user and described in activity data of the first user. A request gets provided to the first user to share the activity data of the first user with the second user.

In response to the request, the method receives a sign from the first user to share a designated part of the activity data of the first user with the second user. The technique causes the activity data of the first user to be accessible to the second user. It provides information for presentation via a device to the second user, where the news gets derived from the shared designated part of the activity data of the first user.

For example, the actions may have gotten performed for a purpose other than accessing the activities and activity data of the first user and may get only related to the activities performed by the first user.

Evaluating Actions Can Include Detecting Data Sent Or Received

Evaluating actions can include detecting searches related to attributes input by the second user to a search engine on a device and selections by the second user of search results related to the points. The search results result from one search input by the second user. Evaluating actions can include detecting data sent or received by the second user on the device, relating to the attributes, and detecting that the second user has visited a location related to the points.

The provided information can get presented in response to the second user using the device and performing at least one action related to at least one of the activities performed by the first user. The information can relate to the action of the second user.

The action can include the second user with a search request related to the first user’s activities. The second user selects, sends, or receives data using the device, including attributes of the first user’s activities. The activity data can reference geographic locations earlier visited by the first user during the exercises. At least one action can include the second user becoming located within a predetermined distance of at least one of the geographic locations.

The first user can have a social connection to the second user. The activity data can include opinions or ratings by the first user about attributes related to the activities and pictures or messages captured by the first user about points related to the actions. It can further include providing at least a part of the first user activity data for display by the second user’s device by input from the second user requesting such collection. The method can include receiving selections from the second user of items in the first user activity data and instructing that notification gets output on the device to the second user in response to the second user performing activities related to the selected items.

This method includes evaluating the actions of a second user to detect that a second user gets interested in attributes of activities performed by a first user and described in activity data of the first user. The actions got performed for a purpose other than accessing the movements and activity data of the first user, and the first user has a social connection to the second user. The method provides a request to the first user to share at least a part of the activity data of the first user with the second user.

The method provides information to the device for presentation to the second user. The data is presented in response to the second user using a machine and performing at least one action related to the activities performed by the first user. In response to the request, the method receives a sign from the first user to share a designated part of the activity data of the first user with the second user. It causes the activity data of the first user to get made accessible to the second user. The method provides information to the device for presentation to the second user. The data is presented in response to the second user using a machine and performing at least one action related to the activities performed by the first user. The information gets related to at least one measure of the second user and gets derived from the shared designated part of the activity data of the first user.

A system can include a storage device and at least one processor accessing the storage device and operative to perform operations. The operations involve evaluating the actions of a second user to detect that a second user gets interested in attributes of the activities performed by a first user and described in activity data of the first user.

The operations include providing a request to the first user to share at least a part of the activity data of the first user with the second user. In response to the request, the method receives a sign from the first user to share a designated part of the activity data of the first user with the second user. The operations include causing the activity data of the first user to get made accessible to the second user and providing information for presentation via a device used by the second user, where the data gets derived from the shared designated part of the activity data of the first user.

Evaluating Actions Can Also Include

The system can include various features. For example, the operation of evaluating actions can include:

• Detecting searches related to the attributes and input by the second user to a search engine implemented on a device
• Detecting selections by the second user of search results related to the attributes, where the search results result from at least one search input by the user in the search engine

Evaluating actions can include detecting data sent or received by the second user on the device, relating to the attributes, and detecting that the second user has visited a location related to the points.

The provided information can be presented in response to the second user using a device and performing at least one action related to at least one of the activities performed by the first user. The information gets related to the least one action of the second user.

The second user with a search request related to one of the first user’s activities, selecting, sending, or receiving data using the device (using attributes like the first user’s activities). It gets located within the distance of a geographic location, where the activity data references the geographic area. A further operation can provide the first user activity data for display by the second user’s device received from the second user requesting such collection.

Sharing user activity data with other users
Inventors: Mario Ventura Anima, Irene Chung, James Kevin Rodgers, and Brandon Herring
Assignee: Google LLC
US Patent: 11,157,572
Granted: October 26, 2021
Filed: August 12, 2014

Abstract

Implementations relate to sharing user activity data with other users. A method includes evaluating the actions of a second user to detect that the second user gets interested in attributes of activities performed by a first user and described in activity data of the first user.

A request gets provided to the first user to share at least a part of the activity data with the second user.

In response to the request, the method receives a sign from the first user to share a designated amount of the activity data with the second user.

The technique causes the activity data to be accessible to the second user and provides information for presentation via a device to the second user.

The presented information gets derived from the shared portion of the activity data.

Sharing User Activity Data With Other Users

This patent is about sharing user activity data with other users. Activity data describing the activities of a first user can get stored. The system can determine whether a second user gets interested in activities that get included in the activity data of the first user based on the actions of the second user.

If there is such interest, the system can request the first user to access relevant activity data by the second user. If the first user grants permission, the system can use this shared activity data to enhance output on devices used by the second user.

For example, information derived from the first user’s activity data can be provided to the second user’s device and displayed to the second user on a device if the second user performs actions related to the first user’s activities included in the accessible activity data.

Furthermore, users can command the display of activity data in convenient and easy-to-understand formats and interfaces, allowing the user to test activity data and perform other functions such as sharing and editing the activity data.

These and other described features can allow automatic, convenient, and relevant sharing and viewing of activity data between users such as socially-connected users.

For example, described features allow a user to get helpful information from friends and other users who have earlier performed activities that are the same or like the activities that the second user is now doing, e.g., who went to the same places and performed the same activities. This helpful information can provide convenient, relevant, and valued opinions, reviews, comments, and guiding information from people the user knows and trusts.

Described features also can provide automatic detection of interest of a user in activities of another user. The system can test a user’s actions that include actions performed for a second user’s other than accessing activities and activity data. These include searching for subjects or topics using a search engine, selecting, viewing, sending content data, or visiting locations.

The system can determine whether the actions get related to attributes of recorded activities of other users. In this way, the system can, in some implementations, check for and infer user interest in other users’ activities in the background. In contrast, the user performs actions for purposes only related to activity data, thus determining which other users’ activity data may become relevant to the user’s interests as an extra (and in some cases, user-unintended) benefit to the user’s actions.

The system also can cause the prompting of the appropriate other users for permission to share their relevant activity data with the user. Described features also include detecting user actions and providing output information based on the shared activity data relevant to the user’s actions.

The output of such information can also become incidental to the user’s purpose in performing the actions. For example, the movements can become intended for other purposes such as viewing search results on topics or subjects, selecting, viewing, sending content data, or visiting locations. The user receives more benefit from the output based on other user activity data.

Such features ease users from asking for and remembering to provide relevant activity data between socially-connected users. Furthermore, users do not need to search for such relevant activity data. Thus, technical effects of sharing activity data as disclosed herein include reducing user time searching for and providing activity data of other users, thus saving a user time, energy, and resources.

Another technical effect is a higher quality of provided information relevant to user interests in activities. It gets presented close in time to when the user has expressed such interest while using a device.

An “activity attribute” or “attribute of activity” as referred to herein can become any characteristics, subjects, or topics that can get included in activity data to describe an activity.

For example, activity attributes can include:

• Location names, addresses, or coordinates (e.g., describing activities performed at the location)
• Specific work or hobby names (describing these activities, e.g., estate planning, comic book collecting, bowling, etc.)
• More general activity names (e.g., traveling, going on vacation, riding a car, viewing content)
• Specific names of works of content getting used in activities such as sending, receiving, playing, or referring to content (e.g., titles of movies, books, games, articles, essays, songs, monument or tour pictures, etc.)
• Particular names of characters, events, objects, concepts, or other characteristics described in a sent/received/referred work of content, etc.

The term “content” or “work of content” can refer to:

• Information can get read, viewed or otherwise experienced by users and including types of information, such as text (books, magazines, papers, letters, text comments, text ratings, etc.)
• Images (photos, drawings, etc.)
• Video (movies, shorts, clips, etc.)
• Audio (e.g., music, voice recordings, sound excerpts from a video source, etc.)
• Games (e.g., a video game, a recording of game play, or other game-related information)
• Geographic location (e.g., map data, GPS data, images of scenery, etc.)
• A mixture of two or more of these types

A work of content is a unit or piece of content. An “image” as referred to herein is a still image, single image, or standalone image, or is an image extracted from a sequence of images, e.g., a frame in a video sequence of video frames.

An example network environment

For ease of illustration, the first figure shows a server system, server device, and database and shows four client devices. The Server may represent many systems, server devices, and network databases and can get provided in different configurations than shown.

For example, a server system can represent many server systems that communicate with other server systems via the network. Another example can provide databases and other storage devices in server system block separate from the servers. They can communicate with a server device and other server systems via a network.

Also, there may be any number of client devices. Each client device can be any type of electronic device, such as a computer system, laptop computer, portable device, cell phone, smartphone, tablet computer, television, TV set-top box or entertainment device, wearable devices (e.g., display glasses or goggles, wristwatch, etc.), personal digital assistant (PDA), media player, game device, etc. In other implementations, the network environment may not have all smartphone components get shown and additional elements, including different types of stuff instead of those described herein.

End Users As Part of A Communications System

End-users U1, U2, U3, and U4 may communicate with the server system and each other using respective client devices. In some examples, users U1-U4 may interact via a social network service implemented on the server system, where separate client devices send communications and data to server systems such as the system. The server system provides appropriate data to the client devices such that each client device can receive content uploaded to the social network service via the server system.

The social network service can include any system allowing users to perform communications, form links and associations, take part in live video, audio, and text chat with other service users. They can also upload and post shared content, including text, images, video sequences, audio sequences or recordings, or different types of content for access by designated sets of users of the social network service, and perform other socially-related functions.

For example, the social network service can allow users to send messages to particular or many other users, form social links in associations to other users within the social network system, group other users in user groups, etc. A user can choose user groups to allow users in the designated user groups to access or receive the user’s posted content, profile information, and other information associated with the user on the social networking service.

Users Access to User Activity Data

Users’ access to user information can get designated larger groups, such as a “public” setting selecting all the users of the social network service, “acquaintances” to state friends of friends, or a different privacy level setting. Some implementations of a social networking service allow the user to choose groups of users, including extended or more socially-linked levels (degrees of separation) of users (e.g., friends of the friends).

A user may also choose other groups or sets of users for access regardless of whether those other users are in the user’s groups. For example, the user may choose users belonging to a group or list or having specified characteristics, such as age, membership in a designated organization, eye color, designated hobbies or interests, members of a selected organization since a particular time or date, etc.

For example, the first user and second user described herein can be users of a social network service or another network service. Other network services or applications can also get another network service implemented using the network environment ten and used by users to communicate messages, content, and information, e.g., photo collection services, web page forums, etc.

A user interface can enable displaying images and other content and communications, privacy settings, notifications, and other data on a client device. Such an interface can get displayed using software on the client device, such as application software or client software in communication with the server system. The interface can get displayed on an output device of a client device, such as a display screen.

Other implementations of features described here can use any type of system and service. For example, any kind of electronic device can make use of the features described herein. Some implementations can process client or server systems disconnected from or intermittently connected to other devices or computer networks. Parts can then be provided to connected devices. In some examples, a client device having a display screen can display images and provide features and results as described herein that are viewable to a user.

A Method For Sharing User Activity Data With Other Users

Some of the methods can get implemented on client devices and both a server system and a client system. The implementing system includes processors or processing circuitry and storage devices such as a database or other storage. Different components of servers and clients can perform other parts of the method.

The method can get implemented by computer program instructions or code executed on a computer. It can get stored on a computer program product, including a computer-readable medium.

The program instructions can also be contained in and provided as an electronic signal, such as software as a service (SaaS) delivered from a server (e.g., a distributed system and a cloud computing system). Or, the method can get implemented in hardware (logic gates, etc.) or a combination of hardware and software.

The method can get based on user input. A user may have selected from an interface such as an application, a social networking interface. The way can get initiated by a system.

For example, the method (or portions thereof) can be continuously or periodically performed based on particular events or conditions such as a user opening an application such as a communication application or editing application, detecting particular user activity (e.g., as described below), etc.

Such conditions can get specified by a user in the custom preferences of the user. The method or portions thereof can get performed with guidance by the user. For example, a user can get the activity data processed by blocks in the plan.

The method (or portions thereof) can get performed on a cell phone, tablet computer, wearable device, or another client device that can send or receive data. In addition, a client device can send data to a server over a network. The Server can process the data using a method or receive data from a different source (a different server, etc.).

What Does Double User Activity Data Describe?

The method determines user activity data, including first, second, and other users. The user activity data can describe activities, conditions, or events that the user performed or occurred about the user. Other related data can also get included in activity data.

Activity data can describe places and locations where the user moved or traveled. It can define types of transportation used by the user or activities performed by the user (sports, games, watching or listening to content data, going to meetings, working, staying at home, etc.). It is also about content data input, posted, and submitted by the user using electronic devices.

Processing activity data definitions are described as follows. A first user’s activity data get determined. A second user’s activity data can also get committed, and the activity data of many other users. Each user can have a set of stored activity data that the associated users can access.

The method provides the second user access to the selected user activity data of the first user. The desired user activity data can include activity data chosen by the first user for access by the second user. They can consist of user activity data selected by the system to be relevant to the second user. For example, the system can detect the interest of the second user in particular attributes of activities and provide dual user access to related activity data of the first user.

For example, the method detects second user interest by evaluating double user actions.

These actions can have a purpose other than accessing activity data, such as performing a search, selecting, viewing, or receiving content data, or visiting a location. Some examples include detecting second user interest, setting first user activity data, and sharing the data for access by a second user.

The method augments what the second user’s device outputs with at least a part of the selected activity data of the first user. For example, this enhancement of production can get performed after the second user has obtained access to the chosen activity data of the first user.

In some examples, enhanced device output can include improved search results, notifications, reminders, content viewing, other production, etc. For example, the enhanced work can be output based on the actions of the second user. These actions can have a purpose other than accessing activity data, such as performing a search, selecting, viewing, or receiving content data, or visiting a location.

The first user and second user can prohibit or otherwise limit the access or use of the first user’s activity data to determine the device output, e.g., turn on or off the enhancement. Some examples of device output enhancement get described.

The features described herein allow activity data describing activities of a user to get processed and provided or accessed by other users for enhancement of output of the devices of those other users. Such features allow one user to be given access to their activity data by other desired users (or other entities), such as friends, family, organizations, or others (all termed “users” herein).

Furthermore, features can include detecting the interest of a second user in the activity data of a first user and providing access to relevant portions of the first user’s activity data, allowing the second user convenient and appropriate access to information with reduced effort in acquiring the data.

The Method Determines User Activity Data

The method can get implemented by the same or different system than a system describing system blocks. For example, a plan can get implemented regularly or periodically on a user’s device or devices, independently of the user’s communications, so that new activity data gets determined for new activities performed or occurring to the user. Activity data can get collected for any user, such as the first user.

User activity can get detected and monitored on any devices used by the user. The method can use a system, not the same machine where the user is performing an activity. A server or client device may be operating on standby and can detect via wired or wireless signals if the user uses other client devices, such as a cell phone or tablet computer.

Such a server or client device can perform a method based on user activity sensed by the remote client device that the user is using (e.g., operating or carrying). The technique gets described as getting run on the same machine the user is performing.

The obtained activity data described below can be stored on local and remote, network-connected storage device(s) accessible to the method. Activity data can get accumulated on storage devices that are also accessible to practices described herein.

The method obtains data collected during activities (including environmental conditions) experienced by the user and data describing such activities. This data gets referred to as “activity data” herein. In some examples, the activity data can include data sensed by a device based on the user’s active or passive device operation.

The user activity data can include written text, audio, image, video, or other types of content data that get recorded or otherwise input, selected, or captured by the user using the device. The user can input text using a keyboard, keypad, or microphone for voice recognition.

A user can input audio data using a microphone or input video or image data by using a camera that gets included as part of the device or a camera that gets coupled to the device.

The input or selected data can include notes, comments, articles, calendar event descriptions, lists, blogs, journal entries, ratings, etc. That data can get uploaded to a network service, such as a social networking service, photo collection service, etc.

Activity data can also include data sensed or received during passive use of the device by the user. Such inactive service can carry a portable device that can receive data from sensors and other devices coupled to the device. The passive use can include designating a device to watch conditions, execute actions based on needs, and send/receive data to/from other devices.

In some examples, the activity data can include location data (e.g., latitude and longitude, or other geographical location data such as locations) sensed by a device or received by a machine from different sensors indicating a geographic or physical location of a user at particular times.

The activity data can also include weather data describing a current weather condition at the user’s location at a particular time or particular types of received data from other sources (e.g., content data, stock market information, news, etc.). In some cases, such activity data can include the times and durations that the user viewed the received data on the device.

More examples of activity data can include time and description of telephone calls, emails, messages, video conference calls, and other communication data sent to and received from specific people or users.

Examples of User Activity Data

More examples include search requests and search results selected by the user, time and selections made during web page browsing or other activity performed in a web browser or other network application, identifications of persons met in physical meetings, such as detections of other persons’ devices in proximity to the user. Those examples include:

• Descriptions of events and activities in calendars
• To-do lists
• Schedules
• Other user lists (meetings, birthdays, vacations, etc.)
• Alarm clock settings
• Sensed mode and duration of transport (bicycle, car, train, walking, etc.)
• Online shopping activities performed through devices (purchases, deliveries, offers, bids, etc.)
• Registrations for network services and web pages
• Physical or biometric data of the user sensed by personal devices
• Sports, and other physical activity data sensed via sensors carried by the user or monitoring user activity
• Etc.

Some implementations can organize activity data into types, categories, and sections. In one example, activity data can get divided into sections. Some areas of data relating to vacations or trips taken by the user and other neighborhoods relate to normal, at-home activities.

Such user activity data can get obtained by the method in various ways. For example, the method can get user activity data via sensors on a device, checking user input on a device, etc. The method can get some or all the activity data from other sources, e.g., other sensors or devices connected over a network that has sensed or obtained the activity data, and online or local databases and services storing reference information such as place names, event names and times, weather data, etc. as well as specific information for the user in user accounts.

The user activity data can get stored on storage devices of the system or otherwise accessible to the system implementing the method, such as a connected storage device, e.g., a local storage device or storage device connected over a network.

Generating More Information About User Activity Data

The method can process the activity data to generate more information about the user and his or her activities. For example, the processing can include aggregating the activity data into summaries in different categories, such as aggregating the time the user spent traveling by car, traveling by bicycle, working at a workplace, going out to lunch, spent at home, etc. In another example, the method can total the number of times the user performed a particular activity, received a particular type of data, etc.

The method can cause the activity data presented to the user if desired, such as activities related to a particular attribute (subject, topic, characteristic, etc.) input by the user, or showing summaries of activities occurring in a particular time span or time designated by the user.

The method can also look for patterns in the activity data to determine which activities the user performs (e.g., occurs more than a threshold number of times within a particular time), and which activities got performed by the user more rarely (e.g., occurs less than the threshold number of times).

The method can cause the activity data presented to the user. For example, the user can view presented activity data on a display, such as a display screen, of a client device or server device.

In some examples, the method can display a list or other display form of activities of the user according to parameters input by the user, such as time, attribute, etc. For example, if the user inputs the most recent day as a time, the method displays the most recent day’s recorded activity data.

The user can request a display of activity data according to user-specified parameters such as a particular time (day, a week, month, etc.), attribute, and another category. Some implementations can display the processed information of the block.

For example, aggregated totals of particular activities in different categories can get displayed, such as the total amount of time that the user spent in different particular activities (walking, riding a bicycle, working at an office, sleeping at home).

Pointing Out Deviations in a User’s Routine

More rare activities that the method found among the user’s activity data in a block can get displayed to point out deviations from the user’s routine. In some examples, a server can store activity data and provide the activity data that is to get displayed by a client device to the user, and in other examples, the client can store activity data to the client and display this data as needed.

In another example, the method can cause a graphical timeline of the desired activity data to get displayed for the user. For example, the timeline can show graphical icons, labels, and other items representing different activities of the user at particular times on the timeline.

A More Detailed Timeline

The items can get selected by the user to show more detail about the selected activity. For example, a more-detailed timeline can get displayed for the selected activity, showing a time scale of greater resolution and activities broken down into many more detailed items.

The method edits activity data based on user input. For example, the user can edit existing activity data, delete activity data, or add to existing activity data using an interface. Block can get performed at any time by the user, e.g., after activity data has gotten obtained, during the acquisition of extra activity data, etc.

The user can select items presented by a block that represent activities, and edit the selected activities. In some examples, the user can add notes or descriptions to particular activities which may not have been automatically determined, such as a description of a movie that got seen, a summary of conversation topics between a group of people, a description of food eaten at a meal, feelings, ratings, or comments about activities, reminders, etc.

The user can also add pictures, audio files, text, or other types of content to any activity represented in the activity data.

The activity data, or portions thereof, may also get edited by the user to be shared with other particular users, or groups that each include users, such as user groups defined by the user in a social networking service. Such directly-shared data can get accessed by designated other users as shared activity data gets accessed by the second user described below.

The user can decide to share certain portions of his or her activity data based on time, subject, type, or other criteria. For example, the user may decide to share activity data that describes activities performed over a time such as a month.

The user can share activity data about an activity attribute, such as a type of location visited (e.g., restaurants, stores, parks, movie theaters, etc.), a certain type of content or hobby (e.g., science fiction movies, gardening, a certain sport), etc., where all the stored activity data of the user relating to that type get shared. For example, the user can select and share visited locations relating to an attribute, comments or ratings of content relating to an attribute, photos depicting content relating to the attribute, etc.

The Method Provides The Second User Access To Selected Activity Data Of The First User

The method can get implemented by a system that is or includes, for example, a server and client device as described above for the method. In this example, the method determines or infers whether a second user gets interested in attributes of activities included in activity data of a first user (or other users), based on actions of the second user.

The actions of the second user can have a purpose other than accessing activity data, e.g., actions that get directed by the second user to a goal other than accessing activity data. Based on that user interest, the method requests the first user to allow access to relevant activity data by the second user and provides that access if permission gets granted. Besides, the method checks if the first user has shared activity data with the second user, e.g., as initiated by the first user.

the method checks (e.g., evaluates) whether the second user has performed actions related to attributes of activities (also referred to as “activity attributes”). The actions can be in any of a variety of forms, and the actions need not get directed by the second user to access activities or activity data of other users.

For example, the actions can get performed by the second user for a purpose and to a goal other than accessing activities or activity data, where the method checks whether the actions are related to attributes of activities. In this way, the method can check for user interest of activity attributes in the background while the second user performs actions for other purposes.

In some examples, the action can be in the form of a search request or search query input into a search engine or other search application, where the method can watch such search requests. The second user can enter the search request in textual form, e.g., using keywords.

For example, the user can input text using an input device of a client device, such as a keyboard, touchscreen, mouse, etc. In other cases, the search request may be in a different form, such as graphical (e.g., image that the user selects for search), audio (e.g., audio file or item that the user selects for search), etc.

In some examples, the search request can include a search query having topics or subjects and has a purpose to get search results from data sources including Internet web pages, databases, local databases or storage devices, etc. The search can get intended by the user to get search results from such data sources that do not store (or have accessible) activity data of particular users, e.g., the second user is not intending to receive results based on activity data of other users.

The Method Checks Whether Search Gets Related To Activity Attributes

Activity attributes can be any characteristics, subjects, or topics which can get included in activity data to describe an activity.

For example, activity attributes can include location names, addresses, or coordinates (describing activities performed at the location), specific work or hobby names (describing these activities, e.g., estate planning, comic book collecting, bowling, etc.), more general activity names (e.g., traveling, going on vacation, riding a car, viewing content), specific names of works of content getting used in activities such as sending, receiving, playing, or referring to sent content (e.g., titles of movies, books, games, articles, essays, songs, monument or tour pictures, etc.), particular names of characters, events, objects, concepts, or other characteristics described in a sent/received/referred work of content, etc.

To determine whether the second user’s search action gets related to a particular activity attribute, the method can parse the search request for attribute words. For example, names of places, countries, cities, monuments, historical entities, occupations, hobbies, etc., can relate a search request to a particular type of activity, such as visiting a named location, viewing a named work of content, etc.

Words in the search request such as nouns, topics, or subjects can be attributes relating to an activity. For example, words describing eating and food topics can relate to going to restaurants, going to parties, or similar activities.

Historical words can relate to going to a museum, viewing a documentary film or movie, etc. Sports scores, team names, stadium names, etc. can get related to sports activities such as spectator sports or participating in sports. Words including hotel reservations or car rentals can get related to travel activities. Products and prices can get related to purchasing or selling activities.

Non-textual search requests can get evaluated as well. For example, a search request using an image can get evaluated by performing recognition techniques on the image to find possible activity attributes such as recognizable objects, such as faces, people, animals, landscapes, monuments, or other recognizable objects. Such recognized objects can get checked for relation to activities. For example, a soccer ball can get related to sports activities, or a monument can get related to travel activities.

The User’s Selection Of Links Can Help To Further Define The User’s Interest

The evaluated actions of the second user can also include a selection of search results by the second user. In some cases, the user’s selection of links can help to further define the user’s interest. For example, if in response to the second user’s search request the system provides search result links to several different types of search results (e.g., web pages found over the Internet having different types of information), and the second user selects one type of search result, this selected result can get considered by the method as whether it gets related to an activity.

For example, the second user may input a search query of “Paris” and receive several different results, including links to an article about the history of the city Paris, a link to travel sites to see in Paris, and a link to a site describing a celebrity named Paris. If the second user selects the city link or travel link, then this can get evaluated by block as an action related to a travel activity.

Selection of other data can also get used as a user action that gets evaluated. For example, the user can select to receive or display content on a device, such as a book, article, movie, song, comments, ratings, or other content.

Such content can include activity attributes such as the names of described locations, persons, actors, characters, objects, locations, themes, organizations or entities, occupations and businesses, feelings, or other subjects.

Such attributes can get related to activities such as viewing, sending, or receiving content, or visiting a theme park or other location including such attributes, for example. Purchases or sales of goods or services (or related actions, such as bidding) by the user can also be user actions that can get evaluated.

Certain Products Purchased Can Relate to User Activity Data

For example, certain products purchased such as sporting equipment can be attributes related to sports activity, while purchasing luggage, tour books, travel accessories, etc., can be attributes related to travel activities.

In cases such as the sale of tour books and similar items that state a particular place or location, an activity related to the place can get inferred. Similarly, as with the search query, these selections can be for selecting or receiving data that is not activity data of other users.

Comments, Ratings, Social Networking, Forums, Photo collections, can all Relate to User Activity Data

In other examples, the action of the second user can be sending or input of comments, ratings, or other content provided by the second user. For example, this content can be displayed, stored, or viewed by the second user or other users, such as other users of a networked service (e.g., social networking service, website forums or community, photo collection service, etc.). In some examples, the second user can write notes on a portable device in a list or journal to read later.

In another example, the second user can write comments on a friend user’s content, such as a positive comment about the subject of a friend’s photo, such as a pet or a place pictured in the photo. A user can rate a friend’s photo, text posting, comment, or other related information with a positive or negative comment.

Such input can get evaluated by the method to determine whether it has attributes related to an activity. For example, input words in a comment or note can get evaluated similarly to the search request described above.

Positive comments or ratings for content can get considered to state interest in the content that is being commented upon. In some other examples, the user can submit an image as posted content, as a comment, or in response to other users’ content.

The image can get evaluated as described above to find recognizable objects that can get related to particular activities. Similarly, as the actions described above, this sending of content data by the second user can be for a purpose other than accessing activity data of other users, e.g., the second user wishes to comment on or rate friends’ postings, content, etc.

Traveling or Visiting Locations Can Also Related To User Activity Data

In other examples, the action of the second user can be traveling to or visiting locations detected by a device, where a location can get related to travel activities or other activities performed at the location.

For example, sensors on a portable device carried by the user, such as GPS sensors or other sensors, can record the latitude and longitude and other location data indicating locations visited by the user.

Sensed new locations can get evaluated for attributes relating to an activity. For example, going to restaurants is related to eating activity, going to a gymnasium or sporting goods store can be related to sports activities, and going to a friend’s house can be related to social activities (party, game, etc.). Such visiting actions can be for purposes other than accessing activity data of other users, e.g., the second user wishes to see a new place, eat at a restaurant, see a movie, etc.

If the method has not detected any actions of the second user related to attributes of activities, then the method continues to block, described below. If the method has detected such actions, then the method continues to block.

The method checks (e.g., evaluates) whether the second user has enough interest in the related activity attributes (e.g., whether the method can infer such enough interest) such that the second user can get considered likely to want to see other users’ activity data having similar activity attributes using features described herein.

Interest In The Second User About the User Activity Data

The method can check for enough interest from the second user in a variety of ways. For example, the method can check whether the second user has indicated an interest in the related activity or activity attribute by a threshold amount, e.g., for a threshold number of times or more.

In some examples, if the threshold number is three, then the method checks for user interest in the activity attribute in separate searches, comments, postings, or other input for at least three times over a predetermined time, such as the last two weeks.

If the threshold number gets met, then the user gets considered interested in the activity attribute. Active interest can get measured in other ways, such as by checking the duration that a user engages in search activity related to the activity attribute, the amount or degree of positive ratings and comments for the activity attribute, etc.

Such parameters as threshold(s), time periods, and durations can get varied or user-specified as desired and can get specified for different types of activities or activity attributes.

The method can prompt the second user to determine the second user’s interest. For example, the method can display a question such as, “Do you have more interest in this activity or topic?” The method can perform this direct prompt in response to not receiving clear signals from the second user as interest in related activities or activity attributes.

If the method has not detected enough interest for the related activity attributes from the second user, then the method continues to block, described below. If the method has detected enough interest, then the method continues to block. The method checks (e.g., evaluates) whether the activity or any of the activity attributes of interest are like attributes includes in the stored activity data of the first user.

Checking For A Social Graph Defined In A Social Networking Service

The method searches stored activity data of users that have a social connection to the second user. For example, the first user can be connected to the second user, e.g., by a social graph defined in a social networking service or another network service, by listing the first user in a contact list of the second user (and vice versa), or by some other defined relationship that connects the first user to the second user and is accessible to the method. All the first user’s activity data can be accessible to the method, e.g., via a social network service, another service, or Server.

Checking For Similarity In User Activity Data

The method can check for the similarity in activity data of each of many users. For example, the many users can be in a social graph, contacts list, or other user group defined by the second user, e.g., indicating a social relationship or other defined relationship with the second user. Each user’s activity data can get examined similarly to the first user’s activity data in the blocks of method.

Checking for similarity can include comparing the activity attributes of interest of the second user with stored data in the activity data of the first user.

For example, the method can check for similar words in the descriptions of activities in the activity data. Particular attributes such as subject or topic words can get compared for similarities, such as the particular names of travel destinations or points of interest (cities, monuments, museums, restaurants, stores, etc.), or the particular names of content or categories of content, such as titles of books, plays, movies, etc. or category names such as comic books, horror movies, romances, etc., or the particular names of products, services, occupations, hobbies, etc.

The method can also check for words, terms, or names that are semantically similar and syntactically similar to these words or are otherwise associated with the words (e.g., a similar product by the same company, a close-by point of interest to a named monument, another item on a menu of the same restaurant, a sequel movie to a named movie, etc.).

Non-textual attributes can get searched for similarity, such as similarity in image content, audio file output, etc. for location attributes, the method can find matches to locations in the first user’s activity data that get located within a predetermined distance of the location of interest to the second user.

In some examples, if the second user expressed a detected enough interest in the city Paris or traveling to Paris, the method can check the first user’s activity data for attributes of the text “Paris”, as well as for attributes related to Paris, such as monument names (e.g., Eiffel Tower), restaurants located in Paris, plane flights to cities near Paris, photos taken at Paris, and other activity data related to travel to France.

In another example, if the second user expressed a detected active interest in eating sushi, then the method can check the first user’s activity data for references to visiting, reviewing, and eating at known sushi restaurants, posting sushi recipes, posting photos related to sushi, or other activities related to sushi.

If the method does not find similar activities or attributes in the activity data of the first user, then the method continues to block, described below. If the method has found similar activities or attributes in the activity data of the first user, then in the block, the method then sends a notification to the first user to request permission for the second user to access a selected part of the first user’s activity data.

In some examples, the selected part can get selected by the method as the part of the first user’s activity data that is relevant, e.g., includes the activities and attributes that got found to be like the activities or attributes of interest to the second user.

Any Other User with Similar User Activity Data Can Get Prompted For Permission For The Second User To Access That User’s Relevant Activity Data

The notification can get implemented in any of several ways. For example, devices of the first user can receive data provided by a device of the second user, e.g., over communication networks, that cause the first user’s device(s) to display the notification that includes the request for permission. The first user devices can provide notification via sound output, tactile output, etc.

The notification can also display the selected portions of the first user’s activity data that is being requested for permission to get shared with the second user. The first user can choose any part of the selected portions of activity data to get denied access by the second user, e.g., refusing permission about a particular designated part of the selected part. One example of notification or prompt gets shown below.

The method can prompt the second user to check whether the second user would like to notify and ask permission of the first user. For example, the method can display a question such as, “Do you want to ask permission to access friends’ activity data related to this activity or topic?” or, “Click Yes to search for other friends or contacts having activity data related to this topic.”

Such prompts can get omitted if the second user has set preferences or settings data that the method can access to state the second user’s preferences for such actions.

The method checks whether permission has gotten received from the first user to allow the second user access to the selected part of activity data (e.g., where the selected part at this stage can be a full, original selected part selected by the method, or a sub-part of the selected part designated by the first user). In some examples, the method can wait for permission from the first user for a predetermined period of time.

If no permission gets received, then the method continues to block, described below. If permission gets received, then the method continues to block, in which the method causes the selected part of the first user’s activity data to get made accessible to the second user. For example, the method shares the selected part of activity data so to provide the second user access to the selected part of activity data.

For example, the method can send the selected part of activity data to a storage device that is accessible to accounts and devices of the second user. Additionally, the method can set permission flags or other controls in association with the first user’s activity data to allow the second user’s accounts and devices to access the selected part of activity data.

If Permission Is Received To Share User Activity Data

The second user can get notified that the first user has granted the second user access to the designated activity data of the first user. For example, the method can cause the granted activity data of the first user to be displayed on the devices of the second user or can cause display of a summary of the accessible activity data, e.g., the time periods the data covers or other attributes of the data (e.g., subjects or topics, names of locations in the data, etc.).

In block, the method checks whether the first user would like to share any of his or her activity data. This block can get performed at any time of the method and can be independent of any request for permission to share as provided in the block. For example, the first user can command that a part of his or her activity data get displayed. If the method does not detect that the first user wishes to share activity data, then the method ends (or can return to block to continue checks).

Otherwise, the method continues to block, in which the method receives input from the first user selecting portions of the displayed activity data to get shared to other designated persons, e.g., to other users of a networked service such as a social networking service, etc.

In some examples, the selections can get provided by the first user with reference to blocks. For example, the first user can specify criteria such as a particular time and subjects to select all his or her activity data that meets the criteria. In other examples, the first user need not share activity data that gets displayed and can select the shared activity data in other ways.

For example, the first user can input a voice command to select all of his or her activity data related to a particular spoken time and subject (e.g., restaurants visited in the last year). Such commanded sharing allows the first user to share particular activity data if he or she believes the second user would get interested or learns from other sources that the second user would get interested in.

The method then continues to block to provide access to the selected part by the second user and any other users specified by the first user. For example, the sharing of the activity data can get performed as described above about block. In some examples, the method can return to block to continue checks.

the method enhances device output for the second user based on accessible activity data of the first user

The method can get implemented by a system that is or includes, for example, a server and client device as described above for method.

In some examples, the method can enhance (e.g., augment) device output with activity data (or with data derived from activity data) in connection with actions performed by the second user as determined by a device, where the actions get method can relate to activities or activity attributes of the first user as described in the activity data.

Some examples of user actions and their detection and enhancement of device output by the method get described below. The actions of the second user evaluated for determining enhanced output can be similar to the actions evaluated for determining user interest.

For example, in some implementations these actions can get performed for purposes other than obtaining output enhanced with the first user’s (or other users’) activity data, e.g., to get search results from general web pages, comment on friends’ content, visit a place, etc., as described above, and the enhanced output from the method is extra output for the second user in beside any intended output.

The method checks whether the second user has input a search request or search query. For example, the second user may have input a search request in a search engine of a website that performs searches in web pages on the World Wide Web. In other cases, the second user may have input a search in a website’s search field to search for results only within the website.

In other cases, the second user may have input a search in a search field of a search engine of an application to search for results in local storage (e.g., files and documents stored to a client device, or in a particular directory of storage), in a particular set of documents or files, in devices connected in a LAN or other contained network, etc.

If the second user has not input a search, then the method continues to block. If a search has been input, then in the block the method enhances the search results of the search to include matches (if any) from the part of the first user’s activity data that has gotten shared with the second user.

If User Activity Data Is Found

For example, the method determines activity attributes in the search request that match or are like activity attributes in the first user’s activity data, e.g., the search request attributes get related to the activities of the first user.

Normal search results get obtained and output using the standard search techniques for the search query (e.g., standard techniques used by an application controlling the search), and the method also performs a search of the part of the first user’s activity data that has gotten shared with the second user, using the input search query.

The method can use the same search process and techniques used by the standard search techniques to search the activity data, if available and appropriate. Or additionally, the method can search the activity data using different search techniques. For example, the method can find not only direct matches to the search query but also activity data like activity attributes in the search query.

For location-based matches, the method can find matches in the first user’s activity data that get located within a predetermined distance of the current location of the second user, while other matches outside that distance can get ignored. Search parameters such as the predetermined distance can also get made configurable by the second user.

The search results obtained from the first user’s activity data can get added to the normal or standard (e.g., intended) search results and presented in line with the normal search results. For example, displayed search result matches from web pages on the Internet can get augmented with matches from the first user’s accessible activity data displayed amongst the other matches.

Distinquishing User Activity Data

Or, the search results can get highlighted or indicated in a particular way to distinguish the search results from the standard search results. For example, the search results can get labeled with the name of the first user to state the origin of those search results.

In one example, the second user requests a search with a search query of “Paris restaurants” in a search field, such as a search engine. The method also receives the search query and checks whether a shared part of a first user’s activity data has any matches to the query. For example, the method determines that the accessible activity data describes seven restaurants that the first user visited while in Paris three months ago. The names and addresses of these restaurants get included in the activity data.

The activity data also includes notes or comments from the first user that state his or her opinion of the meals he or she ate at two of the restaurants, and also includes photos taken by the first user at three of the restaurants. The method can collect all this activity data related to the restaurants and cause it to be output as search results with the normal results obtained by the search engine.

For example, the results from the activity data can get presented as links on a display screen above, below, or to the side of the display of the normal search results. Some examples get described below. In this way, the second user obtains comments, pictures, and other data from the first user’s previous activities, which the second user may be more interested in than generic search results from a search engine.

The method checks whether the second user has sent, received, and selected data related to the accessible first user’s activity data. For example, the data can include attributes like attributes in the accessible first user activity data. The sending, receiving, or selecting can include any of a variety of actions taken by the second user or by the device of the second user.

For example, the second user can select content to view, send, or receive on his or her device, such as text content, image or video content, or audio content, which provide reviews, comments, ratings, books, news, photos, movies, video shorts, chat or other communication content, etc.

Or, the second user can select items such as products or services to buy, sell, bid on, etc. The attributes of such data can get determined by the method by searching for words or phrases in the data (or searching available summaries or synopses of the data), for example.

Searching for Similar Attributes in User Activity Data

The method can then search for similar attributes in the accessible first user activity data, such as similar locations visited by the first user (if the second user data includes locations), comments, ratings, or notes created by the first user that include similar attributes to the second user data, pictures taken by the first user that get related to the second user data, or other activity data related to the second user data.

If the second user did not send, receive, or select data related to the accessible activity data, then the method continues to block, described below. Otherwise, in the block, the method enhances the output of second user devices with a part of the first user activity data that is relevant to the second user’s data. The enhanced output can take any of a variety of forms.

For example, the method can cause an output of a list of the activity items that have similar activity attributes on a display of a second user device. In one example, if the second user is reading a text selection or is viewing an image or video selection, the method can cause a display of links to one side of the screen, labeled with the related first user activity data such as items (e.g., text links, links showing related photos or other content data, etc.).

The second user can select one of the items to view greater detail of the associated activity data. In another example, the selected data is a product that the second user intends to buy, and the method causes a display of a similar product purchased by the first user, the date of sale, and the price paid by the first user, e.g., in a separate window or area of a display screen from the second user’s buy activity.

The method checks whether the second user is currently located at a location close to any locations described in the shared part of the first user’s activity data. For example, raw latitude and longitude data from GPS sensors on a portable device carried (e.g., held, pocketed, worn, etc.) by the second user can state the second user’s current location, and an address, name of a business at the current location, or another descriptor that can get obtained by the method from accessible information sources.

The method can compare the second user’s current location to locations described in the accessible part of the first user’s activity data, e.g., looking for matching coordinates, addresses, business names, place names, etc.

The method can look to see whether any locations in the accessible activity data is within a predetermined distance of the second user’s current location. The method can examine pictures captured at the current location of the second user and compare the second user’s pictures to pictures included in the accessible activity data of the first user.

For example, the method can compare location metadata of the pictures, and compare the content of the pictures to the first user’s pictures, e.g., to try to find similar image content within a predetermined similarity threshold.

If the second user is not located close to any locations in the accessible activity data, then the method can continue to block, described below. If the second user gets located close to any such locations, then in the block the method outputs portions of the accessible activity data that is relevant to the current location of the second user.

Results of User Activity Data Information Can Take Many Forms

The output can take any of a variety of forms. For example, the method may have found pictures taken by the first user at the close-by location, and such pictures can get displayed, e.g., in a separate part or window of the display screen, with labels indicating the first user’s involvement. The method may have found comments or ratings about the location from the first user’s activity data, and can display this content or can display links or icons that will cause the full comments or ratings to get displayed in response to the second user selecting the links or icons. The method can then continue to block.

In one example, the user gets located next to a restaurant, and the method finds that the same restaurant got visited by the first user (e.g., close-by coordinates get included in the first user activity data), as well as a review of the restaurant by the first user and a picture taken by the first user at the restaurant as included in the activity data.

This review and pictures can get displayed by the method on the second user’s device with, for example, an alert or notification indicating that the first user has been to this location.

In this way, the second user can get notified about each location that the first user visited at the time that the second user also visits close to the corresponding locations. The second user can also getearlier notified about the next location that the first user earlier visited after the current location, so that the second user can “replay” the first user’s tour or vacation in particular areas if desired (e.g., if the first user gave positive recommendations for the tour in the accessible activity data).

The method checks whether the second user has requested the output of the first user’s accessible activity data. For example, the second user can in some implementations input commands to display accessible portions of the first user’s activity data on a display screen of a device of the second user.

If such commands are not received, then the method can return to block to continue checks. If the second user has input commands to output activity data, then in the block the method causes the output of portions of the accessible first user’s activity data that get requested by the second user. For example, the second user can input commands to display the activity data in desired formats, such as lists, timelines, etc. Some examples of activity data output. The method can then return to block for continuing checks.

The second user can select notifications to be output to the second user from second user devices in response to events or conditions related to activity data. For example, the second user can state items in the accessible first user activity data for notification, such that a notification is an output on the device to the second user in response to the second user performing actions related to the indicated items (e.g., visiting a location, performing a search, etc., related to attributes of first user activities).

The first user and second user can prohibit or otherwise limit the access or use of the first user’s activity data in the determination and enhancement of the device output, e.g., turn on or off any enhancement, or limit the enhancement to particular activities or activity attributes, etc.

It should get noted that the searching of the first user’s activity data to enhance output on the second user’s devices can also get performed for other users’ activity data that may have gotten shared with the second user.

Such enhancement of second user device output can also get applied based on the second user’s own stored activity data, e.g., if the second user would like a reminder as related activities that the second user performed in the past.

The output of data as described for the method can get performed using a server and a client device. For example, a server can perform of the checks of the method, can store activity data, and can provide actionable data to a client device, the activity data to get included in enhanced output by a client device used by the second user. In other examples, a client device can store activity data, perform the checks, and provide or include activity data in enhanced output that is output from the client device to the second user.

Various blocks and operations of methods can get performed in a different order than shown and at least where appropriate. For example, some implementations can perform blocks of the methods at various times and based on events not related to a user editing an image. Blocks or operations of methods can occur many times, in a different order, and at different times in the methods. The methods can get implemented, for example, on a server system. Client devices can perform blocks instead of or beside a server system performing those blocks.

One example of displayed activity data of a user

In this example, some of the activity data get displayed in a timeline format for easy reference by a user. This timeline can get displayed on a display screen of an electronic device, e.g., on a client device such as a phone, tablet computer, etc.

For example, the activity data at the top left can be the beginning of the data displayed in a vertical timeline, which continues down the left side, continues at the top right side, and continues down the right side. In some cases, a part of the timeline can get displayed at one time on the screen of a device, with the rest of the timeline displayed by being scrolled onto the screen based on user commands.

The activity data can also get provided in a user interface that allows user interaction with portions of the displayed data, as described below.

Activity data can get displayed by a user owning the data, e.g., the first user in the examples described above. For example, the first user can display the timeline to review his or her own activities that occurred in the past.

The activity data and interface can also get displayed in a timeline format for a second user (and other users) that have gotten given permission and access to the shown part of the first user’s activity data by the first user, such as in ways described above.

Activity data includes aggregated summaries of various categories of activity data, including total time spent at work, at home, and bicycling during the previous day, as well as the number of notes received from other users during the previous day.

The timeline shows a chronological view of some activities of the first user during the past day, displayed as icons or items in the timeline and some of which also have timestamps displayed. Such items include locations at which the user got located (e.g., work, home, restaurant, cafe), and modes of travel (e.g., bicycling and walking).

Photos taken by the user can get displayed, such as photos, after the items that state the locations and times, they got captured along the timeline. These photos can get shared by the first user with friends (such as the second user and other users defined in user groups of a social networking service or another service), e.g., by selecting options.

In other cases, the first user may have input text content, audio content, or other content that can also get displayed in the timeline at the appropriate locations and times and shared if desired.

The timeline also can include notes that have gotten, shared or made discoverable by other users, such as friends of the first user. For example, a note may have gotten made shareable by a friend if the first user travels to a particular location, such as the cafe shown, as detected by the device carried by the first user (e.g., using GPS sensors or other sensors/signals).

The timeline can also display the contents of the note, such as text comments or reviews and a photo (or other content) captured or created by the friend at the location or related to the location in some way (e.g., showing a product sold there, a sight seen there, etc.). The user can also select an option to get reminded on his or her device to display the note when the first user next goes to this location, e.g., to get reminded of a friend’s recommendation for a particular product.

Further details can get viewed by the user accessing the activity data. For example, any of the items in the timeline can get selected by the user using an input device (e.g., touchscreen, mouse, etc.) to view greater details such as a timeline at a higher resolution of time, a breakdown of activities during that time, further descriptions, etc.

The first user can share activity data based on the displayed activity data. For example, the first user can select a part of the displayed activity data and share that with other users, such as the second user. For example, a box can get drawn around a part of the timeline and all the first user’s activity data within that time (or only the displayed activity data within that time) can get shared with other users specified by the first user.

The First User Requesting Permission From The Second User To Access The First User’s Activity Data

For example, the second user can be “Sarah” in this example and the part of user activity data gets related to “Paris.”

The first user can select the displayed buttons to permit or deny access to the part of activity data.

The first user can select a button to view the user activity data requested for permission to share, and the first user can select designated user activity data to get shared in more detail, e.g., using a displayed graphical interface.

An Example Ddisplayed Interface Demonstrating of Features

The interface can get displayed by a server or client device, such as on a display screen, via audio output, etc. In this example, the second user has access to a selected part of the user activity data as described above, and device output for the second user can get enhanced with that user activity data. For example, the second user may have input searches of various subjects in search engines before having gotten granted access to the user activity data matched attributes of those search queries to portions of the first user’s activity data and made those activity data portions accessible to the second user and to new searches performed by the second user, as described above.

The second user has input a search query to a search application, such as a web search engine of a web page displayed in a browser or other application. For example, the second user has a purpose in finding matches to the query from web pages located over the Internet and from other publicly-accessible data sources on the Internet.

In response to the search query, the search engine has found search results from such standard Internet data sources, which it displays underneath the search query in a left part of the interface. In this example, the search results are links to web pages accessible on the Internet that have gotten matched to the search query by the search engine.

Besides, according to the features described herein, the search application also displays output information to enhance the search results. Information includes a notification that users known to the second user have user activity data including activity attributes related to the search query.

In this case, the notification indicates that the first user Dan has Paris restaurant data in his activity data that is accessible to the second user. This gets displayed, for example, after the method has determined that accessible activity data of the first user has activity attributes matching attributes of the search query. Some implementations can also display the time of the first user’s activity data that got searched (not shown).

More information and options can get displayed to the second user in information. For example, links can get selected by the second user to display details about the first user’s activity data as described by the links.

The user can select to view the matching attributes themselves (e.g., restaurants visited), or just comments made by the first user about those attributes or activities, just pictures taken by the first user related to the matching attributes, or a timeline of the first user’s activities related to the matching attributes. Some implementations can also display a prompt that allows the second user to input a different query or attribute (e.g., topic or subject) to search for in the activity data of the first user, and which can cause the output to change based on the new query.

User Activity Data Involving Links Displayed In The Information

If the user selected the “visited restaurants” link, then the display in the left part of the interface can get changed to display information related to the selected link. Matched items related to the search query get displayed in a list format, which is:

• The restaurants visited in the first user’s activity data
• The date that the first user visited each restaurant
• Ratings from the first user for each restaurant
• Any other metadata or related content associated with each listed item

For example, icons can allow the user to select to view text notes, image data such as photos, and audio data such as voice recordings that genre associated with each restaurant. For example, the first user may have created this associated content when engaging in the related activity (e.g., going to and eating at the restaurant).

The second user can also access other related data. For example, a particular item can get selected to view greater detail about the item, such as the particular times at which the first user got located at the restaurant or performed the activity (eating), or other really known information about the item from other accessible information sources (e.g., Internet web sites and other sources). The second user can cause a map to get displayed showing the selected item. For example, a map button can get selected to display a map showing all the matched items in their respective relative locations.

About Bill Slawski

With more than 26 years of SEO experience and a Juris Doctor Degree, Bill Slawski is the foremost expert on Google’s patents as related to SEO. Patent Exploration is one of the quickest and most detailed ways to find new information about SEO. Bill is the Editor of SEO by the Sea, a prominent search engine optimization blog, where he is the author of over 1,300 posts. Bill’s experience includes Fortune 500 brands and some of the largest websites in the world. Bill is a contributing author for Moz, Search Engine Land, and Search Engine Journal. In 2014-2021, he spoke at industry-leading international conferences about topics including search engine algorithms, universal and blended search, personalization in search, search and social, and duplicate content problems, structured data, and schema