As reported by Danny Sullivan on Search Engine Land, Bing has announced and rolled out an update to its Satori-fueled Snapshot today.

As with Google's Knowledge Graph, Bing's Snapshot (or "Snapshots" – the label is awkward one) is entity-focused, and the update extends the number of entities being shown, improves and extends the relationships displayed between entities, and displays Bing's prowess with entity disambiguation.

While this is resulting in more detailed and better-connected results being shown for all named entities, the biggest impact is certainly in the display of personal named entities (you know, "people") in Snapshot verticals.  And in this they're arguably now doing a much better job than Google is with their Knowledge Graph (and so my reference to a "personal offensive" in the title). With this latest update Bing is more than ever taking advantage of its social partnerships, and is in general exploiting the availability of social information for (living) people.

This is an interesting – and I think substantial – challenge to the Knowledge Graph, which draws its information from a more limited set of ("trusted") sources, and so has a much higher "fame" threshold that a personal entity has to overcome before it can appear in this type of vertical.  It may turn out that the Snapshot barrier to entry is as low as possessing a LinkedIn profile.

Bing appears to have effectively removed this barrier by doing a really good job at disambiguating personal entities:  if they're displaying information from the right Twitter, LinkedIn, Facebook and Klout accounts, then (regardless of the veracity of the information) they're linking to "trusted" social sources insofar as they're accounts that are owned by the same person.

Bing search results for the query "dan brickley". While Dan is well-known in the semantic web community, he is not a "famous person" in the sense that John Kerry (the example used in the Bing post) is, and to the best of my knowledge (and basic testing) he's never appeared in a Google Knowledge Graph vertical.
(Click to enlarge)

Practical takeaways?

• From a personal branding perspective, LinkedIn is more important than ever.  Aside from the large amount of information that's pulled from LinkedIn into the Snapshot vertical for a personal entity, it also seems to be the location from which the entity's photograph is pulled:  no LinkedIn photo, no Bing Snapshot photo.

• Marketers have long pushed back against Klout, arguing (fairly persuasively) that Klout is a pretty blunt measure of social influence.  That is to say, the message has ended up being "your Klout score doesn't matter."  Guess what?  Your Klout score matters – maybe not to you, maybe not to those "in the know," but from a raw optics point of view it's now closely associated with your Snapshot in Bing.

These takeaways are only "practical," of course, insofar as it impacts those who see such results in Bing:  currently about 17% of searchers in the United States, and a negligible number elsewhere in the world.

This limited reach may not be the case for long, however, depending on whether or not Google responds in kind.  This represents – for the first time, I think – a real challenge to Google's Knowledge Graph results, and I can't see Google sitting on their hands while Bing is now producing better semantically-fueled and interlinked results for non-celebrity personal entities than them.

And Bing is seemingly doing at least as good a job as Google for "famous" people too (for Google and Bing the working definition of "famous" or "celebrity" seems to be "a person with an established page about them on Wikipedia").

Google Knowledge Graph (left) and Bing Snapshot results (right) for "john kerry".
(Click to enlarge)

Bing might arguably be returning more useful Snapshot results than the equivalent Knowledge Graph results because of the additional social properties (Twitter, LinkedIn, Klout) that are displayed for living celebrities in Snapshot, but not in the Knowledge Graph.

Again, I can't see Google taking this lying down.  Bing's socially-connected search results were at one time derided as being not particularly appetizing icing on the search results cake ("I don't care what movies my friends like").  Now the labors of socially integrating entities are beginning to bear fruit, and Google is starting to appear woefully behind in this realm.  Google is certainly promoting and expanding Google+ with nothing less than frenetic energy, but even though this isn't a walled garden in the Facebook sense, it is one garden:  Bing is now foraging far further afield for social produce to include in its organic basket.

All of this is made possible by Bing's Satori graph-based repository that uses RDF and SPARQL, and has been designed to handle a huge number of triples.  Having (at least until now) flown lower under the radar than Google's Knowledge Graph technologies, not a ton has been published on Satori, but here's a couple of articles with interesting insights.

• How Google and Microsoft taught search to “understand” the Web
  Sean Gallagher, Arts Technica, 6 June 2012
• Bing Now Knows Much More about People and Places Thanks to LinkedIn and its Satori Entity Engine
  Frederic Lardinois, TechCrunch, 21 March 2013
• Knowledge Graph, Satori, and Unicorn
  Laurie Sullivan, 21 March 2013

[Update, 24 March 2013:  Added the last link above, which also put me onto more Satori information from the Microsoft Trinity project website, which I've summarized here.]

Completely independent of search market share wars, it's really great to see Bing putting competitive pressure on Google through improvements to their products rather than shrill and aggressive marketing campaigns.  Search engine users will ultimately be the beneficiaries of more competition in the semantic search results space.

The rallying cry from the search engines to encourage the use of semantic markup has been, so far, centered on rich snippets.  Markup you site with structured data, they say, and your search result snippet might be enhanced with information and graphics that will increase the number of searchers that click through from the results to your website.

Absolutely true.

But the benefits of employing structured data and associated semantic web technologies extend past rich snippet generation, and that's what I'll be exploring here.This post is an annotated version of slides I presented at SearchFest 2013 in Portland, Oregon.  It was a fabulous conference, and I was honored by the invitation from Matthew Brown to appear there, and by the opportunity to present alongside Jeff Preston of Disney Interactive – thanks guys!

So what was that about rich snippets?  Let's start with a little history.

Building the foundations:  1999-2005

The Dublin Core Metadata Initiative (DCMI, or simply "Dublin Core") was among the first efforts at developing metadata standards for describing both physical and online resources (1995).   Microformats (2004) enabled semantic markup of particular types of information by leveraging the attributes of markup tags.

Specific protocols like XML sitemaps (2005) and product feeds like Google Base (2005) were among the earliest efforts by the search engines to better understand and index with structured data.

Underpinning all of these developments was the Resource Description Framework, or RDF (1999), which provided the general mechanism for describing resources and modeling information on the web.  FOAF ("Friend of a Friend," 2000) built a simple but powerful ontology for describing persons on this framework.  And RSS (originally "RDF Site Summary," 1999)" – a feed format for describing frequently updated resources – rapidly became one of the most widely-employed XML applications on the web.

Rise of the vocabularies:  2005-2011

By mid-decade the search engines (and in particular Google) were producing rich snippets and displaying topical verticals in search results by parsing and structuring the content of specific sites and, to a certain extend, making sense of data provided by microformats.

But it was the introduction of RDFa (Resource Description Framework in attributes, 2007) and microdata in HTML5 (2005-2008) that really allowed the search engines to leverage structured data.  These protocols provided a mechanism for marking up the visible content of web pages with structured data (without relying on class attributes), allowing rich metadata from external vocabularies to be embedded in machine-readable code.

GoodRelations (2008) and rNews (2011) leveraged the power of RDFa to build specialist vocabularies for ecommerce and news content, respectively.  The introduction of schema.org (2011), though, provided webmasters with the first general-purpose set of schemas that were officially sanctioned by the search engines.

Freebase (2007) developed as a large collaborative knowledge base of metadata, and DBpedia (2007) evolved to extract structured content from Wikipedia.  The Open Graph protocol (2010) grew out of the Facebook Platform, and enabled developers to integrate web pages into the social graph (a graph of the relationships between internet users).  All of these initiatives were to play a role in the semantic evolution of search engines in the years ahead.

Consolidation and growth:  2011-present

Since 2011 the schema.org vocabulary has continued to grow, both by the addition of extensions built for schema.org, and by incorporating types and properties from rNews and GoodRelations.  This has meant that a increasingly richer, search engine-sanctioned vocabulary is available to webmasters, even if not all schema.org types and properties currently result in the production of rich snippets in the search results, or other clearly demonstrable benefits for search visibility.

Google introduced its Knowledge Graph in 2012, which draws on a number of structured data sources to populate this knowledge base that currently resides alongside Google's main search results.  Google's 2010 acquisition of Metaweb (developers of Freebase) turned out to be fundamental to the Knowledge Graph.  Bing's own version of the Knowledge Graph, Bing Snapshots, references many of the same sources as the Knowledge Graph.

Just as Google leaned on structured data sources to build its knowledge graph, Facebook Platform technologies facilitated the release of Facebook Graph Search in 2013.

So the search engines are using structured data to improve their search results and to build complementary products, while the power of the social graph is allowing sites like Facebook to challenge the search engines with their own, socially-informed search products.

Specialty search and structured data

Along with reviews, one of the earliest types of rich snippets to appear in search results based on structured data markup were recipes.  Building on this, and starting with recipes, Google has constructed a number of specialty search engines (or, depending on how you look at, subsets of its main results) that allow searches restricted to be restricted to specific topical domains.

Structured data markup is the price of admission to obtain a presence in these specialty engines, as the underlying markup for pages that appear in recipe search readily attest.

Chicken?  Egg?

The fundamental piece of information that recipe markup provides to the search engines is the formal declaration "this resource is a recipe."

Does this information help the search engines return relevant resources when they judge that the intent for a particular query might be a recipe, even if a keyword trigger like "recipe" does not appear in the query?  Judging from the similarity of the web search results for "chicken cacciatore" this might well be the case.

Even the image vertical for this query is dominated by results which might be informed by the use of structured data, the exception being a highly-optimized page containing only a highly-optimized picture of chicken cacciatore.

Application search and recipe lessons learned

As with recipe search, Google's application search relies on structured data markup to return results about, and only about, software applications.  Obviously sites with one or more pages dedicated to software have the ability to improve their search visibility by using application markup.

[SIDEBAR] Google and the iTunes store

If the price of admission to a Google specialty search engine is structured data, what is a page from the iTunes store – which does not contain schema.org/MobileApplication markup – doing in the applications search results?

Google knows that any iTunes store app is a mobile application.  And as the iTunes store is built on standard templates, it's straightforward for Google to parse and structure the data it finds there.

The lesson here is that when Google considers a source of information to be important enough to provide value to its search results, it will go to exceptional lengths to parse and structure unstructured or semi-structured data.  It has previously done this with Yelp and pages with BazaarVoice reviews to generate review rich snippets in the SERPs, and has returned product and review rich snippets for Amazon, which is devoid of any Google-sanctioned structured data.  The exceptional lengths to which Google will go to structure data provides a pretty strong clue about the value it places in it.

Web search and recipe lessons learned

Google application search is newer than recipe search, and the application schema is much newer than hRecipe, data-vocabulary.org/Recipe or schema.org/Recipe.  Accordingly, adoption of application semantic markup has lagged behind that of recipe markup.

This relatively poor adoption means there's a competitive advantage available for those that choose to markup software pages.  Aside from the fact that this markup might generate rich snippets in the main SERPs, and that it is required to appear in application search, there is the possibility of the sort of knock-on effect from specialty search observed in the web recipe results.  And if Google determines that a particular query's intent is software-related, there's no clearer signal that can be provided to Google here than saying unambiguously in the code that "this page is about a software application."

Web search and the Knowledge Graph

Google's Knowledge Graph has typically contained information derived from major trustworthy structured (or structure-able) sources like Wikipedia or IMDb.  Recently, though, results for events have begun to appear for city queries.

Clicking through from the Knowledge Graph

While there is no specialty events engine where semantic markup is the price of admission, the results displayed when clicking through on Knowledge Graph event results suggests that Google favors structured data sources for these results, and might be relying on them in some form to populate the Knowledge Graph events vertical in the first place.

[SIDEBAR] The Google Data Highlighter

Around the time that the Knowledge Graph events vertical started to appear, Google introduced the Data Highlighter, which allows webmasters to visually match visible website information with properties supported by the Highlighter.  In other words, it provides a mechanism to provide Google with structured information about a resource without marking it up in the code.

The first (and at time of writing, only) content type supported by the Highlighter?  Events.

If the Knowledge Graph events vertical is in some way predicated on the presence structured events data, this suggests that Google created the Highlighter to help grow the amount of structured data available to it about events.  It stands to reason that Google doesn't want to throw out the baby with the bathwater by restricting Knowledge Graph event listings to sites that have marked up their events, and has introduced the Highlighter to ensure that unstructured pages about events still have a presence in the events vertical and their linked results.

It also stands to reason that webmasters concerned with search visibility will want to pay attention to any new types supported by the Highlighter, even if they don't actually employ the Highlighter; if, say, software applications are supported by the Highlighter, webmasters with the wherewithal to do so would be well advised to markup their software pages.

[SIDEBAR] An event thought experiment

The conference at which these slides were presented, SearchFest, did not appear in a Knowledge Graph search vertical for a "portland" query.  The SearchFest site did not feature event markup.  While there are doubtlessly a number of factors that go into Google's determination of what should appear in the Knowledge Graph events vertical (such as, perhaps, the number of times an event is cited, or the authority of the sites on which a given event appears), might SearchFest have appeared in the "portland" events vertical if the SearchFest site had carried events markup?

I subsequently learned that a SearchFest organizer had used the Data Highlighter to provide event data about the conference, but – aside from not appearing in the events vertical – the effort did not result in a rich snippet.  With so many unknowns and variables its hard to say whether this is due to the lack of events markup or any number of other factors.

I can report, however, in regard to the slide above used to demonstrate the Google Data Highlighter, that some two weeks after using the Highlighter that this events rich snippet has started to appear for the query "paste 2013":

No rich snippets and no specialty search?

There are myriad schema.org types that currently do not produce rich snippets, have no specialty search engine associated with them, and have no visibility in the Knowledge Graph.  What's the point of adding schema.org RDFa or microdata markup for these types?

The history of recipes, applications, events, products and reviews suggests both that pages with semantic markup can suddenly make an enhanced appearance in the search results, and that the types most likely to appear there are related to the types of information that people frequently search or browse for on large, well-used, topically-specific sites.

So marking up pages with structured data today may prove to be a competitive advantage tomorrow, and that return on investment may come sooner rather than later for certain types of information.  Absent a crystal ball, one can still make educated guesses about what some of the more important types might be.

• schema.org/JobPosting
  Already in use at the US National Resource Directory, bringing job postings directly to the SERPs would obviously be useful to searchers (however much a full-fledged Google or Bing jobs specialty engine might present perhaps too much of a politically-charged challenge to the likes of Monster or LinkedIn).
• schema.org/Organization, Person, Place, etc.
  Named entities are the life's blood of the Knowledge Graph and Bing Snapshots.  And even if marking up named entities don't end up producing fireworks in the SERPs, I'd argue that any structured information provided to the search engines about entities helps them better understand those entities.
• schema.org/Product, schema.org/Offer
  At this point its remarkable to me how little the search engines have done with ecommerce information in the web results.  Even if this is a result of a deliberate effort to protect product search (either to retain its monetary potential, or to promote the submission of account-linked and verified product feeds), sooner or later somebody is going to make hay of structured product and offer information.

[SIDEBAR] No feeds required

Not that long ago, the only way to provide reliable, detailed product and offer information to search engines (and other data consumers) was through XML product feeds.

Ecommerce-related structured data has changed this.  Using schema.org, not only can ecommerce sites expose the same information in markup that is supplied in feeds, but – with GoodRelations integration – websites can now markup more detailed information about their products, offers and services than are supported by product feeds.

So there might be a future where product feeds might not be required for sites to appear in product search (data cleanliness and veracity being the chief impediment to that right now).  Might Bing – already trying to gain favor with consumers by publicly attacking Google for moving to paid product listings – circumvent the restrictiveness of product feeds altogether?  Or might some other player do an end run around them both by creating their own product search engine?

[SIDEBAR] Structured markup and Google CSE

Sites employing Google Site Search (the paid version of Google Custom Search) can leverage almost any sort of structured data in their search results.  Certain data can be used for filtering results; other types of data can be used in the sorting or biasing (ranking) of Site Search results.

No schema.org types that are relevant to you?

schema.org is a large general-purpose vocabulary, but is not comprehensive. There are many types of information found on websites that cannot describe be described using schema.org.

If a website owner judges that there's potential value in doing so, they can create an extension to schema.org for their content.  For example, there's no type for video games in the current schema, and so a site dedicated to video games might build an extension in order to be able to provide detailed semantic information about video games listed in their (publicly-consumable) code.

One could build such an extension, employ it one's site and make internal use of the types and properties developed.  One might further hope that other video game sites use the extension, and so extend its utility.  For a search marketer, however, ultimately the wisdom decision to spend time and money on developing an extension has to hinge on improved search visibility.

Different approaches to schema.org extensions

There are countless well-developed formal vocabularies and ontologies available on the web, covering a large number of specialist domains.  Rather than extend schema.org by adding more types and properties to it, semantic web developers are more inclined to link schema.org markup to controlled vocabularies, standards and datasets where they exist.  This makes a lot of sense, as it keeps the maintenance and development of these vocabularies singularly in the hands of the specialists in the domains to which they are related.

For search marketers, though, types and properties formally integrated into schema.org have a much better chance of eventually surfacing in search engine results.  And while there still might be search benefits for a site employing external lists – insofar as the search engines might derive a better understanding of that site's content – the nebulous and unproven nature of this value proposition is unlikely to result in resources being committed to vocabulary building (at least where the impetuous is SEO).

Accordingly, search marketers with a terrific use case for an extension might find the best route is to lobby for its inclusion in schema.org directly.  This is more-or-less how MedicalEntity and its sub-types ended up in schema.org.

Proposed extensions to schema.org will have the greatest chance of success if the following steps are followed:

• Conduct research on vocabularies and datasets that may have already been developed in your topical domain, allowing you to build on them and to avoid unnecessary duplication of effort
• World collaboratively with other leaders in your industry to develop and promote your extension
• Make your proposed extension publicly available and solicit feedback from interested parties, refining and improving your extension based on this feedback
• Formally propose adoption of your extension, citing need, use cases and the benefits to site owners, data consumers (both human and machine varieties) and search engines

The proposal from Jindrich Mynarz for a JobMarket extension to schema.org, featured below, is an excellent example of a well thought-out and executed extension strategy.

An example of a well thought-out extension proposal
(Click to enlarge)

Structured data for structure’s sake

Even breadcrumb markup can provide the search engines with valuable clues about a website's structure
(Click to enlarge)

As I've already alluded to more than once, adding relevant, consistent, valid structured data to your site helps the search engines better understand the content and structure of your site, regardless of any special visibility you may or may not receive in the SERPs, such as rich snippets.  Let me reiterate this with some bold text for extra emphasis:  structured data helps the search engines better understand your site.

A breadcrumb rich snippet in the SERPs is hardly going to cause a doubling of your click-through rate from search.  But what if the underlying code helps the search engines more clearly understand the hierarchy of your site and the relationship between pages?  In the absence of the breadcrumbs I marked up with structured data, might Google nonetheless generate the ideal breadcrumbs, sitelinks and mini-sitelinks in the SERPs featured above?  Perhaps.  But given the results I don't feel in any way that this was wasted effort.

In delivering the SearchFest closing keynote, Bing's Duane Forrester urged webmasters to employ structured data, and said that doing so would not immediately boost your rankings, but would pay off in the long run.  Why?  Because structured data helps the search engines better understand your site.

Structured data for greater data fidelity

Providing consistent data values helps search engines and other data consumers
(Click to enlarge)

Well-executed structured data can help provide a more consistent and positive experience for website users, whether their exposure to your content is through search results, social media or third-party applications.

Employing structured data can help developers and optimization specialists see what underlies a resource through a data lens – that is, as related data points and data values, rather than disconnected widgets and disparate pieces of information.  For example, a page's "like" button seen through this lens is not generically a widget for Facebook, but a mechanism to provide Facebook users and Open Graph consumers with precisely crafted information.

Data fidelity as a trust-building measure

Data fidelity between a product feed, visible page content and structured data markup
(Click to enlarge)

Consistency of data is important for search engines.  In particular, as this principle applies to structured data, search engines will trust your content more if it can see that your visible content aligns with the data provided in your markup.

This is, in fact, almost certainly one of the reasons that the search engines put their weight behind schema.org, which was literally designed for attribute-based markup, rather than opting for Open Graph-like invisible metadata (remember <meta> keywords?).  And this is also the reason Google makes a point of advising against marking up non-visible content except in situations where a very precise data type is required but is not available on a page, such as a numeric representation of review star ratings, or event durations in ISO 8601 date format.

An additional point of data reference for the search engines are XML product feeds.  As in the case of the toaster oven pictured above, the search engines will accord greater trust to ecommerce pages if the product feed, markup and visible content are in sync.

If it looks like a duck, quacks like a duck and walks like a duck Google might feel reasonably confident that what they're looking at is a duck; if it looks like a duck, honks like a goose and does the Harlem Shuffle … not so much.

The inspiration for the slide above was from an excellent piece by Barbara Starr on leveraging data quality in Google Shopping, in which she says:

Paid inclusion is now a fact of life in Google Shopping, and Google made a simultaneous mandate requiring clean, relevant, rich data inputs to product feeds, data that is verifiable by cross referencing other sources. That means the following data must all be the same and in sync:

• Data on the webpage visible to humans
• On-page semantic markup
• Data in the product feed

Google’s mandate is strongly reminiscent of Semantic Web philosophy for dealing with data quality and provenance. As is the fact that Google uses some forms of Rich Snippets to expand its Knowledge Graph.

Provenance – knowing precisely from where a particular piece of data originated – is an important semantic web concept, and the W3C has been developing the PROV model to support the "inter-operable interchange of provenance information" on the web and elsewhere.

Where an SEO, in trying to assess whether a particular document is trustworthy, might look at the absence or presence of spam signals, a semantic web type is more liable to first ask the question, "where did it come from?"

[SIDEBAR] Provenance and profiles

Initial assessments of Google+ have tended to focus on its traction to date as a social network:  how many users does it have, how engaged are its users, how much information is shared over the platform, and so on.

This glosses over what I think is the true strength and ultimately the great potential of Google+, namely, as a interlinked network of verified, canonical, named entities.  The role that Google+ could potentially play in determining provenance is obvious.  The combination of a Google+ Profile or Page and structured data means Google can connect the dots between a web resource and the entity that produced that resource.

[SIDEBAR] Provenance and a/Authorship

A possible linked data use of a verified personal entity profile
(Click to enlarge)

Much has been said about the role of authorship since the introduction of author rich snippets in Google search results, which – of course – are predicated on the existence of a Google+ Profile.

What I find interesting about these linkages is less the role that authorship ("Author Rank") might play in the display and ranking of search results (which might, indeed, turn out to be quite important), but the potential that the combination of structured data and verified Google+ Profiles and Pages might have in supporting complex queries based on linked data.  I could, for example, state on a page I authored (and which Google can verify I authored) that I work for InfoMine, even though that information might not appear on my Google profile.  From that Google could, with a high degree of confidence, include me in the results for the query "people that work for InfoMine."

Using semantic tools to improve site structure

Entity disambiguation: is the name of this Company "IBM" or "International Business Machines"?
(Click to enlarge)

As much as adding structured data may help improve your site's visibility in search, semantic web technologies can be leveraged to help improve your site in general ways, even if these improvements don't result in the creation or manipulation of structured data on your site.

Extracting and disambiguating entities using APIs is one such use of semantic web technologies.  Categorizing topically identical content in two or more places because of entity variants may result in duplicate content, keyword cannibalization, or both.  On a news site, for examples, news stories pertaining to IBM might be listed either on a page about "IBM," a page about "International Business Machines," or both.  By using an API to identify stories about IBM, all these stories are associated with a single resource page, regardless of whether "IBM," "International Business Machines" or some other variant on the name appears in the content itself – as in this example of the IBM page on Wikipedia put through Calais.

Entity extraction and disambiguation: sample Calais output
(Click to enlarge)

Rich applications can be built on these technologies to help produce well-organized, meaningfully-linked, content-rich pages – all attributes that curry favor with the search engines regardless of the presence or absence of structured data.  Zemanta, for example, has used semantic web technologies to provide bloggers with tags, multimedia resources, and relevant inline links based on a post's content.

Entity extraction and disambiguation: sample Zemanta output
(Click to enlarge)

[SIDEBAR] Linked (Open) Data and search

While the principles of linked data may not be directly applicable to SEO, there's certainly analogies that can be made between best practices in the two realms.  While I won't dwell on this, most SEOs would benefit from getting better acquainted with linked data, and understanding its importance in the semantic web world.  No time?  Purchase the linked data coffee cup and keep it at your desk as a reminder.

Structured data and semantic architecture

"Once we get the site built we'll SEO it."  Any seasoned search marketer has probably heard a variation on this declaration at least once in their career, and knows the folly the statement engenders.  While it's perfectly possible to markup a site with structured data after it's been built, a site that's constructed with an eye to semantic structure will fare better in the long run than one that's not.

The BBC World Cup 2010 website did not have structured data applied to it, but was actually assembled with the aid of semantic web technologies.  The result is a rock-solid resource that at once reliably meets human visitors' needs, and at the same time provides search engines with explicit, utterly unambiguous data (the BBC has taken a lead role this sort of semantic architecture).

Fine for the megalith that is the BBC, but impractical for your site?  Perhaps.  But in the future expect more platforms and tools to emerge that can make this sort of architecture more accessible.  From a data perspective, after all, what is a standard ecommerce site if not a collection of entities with associated properties, all of which may be fully described using existing syntaxes and vocabularies?

On Beet.TV, Andy Plesser recently featured a short but fascinating video of Michael Zimbalist, Vice President of Research and Development Operations at the New York Times, talking with Joanna O'Connell of Forrester about a prototype linked data search engine being developed by the Times.

Zimbalist begins by talking about the great asset that is the New York Times Index, and the relationship between the Index's metadata and linked data.

For almost the entire life of the newspaper … we've been annotating that.  We've had the Times Index – you've probably seen it at the library, these big fat red books so you could go and look up the Index and find what issue of the Times and what section there was a story about that.  So that really provides the basis for some exceptionally rich metadata that's been consistent over the history of the organization.

So we're able to look at the text of articles as data, as very unstructured data, and begin to put some structure around it.  And we do it in a lot of really different ways.  So we have our metadata and our index now rationalized.  So all the terms in the index match up to all the terms in our metadata, and we've connected that to this linked data cloud.  There's this linked data movement that's trying to treat the entire web as this database of information.

Zimbalist goes on to talk about entities and how, for example, the New York Times may have one way of referencing the named entity "Barack Obama" and Amazon another.

And through this linked data movement we're able to say that their Barack Obama is the same as our Barack Obama and create these new editorial products that fuse these different bits of information.

So, for example, one the things we have built as a tool, just as a prototye example of the kind of services we might be able to deliver in the future, is a search engine where you can type in a college or university and we will go look through the articles that we've written and surface any articles that mention alumni or alumnae from that university even though we might not have mentioned in the article that that person attended that school. Because we're able to link up the person's name with a larger database of college graduates.

This is a great example of a leveraging linked data to create a product that is obviously useful in tying together disparate pieces of information that aren't available from a single source.

In response to a question from O'Connell, Zimbalist agreed that "normalization … does matter," but said that the bigger challenge was broad-based indexation of content.

I feel like the bigger problem is getting all this stuff indexed. Rationalizing among the indices seems like a more solvable problem than getting everybody who's publishing on the Internet to adequately index their content, but I don't know.

Some definite food for thought here for data producers. That is, how useful is that data outside your organization if it is not exposed? And while Zimbalist does not explain the difference between "adequate" and "inadequate" indexation, any student of the semantic web could reasonably infer that "adequately" indexed data is ultimately some form of structured data.

Members of the "linked data movement" to which Zimbalist refers would likely also argue that maximum benefit is derived from data that is not only structured but open. In discussing the prototype engine, O'Connell makes an oblique reference to this in terms of the possible data ownership issues.

Well, I was just thinking about monetization challenges that it might cause, or rights challenges … how does that work when you have everything linked together like that doesn't it create a who owns what and who gets paid for what question?

While Zimbalist concurs that this is "an interesting question," one doesn't get the impression that, as O'Connell suggests, that this is a question that the Times thinks about "every day." To me, in fact, the prototype engine is something of a poster child for how useful applications may be built when structured data is made freely accessible to all data consumers. The degree to which an application harnessing these data may be monetized is dependent on that data existing in a consumable format in the first place; that is, linked open data is itself a condition for creating products that have a monetization potentional.

Watch the entire interview:

Google announced today a new, potentially powerful addition to its Webmaster Tools options today:  the Data Highlighter.

The Data Highlighter allows webmasters to select text on a web page and associate it with properties for a particular data type.  At this time the only supported data type is for events, but (perhaps based on lessons learned from this initial rollout) expect more to follow.  Indeed, in the Data Highlighter help section on Google, there is already a page listing "data types supported by Data Highlighter."

While marking up – I mean "highlighting" – a single page with data related to an event without needing to add any additional code to a site is pretty useful in itself, what makes the Data Highlighter a potentially important leap forward in the the provision of structured information to Google is the ability to highlight patterns that pertain similar pages on a site.  In the sphere of events, this means that if your site has multiple, similarly structured pages containing event data, you can highlight event properties on a single page and have Google apply the same logic to pages that have "a consistent format."

The utility of this move is obvious:  it allows webmasters to inform Google of structured information on a site without the need to add additional markup to a page, or even without knowledge of any vocabularies that pertain to the information at hand.   This in turn facilitates the production of rich snippets in Google search results, and provides Google with de facto structured data that it can use in all sorts of situations.

Put another way, if yesterday I wanted to reliably provide Google with structured information about an upcoming event listed on my website, I would need to know a fair amount about schema.org/Events and its properties, encode that information using either microdata or RDFa, and publish or republish the page.  Today, I can trot on over to Webmaster Tools, spend a few minutes highlighting some text and press "publish."

Conceptually, I personally find the Data Highlighter fascinating because the tool replicates for webmasters what Google has been doing behind the scenes for many, many years:  making sense of unstructured and semi-structured data by uncovering consistent patterns on web pages.  Now everyday webmasters have the ability to "train" Google about the presence of consistently-formatted on their site.

Before I get to a walk-through of the Highlighter, a couple of quick initial thoughts about some of the Highlighter features and benefits.

• The Data Highlighter explicitly names required properties for events.  As schema.org is parser-agnostic, previously the only way webmasters could learn what properties Google required for any given data type was to run a page or some code through the Structured Data Testing Tool and try to decipher the resulting error messages.
• While the event properties available in the Data Highlighter are clearly based on schema.org/Events it is apparently not a faithful one-to-one mapping (they say that the "information that Data Highlighter can extract for events is slightly different from the event data that you can specify with HTML markup").  Though in their enumeration of the differences between the two, the only thing that appears radically different is the date formats that Data Highlighter allows.  This makes sense as it's unlikely that many websites reliably publish dates on-page in ISO 8601 format (when encoded in microdata or RDFa date information is almost always placed in a non-displaying <meta> tag).
• Knowledge Graph sources are being extended.  That is, in their video and documentation Google specifically references the Knowledge Graph, and even provides a graphic of how successfully-extracted event information will appear in a Knowledge Graph vertical.  This may not mean a change in Google's Knowledge Graph aim – in the words of Google's Emily Moxley – to "purposely try to show things that are definitely true," but it does represent a vast expansion of what sources Google might consider to be "definitely true."
• From a trust and proof point of view, "data highlighting" may circumvent the ability of websites to game Google with inaccurate structured data.  While the tool allows for the addition of "missing data" it is still largely predicated on the highlighting of existing, visible content on a website (and one would certainly think that Google would have a higher confidence in highlighted, visible on-page data than that supplied through the "missing tags" interface).  Which may be why (to my point above) Google is willing to push Data Highlighter-supplied information to the Knowledge Graph.

Adding event information to using the Data Highlighter

To access the data highlighter select "Data Highlighter" in the "Optimization" section of Google Webmaster Tools.

When you click on the big blue "Start Highlighting" button you're required to enter the URL of a page, and to select whether or not your want to tag just the page in question, or the page "and others like it."  Here I selected the single page option.

Once the page is loaded and a portion of text is highlighted, you assign it to a predefined tag (equivalent to a schema.org property).  As you'll see, I used what isn't really an event page per se (and it happened in the past), but had most of the elements of a "proper" event page.

Once a piece of information has been tagged, a label appears next to the highlighted text, and the tag content appears in a sidebar.  Sidebar items that Google regards as problematic appear with an "attention" icon next to them.

The Data Highlighter will allow data to be selected from any portion of the page.  Here I've selected my name to populate the "Performer" tag by using the anchor text of my linked WordPress author byline.

Data that isn't visually present on the page can be added to a tag by selecting "Add missing tags" from the settings icon at the upper right, and then following the prompts.

Once you click "Publish" information about the page or pages you've tagged using the Data Highlighter appear in the Data Highligther section of Webmaster Tools.  What data "will become available" once Google has recrawled the site remains to be seen (for example, whether or not successfully published and crawled Highlighter-entered data will appear in the Structured Data report).

The interface for tagging multiple, pattern-based pages is similar, but includes a progress meter and additional steps (I haven't fully explored this yet).

At first blush the highlighting interface is intuitive and seems to work well.  Now it's a waiting game to see when and if rich snippets appear for marked up events, how long it takes for rich snippets or Webmaster Tools data about an event to appear, and whether or not an event tagged through the Highlighter makes its way to the Knowledge Graph.

All of this unlikely with my example event that occurred in the past, but I have many other (and arguably more important) pages to tag with event properties.  I'll keep you posted.

In May of 2009 I wrote a post on Google's paid link reporting mechanism, opining that the introduction of a manual link review process undermined Google's traditional and successful reliance on machine-based algorithms to rank web pages.

Fast forward to 16 October 2012, when suddenly the airwaves are buzzing with Matt Cutts' announcement of Google's "disavow links tool" at PubCon.

To a certain extent this is a déjà vu moment. Like the paid links reporting tool it is at once an admission that Google's mighty machines aren't up to the task of assessing the native value of hyperlinks without some degree of human intervention. And as with paid link reporting, it also "formally makes the SEO industry necessary" as an auxiliary force to police Google's webmaster guidelines.

However, the possible necessity of link disavowal seems more nefarious to me in two respects.

First, while paid link reporting ultimately put the onus on those abusing Google's reliance on PageRank to clean up their acts (at least those identified through the paid link reporting tool), link disavowal means puts webmasters on notice that they're now on the hook for behavior by others over which they have no control. To use a legal analogy, the presumption of innocence is no more.

And a mechanism for reporting "bad links" does not place webmasters in control, except insofar as its an explicit admission that competitors can influence your search visibility (simultaneously placing a nail in the coffin of the "is there such a thing as 'negative SEO' debate – cased closed). Yes, a webmaster now can combat a competitor's malfeasance – if they opt into Google Webmaster Tools so they may receive "unnatural link" notifications, and if they have the time, knowledge and motivation to disavow linkspam.

(Google's announcement sports a marvelously ambiguous adverb in it's second sentence. "If you haven't gotten this ['unnatural links'] notification, this tool generally isn't something you need to worry about." Emphasis mine.)

In short, where one could previously garner something like a competitive advantage by optionally ratting out nefarious competitors, now one is seemingly required to protect one's reputation for purity by formally distancing oneself from slanderers.

Which leads me to my second point – namely that this sort of esoteric search engine optimization activity favors sites with bigger budgets and well-informed marketers. A small business with a site that offers "great content" to visitors ("great content" having become almost a sacred mantra in Matt Cutts' missives) may now, apparently, fall prey to the evil intentions of unscrupulous competitors unless they employ professional help to keep the vampires at bay.

To a large extent much of the concern over "negative SEO" is probably a tempest in a teapot, and really most webmasters shouldn't worry their pretty little heads over "unnatural links." As the announcement post says, the "[v]ast, vast majority of sites do not need to use this tool," that's "primary purpose is to help clean up if you've hired a bad SEO or made mistakes in your own link-building" and that one "[t]ypically" does not have to clean up links that a webmaster didn't create. Still, the profusion of qualifiers employed (emphasis, again, mine) leaves Google a lot of wiggle room to disavow their own advice.

The fact of the matter is that bad links, like good content, ultimately have a human behind the wheel. And one thing I've always considered a primary reason for Google's success and strategic brilliance is to remove humans from the process of ranking websites (keep in mind, for example, that the primary purpose of Google's "quality raters" is not to circumvent the algorithm, but to help hone it). There are probably in the neighborhood of 40 to 50 billion pages in Google's index: any significant human intercessory process to determine the relative value of these pages obviously doesn't scale.

So – to me at least – further crowd-sourcing of Google quality guidelines enforcement further undermines the foundation of their algorithmic success. All of which may be simply to say that the age of the link-as-an-impartial-vote may soon come to an end (link fraud, unlike voter fraud, is an actual problem). And maybe what rises from the ashes may once again allow Google to return to its mechanical roots, at least until pesky humans again upset the applecart.

Ready to disavow your social connections, y'all?

Google has just announced a welcome new addition to the information available in Webmaster Tools:  the Structured Data Report.

While Google has long been consuming structured data markup, until now there's been no mechanism to verify that Google had actually consumed the structured markup that webmasters had been providing.  With the Structured Data Report, Google is now reports on key pieces of information about structured data present on a website:

1. The type and corresponding schema of structured data Google has discovered on a website.
2. The total number of occurrences of that structured data type Google has discovered.
3. The total number of pages containing each type listed.
4. The specific pages on which each structured data type is present.
5. Information about the properties present on each page for each structured data type.

Structured Data Type, Schema, Items and Pages

The main report gives an overview of structured data for a given domain, and shows the presence of different items and pages over time (all Google Webmaster Tools reports are live data from seocooking.com).

Google Webmaster Tools Structured Data Overview Report

One of the interesting things about this overview report is that webmasters will be able to see the lag time (if any) between page indexing and the appearance of structured data on a page as reported by Webmaster Tools.  I haven't yet created new content containing structured data, but I look forward to finding out whether the indexing of structured data occurs at the same time as page indexing, or if structured data is separately indexed.

While this is likely simultaneous, one will nonetheless be able to see how long it takes Google to index structured data once it has been added to a page, and whether or not the addition or removal of structured data without visible content changes is picked up by Google.

While downloads are always nice, the main utility I see for the data export is to calculate the data type distribution – really only useful for sites with a large number of types and pages.

Note that where there are a greater number of items reported than the total number of pages for that structured data type, this is because one or more pages contain multiple instances of that type.

Information about Specific Structured Data Types

Clicking on a type from the overview report will display a report showing the pages that contain that structured data type.  As noted above, pages containing more than one item contain multiple instances of that item.  The report displayed below is the result of clicking on the linked "VideoObject" type in the overview report displayed above.

Google Webmaster Tools Structured Data Report for a Specific Type

Like the overview report, the type report shows the instances of that type recorded over time.  It also displays the specific pages on which that structured data appears and the number of occurrences over time.

Here the data download is potentially more useful, as a webmaster can compare the pages on which he or she thinks that typeshould be present against pages Google actually acknowledges containing that type.

It also alerts webmasters to the presence of structured data they may not have been aware that they were publishing.  Many WordPress site owners, for example, will be surprised to see that the platform and certain plugins are producing structured data of their own accord.

Clicking on a specific page will then provide details of the properties present for that structured data type, as well as a link to the Rich Snippets Testing tool (that provides that same information in a different format, as well as warnings about malformed syntax or invalid or missing properties).

Structured Data Report for a Specific Type on a Specific Page

Interestingly, Open Graph properties for this page are appearing under the VideoObject itemtype for this page, even if those these properties are not a part of this schema, and are fully separated in the markup of the page (the do not appeared so grouped in the results of the Rich Snippets Testing Tool.

Open Graph and schema.org/VideoObject Properties on the Referenced Page

This doesn't make a lot of sense to me, but I'll dig deeper to see whether this is a syntax problem on the publishing side or a bug on Google's reporting side.

In any case, I just wanted to provide a quick, first-blush overview of a new Webmaster Tool report that's obviously useful to webmasters and marketers, like me, that have been actively adding structured data markup to websites.  I welcome any thoughts or observations from others active in the semantic web as it pertains to search engines.

The tweet button allows users to tweet a link to a page with a single click, and is one of the most ubiquitous social sharing buttons on the web.  This site uses it, your favorite sites almost certainly carry it, and if your mom has a blog you'll probably find a button there sporting a picture of a little bird with the word "Tweet" next to it.

As easy as it is to add the tweet button to a website, most users do not avail themselves of the full functionality that the tweet button offers.  By making three relatively simple implementation tweaks you can create more engaging tweets, improve tracking, provide better exposure for your Twitter username and increase the number of your followers.

In the examples below I mostly reference the WP Tweet Button plugin for WordPress, as it is the most commonly-used tweet button utility for the most commonly-used blog platform.  However, there are other plugins available for other platforms, and Twitter itself offers both a button configurator and detailed tweet button documentation for developers.

Proper configuration of the tweet button is straightforward regardless of the method you use, and the best practices described below should not be difficult to implement.  By following the guidelines below you'll get more mileage out of your button-based tweets, and find yourself unsullied by any of the three cardinal sins of tweet button implementations!

Sin #1:  Failing to Shorten the URL

The primary advantage of tweet button URL shortening is that much less of the 140 characters Twitter allows is eaten up by the web page address, freeing it up for use by the tweeter to add comments, and for you to add a Twitter @name to the end of the tweet (discussed under sin #2).  It also renders the resulting tweet easier to read, which may increase both the number of clicks on the link and the number of retweets.

For a (fairly standard) WordPress site like mine, which uses dash-separated words in the title to generate a post file name, a long title generates an equally long URL.  Longer URLs are also a consequence, of course, of sites with long domain names.

An out-of-the-box, unmodified tweet button implementation on WordPress using the plugin produces share windows like this when the tweet button is clicked.

 An even longer title will result in truncation of that post's title.

The tweet button does do some URL truncation of its own accord – you may have observed that for both those titles the output exceeds Twitter's 140-character limit (the first tweet is actually 243 characters long!).  However, this still leaves little space for the addition of any commentary from the person posting the tweet, and the fact that the visible area of the share window is occupied with the post title and URL in any case discourages such additions.  And one way or another, when the tweet is posted a large amount of real estate is necessarily given over simply to the URL.

Sin not!  Shorten your URLs!  Using the plugin, this is readily accomplished by selecting one of the available shortener options.

Some of these shorteners may be used as is, whereas some shortening services require an API, for which credentials may be entered in the appropriate fields in the plugin's settings (for the sake of conciseness – here, as elsewhere in this post – additional plugin options are not displayed or discussed in detail).

Using a shortening service linked to your account is recommended, as the service (such as bit.ly, my quick and easy shortener of choice) may provide you with aggregated metrics about links you've shortened.  But using any shortener is better than not using one at all, as you can see from the share window output once a shortener has been enabled.

The would-be tweeter has more space to add their contribution (if only conceptually), and the tweet is now more fully focused around the more important of its two elements:  the post title.

Of course, few SEOs regularly create post titles 103 characters in length.  A properly length-optimized title (i.e. of 70 characters or less) makes for a very tidy share snippet indeed.

Hey, who wrote that post, anyway?

Sin #2:  Failing to Attribute the Tweet to a User

When I tweet a link to an article, I go out of my way find out if the author has a twitter account so I can attribute the piece to its creator ("I Head a Fly Buzz – poet.com/fly/ @emilydickinson").  I am the exception:  users of tweet buttons tend to accept the defaults, and if your @account doesn't appear in the share window its unlikely to make its way into the final tweet.

Attribution has some obvious benefits.  Whether by clicking on Twitter's "Connect" link or being alerted of @mentions by utilities like TweetDeck, you can more readily see activity on something you've created.  And both corporate and individual creators of content see branding and name-recognition benefits of having an @account more frequently exposed.

Like shortening, adding a user attribution to a tweet is dead simple using the WP Tweet Button plugin (and also dead simple when using Twitter's tweet button configurator).  Just add your user name in the appropriate spot in the plugin's settings panel.

By specifying my username in this box, now all tweets are appended "via @aaranged".

Again, there are more plugin options available for how usernames can be configured, and developers can modify the tweet button code to accommodate @user accounts in complex ways (" … by @neptune from @romangods").  I will note, however, that for WordPress sites it's possible to leverage author accounts to automatically populate posts with the right @user, and to override the default attribution on a post-by-post basis.

Sweet, eh?  Now if you only had more followers….

Sin #3:  Failing to Recommend Users to Follow

You've shortened your URL.  You've attributed the link to a particular Twitter user.  Your loyal reader, enamored of your prose, readily clicks the tweet button on your post, accepts the default content of the share box and clicks the final "tweet" button there.

But unless that user already follows you that's not the end of the story, but rather a missed opportunity.

The tweet button allows you to specify up to two users that will be recommended to the tweeter after their tweet has been sent.  The benefit should be obvious:  it's a contextually relevant call-to-action that can help you acquire new followers.

Yet again, configuring a recommended user is straightforward.  For the WP Tweet Button plugin simply enter the account name of the user you wish to recommend (without going into details, I find the description field a little problematic – if you have a decent Twitter bio this should more than suffice as an enticement).

When a default recommended Twitter user is specified, sharers will be invited to follow this user after they send their tweet (unless they follow the recommended user already).

As with @attribution, there are other plugin options for user recommendations, including deriving the suggestion from the Twitter account associated with an author.  At the post level, an additional user recommendation can be entered.

If the tweet button user follows neither the default user nor the one specified at the post level, both will be recommended after the tweet is sent.

That's all there is to it.  For the WordPress plugin configuring URLs to be shortened, assigning a default Twitter user to generate @user notices and assigning a default recommended Twitter to follow takes only a few minutes, yet the majority of publishers sin by neglecting one or more of these simple steps.  As I hope you can see, if you're one of these sinners repentance is readily at hand.

My SMX Toronto presentation on using structured markup for search engine optimization.

Includes a brief overview of semantic web principles, a comparison of the different flavors of structured markup (microformats, microdata and RDFa) and examples of rich snippets generated from structured markup.

Because of its obvious relevance to SEO, schema.org is the primary
vocabulary discussed, although the importance of GoodRelations for ecommerce sites is also acknowledged.

See the section below the embedded presentation for a brief addendum.

 
I wasn't able to squeeze these into the presentation list of relevant people to follow on Twitter, but here are some great search marketers that sometimes tweet about schema.org and related topics:

• AJ Kohn
• Doc Sheldon
• Phil Buckley
• Ani Lopez
• David Harry

And here's the main list from the presentation, reiterated here because somehow SlideShare has pointed all these links simply to twitter.com (Twitter's damnable implementation of rel="canonical" perhaps?):

• Dan Brickley
  Currently under contract from Google to oversee schema.org matters
• Martin Hepp
  The irrepressibly enthusiastic GoodRelations lead
• Jay Myers
  Web architect for Best Buy, responsible for the first major implementation of RDFa in an ecommerce enviroment
• Matthew Brown
  SEO and news optimization expert at AudienceWise, previously Director of Search Strategy at the New York Times
• Manu Sporny
  RDFa/RDF WebApps chair at W3C and one of the world's key semantic web technologists
• Gregg Kellogg
  Semantic media consultant and a part of the W3C's RDF Web Applications Working Group

Finally, in determining what sort of sites and topics might experience search visibility benefits by employing structured markup, I'd like to stress these two broad categories:

1. Any and all entities (people, places and organizations)
2. Anything relevant to the vending of goods and services, from online stores to brick-and-mortar local businesses

This SMX Toronto presentation explores how an agile approach to SEO can be used to address algorithmic and other changes made by Google on an ongoing basis.

In particular, "agile SEO" facilitates:

• The identification of opportunities that Google changes might represent.
• Responding in an effective way to big decreases or increases that come about as a result of Google algorithm changes.
• Anticipating what changes Google might make in the future.

I've used "agile" here in the everyday sense of "nimble." I know this may result in some confusion with the term "agile development," which has quite a different meaning, but it really was the right word for what I wanted to convey.

In a Wall St. Journal article published recently, Google's Amit Singhal suggested that changes were afoot at the world's dominant search engine.

While the Journal piece might overstate the magnitude of possible changes ahead (it is article author Amir Efrati who calls this "a makeover" to Google's formula, not Singhal himself) and while much of the substance of what Singhal is reported as having said is not new, there are indications that there is at least a substantial retooling of Google's search technology underway, and that the nature of these changes are related to Google's embrace of semantic web technologies.  As Efrati puts it, "the company is aiming to provide more relevant results by incorporating technology called 'semantic search,' which refers to the process of understanding the actual meaning of words."

What shape can we expect the nature of that retooling to take?  What changes might Google make to both better utilize semantic technologies and encourage their use by webmasters?  And what are the implications for search marketers and the SEO industry?

Provenance, Please!

In determining the relevance of thousands of resources, one of the most important things Google does is weeding out maliciously irrelevant resources – spam detection and filtering.  By "spam" I'm talking the whole gambit, from pages that deliberately misrepresent themselves (e.g. a page built to match the query "golf clubs" that redirects the user to an online poker site) to pages that try to exaggerate their actual relevance (e.g. an on-topic but keyword-stuffed page on golf clubs).  Google needs pick the gems out of the goo, and this procedure is still a requirement when it comes to assessing the value of structured data.

You won't find much discussion about spam in the semantic web world.  It's not that the semantic web framework does not account for the necessity to validate the veracity of documents, but that it takes a different tack.  At the top of the classic semantic web "layer cake" lays the "trust and proof" layer.  The chief mechanisms being worked on in this layer surround issues of provenance:  the origin and chronology of a document.

The W3C Provenance Working Group recently published the third working draft of its provenance data model; the introductory paragraph of this document provides an excellent overview of provenance issues:

For the purpose of this specification, provenance is defined as a record that describes the people, institutions, entities, and activities, involved in producing, influencing, or delivering a piece of data or a thing in the world. In particular, the provenance of information is crucial in deciding whether information is to be trusted, how it should be integrated with other diverse information sources, and how to give credit to its originators when reusing it. In an open and inclusive environment such as the Web, users find information that is often contradictory or questionable: provenance can help those users to make trust judgments.

(See an earlier post on "trust and proof" if you're dying to know more about approaches to this topic in the semantic web world.)

So how is this relevant to Google and its "semantic search" intentions?

As stated above, Google is very much in the trust and proof business, and if provenance proves to be a useful method to help Google determine to what extent resources can be trusted, and to verify the source of those resources, they'll add provenance mechanisms to their toolkit.

Google has previously stuck its toe in the provenance water in the form of the Google News meta tags original-source and syndication-source.  Interestingly, Google News seems to have (without fanfare) dropped support for original-source (it is detailed in the original announcement, but no long appears on the publisher metadata tags help page, having been "replaced" by the more limited standout tag).  That original-source was a bust makes sense, as it was a unilateral publisher declaration where publishers could readily unilaterally lie (and continued support for syndication-source makes sense in this context, as there's limited benefit to lying about not being the original source of a news story).

Not so with Google's big entrance into the world of provenance:  rel="author" and rel="publisher tags, combined with Google+ profile pages.

What does Google have to say about your author profile (emphasis mine)?

A rich profile is not only a great way to share information with users, but it also gives Google information we need to better identify you as the author of web content.

Your publisher profile (emphasis mine)?

Linking your Google+ page and your site like this not only helps you build relationships with friends and followers, but also gives Google information we can use to determine the relevancy of your site to a user query in Google Web Search.

The combination of a meta tag and a profile page provides Google with something that unilateral publisher meta tags cannot:  an identity verification mechanism.  Both author and publisher markup use Google+ profiles to validate the identity of a publisher, enabling them to establish provenance for resources linked to verified sources.  (In all the hoopla about Google's failure or success so far to establish Google+ as a social network, it's easy to overlook how the introduction of Google+ profile pages is an important structural improvement to previous Google profile pages.  There's now value for authors and publishers to have a correctly-linked Google+ page, even if Google+ profile owners never create or consume a Google+ post).

If Google's "semantic search" plans include facilitating the propagation of more structured data, whether by encouraging structured markup or by other mechanisms, it makes sense that they're going to continue to lean on provenance measures to help make sense of it.  Being able to determine the provenance of resources makes it easier to produce something akin to a "author graph" where queries return pages based not just on the authority of the page or site, but the authority of the author as well.

Not just what you say, but who you are, will almost certainly play an increasingly important role in the search results.

Vocabularies Big and Small

In the biggest move to date by the search engines to facilitate a more semantic web, Google, Bing and Yahoo did two things when they unveiled schema.org:  they stipulated a preferred markup specification for structured data (microdata) and provided a vocabulary to use with this markup (schema.org).

The bulk of analysis surrounding the introduction of schema.org has focused on the markup standard endorsed by Google, and in particular on the relative merits of microdata compared to microformats and RDFa.  From a practical perspective I think the schema.org vocabulary itself is of more importance to publishers, and probably to Google too.  Publishers will obviously produce more uniform – and so more readily-digestible – structured markup if they're provided with specific properties to apply to specific types of things.  And the more extensive that vocabulary, the more it allows for a greater volume and greater topical breadth of structured data.

Right now, the schema.org vocabulary is extremely useful marking up three types of data:

1. Named entities (people, places, organizations, etc.)
2. Media (information about types of web pages, images, videos, etc.)
3. Things that are bought and sold (especially on the Internet)

For this last type, schema.org is good about providing information about products and offers in the abstract, without defining domain-specific properties of the things being bought and sold.  For example, it allows publishers to say very precise things about the price of a specific television, and what consumers think of that television, but doesn't provide a mechanism to classify that television by size, display resolution, or any other property specifically relevant to televisions.

Which is at least in part to say that schema.org is not useful for marking up information specifically relevant to:

1. Any vegetable, mineral or animal that is not a named entity
2. Concepts

To a certain degree this is undoubtedly by design, and is certainly in keeping with the thrust of semantic web technologies, which is to provide machine-digestible data about real-life objects in the world (objects that can be represented by URIs, if you want to get technical about it).  Certainly the bulk of what Singhal apparently conveyed to the Journal surrounded improved recognition and inclusion of entity-based information (and, further to my previous parenthetical comment, of "identifying information about specific entities referenced" on web pages – which works hand-in-hand with linked URIs).

However, the consequences of a limited vocabulary are, well, limited information.  The best that a site about televisions that doesn't actually sell them can offer to Google now using schema.org is information about the web pages that house television information, but – again – no domain-specific information about the properties of any of those televisions.  As I've often contended, structured markup will not be truly useful to content product producers until it allows them to accurately describe a cat video.  Using schema.org the properties of the video itself may indeed by accurately described, but – aside from being able to declare the subject of the video as the entity "Mittens" – cannot provide any structured information about the adorable feline.

Among those actively building and finessing schema.org you'll find lively discussions about which new types should be added to the vocabulary, and the limits of extending the vocabulary (see the W3C vocabularies mailing list).  Semantic web types mostly caution against efforts at building all-inclusive vocabularies, and the foolhardiness of pursuing an "ontology of everything" – pointing out, sensibly, that domain-specific vocabularies should be reused by publishers, linking vocabularies when necessary.

This has a great deal of merit technologically and is sensible conceptually, but leaves the everyday webmaster (even a very technically adept webmaster) at a loss if he or she wants to express, in a structured way, some specific property of a topic not covered by schema.org.  However inelegant and monstrous it might be, building an "ontology of everything" – or at least a vastly expanded (and more readily extensible) schema.org vocabulary – might be in Google's best interest to facilitate and promote the production of structured data to improve its search capability and results.

The point is, from a taxonomic perspective, if Google hopes to exploit the benefits of classified data it will be in its best interest to support that classification by building or extending vocabularies.  They've been doing this on an ad hoc basis since the introduction of schema.org (June 2011), extending it to include sports (Aug. 2011) and then software applications (Sept. 2011).

Entities Rule!

What I've suggested above about Google extending vocabulary support to cover a broader range of non-entity types is highly speculative.  That entities will play a pivotal role in Google "semantic search" is much less so.

In an interview with Mashable, Singhal stressed – just as he subsequently did in the Journal interview – that entities will play a critical role in the road ahead for Google:

Google is "building a huge, in-house understanding of what an entity is and a repository of what entities are in the world and what should you know about those entities," said Singhal.

In 2010, Google purchased Freebase, a community-built knowledge base packed with some 12 million canonical entities. Twelve million is a good start, but Google has, according to Singhal, invested dramatically to "build a huge knowledge graph of interconnected entities and their attributes."

Apparently the work on entities has been continuing at a fevered pace.  According to the Journal:

Mr. Singhal said Google and the Metaweb team, which then numbered around 50 software engineers, have since expanded the size of the index to more than 200 million entities, partly by developing "extraction algorithms," or mathematical formulas that can organize data scattered across the Web.

Bill Slawski has argued – I think convincingly – that what some observers have classified as "brand bias" is in fact Google endeavoring to make sense of queries as they might be related to named entities, and returning information relevant to an identified brand whenever it can (Slawski recently named Google's Entity Detection patent as one of "the 10 most important SEO patents – see the end of the article on the patent for more resources about Google and entities).

Between Google's filings of entity-related patents, it's 2010 purchase of Metaweb, the introduction of shema.org (supporting the structured markup of named entities) and Singhal's own unambiguous statements, it is clear that entities will play an important role in Google's "semantic search" initiative.

Does Retooling Mean More Tools?

Whether Google continues to build out from schema.org or introduces an entirely different knowledge organization scheme, the best vocabulary in the world isn't going to help if nobody uses it.  While it's not rocket science for developers, even relatively code-savvy webmasters and SEOs (these days, often the ones actually marking up HTML with microdata) find it difficult to get microdata right.  How might Google make it easier for webmasters to use structured markup?

One of the reasons often cited for the slow adoption of semantic web technologies is a paucity of useful, relatively easy-to-use tools.  Certainly there's very little in the way of microdata authoring tools, and the most widely-used content management systems (Drupal notwithstanding) lack native microdata support.  Were Google to introduce authoring tools that made structured markup easy that would probably go a long way toward improving adoption, with the added benefit that the markup would be uniform and syntactically sound.

Are "Direct Answers" (Or Much Else Here) New?

Many commentators in the search marketing industry have remarked that a "semantic" Google might answer more user queries directly in the search results, obviating the need for searchers to visit a site for that information, and so depriving publishers of (search-derived) traffic.  The Journal recounts:

Over the next few months, Google's search engine will begin spitting out more than a list of blue Web links. It will also present more facts and direct answers to queries at the top of the search-results page.

A tweet from Craig Fifield summarizes the reaction of many search marketers:

Should Google force a user to click through to a site to consume a piece of information that it thinks it can accurately return in the search results?  As long as the source or sources of that information is cited and linked (a SERP-level provenance measure) it seems to me a user's best interests are served without requiring them to jump through that particular hoop.

While I recognize that there's lots of nuance involved in debates about information ownership and Google's use of information published by others, I think that SEOs worry far too much about "direct answers" poaching traffic to their sites.  The nature of such information is, first, limited to the sort of facts that can be encapsulated in relatively brief form (e.g. "when was William Shakespeare born") and, second, as likely or not be found on a non-commercial authority site that's going to outrank you anyway (e.g. "when was William Shakespeare born").  Sites with rich content are still likely to fare well, and do any of us bemoan the loss of content farm pages in the search results that were written specifically to answer queries like "when is Memorial Day 2012?".

In any case, as ReadWriteWeb's Jon Mitchell correctly notes, such "direct answers" are not new.  Perhaps we'll see more of these, or perhaps even the development of user interfaces that permit searches to delve deeper into the response to their question without leaving a Google page.  Were this ever to present an egregious challenge to a publisher's traffic, that publisher could always prevent Google from indexing their content – though of course, that would also prevent users from discovering that content from search, and almost certainly end up being a case of cutting off one's nose to spite one's face.

As I never tire of telling people (but as they almost certainly tire of hearing), Google has been employing semantic web technologies for a long time (this earlier post on SEO and the semantic web provides examples).  It's possible that what's on the Google horizon when it comes to "direct answers," the use of named entities and other semantic web technologies may end up be being big changes under the hood, but rather less evident in the way actual search results are displayed.

The Big Takeaways for Search Marketers

Sites with structured markup will appear in the results of more queries, rank better than non-structured web pages for those queries, and have a greater visibility both in linked and "direct answer" verticals.

Sites that can successfully identify and interlink entities in a fashion that Google can readily understand, whether by the use of structured data or otherwise, will find themselves particularly favored, both in linked search results and as sources of information extracted by Google and presented directly in responses to queries.

Up until now Google has been insistent upon the fact that structured markup does not impact the ranking of a site in its search results.  To channel my inner Jan Brady, it's all rich snippets, rich snippets, rich snippets!  But here comes a point where it becomes disingenuous to leverage structured data solely to manipulate the way in which it is represented.  More to the point as it pertains to Google's business model, there comes a point where willfully ignoring the information structured data provides results an inferior user experience.

If a site – by virtue of declaring and linking named entities, employing structured markup and providing verifiable provenance information – helps Google understand that a given resource is a relevant match for a given query, Google will inevitably favor relevant results over a fear that, in doing so, it may be exhibiting bias against more loosely structured sites.  Google will absolutely continue to try to understand sites with unstructured data, but at the end of the day the better Google understands a resource, the more use it can – and will – make of it.