Yesterday I was doing a bit of site crawling with Screaming Frog. Since the crawl was for the purpose of analysing authority distribution, I was most interested in the canonical URLs and meta robots information, which is one of the useful features of the crawler.
I’m used to seeing ‘noindex’ and ‘nofollow’; I’ve written about this before in my robots txt guide. But when crawling one site, I found this:
My first thought: wtf is a noodp and a doydir. I’d never seen it before.
Or had I? Ages ago, I remembered I’d overheard a conversation about site-owned meta descriptions not appearing in the SERPs – and that’s what these robots tags are responsible for fixing.
What’s a noodp?
‘Noodp’ stands for ‘No Open Directory Project’. It’s used by Google, Yahoo and Bing so that the website owner can specify that their own written meta descriptions are used, rather than a meta description taken from DMOZ. It was introduced in 2006, initially by Bing in May, while Google and Yahoo followed suit in July and October respectively.
Google will sometimes choose to offer up a different title and description to the one you have specified by using on-page text or anchors; or DMOZ. The intention is to provide results with more relevance to the query. Google has plenty of reasons it lists to provide these alternative titles and descriptions in the snippets here; so it’s worth taking a look at the descriptions and titles you have specified and working out why Google didn’t think they were worth showing.
The tag specifically stops page titles and descriptions from DMOZ from appearing in the SERPs – especially useful if you haven’t updated your DMOZ information for ages.
What is a noydir?
Very much like the noodp tag, the noydir tag is specific to Yahoo and prevents the search engine from presenting users with snippets from the Yahoo Directory in the SERPs.
Adding the ‘noydir’ tag will get your specified meta description and title tag visible again for whichever pages are currently showing the Yahoo Directory entry.
How to Use Noodp and Noydir
Just like another other robots directive, really. Just chuck it in the head tag of your source code.
As is standard with robots directives, you can specify rules for Google, Yahoo and Bing specifically or separately.
You can also add it to WordPress self-hosted sites by adding the tag within the header.php file in the editor section; but if you’re a bit nervous about coding or you don’t have a self-hosted account, you can use the SEO Yoast WordPress plugin to apply sitewide noodp and noydir meta robots tags.
One thing to note; it can take some time for these robots directives to notice these changes – I’ve had reports that it can take a couple of days.