Adult SEO Blog

2/13/2005

• Adult SEM Map
  • Adult SEM Forum
• Recommended Blogs
  • Baddog's Bytes
  • Business Voyeur
  • Free Adult Website Traffic
• Recommended Links
  • AVN Online
  • Herbal Affiliate Program
  • Multiple Class C Hosting
• Categories:
  • general
  • adult seo tactics
  • affiliate marketing
  • link strategies
  • adult seo tutorials
  • adult seo training
  • adult content optimization
  • adult seo articles
  • search engine news
• Search:
  
  
• Archives:
  • September 2005
  • August 2005
  • July 2005
  • May 2005
  • April 2005
  • March 2005
  • February 2005
  • January 2005
• Adult Search Engine Newsletter
  
  name:
  email:
  Yes! I want to recieve Adult SEM Newsletter. I agree with AdultSEM.com's Privacy Statement and Terms. I also understand that I will recieve the newsletter by email.
  
• 

• February 2005

  S	M	T	W	T	F	S
  « Jan				Mar »
  		1	2	3	4	5
  6	7	8	9	10	11	12
  13	14	15	16	17	18	19
  20	21	22	23	24	25	26
  27	28

• Other:
  • Login
  • Register
• Meta:
  • 
  • 
  • 
  • 
  • Comments RSS 2.0
  • Valid XHTML
  • WP

  • Video Search: Now more Flashy and Fresh than ever before
    In our quest for video nirvana, we've performed a major update to our Video Search index to bring you even more video content, and increased reach for video content worldwide. So now you'll be able to find even more relevant...

    SDForum Search SIG on Tagging and Social Bookmarking at Yahoo! Next We
    Just a quick heads-up. If you're in the Bay Area (or are exceptionally skilled at finding cheap flights here), drop by Yahoo! next Tuesday evening for another SDForum Search SIG. The topics this time are Tagging and Social Bookmarking. As...

    NEW LIGHTSPEEDCASH SITE: Britney Lightspeed
    

    It's Summer Time -- Happy Travels
    

    Summer is in full swing, and it's getting harder and harder to focus on work. My mind constantly wanders off to possible destinations for my next vacation. I'm thinking a two-week European getaway -- Barcelona for the architecture, Provence and French Riviera for some great food and relaxation by the beach and finally London for some shopping and Broadway shows.

    It's getting easier than ever to plan a vacation with the help of Yahoo! Search. Whether you're a budget traveler or a luxury traveler, you can quickly find the right hotel. After a quick search using 'Hotels Barcelona,' prices look pretty affordable.

    

    But for me, I'm somewhere in the middle. I can't rough it in hostels but I'm also not willing to pay premium prices for a 5 star hotel. So, I started by checking the 4 star hotels that came up in my Yahoo! Shortcuts results, hoping to find something nice for around $150. Thanks to Yahoo! Travel, the narrowed results gave me the photos, reviews and prices all in one place, which makes the process of choosing a hotel much easier.

    
    

    
    Another great way to save time is to go with a trusted name. For the London portion of my trip, I searched for 'Hilton London' in Yahoo! Search and came up with this helpful Shortcut.

    
    

    
    The third option I clicked on looks pretty promising -- near a Tube stop, good user ratings, almost within desired price range, and the photos look great!

    
    

    
    Two searches on Yahoo! Search and I've knocked out two major destinations on my list. Now, I just need to get a feel for Provence and the French Riviera. My vacation planning is coming along nicely!

    Wherever your travels may take you -- Shanghai, Florence, or New York -- try finding the perfect hotel for your next trip on Yahoo! Search. Try hotels Shanghai, hotels Florence or hotels New York.

    Happy travels!

    
    Lingxian Ding
    Yahoo! Search

    The Santa SmackDown in Search: You Better Watch Out, You Better Not Cr
    

    The holiday season is chock full of characters, stories and traditions - all of which see their fair share of search activity. In fact, we hear there's a bit of competition among Santa's cast and crew for the top billing in search. To relieve a bit of pressure (December is a stressful time of year for these guys), Santa Claus returned to the frozen stage this month to oversee some one-on-one action in the Search Smackdown. He got out his list and set up some Yuletide matches in the Buzz Arena...and man, will the reindeer fur fly.

    Based on this season's search data, we've got the Search odds right here, and a few highlights from their past matches. All we gotta say is, when Jack "The Cannibal" Frost comes nipping at your nose, you better not cry, you better watch out.

    Smack Down	Red Corner	Green Corner	And the Buzz Goes To...	Odds
    Vengeance	Rudolph "Red-Nosed" Reindeer	Dasher and Dancer et al	Vengeance is sweet. The reject reindeer outshined the tag team by way more than a nose, although that was too bad about Grandma getting in the way.	2:1 Rudolph
    Snowblind	Jack "The Cannibal" Frost	Frosty the Snowman	Frosty proved too slippery for the nimble Jack, although he better watch his addiction to that corn-cob pipe.	9:1 Frosty
    Extreme Fighting	Blister the Heat Miser	Snow Miser aka Mr. Icicle	Mr. Icicle may have the cooler ragtime routine, but his hotheaded half-brother burns him in Buzz by nine-fold. Now we know the reason behind global warming.	11:1 Heat Miser
    Buzz Office Rumble	"Home Alone"	"A Christmas Story"	The McAllister kid can lay a mean booby trap, but all the blow torches in the basement can't match a Red Ryder -- plus that underhanded tongue-on-flagpole trick should freeze any loner in his tracks.	6:1 Christmas Story
    The Unforgiven	Grinch	Scrooge	A face-off for the ages: the cave-dweller with the super-shrunken heart against that squeezing, wrenching, grasping, scraping, clutching, covetous, old sinner. Ah, but that Broadway endorsement clinched the Buzz for the Grinch.	6:1 Grinch

    B-Ball Madness, Web Video Style…
    These guys are hungry, that’s why we watch them. No contract disputes, no ego tirades – at least not yet – just pure college basketball and lots of it. Starting this week, we’re pulling together as much NCAA tournament coverage...

    The Good, The Bad and the Incorrigible
    

    Recruiting webmasters to your small affiliate program (a lower cost To
    

    One Standard Fits All: Robots Exclusion Protocol for Yahoo!, Google an
    

    Over the last couple of years, we've been collaborating with Google and Microsoft to make webmasters' efforts more effective across the major search engines. By bringing standards such as Sitemaps and improvements like auto-discovery and cross-host submission, webmasters can simplify their account management across the different search engines.

    The Robots Exclusion Protocol (REP) lets content publishers specify which parts of their site they want public and which parts they want to keep private from robots, whether it's controlling the visibility of their content across their site (via robots.txt) or at the level of individual pages (via META tags). REP was introduced in the early 1990's and is the de facto standard. Its strengths lie in its flexibility to evolve in parallel with the web, its universal implementation across major search engines and all major robots and in the way it works for any publisher, no matter how large or small. We've heard that there is some confusion around the specific implementation of REP supported by each engine. Since we've never detailed the specifics of implementing the protocol, today we're releasing detailed documentation on how REP directives will be handled by the three major search providers.

    Common REP Directives
    The following are all the major REP features currently implemented by Google, Microsoft and Yahoo!. Each of these directives can be specified to be applicable for all crawlers or for specific crawlers by targeting them to specific user-agents, which is how any crawler identifies itself. Each of us also supports Reverse DNS based authentication of our crawler, and you can use this validate the identity of any crawlers claiming a particular user-agent.

    1. Robot.txt Directives

    DIRECTIVE	IMPACT	USE CASE(s)
    Disallow	Tells a crawler not to crawl your site or parts of your site -- your site's robots.txt still needs to be crawled to find this directive, but the disallowed pages will not be crawled.	'No crawl' pages from a site. This directive in the default syntax prevents specific path(s) of a site from crawling.
    Allow	Tells a crawler the specific pages on your site you want indexed so you can use this in combination with Disallow. If both Disallow and Allow clauses apply to a URL, the most specific rule - the longest rule - applies.	This is useful in particular in conjunction with Disallow clauses, where a large section of a site is disallowed, except a small section within it.
    $ Wildcard Support	Tells a crawler to match everything from the end of a URL -- large number of directories without specifying specific pages.	'No Crawl' files with specific patterns, for eg., files with certain filetypes that always have a certain extension, say pdf; etc.
    Sitemap Location	Tells a crawler where it can find your sitemaps.	Point to other locations where feeds exist to point the crawlers to the site's content.

    2. HTML META Directives

    These directives can either be placed in the HTML of a page or in the HTTP header for non-HTML content like PDF, video, etc. using an X-Robots-Tag. The X-Robots-Tag mechanism allows these directives to be available for all types of documents -- HTML or otherwise. If both forms of the tag, HTML META and X-Robots-Tag in the header are present, the most restrictive one applies.

    DIRECTIVE	IMPACT	USE CASE(s)
    NOINDEX META Tag	Tells a crawler not to index a given page.	Don't index the page. This allows pages that are crawled to be kept out of the index.
    NOFOLLOW META Tag	Tells a crawler not to follow a link to other content on a given page.	Prevent publicly writeable areas from being abused by spammers looking for link credit. By NOFOLLOW, you let the robot know that you are discounting all outgoing links from this page.
    NOSNIPPET META Tag	Tells a crawler not to display snippets in the search results for a given page.	Present no abstract for the page on search results.
    NOARCHIVE META Tag	Tells a search engine not to show a "cached" link for a given page.	Do not make a copy of the page available to users from the search engine cache.
    NOODP META Tag	Tells a crawler not to use a title and snippet from the Open Directory Project for a given page.	Do not use the ODP (Open Directory Project) title and abstract for this page in Search.

    Other REP Directives
    Yahoo!-specific REP directives that are not supported by Microsoft and Google include:

    • Crawl-Delay: Allows a site to delay the frequency with which a crawler checks for new content
    • NOYDIR META Tag: This is similar to the NOODP META Tag above but applies to the Yahoo! Directory, instead of the Open Directory Project
    • Robots-nocontent Tag: Allows you to identify the main content of your page so that our crawler targets the right pages on your site for specific search queries by marking out non content parts of your page. We won't use the sections tagged as such for indexing the page or for the abstract in the search results.

    Apart from these tools in the REP, Yahoo! Site Explorer also provides further ways to tell Yahoo! to Delete URLs, or Rewrite Dynamic URLs to remove spurious parameters. You can learn more about our crawler at the Slurp Help page.

    We plan to continue coordinating with the leading search engines to ensure simplicity for webmasters, so stay tuned for more developments in the future. You can also find information on the Official Google Webmaster Central Blog or the Microsoft Live Search Webmaster Center Blog. And, feel free to share your thoughts with the webmaster community on the Site Explorer Suggestion Board.

    
    Priyank Garg
    Director, Product Management
    Yahoo! Search

    One Standard Fits All: Robots Exclusion Protocol for Yahoo!, Google an
    

    Over the last couple of years, we've been collaborating with Google and Microsoft to make webmasters' efforts more effective across the major search engines. By bringing standards such as Sitemaps and improvements like auto-discovery and cross-host submission, webmasters can simplify their account management across the different search engines.

    The Robots Exclusion Protocol (REP) lets content publishers specify which parts of their site they want public and which parts they want to keep private from robots, whether it's controlling the visibility of their content across their site (via robots.txt) or at the level of individual pages (via META tags). REP was introduced in the early 1990's and is the de facto standard. Its strengths lie in its flexibility to evolve in parallel with the web, its universal implementation across major search engines and all major robots and in the way it works for any publisher, no matter how large or small. We've heard that there is some confusion around the specific implementation of REP supported by each engine. Since we've never detailed the specifics of implementing the protocol, today we're releasing detailed documentation on how REP directives will be handled by the three major search providers.

    Common REP Directives
    The following are all the major REP features currently implemented by Google, Microsoft and Yahoo!. Each of these directives can be specified to be applicable for all crawlers or for specific crawlers by targeting them to specific user-agents, which is how any crawler identifies itself. Each of us also supports Reverse DNS based authentication of our crawler, and you can use this validate the identity of any crawlers claiming a particular user-agent.

    1. Robot.txt Directives

    DIRECTIVE	IMPACT	USE CASE(s)
    Disallow	Tells a crawler not to crawl your site or parts of your site -- your site's robots.txt still needs to be crawled to find this directive, but the disallowed pages will not be crawled.	'No crawl' pages from a site. This directive in the default syntax prevents specific path(s) of a site from crawling.
    Allow	Tells a crawler the specific pages on your site you want indexed so you can use this in combination with Disallow. If both Disallow and Allow clauses apply to a URL, the most specific rule - the longest rule - applies.	This is useful in particular in conjunction with Disallow clauses, where a large section of a site is disallowed, except a small section within it.
    $ Wildcard Support	Tells a crawler to match everything from the end of a URL -- large number of directories without specifying specific pages.	'No Crawl' files with specific patterns, for eg., files with certain filetypes that always have a certain extension, say pdf; etc.
    Sitemap Location	Tells a crawler where it can find your sitemaps.	Point to other locations where feeds exist to point the crawlers to the site's content.

    2. HTML META Directives

    These directives can either be placed in the HTML of a page or in the HTTP header for non-HTML content like PDF, video, etc. using an X-Robots-Tag. The X-Robots-Tag mechanism allows these directives to be available for all types of documents -- HTML or otherwise. If both forms of the tag, HTML META and X-Robots-Tag in the header are present, the most restrictive one applies.

    DIRECTIVE	IMPACT	USE CASE(s)
    NOINDEX META Tag	Tells a crawler not to index a given page.	Don't index the page. This allows pages that are crawled to be kept out of the index.
    NOFOLLOW META Tag	Tells a crawler not to follow a link to other content on a given page.	Prevent publicly writeable areas from being abused by spammers looking for link credit. By NOFOLLOW, you let the robot know that you are discounting all outgoing links from this page.
    NOSNIPPET META Tag	Tells a crawler not to display snippets in the search results for a given page.	Present no abstract for the page on search results.
    NOARCHIVE META Tag	Tells a search engine not to show a "cached" link for a given page.	Do not make a copy of the page available to users from the search engine cache.
    NOODP META Tag	Tells a crawler not to use a title and snippet from the Open Directory Project for a given page.	Do not use the ODP (Open Directory Project) title and abstract for this page in Search.

    Other REP Directives
    Yahoo!-specific REP directives that are not supported by Microsoft and Google include:

    • Crawl-Delay: Allows a site to delay the frequency with which a crawler checks for new content
    • NOYDIR META Tag: This is similar to the NOODP META Tag above but applies to the Yahoo! Directory, instead of the Open Directory Project
    • Robots-nocontent Tag: Allows you to identify the main content of your page so that our crawler targets the right pages on your site for specific search queries by marking out non content parts of your page. We won't use the sections tagged as such for indexing the page or for the abstract in the search results.

    Apart from these tools in the REP, Yahoo! Site Explorer also provides further ways to tell Yahoo! to Delete URLs, or Rewrite Dynamic URLs to remove spurious parameters. You can learn more about our crawler at the Slurp Help page.

    We plan to continue coordinating with the leading search engines to ensure simplicity for webmasters, so stay tuned for more developments in the future. You can also find information on the Official Google Webmaster Central Blog or the Microsoft Live Search Webmaster Center Blog. And, feel free to share your thoughts with the webmaster community on the Site Explorer Suggestion Board.

    
    Priyank Garg
    Director, Product Management
    Yahoo! Search