<?xml version="1.0"?>
<oembed><version>1.0</version><provider_name>Adwords Consultant India : DigitalSRC</provider_name><provider_url>https://digitalsrc.com/blog</provider_url><author_name>digitalSRC</author_name><author_url>https://digitalsrc.com/blog/author/saptarshi/</author_url><title>Robots.txt and Google - Adwords Consultant India : DigitalSRC</title><type>rich</type><width>600</width><height>338</height><html>&lt;blockquote class="wp-embedded-content"&gt;&lt;a href="https://digitalsrc.com/blog/robots-txt-and-google/"&gt;Robots.txt and Google&lt;/a&gt;&lt;/blockquote&gt;
&lt;script type='text/javascript'&gt;
&lt;!--//--&gt;&lt;![CDATA[//&gt;&lt;!--
		/*! This file is auto-generated */
		!function(d,l){"use strict";var e=!1,n=!1;if(l.querySelector)if(d.addEventListener)e=!0;if(d.wp=d.wp||{},!d.wp.receiveEmbedMessage)if(d.wp.receiveEmbedMessage=function(e){var t=e.data;if(t)if(t.secret||t.message||t.value)if(!/[^a-zA-Z0-9]/.test(t.secret)){for(var r,i,a,s=l.querySelectorAll('iframe[data-secret="'+t.secret+'"]'),n=l.querySelectorAll('blockquote[data-secret="'+t.secret+'"]'),o=new RegExp("^https?:$","i"),c=0;c&lt;n.length;c++)n[c].style.display="none";for(c=0;c&lt;s.length;c++)if(r=s[c],e.source===r.contentWindow){if(r.removeAttribute("style"),"height"===t.message){if(1e3&lt;(a=parseInt(t.value,10)))a=1e3;else if(~~a&lt;200)a=200;r.height=a}if("link"===t.message)if(i=l.createElement("a"),a=l.createElement("a"),i.href=r.getAttribute("src"),a.href=t.value,o.test(a.protocol))if(a.host===i.host)if(l.activeElement===r)d.top.location.href=t.value}}},e)d.addEventListener("message",d.wp.receiveEmbedMessage,!1),l.addEventListener("DOMContentLoaded",t,!1),d.addEventListener("load",t,!1);function t(){if(!n){n=!0;for(var e,t,r=-1!==navigator.appVersion.indexOf("MSIE 10"),i=!!navigator.userAgent.match(/Trident.*rv:11\./),a=l.querySelectorAll("iframe.wp-embedded-content"),s=0;s&lt;a.length;s++){if(!(e=a[s]).getAttribute("data-secret"))t=Math.random().toString(36).substr(2,10),e.src+="#?secret="+t,e.setAttribute("data-secret",t);if(r||i)(t=e.cloneNode(!0)).removeAttribute("security"),e.parentNode.replaceChild(t,e)}}}}(window,document);
//--&gt;&lt;!]]&gt;
&lt;/script&gt;&lt;iframe sandbox="allow-scripts" security="restricted" src="https://digitalsrc.com/blog/robots-txt-and-google/embed/" width="600" height="338" title="&#x201C;Robots.txt and Google&#x201D; &#x2014; Adwords Consultant India : DigitalSRC" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" class="wp-embedded-content"&gt;&lt;/iframe&gt;</html><description>Google&#x2019;s recent declaration makes robots.txt all the more important. John Muller from Google informed that in case Google bot requests for robots.txt for your website and finds it unreachable, Google would prefer to stay on safe side in such cases and do not index the website. By &#x201C;unreachable&#x201D; it was meant, when the server times [&hellip;]</description></oembed>

<!--
Performance optimized by W3 Total Cache. Learn more: https://www.boldgrid.com/w3-total-cache/

Object Caching 4/62 objects using disk
Page Caching using disk: enhanced (Requested URI contains query) 
Minified using disk
Database Caching 10/33 queries in 0.008 seconds using disk

Served from: digitalsrc.com @ 2026-04-27 05:48:24 by W3 Total Cache
-->