Google is calling into question a long-held belief about the location of robots.txt files (which celebrate their 30th anniversary this year). Gary Illyes, an analyst at Google, revealed that these files can be centralized on CDNs, offering new flexibility to site administrators. Key takeaway: Robots.txt: new flexibility according to [...]
The article “Robots.txt file: no need to place it on the root domain anymore!” was published on the site Abundance.
