git.shiar.nl / sheet.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: 03275d1) | patch
robots: host directive to reduce mirror crawling
authorMischa POSLAWSKY <perl@shiar.org>
Tue, 2 May 2017 21:35:02 +0000 (23:35 +0200)
committerMischa POSLAWSKY <perl@shiar.org>
Mon, 29 May 2017 17:23:12 +0000 (19:23 +0200)
commitff54b2f125dada0899ac2b12bf4fa9f496d03762
treee7cc8a68952772c73082c18fcb5dc43c3fb3cf4btree | snapshot
parent03275d12a6e5b198a3751318743a6fdd338583ffcommit | diff
robots: host directive to reduce mirror crawling

Preferred domain to indicate main site for at least Yandex bots:
<https://yandex.com/support/webmaster/controlling-robot/robots-txt.xml#host>
robots.txt diff | blob | history
sheet.shiar.nl - software cheat sheets