robots: host directive to reduce mirror crawling

Preferred domain to indicate main site for at least Yandex bots:
<https://yandex.com/support/webmaster/controlling-robot/robots-txt.xml#host>

diff --git a/robots.txt b/robots.txt
index 4ba41488a278f873796b611151ab7564588f1ab8..15b6ecb586ed18dd64f57c40e774dab89ddd2bc9 100644 (file)
--- a/robots.txt
+++ b/robots.txt
@@ -2,3 +2,4 @@ User-agent: *
 Disallow: /source/*::*
 
 Sitemap: http://sheet.shiar.nl/sitemap.xml
+Host: sheet.shiar.nl