From b64ac3ada938f8c288c0a7b6be43979fc448812f Mon Sep 17 00:00:00 2001 From: Ralph Siemsen Date: Fri, 26 Nov 2021 21:27:49 -0500 Subject: [PATCH] robots.txt: block the archive subdir Add a robots.txt file (http://www.robotstxt.org/robotstxt.html) to block web crawlers from indexing the old website archive. Signed-off-by: Ralph Siemsen Signed-off-by: David Oberhollenzer --- robots.txt | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 robots.txt diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..1cd4556 --- /dev/null +++ b/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Disallow: /archive -- 2.50.1