User-agent: * Disallow: /wp-admini/ Allow: /wp-admin/admin-ajax.php Disallow: /wp-includes/ Disallow: /wp-uploads/ Disallow: *.pdf$ # Block pdf files from all bots. Albeit non-standard, it works for major search engines. Sitemap: https://freiesradio.at/wp-sitemap.xml User-agent: Googlebot Disallow: User-agent: googlebot-image Disallow: / User-agent: googlebot-mobile Disallow: User-agent: MSNBot Disallow: User-agent: Slurp Disallow: User-agent: Teoma Disallow: / User-agent: Gigabot Disallow: / User-agent: Robozilla Disallow: / User-agent: Nutch Disallow: / User-agent: ia_archiver Disallow: / User-agent: baiduspider Disallow: / User-agent: naverbot Disallow: / User-agent: yeti Disallow: / User-agent: yahoo-mmcrawler Disallow: / User-agent: psbot Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / User-agent: CCBot Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: /