User-agent: Baiduspider Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Yandex Disallow: / User-agent: AhrefsBot Disallow: / User-Agent: MJ12bot Disallow: / User-agent: CCBot Disallow: / User-agent: * # only friendly URLs to be indexed Disallow: /api/ Disallow: /basket Disallow: /browserconfig.xml Disallow: /checkout Disallow: /cms/ Disallow: /my-profile Disallow: /my-products Disallow: /site/api/ Disallow: /site/cms/ Disallow: /site/user/ Disallow: /site/user/currency.phtml Disallow: /temp/ # no statics other than PDF Disallow: /*.xls$ Disallow: /*.xlsx$ Disallow: /*.ppt$ Disallow: /*.pptx$ Disallow: /*.doc$ Disallow: /*.docx$ Disallow: /*.rtf$ Disallow: /*.swf$ # main sitemap - must be fully qualified URL Sitemap: https://www.bioscience.co.uk/sitemaps/cbio-map-index.xml