# robots.txt for UK Logistics Company Website # ============================================= # This file tells search engine crawlers which pages they can access. # # SEO Notes: # - Allow all crawlers to access all content (we want to be indexed) # - Reference the sitemap so crawlers can discover all pages efficiently # - TODO: Update the sitemap URL to your production domain # Allow all web crawlers User-agent: * Allow: / # Sitemap location - helps search engines discover all pages Sitemap: https://nhtlogistics.co.uk/sitemap.xml # Block any sensitive paths if needed (uncomment and modify as needed) # Disallow: /admin/ # Disallow: /private/