# Robots.txt file to crawl all pages

User-agent: *
Disallow: /*?