Indexing and robots.txt

• noindex in a robots meta tag
• noindex in an X-Robots-Tag header
• nofollow in a robots meta tag
• Disallowed for every crawler but ZaaganBot
• Disallowed for ZaaganBot
• A client-rendered app shell