User-agent: * Disallow: /login Disallow: /pages/app/ Disallow: /pages/404.html Disallow: /api/