robots.txt and sitemap

← Back

The sitemap is generated automatically at /sitemap.xml, and robots.txt is edited from the admin panel at its own address.

The sitemap is located at /sitemap.xml and is generated automatically on each request — there is no file on disk and you don't need to create one. Only pages with access level "Everyone" are included; language versions are listed within a single entry, as Google requires.

robots.txt is opened at /robots.txt: a visitor and a search bot will see the text, while you see the editor. An empty field here is meaningful: it means "generate automatically", and the engine will write a sensible file with a link to the sitemap.

The robots.txt section exists on every site, but it is not shown in the menu — that's normal.

If it didn't work

  • Changes to robots.txt are not visible. If there is a real robots.txt file in the site root on the server, the web server serves it and the request doesn't reach the engine. The file must be deleted.
  • Page is not in the sitemap. Check the access level: everything except "Everyone" is not included in the sitemap. Addresses with extensions and informers are also excluded.
Did the article help?

See also