robots.txt is a plain text file at the root of your site that tells crawlers which paths they may and may not fetch. Each block names a user agent, then lists the paths to disallow or allow. A crawler reads it before it fetches anything else.
It controls crawling, not indexing. A page blocked in robots.txt is not fetched, but it can still be listed if other sites link to it, just with no description. To keep a page out of the index you use a noindex directive, which an engine can only see if it is allowed to crawl the page. The two pull in opposite directions, so you rarely want both on one page.
You can point crawlers at your sitemap from robots.txt, and set rules per bot, so you can allow Googlebot while disallowing a particular AI crawler.
A common mistake is listing a URL in your sitemap that robots.txt also blocks, which tells engines to crawl a page you have shut. Kenovar's robots tester fetches your rules, lets you edit and test paths against them live and cross checks the sitemap for that conflict.