Be careful, you use the robots.txt generator at your very own possibility. It can be advisable to validate that your robots file would not exclude pages by miscalculation. There are plenty of tools to check this such as Google tool. Should you be creating the file manually, then you'll want https://seotoolstube.com/