In robots.txt, should I use a wildcard at the end of a disallow directive?

I want to disallow a specific folder and all of its files and subdirectories but I don’t know the difference between Disallow: /somedir/ and Disallow: /somedir/*. which one of these lines should I use?

By the way, what does Disallow: /somedir? mean? should I use it too?

Here is Solutions:

We have many solutions to this problem, But we recommend you to use the first solution because it is tested & true solution that will 100% work for you.

Solution 1

Disallow: /somedir/ and Disallow: /somedir/* Both are same, here is what google dev said about it.

/* is Equivalent to /. The trailing wildcard is ignored.

And Disallow: /somedir? will block only directory like /somedir?id=1, and /somedir?id=2.

Note: Use and implement solution 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply