Hacker Newsnew | past | comments | ask | show | jobs | submit | prokizzle's commentslogin

Screenshot of original robots.txt for when the changes get pushed to production:

http://imgur.com/8zkK8w2


Also, in

   Disallow: /sr/mattress* Disallow: /sr/mattresses*
Doesn't A imply B?


maybe, but why would THAT be the honey pot? it could be any phrase, really.


Well, you'd want them to try to crawl it. Perhaps there was a bot crawling their site for such links and they wanted to identify specifically that one. It seems most likely that such URL's never existed, but this would be a decent way to identify a bot that's otherwise not distinguishable.


Maybe that was their intention, but that's not how it's done.


Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: