I got an email from Google Analytics that Googlebot couldn’t crawl my site. Any ideas how to fix this?
Here is the error:
Googlebot couldn’t crawl your URL because your server either requires authentication to access the page, or it is blocking Googlebot from accessing your site.
The folder causing the error is: /wp-content/sp-resources/forum-smilies
why are you trying to crawl that directory??? no forum content there… no real reason to crawl anything below wp-content… sounds like you need a robots.txt file…
*I’m* not trying to crawl that directory, lol, Googlebot is! I am not privy to the ways of Googlebot, he goes where he wants 
I have never gotten this error before installing Simple Press, why would I need a robots.txt file?
yeah, I meant why are you letting bots crawl it… if you dont want to make your own robots.txt, there are many wp plugins that will do it for you…
you may be getting it now, because bots are detecting that plugin because of smilies in your forum content… but having a robots.txt to control how bots behave on your site has always been a google recommendation…
