Google

Newbie Joined: Nov 20, 2008 Posts: 55

Not sure if you are aware of this, but if you do a search for "code authors" in Google your site comes up first. The thing about it is the description just says that you have been blocked. So you must be blocking their search bot or something.

Site Admin Joined: Jan 06, 2006 Posts: 311

Oh my... hey, look on the bright side, they will still get to your site through your news article on my site showing up in third place. Laughing

I wonder if more of us have this issue. Would sure like to know what caused (or still is causing it). Hope you can get it cleared up. I just hate trying to work with Google on problems like this. Communication with them is one way... assuming you can even find a way TO contact them.

Newbie Joined: May 12, 2009 Posts: 74 Location: TX

This is what the cached version of the page is saying if it helps...

Quote:

Our systems have detected unusual traffic from your computer network. Please try your request again later. Why did this happen?

This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop.

This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests. If you share your network connection, ask your administrator for help � a different computer using the same IP address may be responsible. Learn more

Sometimes you may see this page if you are using advanced terms that robots are known to use, or sending requests very quickly.

Posted: Sat Dec 31, 2011 5:57 pm

That's hillarious!
I checked Google and under the top result which you mentioned, it saus "You shared this" hahahaha

Looks like I shared a page that is no longer public or something. Thanks for letting me know, I'll ping Google and get it updated.

Posted: Sat Dec 31, 2011 6:06 pm

Hmm, according to NS, Google performed a script attack on my site from the IP 66.249.71.202
I checked further and the UA was for Google-Mediapartners which is the adsense bot. This makes absolutely NO sense at all so I have to assume it was some script kiddie spoofing the IP and UA. I'll investigate further but thank you all for the heads-up.

Posted: Sat Dec 31, 2011 8:29 pm

montego wrote:

Oh my... hey, look on the bright side, they will still get to your site through your news article on my site showing up in third place. Laughing

You have another one on the next page too Shocked

Well what a pig this is turning out to be. I have gone through the recorded 1171 hits on this site for that IP that were tracked by NS and there are quite a few hits to links that shouldn't have been there (user account links, admin links etc). I know that Google respects robots.tx so I'm at a complete loss to explain why it would be hitting some of them. Maybe NS is not tracking correctly or maybe as I originally thought, the IP was spoofed, got blocked and then when the real Google paid a visit, got the blocked page.
All I really know is I wish I hadn't turned the darn thing back on in December so now it's disabled again.

Posted: Fri Jan 06, 2012 7:08 pm

That's better now, got a nice block of 5 links on the top result Smile

Site Admin Joined: Jan 06, 2006 Posts: 311

Was thinking this thread might be useful to others in case something like this happens to them. Are you able and willing to maybe post a bit about what happened you think and what you did to correct?

Posted: Sat Jan 07, 2012 3:50 pm

I'm not entirely sure what happened Sad

Since Google obeys robots.txt and knows to stay clear of 'admin' I cannot think of how it might have crawled something that tripped NS's 'script attack' blocker.
And like a complete idiot, I cleared the block before saving off the data for further analysis though I didn't actually see anything in the referring URL or the page that was accessed that jumped out at me.

Clearing up the indexed gremlin page is pretty straightforward, I just pinged Google to tell it my sitemap had been updated to make sure it stopped by and did a re-index. Since what it had indexed before (the blocker page) had now changed (because the block was removed) it automatically got re-indexed because it was different data to what it had previously.
Not entirely sure how long it took but I think 24 to 48 hours before I noticed the change - I didn't post right away as I was a tad busy.

Site Admin Joined: Jan 06, 2006 Posts: 311

This is still useful... Wink

How did you "just ping" Google? Is this in your webmaster account for the site and there is something there to do this? Just curious...

Posted: Sun Jan 08, 2012 5:29 pm

The software I use to create the sitemaps here automatically pings Google to update the XML based sitemaps. I currently have three; one for this site, one for the Shop and another for GoogleBase, which is like a sitemap for products.

You can do it from your Google webmaster tools account
https://www.google.com/webmasters/tools/sitemap-list?hl=en&siteUrl=http://YOURDOMAIN.COM
There is an option there to ask google to manually refresh the sitemap and while it's doing that it usually crawls the site as well.
It isn't a perfect method but it is about as close as we'll probably ever get to bending Google to our will (metaphorically speaking).

Of course it would most likely update it's index any way at some point but my need in this instance was a little more urgent Smile

Site Admin Joined: Jan 06, 2006 Posts: 311

Nice!