• RSS
  • Twitter
  • FaceBook

Security Forums

Log in

FAQ | Search | Usergroups | Profile | Register | RSS | Posting Guidelines | Recent Posts

google spiders

Users browsing this topic:0 Security Fans, 0 Stealth Security Fans
Registered Security Fans: None
Post new topic   Reply to topic   Printer-friendly version    Networking/Security Forums Index -> Beginners // Misc. Computer Questions

View previous topic :: View next topic  
Author Message
daffyduck
Just Arrived
Just Arrived


Joined: 08 Dec 2010
Posts: 0


Offline

PostPosted: Wed Dec 08, 2010 6:41 pm    Post subject: google spiders Reply with quote

hi can anyone explain to me how google spiders work??
Back to top
View user's profile Send private message
PhiBer
SF Mod
SF Mod


Joined: 11 Mar 2003
Posts: 20
Location: Your MBR

Offline

PostPosted: Wed Dec 08, 2010 6:56 pm    Post subject: Reply with quote

Sure, someone can explain but it is easier to simply search Google for what has already been written:

How Internet Spiders Work
How Spiders Work
Back to top
View user's profile Send private message
DaveShield
Just Arrived
Just Arrived


Joined: 23 Dec 2010
Posts: 0


Offline

PostPosted: Thu Dec 23, 2010 12:44 am    Post subject: Reply with quote

Hey PhiBer - I'll take you up on that and give explanation a shot... Smile

First, GoogleBot starts by fetching a specific url. The bot examines the code of the page, paying particular attention to title, meta description, headers, body copy and ALT text. The info in each of these elements is used to determine relevancy.

Next, the Bot gathers up all the links on the page - looks for <a href="etc, etc." Those links are put into the queue so that each of those urls can, in turn, be analyzed - this is the act of "crawling." The bot does the same with each of those urls.

After the bot examines the page code, notes the keyword relevancy, puts the links on the page into queue, the page code is then put through a SPAM filter. This looks for anomalous code or any attempt to game the engine, i.e., white text on white background, for a simple example.

Once through the SPAM filter, the url/page is categorized based on keyword relevancy and placed into the Google Index - the page is now said to have been "indexed."

After a page has been indexed, it is available to show up in search engine results. Just being indexed, however is no guarantee that the page will show up at all, much less high up in the results. A site must meet relevancy criteria to be displayed. Google uses various criteria to rank sites - a completely separate issue than how the bot works.

I'm sure I'm missing some technical details, but those are the basics:

Look at the page => examine title, meta desc, header, body copy, ALT => grab/queue links => SPAM scrubber => Index.

Hope that helps!

Dave
Back to top
View user's profile Send private message
Fire Ant
Trusted SF Member
Trusted SF Member


Joined: 27 Jun 2008
Posts: 3
Location: London

Offline

PostPosted: Wed Feb 09, 2011 2:47 pm    Post subject: Reply with quote

Hi Carl,

Thanks for posting about Fetch Googlebot, not only did I learn something but I can now include it in my current research.

Cheers,

Fire Ant
Back to top
View user's profile Send private message
Allnsmth
Just Arrived
Just Arrived


Joined: 15 Feb 2011
Posts: 0


Offline

PostPosted: Mon Feb 21, 2011 11:59 am    Post subject: Reply with quote

Hi,
Google spider read meta tag from your site and index your site if you have unique and related content your keywords..!
Back to top
View user's profile Send private message
Display posts from previous:   

Post new topic   Reply to topic   Printer-friendly version    Networking/Security Forums Index -> Beginners // Misc. Computer Questions All times are GMT + 2 Hours
Page 1 of 1


 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Community Area

Log in | Register