linuxtv.org Wiki pages not found by Google

linuxtv.org Wiki pages not found by Google

[Jan-Pascal van Best]

Hi all,

Not sure if this is the right list, but here it goes: Google does not seem
to find pages in the linuxtv.org wiki. Nor does any of the other search
engines I've tried. It seems indexers are blocker by robots.txt:

User-agent: *
Disallow: /cgi-bin/
Disallow: /hg/
Disallow: /git/
Disallow: /wiki/
Disallow: /irc/
Disallow: /irc/linuxtv/
Disallow: /irc/v4l/

I would think information in the wiki (such as on a DVB-C USB box I'm
researching,
http://www.linuxtv.org/wiki/index.php/TechniSat_CableStar_Combo_HD_CI )
should be findable by search engines!

Could the "Disallow: /wiki/" line please be removed from robots.txt?

Cheers

Jan-Pascal

Re: linuxtv.org Wiki pages not found by Google

[Samuel Rakitničan]

On Mon, 07 Jun 2010 09:07:24 +0200, Jan-Pascal van Best  
<janpascal@vanbest.org> wrote:

> Hi all,
>
> Not sure if this is the right list, but here it goes: Google does not  
> seem
> to find pages in the linuxtv.org wiki. Nor does any of the other search

Hi,

I'm not an expert, but I read somewhere that's happening because there's  
to many subbranches. If there's more than tree search engines will not  
look for pages behind it.

I know one site that dealed with it by removing index.php from the wiki  
path and that way allowing search engine to find the pages.


Best regards,
Samuel