why search so slow?

[NimS]

Hiya,

Why is it that if I search for a filename in 100G, it takes up to 10s of seconds and if I search for content it can take minutes. Yet if I search on google it searches trillions of gigs in miliseconds.

Is there something I can learn and implement from what google does? Or would it ultimately just be faster to upload all my files to the net and then get google to search them in a split second?

It doesn't make sense how this works! Any clues?

[xenopeek]

What tool are you using to search file content?

Searching for file names uses an indexed database, which is fast. Searching file content, is not indexed and needs to go through each file you search entirely. Assuming you have a normal harddisk, this will be tediously slow. Google has huge data centers, each of which costs multiple hundred million USD. I guess you don't have that at home either Google also uses indexed databases (it doesn't search the entire web real-time; the Google bot travels the web and updates the database, but you only search the database). They use highly optimized databases

[NimS]

yeah, I guess my question really was about indexing.

Being an escapee from osx (not quite complete -- I still need to do video and motion graphics work on my mac) I do miss the content search the spotlight indexing provided, relatively quickly.

I was being a little tongue in cheek about google, though data recovery (or search or whatever it would be called) seems to be an increasingly integral part of the ever larger amounts of personal data. I am surprised content indexing is not becoming part of every file system.

Oh, and I'm using the search in nautilus. Is there a better, faster way?

thanks!

[xenopeek]

If you frequently are searching through large files, you may want to install a desktop search engine. One that indexes the content of the files so you can search faster.

Recoll is reportedly very good: http://www.lesbonscomptes.com/recoll/index.html

Some alternatives to Recoll: http://alternativeto.net/software/recol ... form=linux

[NimS]

cool, thanks Vincent, I'll check them out.

And thanks for your ever-present and generous advice!