Intelligent agents (knowledge bots), where are they now?

We all know that the search engines use Robots to scour the web and catalogue new and exisiting pages in order to provide an index that we can traverse but whatever has happened to the personal bots that were flourishing in the late 1990's?

One of my earliest jobs in the online world was managing the data returned by a group of knowledge bots that were tasked with finding competitive intelligence for clients, these were very successful in their day and if the search query was fine tuned constantly they nearly always returned results with high relevance.

Autonomy had their personal search agent; which while masquerading as a doglike character went off to scour the web looking for relevant links and content. Hotbot (the search engine) had a service called Newsbot which scoured the news sites constantly looking for articles containing keywords you'd chosen. Both these services died a death (along with the agents I was at one time responsible for).

These days search engines are much easier to use (although I still hanker for a large text area to enter full boolean queries in a nostalgic kind of way) and it is easier to find relevant links. RSS feeds enable us to subscribe to sites of interest and news search engines allow us to set up alerts on chosen search parameters. But is this enough?

The web is huge, lets face it, the ratio of spam to good content is growing in spams favour all the time and finding everything on a very specific subject is nigh on impossible. With so many users coming online every day a lot of them really don't understand search engines or the web for that matter (and why should they, proven by all the people who type correct url's into the search box) I believe there must be a place for personal search agents once again. To really see what people are saying about your company for example, you have to visit every search engine, news sites and aggregators, blog searches, forums etc etc. A very time consuming task!

I'd use a service where I could fine tune my results, send a bot out daily to hunt for new links (using links and engine queries to get started) and only returning the new ones each day. Build all this into a really nicely developed interface (ajax if you like, imagine the sliding bars to rate results with the data then being added to the query parameters) and I reckon you'd be on to a winner. Market it in the right way, and to the right groups of people (companies needing intelligence, net newbies, information freaks like me) and I'm sure it would be a viable business model!

Is anyone doing this? I've found many services purporting to offer personalised search services, but none tick all the boxes and none offer the depth of search I'd require. Let me know if you know of anything that will fit the bill!