How search over documents works, and where it breaks

Many companies now promise to "unlock your documents" with AI. If you buy software, you should know what happens behind that promise, because it decides which questions the product can answer and which ones it will get wrong. Here is a short tour. You do not need to know how to build software to follow it.

This is how search worked for decades. You type some words, and the computer looks for those letters in your files. If you type "fruit", it also finds "grapefruit", because the letters match. That is all it does.

Say you search for "sweet fruit". A file about oranges does not show up, because it never uses those two words. The computer does not know what an orange is. It only compares letters. You can loosen the rules so that any one of your words is enough, and more files appear. But the computer still understands nothing. It got lucky on letters.

Newer systems do something cleverer. Before anyone searches, the computer turns each document into a long list of numbers. The numbers describe what the document is about. Documents about similar things get similar numbers.

Now "sweet fruit" finds the file about oranges and the one about apples, even though neither one uses those words. The system does not match words at all. It matches whole documents by how close their meaning is to your question.

The old way matches letters. The new way matches ideas. That is a big improvement.

Search by meaning is still not enough for real work. Good systems also give each document two more things.

The first is tags: what kind of file it is, when it is from, and what place it is about. For example: "rent roll", "June 2025", "Florida".

The second is a short summary of what the document says.

These matter because of how AI assistants answer questions. The assistant cannot read thousands of files for every question. It has to pick the few right ones and read those. Tags and summaries are how it picks.

Picture a made-up company that owns small apartment buildings. Someone asks its assistant:

How much rent did our Florida buildings collect in June?

Behind the scenes, the assistant does something like this:

1. It searches for "rent roll Florida June 2025" and gets back five files.
2. It reads the summary of a bank email: June rent arrived on time.
3. It reads the summary of a lease: one rent goes up in June.
4. It answers: "June rent arrived on time, and one rent is going up."

Search found the files. The summaries told the assistant which ones mattered. It read a few and answered. This works well for questions like this one.

Now ask a harder question:

What was the average rent for a two-bedroom apartment in Florida in June of last year, based on all the rent rolls in our shared drive?

Search plus summaries cannot answer this well, for three reasons.

First, there is too much. The drive holds hundreds of rent rolls, and each one has hundreds of rows. Search returns the top few files that look relevant, not all of them. The word "all" in the question is where it fails.

Second, summaries throw away the numbers. A summary says "monthly totals inside". It does not keep every row of every rent roll. The assistant cannot add up rows it never saw.

Third, the assistant ends up guessing. It averages the few files it did read. That is a sample, not the real number. The answer sounds confident, and nobody can say how far off it is.

Before you buy from a company that promises to unlock your documents, ask them this:

If I gave you every rent roll from the last three years, how would your system work out the average two-bedroom rent in Florida for June, and how would you prove the number is right?

A weak answer sounds like this: "We search the documents, summarize them, and the AI does the rest." That approach misses files and drops rows. It cannot give you a number you can trust.

A strong answer shows the number coming from every row, not a sample. It also shows the work: how many rows went in, what was counted, and how you could check it yourself.

That is the whole tour. Search finds documents. Summaries describe them. Only a system that reads every row can count.