There are a lot of misconceptions about search, so let me explain it a bit. As a practitioner in the field since 1999, having a master degree in this specific area and actually building commercial search engines for a living, I think I know what I'm talking about. I'll tell a bit about search and a bit on patent search in particular.
The main reason I write this down because I think this can help people to better understand how search engines do their job and what (and what not) to expect from them, thereby making your work wrt. patents easier.
First off, search is about humans finding information in text. Note that there are two 'unstructured' pieces of information in there: text and humans (or more precise: user intentions). Search is not something to take lightly; most universities that have Computer Science also have an 'Information retrieval' specialization, which is basically what "search" is. Moreover, there's a ton of research going on in the field of search.
Bit of history
Search originated from databases. Databases make simple 'if it matches it's a result, if it doesn't match it's not a result' decisions. However, text databases grew to be much larger than structured databases, so this quickly became a problem. To narrow down this problem, the first search engines were boolean search engines, which basically means that you can create expressions that narrow down the number of results considerably.
Around 1965 a lot of things were going on with computers. Millions of dollars were invested to make computers understand text, which led to f.ex. the Brown Corpus and most fundamental things we know about language (like Zipf curves) nowadays. Around this time, it was Karen Sparck Jones that first realized how 'term frequency' and the 'inverse document frequency' (the number of documents that contain a word) relate to how 'important' a word is given a certain a document.
This realization gave rise to a new type of search engines, called ranking search engines. Basically ranking search engines mimick the way a human deems a 'document' relevant given a 'search query'. Instead of narrowing down search results, the engine attempts to score the most relevant documents on top. Also, we nowadays work with much more sophisticated language models than the tf.idf model from Karen Sparck Jones.
Now, at this point you also have to realize that this doesn't even have to mean that a word is in the document for the document to be relevant. In fact, it's quite the opposite; for example: academic studies have shown that OR queries perform better than AND queries, especially for tasks like patent search. And if we're really academically, we can even state that all documents are in some way relevant for any given search query (most just have a very low score). Techniques like 'query expansion' literally add more information to your search query based on the internal language models.
Unstructured humans
So far I've just discussed ways how document matches can be improved by using statistics. However, I started with the notion that humans are also 'unstructured' (- or perhaps a better word would be: 'inprecise'). One thing that's particularly difficult is that humans express only a tiny bit of information of what they really mean in their search query. The rest of the information is part of the context or hidden, and therefore (both implicitly or explicitly) left unspecified.
To solve this, tools like Collaborative filtering and the Page Rank model from Larry Page uses a sort-of democratic election model for search. The idea is that if more people find something relevant, that's more likely what you're looking for. Practically all web search engines, things like Amazon recommendations, etc are based on this idea (NOTE: I'm not talking about Google Patent Search!).
Now, as you might also realize at this point, there's a huge gap here between patent search and 'normal' search: in patent search there's no 'democracy' involved, you just want to do 'known item retrieval' or finding 'relevant patents', regardless of how many times it's linked.
Relevance formalized
A lot of research is done on building proper relevance models. Because of all the uncertainties that are part of the core problem of search that I've just explained, evaluation of systems is also something that has been (and still is) thorougly researched.
I'm not going into the details here (it would simply take way too much time); if you're interested search for 'IR evaluation' or if you're specifically interested in patent search evaluation, there's the 'TREC-CHEM' task aimed at patent search in chemistry.
This is just a (very) brief explanation of the relevant info; I'm just skipping over the evaluation metrices like BPREF and MAP here...
Basically tasks are set up like this:
- Given a large collection of documents;
- And given a few retrieval tasks;
- Let a room of experts decide which documents are relevant and which ones are not
Note that this is a lot (and a lot and a lot...) of work. The fact that there is a TREC on patent search also means that there's a LOT of both academic and commercial research going on on this subject. SIGIR is the place to look for academic research; as for commercial applications, I'm sure there are companies out there with some very smart guys that do this all day long.
Now that we have this 'ground truth', we can evaluate search engines by:
- Giving them the same task;
- Figuring out how the search results score w.r.t. the human evaluations.
Basically a search engine works better if it produces early results that match relevant documents. We call this 'precision at N' where N is f.ex. 5, 10 or 100. We also check the percentage of relevant content that we find, which is what we call 'recall'.
Now, for all search engines, precision is relevant. We always check things like precision at 5,10,100, etc (p@5, p@10, ... in papers), but specifically in the case of patent search we also care a lot about recall. After all, missing a relevant patent might result in a claim, which makes it more important than in other areas.
A trade off for performance
Search engines in closed (corporate) environments continuously struggle in a trade-off between performance of the system and relevance of the search results. Most search engines use a pretty basic statistical model for relevance, simply because they don't have the processing power to do do thorough in-depth analysis of the data. Using big volumes of data helps, because that evens out the noise you get from just having only a few documents.
On the other hand, companies like Google and Microsoft have a enormous amount of processing power and data at their disposal, and they use these to do both in-depth analysis and create better models. The trade-off probably also exists there, but at a totally different scale. I have yet to see the first open source search engine that does similar things (personally I think it's simply not going to happen, but who knows).
Feedback on our approach
First off: If it works for you, it's probably going to work for someone else. Don't let me hold you back on open sourcing it; if people pick it up, that's always nice, and since search is all about "creating something that works" -- who knows.
Then more to the subject.
I strongly believe that search is one subject that is best left to the 'real' experts. Search engines are among the most difficult pieces of technology out there, because they are designed to predict human evaluation. Slapping Lucene (or Elastic search) onto a problem will only get you so far. There's a reason universities provide a multi-year study on this very subject and I personally believe that just covers the basics.
Using search to get patent results, and then feeding that to your search engine, means you're going to ruin your language model. Lack of a proper language model means that your relevance scoring has no 'real' meaning anymore that can be reflected to human behavior. Recall how and why search engines calculate scores; what you're basically doing is throwing relevance away.
As such, the end result you describe is similar to a 'boolean search engine' that does an 'and' query between a topic (e.g. '3d printing') and combines that with the user query. A better approach would be to index everything, then do this query explicitly -- but really, the only way to know for sure how good it works, is to evaluate it.
I hope this gave you some more insights into the world of search.
