1

I developed some Python and Java clients that perform the following tasks:

  • querying the EPO OPS service with a list of several queries in order to get the broadest results set for a given subject matter (it gets an average of 2000 / 4000 results for main subjects like 3d printing, etc);
  • collecting, for each of the previous results, bibliographic data, abstract, claims and description (if available) from the EPO OPS and saving them to local file system (XML files);
  • for USPTO documents in the results set locating the right zipped file from the United States Patent and Trademark Office Bulk Data Downloads, unzipping it and splitting the huge XML file to get the required document and save it to local file system (XML file);
  • extracting text from the previous saved xml files;
  • searching through all the saved documents with Apache Lucene with queries of any complexity (all the queries that can be made with Lucene operators);
  • displaying the results in html files that can be opened with a browser.

I'm considering the idea of releasing the above described pieces of software under an open source license.

Would somebody being interested in downloading and using it? If not, why? Please, any critic/suggestion is welcomed.

closed as off-topic by Matthew Haugen Feb 8 '16 at 8:16

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about patents within the scope defined in the help center." – Matthew Haugen
If this question can be reworded to fit the rules in the help center, please edit the question.

  • Since you asked, what you're doing is too simplistic IMO... As a search expert for over a decade, I know that there are companies specialized in patent search that have way better technology. The 'searching' is the difficult part, which is why these companies usually have their own search engine, specifically optimized for patent retrieval. Still, if it works for you, it might also work for other people, so there's no hurt in open sourcing it (and see what happens). – atlaste Aug 21 '15 at 5:44
  • From what I understood, patent search is pretty useless without a specialized search engine, since it's mainly a recall task while most search engines are focused on precision (it's a trade-off). As for the companies, I don't know them by heart, if you search for 'patent search SIGIR' you'll find a ton of both academic and commercial work. F.ex. I know HP/Autonomy does it, but that's also the most expensive search company out there; I'm sure there are much cheaper solutions out there. – atlaste Aug 21 '15 at 7:09
  • I've put this question on hold as it does not appear to be about patents as defined in our help center. Questions asking for brainstorming or review are opinionated and likely to produce discussion in a way that does not speak well to our format. I wish I had a better place to point this, but I don't know what websites, other than potentially some on Reddit, would be a good target for such a question. Sorry for the conclusion. – Matthew Haugen Feb 8 '16 at 8:18
1

There are a lot of misconceptions about search, so let me explain it a bit. As a practitioner in the field since 1999, having a master degree in this specific area and actually building commercial search engines for a living, I think I know what I'm talking about. I'll tell a bit about search and a bit on patent search in particular.

The main reason I write this down because I think this can help people to better understand how search engines do their job and what (and what not) to expect from them, thereby making your work wrt. patents easier.

First off, search is about humans finding information in text. Note that there are two 'unstructured' pieces of information in there: text and humans (or more precise: user intentions). Search is not something to take lightly; most universities that have Computer Science also have an 'Information retrieval' specialization, which is basically what "search" is. Moreover, there's a ton of research going on in the field of search.

Bit of history

Search originated from databases. Databases make simple 'if it matches it's a result, if it doesn't match it's not a result' decisions. However, text databases grew to be much larger than structured databases, so this quickly became a problem. To narrow down this problem, the first search engines were boolean search engines, which basically means that you can create expressions that narrow down the number of results considerably.

Around 1965 a lot of things were going on with computers. Millions of dollars were invested to make computers understand text, which led to f.ex. the Brown Corpus and most fundamental things we know about language (like Zipf curves) nowadays. Around this time, it was Karen Sparck Jones that first realized how 'term frequency' and the 'inverse document frequency' (the number of documents that contain a word) relate to how 'important' a word is given a certain a document.

This realization gave rise to a new type of search engines, called ranking search engines. Basically ranking search engines mimick the way a human deems a 'document' relevant given a 'search query'. Instead of narrowing down search results, the engine attempts to score the most relevant documents on top. Also, we nowadays work with much more sophisticated language models than the tf.idf model from Karen Sparck Jones.

Now, at this point you also have to realize that this doesn't even have to mean that a word is in the document for the document to be relevant. In fact, it's quite the opposite; for example: academic studies have shown that OR queries perform better than AND queries, especially for tasks like patent search. And if we're really academically, we can even state that all documents are in some way relevant for any given search query (most just have a very low score). Techniques like 'query expansion' literally add more information to your search query based on the internal language models.

Unstructured humans

So far I've just discussed ways how document matches can be improved by using statistics. However, I started with the notion that humans are also 'unstructured' (- or perhaps a better word would be: 'inprecise'). One thing that's particularly difficult is that humans express only a tiny bit of information of what they really mean in their search query. The rest of the information is part of the context or hidden, and therefore (both implicitly or explicitly) left unspecified.

To solve this, tools like Collaborative filtering and the Page Rank model from Larry Page uses a sort-of democratic election model for search. The idea is that if more people find something relevant, that's more likely what you're looking for. Practically all web search engines, things like Amazon recommendations, etc are based on this idea (NOTE: I'm not talking about Google Patent Search!).

Now, as you might also realize at this point, there's a huge gap here between patent search and 'normal' search: in patent search there's no 'democracy' involved, you just want to do 'known item retrieval' or finding 'relevant patents', regardless of how many times it's linked.

Relevance formalized

A lot of research is done on building proper relevance models. Because of all the uncertainties that are part of the core problem of search that I've just explained, evaluation of systems is also something that has been (and still is) thorougly researched.

I'm not going into the details here (it would simply take way too much time); if you're interested search for 'IR evaluation' or if you're specifically interested in patent search evaluation, there's the 'TREC-CHEM' task aimed at patent search in chemistry.

This is just a (very) brief explanation of the relevant info; I'm just skipping over the evaluation metrices like BPREF and MAP here...

Basically tasks are set up like this:

  • Given a large collection of documents;
  • And given a few retrieval tasks;
  • Let a room of experts decide which documents are relevant and which ones are not

Note that this is a lot (and a lot and a lot...) of work. The fact that there is a TREC on patent search also means that there's a LOT of both academic and commercial research going on on this subject. SIGIR is the place to look for academic research; as for commercial applications, I'm sure there are companies out there with some very smart guys that do this all day long.

Now that we have this 'ground truth', we can evaluate search engines by:

  • Giving them the same task;
  • Figuring out how the search results score w.r.t. the human evaluations.

Basically a search engine works better if it produces early results that match relevant documents. We call this 'precision at N' where N is f.ex. 5, 10 or 100. We also check the percentage of relevant content that we find, which is what we call 'recall'.

Now, for all search engines, precision is relevant. We always check things like precision at 5,10,100, etc (p@5, p@10, ... in papers), but specifically in the case of patent search we also care a lot about recall. After all, missing a relevant patent might result in a claim, which makes it more important than in other areas.

A trade off for performance

Search engines in closed (corporate) environments continuously struggle in a trade-off between performance of the system and relevance of the search results. Most search engines use a pretty basic statistical model for relevance, simply because they don't have the processing power to do do thorough in-depth analysis of the data. Using big volumes of data helps, because that evens out the noise you get from just having only a few documents.

On the other hand, companies like Google and Microsoft have a enormous amount of processing power and data at their disposal, and they use these to do both in-depth analysis and create better models. The trade-off probably also exists there, but at a totally different scale. I have yet to see the first open source search engine that does similar things (personally I think it's simply not going to happen, but who knows).

Feedback on our approach

First off: If it works for you, it's probably going to work for someone else. Don't let me hold you back on open sourcing it; if people pick it up, that's always nice, and since search is all about "creating something that works" -- who knows.

Then more to the subject.

I strongly believe that search is one subject that is best left to the 'real' experts. Search engines are among the most difficult pieces of technology out there, because they are designed to predict human evaluation. Slapping Lucene (or Elastic search) onto a problem will only get you so far. There's a reason universities provide a multi-year study on this very subject and I personally believe that just covers the basics.

Using search to get patent results, and then feeding that to your search engine, means you're going to ruin your language model. Lack of a proper language model means that your relevance scoring has no 'real' meaning anymore that can be reflected to human behavior. Recall how and why search engines calculate scores; what you're basically doing is throwing relevance away.

As such, the end result you describe is similar to a 'boolean search engine' that does an 'and' query between a topic (e.g. '3d printing') and combines that with the user query. A better approach would be to index everything, then do this query explicitly -- but really, the only way to know for sure how good it works, is to evaluate it.

I hope this gave you some more insights into the world of search.

  • @marcoe You misunderstood. It's not about all patent documents, it's about a substantial database, which enables you to get rid of noise. Quite frankly, your corpus is not substantial, and therefore not able to surpass boolean queries. Now, boolean queries have a lot of applications in small sets, so it might be workable - but IMO there are better solutions. As for extending Lucene - experts like me usually cram more datastructures in an index, which means 'general' frameworks like Lucene naturally won't work. – atlaste Aug 21 '15 at 22:03
  • You have to understand, Lucene works fine for TF.IDF based indexing - perhaps even with some newer models... but that's about the limit. There are plenty of ways to improve on that. If you really are into patent search (as a company), I'm pretty sure you have to build your own search engine, just to ensure you can put all the data necessary for fast and accurate retrieval in the index. I can already think of a few ways to do this, some of which might improve search results quite a bit... – atlaste Aug 21 '15 at 22:06
  • @marcoe I don't think that's fair. You asked for a general opinion, and I gave you exactly that. I even gave you some pointers where to get started. Now you're asking me to put over a decade of experience in a single answer - that's simply not possible, even if I wanted to do that. Once you do that and you have a specific (algorithm, practical or other) question, I suggest you put it on SO, and I'd be happy to help you out. – atlaste Aug 22 '15 at 12:10
-1

It is already there: http://worldwide.espacenet.com/?locale=en_EP No need to have such thing.

  • If it's all about espacenet, then how come did they implement OPS? – mrc-- Mar 14 '16 at 13:30
  • OPS is for automated queries (for software developers for example POXOQ), not for end users. For end user best ops software is ESPACENET – Rahul Patel Mar 15 '16 at 7:59
  • I was just meaning that EPO provided OPS to developers so that they could build apps for end users in alternative to ESPACENET.... incidentally like I did. – mrc-- Mar 20 '16 at 19:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.