API access to USPTO application

Question

I'm trying to programatically access updated application information for USPTO patents including transaction details, continuity information, and dates and codes from the Image File Wrapper (IFW) section. Here's what I've tried so far:

1. The information I'm after is available on https://portal.uspto.gov/pair/PublicPair; however, this is a CAPATCHA website and not conducive to web-scraping.
2. The information also is available for bulk download from http://patents.reedtech.com/ (which apparently is partnered with the USPTO to scrape data). Unfortunately, the data on Reed appears to often be much out of date compared to what is on the PublicPair site. This is the most hopeful path so far because it seems to show the data can be scraped somehow and it gives me everything I need, but is just out of date is many cases.
3. I've tried using the USPTO bulk download API. This has a few problems. First, many of the applications I look for are not found and return "Packaging Error." (others return a package). Even when I do get a package back, I don't get transactions, IFW, continuity, etc.
4. USPTO PAIR API - I can't get this site to work. I emailed the address at the bottom of the page and they responded: "The PAIR Bulk Data (BETA) site is being removed soon." and pointed me towards the PEDS API.
5. PEDS - This API does give transaction detail, but nothing in the IFW or continuity.
6. Bulk Data Downloads - The USPTO generates raw files in XML format. Perhaps I could download ALL of the historical data and generate my own transaction and continuity data? This seems like an overwhelming task, but perhaps is theoretically possible. Maybe this what Reed does? I haven't dug deep enough into this to know whether or not all the information I'm after is even there.

Any suggestions of where to look for this sort of information? Third party sites? Pay for services?

Answer 1

Here are two links to GitHub projects that can be used to help download and parse USPTO data.

https://github.com/USPTO/PatentPublicData

https://github.com/USPTO/bulk-data-tools

I have also built a USPTO bulk data scraping and parsing software in Python3 => https://github.com/rippledj/uspto

If you would like dedicated access to up-to-date USPTO data via remote database connection it is also available: https://www.ripplesoftware.ca/?page_id=249

Answer 2

You should take a look at patent_client! It's a python module that searches the live USPTO and EPO databases using a Django-style API. The results from any query can then be cast into pandas DataFrames or Series with a simple .to_pandas() call.

You can get the information you're looking for like this:

from patent_client import USApplicationn

USApplication.objects.get("<application number>").documents.to_pandas()

A great place to start is the User Guide Introduction

PyPI | GitHub | Docs

(Full disclosure - I'm the author and maintainer of patent_client)