As you said, Bouncy Castle contains many different implementations, some of which may be patented. For example, from personal use, I know Bouncy Castle includes some ECC methods and curves that could be covered by various patents.
However, since BC contains so many implementations, it's a very open-ended (and difficult to answer with certainty) question to ask which parts of it are covered by patents. The better way to go about it would be to look at each specific crypto method you want to use, and check if that is patented. If it turns out to be patented, it won't matter if you're using BC or any other library to implement it, because any unlicensed use would still be infringing.
On the other hand, if your method of interest is not patented, it might be safe to use that specific method, despite the inclusion of other potentially patented methods in the library. The theory is that since you are not using those other methods, you should not have to license them. The problem is, if you distribute the library along with a product, and that library contains patented methods that you do not use, an argument could still be made that you are distributing a product infringing patented methods, since infringement covers "using, making or selling" (and distribution falls under making or selling). I personally wouldn't buy that argument, but it has not been tested in court as far as I know, so it's a bit of a gray area.
The safe bet here would be to ensure the methods you want to use are not patented, and using a tool (such as ClassDep, maybe?) to prune out unused classes and dependencies so that your final jar does not contain any potentially infringing components. This is slightly more difficult for a library like BC because it uses a lot of reflection to instantiate the necessary classes, but it should be tractable with some trial-and-error.
On the other hand, if you are never distributing the library (e.g. using it internally on your servers), you should be fine.
