Is there any way to read the proceedings of a patent prosecution or a re-examination? Public would be great of course, but so far I have not found even a fee-based resource so I wonder if proceedings are confidential. (If so, why?)
Prosecution and re-exam proceedings (often referred to as "file wrappers") are selectively available on Public Pair. There you can search by patent number (e.g., 1,234,567) or publication number (e.g., 20130143080). If the file is available online you'll be able to view the transaction history (e.g, the titles of documents sent back and forth). If the file is newer you'll be able to view the Image File Wrapper (PDFs of all correspondence except for non-patent prior art cited).
Some patent applications can be held in confidence and not open to inspection. See MPEP §103. The MPEP describes in detail all of the situations where a file wrapper may not be available, but in general, many recent patents/applications are available for free online. Some common situations where you won't be able to view the documents online are if the patent/application is older (usually patent numbers lower than 6,400,000 do not have the full images), the patent is new (is not published yet), or the patent is pending and subject to a secrecy request (not that common, but occurs enough to warrant mention).
Note that this only discusses methods to get file wrappers for free online. You can always travel to the patent office archives, although I've not met anyone who's done that.
There are also some paid services that will get file wrappers that are not available online. I can't speak to the exact cost, but some are rather costly and some are based on the number of pages (which can easily run past 500):