At least for now...
if so, would you recommend it? what are the downsides of it? (other than missing a big market)
(Note: seems like bwin.com chose this way (3rd column). It is a huge poker and betting site here in Europe.)
Patents are per territory. If something is excluded from exercise only by a U.S. patent (that has been granted and did not yet expire), then in general you are not in violation of the patent if you do not sell it in the U.S. nor do you manufacture it in the U.S.
Disclaimer: I'm not sure if you would be in some kind of retroactive violation or not, in case it is later extended to the territory where you sell or produce/assemble it. But if it's later extended to those territories you are certainly in violation if you continue after that.
Please note I assume you wrote 'Americans' without giving it much thought, but it's not the nationality of a person that matters to the aspect of the right here, but where the product is sold, used, or manufactured/assembled.
As a final note, practically speaking, you should want to check whether there are international patents or patents of other states linked in patent databases as the equivalents of the U.S. patent in question, before assuming the invention is not patent protected in any given territory. And maybe consider whether what you're doing at all feels ethical or not because the motivation of the question is not very clear...