I am not a lawyer, but here is my opinion.
If the Z component is added between A and B, does his new invention,
A, Z, B, C and D, infringe my invention?
Yes a product with Z added to your patent's claim of A, B, C and D would infringe on your patent since it implements all aspect of at least one of your claims.
For the Case 1, if Z improves (better results, easier for B,C,D
components) the invention a lot, does this new invention infringe my
Still infringes. Just because Z improves things doesn't change the fact that the product infringes if it implements every aspects of at least one of your claims.
For the Case 1, if Z doesn't improve the invention a lot, does this
new invention infringe my invention?
Just like 1b, it still infringes.
If the Z component is added before A or after D, does his new
invention infringe my invention?
Order doesn't matter if the product implements each and every aspect of at least one claim in your patent. Still infringes.
The thing to understand about patents that improve upon an existing patent is that while the improvement can be found to be worthy of a patent, that doesn't imply freedom to operate if an existing patent is infringed upon. If, however your patent's claim implements step A, B, C and D and another person figures out a way to make their product work using only steps A, B and D, then they don't infringe since they don't implement each and every aspect of your patent's claim.
Let me reiterate that I am not a lawyer and this is not a legal opinion. Even if I were, you shouldn't rely on internet Q&A sites for legal decisions. As always, you should consult with a qualified attorney.