Hi,
Welcome to the ehealthforum and I am really glad to help you out.
Oligospermia or low sperm count can be due to multiple causes and it is broadly divided onto pre testicular, testicular and post testicular causes. We need to identify the cause in your husband's case first. It is very difficult to confirm whether oligospermia will be carried to the male offspring after IVF/ICSI because in some causes a genetic cause for oligospermia can be seen. It cannot be tested prenatally but can be done only after the birth of a male offspring.
It is very difficult to precisely confirm a diagnosis without examination and investigations and the answer is based on the medical information provided. For exact diagnosis, you are requested to consult your doctor. I sincerely hope that helps. Take care.