Fingering and touching penis

Fingering is not an usual way for HIV transmission.
Theoretically, this activity poses some risk, but, knowing the fact that the virus is very fragile and its survival outside the body is estimated by scientists to be just a fraction of a second, and, the fact that you haven't touched your mucous or skin (vaginal fluids need to come into contact with your blood and/or mucous membrane), the transmission of HIV is considered impossible in your cases.
Anyway, you can do a HIV test just to ease your mind.
Remember, that you, yourself carry risks for passing infection to the partner while performing this activity (infection can be caused if cuts are created by sharp or jagged nails and if the fingers and areas under the nails are not clean).