I'm hoping that some good websites come up for finding a good caregiver without having to go through an agency. I've heard about this one site for finding caregivers, but it doesn't seem to be active just yet. Maybe it will go into effect soon and it will help?
If not, or if you simply need to find someone sooner rather than later, an agency might be the way to go.
Good luck!