I've always used humane traps and released the mice into the culvert on the corner of our street.
There are many myths around about mice, but no, if you transport them more than a house block away, there's no reason to think they will 'find their way back'. Think about it: the animal is less than two inches long and can see only six inches in front of itself! Unless you leave a scent trail for it, there's no way!
Also, while mice breed very prolifically (a nursing mother can fall pregnant right away and have another litterof fifteen or twenty kittens within three weeks), they don't breed automatically and at 'top volume' as it were. They need to make a colony first and nests. If you clean scrupulously and disrupt wherever it is they're nesting, they'll move along.
The very best thing to keep mice away is to have a large snake living under your house. We had a red bellied black snake called Snidely who kept our bird area nicely clean of mice for over five years (did a darned better job than the lazy cats!). When Snidely moved on, the mice moved back. It was tiresome, but trapping fixed the problem.
Just FYI, an easy cheap humane trap can be made from a large (two litre) plastic bottle with a handle (Australian milk bottle). You just smear the neck *very* thinly with peanut butter, put a peanut or breadcrumbs inside and run a long string from the handle or neck of the bottle up through a bracket or hook (so that when you pull the string, the bottle 'jumps' up to the vertical immediately) - and wait.
I've caught literally hundreds of mice this way. It's pretty hilarious if you enjoy watching animals do what they do. I remember being woken by squeals during a mouse plague and sat up in bed to see six or eight minute mouse kittens actually playing in the middle of the room. They were chasing each other around and jumping about to avoid being caught, squealing with delight. I can't bring myself to kill such charming little creatures, hence the humane traps. It's possible to get funnel-mouthed traps too. They're useful if you have neither the time nor the leisure to wait around for a mouse to walk into your milk bottle (which they do with alacrity, only it does take time). The funnel-mouthed trap will contain quite a few wild mice (they're much, *much* smaller than the fancy mice people keep as pets), but it's pretty inhumane to leave a crowded trap with no food or water for very long. Always check it every few hours and empty it in a drain or near a creek.
PS. I have actually tested the 'finding their way back' theory. My brother-in-law reckoned my humanely caught-and-released mice were simply recaptures because I'd caught so many (eighty-three during the last plague). So, I carefully marked the mice with hot pink nail polish before releasing them. Didn't have a single recapture! No doubt some amazed householders ultimately caught some of the decorated mice and wondered how on earth they managed to acquire a pink dot on the backs of their necks!
PPS. Mice aren't a problem for us any more, since our 'new' house is built on tall foundations (last one was at ground level, which made for many interesting visitors from funnel-web spiders to a blue-tongued lizard). What *is* a huge problem is cockroaches!!! Anyone got any failsafe ideas for those? And no, I'm not into humane trapping in this instance. I just don't want to poison my birds!!!