I would think the sound would probably bother the birds at least as much as the mice.
I have been using something called Mouse Magic, which are peppermint-scented sachets. Along with that, Capture traps (I take the mice Several miles away!), and steel wool in any openings I can find.
Thankfully my landlord was able to locate most likely entry points from the outside, and also blocked them and lined them with steel wool. That was disturbed in the summer when some work was being done, and another immediately appeared in my capture trap. After which landlord re-blocked those openings. That was summer of course; now we are heading into fall and we Shall See.
So anyway the main recommendations are sweep vacuum sweep vacuum -- capture traps -- and block any openings, with steel wool or copper wool if you can. (*Steel wool can be a fire hazard at high temps, due to oils from manufacturing, so do not put steel wool around any pipes or heaters etc that could get flammably hot, nor near any open flame.)