Yes, they do but until female won't lay too many eggs it's not a problem - you can throw out them and babies won't hatch. But if famale will lay too many then it may be so dangerous for her health.
Here's the thing, if eggs are laid and you do not want the babies hatch (which, honestly, you don't unless you have experience with that sort of thing--it can get super expensive and complicated and sometimes violent) then you don't just want to throw them out but you do want to prevent them from hatching. To prevent the eggs from incubating further while you wait for the female to lay the whole clutch and lose interest in the eggs, you can quickly remove the egg and boil it like you would a hard-boiled egg (careful not to crack it---if worried, you could probably even put a washcloth in the water for cushioning once the water is boiling as long as you don't scorch the fabric). Once you have boiled it, mark it with a pen (small dot)--let it cool so that it is safe to touch etc, and return it to the cage. The mark shows you which ones have already been boiled in case new ones show up--parakeets almost always lay multiple eggs over a span of days. Remove, boil, mark, cool, return. You want to try to do this quickly.
You should leave them until the female loses interest, or you can end up stimulating her to lay a new clutch if you remove them before the process is complete.
As Rozalka said, a laying female can have health issues. You want to make sure they have a cuttle-bone or something around in the event that she does start producing eggs, because if a bird lacks nutrition, the shell can either break internally, or get stuck on its way out (egg-binding). This is yet another reason why you don't want her laying eggs if you can help it.
Just an FYI-- in case you ever have a single female bird in the future -----when hormonally triggered, single females can ALSO lay eggs, but unless they have access to a mate, the eggs will be infertile. In the event that a single bird were to lay infertile eggs, you would still want to leave them until she loses interest, but the boiling part would not be necessary, as the eggs would not be viable to begin with.
