We do that, it’s called bombs. We bomb the radar when we can, which makes it not work. If you can get close enough to disperse something, you’re close enough to drop a bomb which has the advantage of preventing the radar from working again later.

Radar sweeps do give away their location, and radiation seeking missiles exist whose only job is to lock on to where radar is coming from and make it blow up even if the radar tries to turn off.

As someone else mentioned chaff can also be effective but it’s mostly used against missiles with little radars in the nose, to confuse that little radar and make the missile miss.

What IS sometimes done is an electronic warfare plane will fly off to where the strike team isn’t and go make a lot of radio noise, so when the strike team comes along the radar is busy looking in the wrong spot. This is how Ukraine sank the Moskva https://en.wikipedia.org/wiki/Sinking_of_the_Moskva , a decoy made noise over at 6 o clock while cruise missiles came in from 9 o clock. Absolutely textbook maneuver!