Um
The head unit wasn't so bad. Way worse than the domestics I've worked with in the past because you have to dismount the A/C controls and remount them to a new bezel plate. And everything (I mean everything) is screwed down (unlike the domestics which just pop in and out) But overall, no biggie.
Pulling new wires was a pain in the ass. As best I can tell there is no way around this if you're like me and like audiophile quality sound- I guess you can try to use the factory wiring, but I recall seeing something about having to do some splicing somewhere else, plus you have to figure out how to tie an aftermarket head unit in with the factory amp. Where the factory amp is located, I have no idea. It is not a one piece deal with the head unit, I can tell you that.
But since you're going to just replace speakers, I guess you could just put them in and tie the factory wiring into them. Shouldn't be too bad..not sure how much power the head unit has though. Underpowering speakers is way worse than overpowering them though, so if there's not enough wattage there you could damage your new goods. The factory speaks I pulled were rated for like 20 watts or something ridiculous.
For me (a huge music person) I just couldn't deal with the factory system. Of course, I do this with every vehicle I own. I am doubtful that you could just replace speakers and get that much difference, but it's possible. I'm just thinking the factory head unit/amp will still be underpowered for what I bet you're trying to achieve. You could always get a line output convertor and put in a sub, and then just hook up aftermarket speaks to the factory wiring and leave the head unit. Might be ok.
llajumpvid and others here could probably offer more...