SCR is certainly right that most of the standard LE stuff is available on an SE (at a price). My point is that once you pay to upgrage an SE to the things you mentioned you wanted, (2nd row captains chairs and leather seating), I would bet that you would be in the ball park of an LE. Then all of the Standard LE stuff like upgraded stereo, temp gauge, hitch, etc, comes as a bonus.
By my calculations, you are sitting at $3K below invoice for the '04 LE verses $1K below invoice for an '05 SE. Since the '04 is a more expensive LE and the '05 is a less expensive SE, I will assume that the invoices were close to begin with. So the difference betweeen deals is about $2K. If these are your only choices, I would go with the '04 LE. You get all the features you want, and you get them for $2K less. Resale value may be a little less, but $2K in your pocket now is better than looking forward to resale value. There weren't all that many problems in the '04 to begin with. Plus, you can take a little comfort in the knowledge that it is a late model '04.
If you just don't want the '04, I would suggest thinking about passing on the '05 SE and going with an '05 LE. That would be the best way to get all the features you said that you wanted (and more).
My last bit of advice is to test drive both if you can. The answer should become clear for you.