I use a sit-on-top kayak with an on-board bait holder, rod holder and room for a few coldies! I figured the sit-on-top (SOT) may leave you a bit more exposed to the elements, but it's way safer (and it's nice to dangle your tozzies over the side). If you do something really crazy and end up falling out, a SOT is a lot easier to get back on to, plus you don't have to bail out a kayak full of water. There's also no need to learn how to Eskimo roll - you don't need it. I've seen a few kayaks set up with on-board GPS and fish finders, so your budget will determine just how far you mod your 'yak.
They are fantastic for beetling around mangroves, estuaries, rivers, lakes etc. Some use bigger sea kayaks for open water, but many just use a SOT, providing the water's not too rough. Speaking of which, I put a couple of sink plugs in my drain holes and now stay perfectly dry unless there's a chop. In bigger waves I just remove the plugs and let the water drain out.
One of the big things to consider is how easy the damn thing is to get on to the roof of your car. Most of the "Angler" 'yaks are around 26kg, which I find a bit difficult solo. A really good investment is a trolley to get the kayak from the car to the water - lugging 26kg gets old pretty quick.
Unfortunately, my kayak is currenlty for sale. Since I smashed my arm just 3 months after I bought it, I can only paddle in circles! So, if you are after a near-new kayak with all the fruit...