As a general rule, PNG is never worse, and often better than GIF because of superior compression. There might be some edge cases where GIF is slightly better (because the PNG format may have a slightly larger overhead from metadata) but it’s really not worth the worry.
It may just be the tools I’m using, but GIF files usually seem to be a bit smaller than a comparible PNG
That may indeed be due to the encoding tool you use.
There’s nothing wrong with GIFs for images with few colours, and as you have noticed they tend to be smaller.
This is a typical encoding mistake and not inherent in the format. You can control the colour depth and make the PNG file as small. Please refer to the relevant section in the Wikipedia article.
If you need transparency and can get by with GIFs, then I’d recommend them because IE6 supports them. IE6 doesn’t do well with transparent PNGs.
That’s wrong. MSIE6 does support PNG transparency. It doesn’t support the alpha channel (without a few hacks), though but this is a different matter since GIFs don’t have it at all.
The only technical reason to use GIFs instead of PNGs is when use need animation and don’t want to rely on other formats.