Why shouldn't I use Unity?
.net, ioc-container, unity-container
Solution
These containers tend to be about taste, more than anything. Ninject has a nice fluent interface for configuring it, but some people like Unity's ability to be configured through config (masochists, I think) as well as code.
I, like you, had these questions when adopting Prism, so I went through the exercise of replacing Unity with both Ninject and Autofac. I found no compelling reason in terms of performance or features to stick with either over Unity. There are certain things that were nice in both, but they all did basically the same thing and did it well.
I would look at the features, style, and syntax of each and decide which one you like best. That's really the only difference... how it feels. There are generally some core feature differences, but the likelihood of you needing these is pretty slim.
Post an update about what you decide... I'm always interested in people's real world findings vs. what passionate users are advocating.
Problem
I'm using the Unity IoC container. It really wasn't a decision I made, it just came with Prism, and I've just stuck with it. I've never used any other IoC frameworks, and I must admit I'm quite happy with Unity. However, the satisfaction may come from ignorance as I don't really know what the other frameworks have got to offer. I keep hearing that I shouldn't use the Unity IoC container. "Use Castle, nInject or StructureMap instead", people are saying, but I still haven't heard any concrete arguments or examples to why I should use a different framework. So, why shouldn't I use Unity? Or maybe I should?