There are many useful Firefox plugins and extensions that are compatible with the Tor Browser Bundle - how do I know which ones are safe to install without compromising my Tor anonymization?
Except for reading through all of the plugin's or the extension's source code, there is unfortunately no way to make sure it is safe.
And some believed-to-be-safe extensions like AdBlock Plus will still reduce your anonymity somewhat compared to other Tor users, by creating a more unique network fingerprint ("this user is the one that doesn't download the ads").
If you don't really really need the plugin or extension, don't install it. By installing it you take a risk. The most obvious IP leaks by proxy bypassing Tor can however be detected using network sniffers like Wireshark, if you want to inspect the plugin somewhat before using it.
Java and Flash is known/believed to be NOT safe, DO NOT install any of them (in your Tor Browser) if you need anonymity.
Some addons can be extremely helpful, f.ex. RequestPolicy. As of https://superuser.com/questions/137931/can-websites-see-my-firefox-addons, you need to distinguish plugins (Java, Flash, etc) and addons.
They can easily be detected.
Detectable using hacks.
Concerning malicious addons: The Mozilla crew is going in the direction of certifying addons.
@Rhin's answer recommended "reading through the source code".
This is, unfortunately, correct, but you might be able to narrow down the search for certain
require(...) lines which might spell problems.