I think N64 games are still very hit or miss. I can get most games to run full speed but there are random graphical and sound glitches on a lot of games. I think N64 is just hard to emulate; a better option would be to download the games using a Nintendo Wii, but then you have to pay for them.
You can try playing around with different plugins under core options, but I think the defaults are the best for the pi. Other plugins might be more accurate but not run as well on the pi.