Theoretically this could work, provided that you only use 2 types of controllers.
You’ll have to redo the ESconfig, with both controllers plugged in at the same time. Set the one up as Player 1 and the other up as Player 2, that way both controllers would be able to navigate through the ES Menus. Unless ESconfig takes controller ID’s into account (which it might) it should work. If not, you’d have to have both controllers plugged in at all times for this to work.
From then on it’s pretty straightforward. You’ll just need to configure a retroarch.cfg file for each emulator.
Alternative, you might be interested in the auto configure functionality of RetroArch: essentially, it changes the input settings depending on the controllers that are currently plugged in and supports hotplugging. You can find more information about that at