I had the same problem yesterday and i found a solution.
I’m not sure if there is a better one though since i’m kinda new to both linux and the raspberry pi.
Go into /opt/retropie/configs/nes
open the retroarch.cfg and clean the “# All settings made here will override the global settings for the current emulator core
” lines out, instead add this line : joypad_autoconfig_dir = /opt/retropie/emulators/RetroArch/configs/nes/ and save the file.
then go into: /opt/retropie/emulators/RetroArch/configs/ and make a folder called nes.
go back to /opt/retropie/emulators/RetroArch/configs/
in that folder there should be an xbox cfg file for your controller (not completely sure about the name since i use the ps3 controller instead).
copy that one into the nes folder you just made.
Open the nes folder and edit the .cfg file and change the buttons like you want them to be for nes.
save the file and launch.