I actually gave up on this project because I couldn’t find a reliable way of getting it to work with low latency input. The best I was able to get was by using dispmanx as the display driver and linuxraw instead of udev for joystick inputs. The problem with linuxraw is…..no Xbox 360 pad support.
I also used the realtime kernel, but beyond that I didn’t really do anything too fancy. Keep in mind in that video I am running fba libretro running Street Fighter Alpha 2 test mode. Games you test may or may not have inherent latency to them.
If you are looking for a pocket sized computer to bring around with you for fighting games like I am. You might want to look into the UP Board running Windows: