HOWTO: Use datfiles to eliminate clones, etc, from Mame/FBA/Neogeo

Home Forums RetroPie Project Everything else related to the RetroPie Project HOWTO: Use datfiles to eliminate clones, etc, from Mame/FBA/Neogeo

RetroPie has a new website and forum. Please visit https://retropie.org.uk/ for the new site. The new forum is located at https://retropie.org.uk/forum/. This forum is left here as a read-only archive.

This topic contains 16 replies, has 6 voices, and was last updated by Profile photo of Floob Floob 1 year, 6 months ago.

Viewing 17 posts - 1 through 17 (of 17 total)
  • Author
    Posts
  • #101538
    Profile photo of dankcushions
    dankcushions
    Participant

    I’ve been using ROMLister to create some dat files to clean up my roms, so I thought these might be useful to people wanting to do the same.

    STEP 1:
    Download the right romset for the emulator you want to use. For example, the default (?) FBA emulator in RetroPie is FBA-libretro, and according to the wiki, that uses the FBA romset v0.2.97.30. Go to google and find that romset.

    STEP 2:
    Download clrmamepro: http://mamedev.emulab.it/clrmamepro/download.htm. This is a tool used to check and refactor romsets. We’ll need it later.

    STEP 3:
    dat files are used by clrmamepro to retrieve a subset of files from within your set. they contain various metadata, including precize file sizes, to do this. if you got the v0.2.97.30 dat file, and use that to rebuild your v0.2.97.30 romset, you should end up with the exact same set. However you can use refined dat files to get rid of all the clones:

    FBA:

    NEOGEO:

      NOTE: for neogeo rebuilds, ensure the ‘separate bios sets’ advanced option is ENABLED, otherwise it will try and add the contents of neogeo.zip to every single rom zip, which increases size and means that hash-based scanning won’t work.

    • Emulator: fba-lr
    • Romset: FBA 0.2.97.30
    • Datfile: http://s000.tinyupload.com/index.php?file_id=05558959831361649927
    • Description: neogeo ONLY, no homebrew/demos/etc

    MAME:

    (to be continued! anyone got any more?)

    STEP 4:
    Use clrmamepro to rebuild the downloaded romset using the datfile above. There’s a guide: https://github.com/RetroPie/RetroPie-Setup/wiki/Managing-ROMs – we’re only interested in steps 4 & 5.

    STEP 5:
    Upload your new set to your pi, making sure to change the default emulator for each system affected to the one in table above.

    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions.
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions.
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions.
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions. Reason: gngeo added
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions.
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions. Reason: pifba added
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions.
    • This topic was modified 1 year, 9 months ago by Profile photo of dankcushions dankcushions. Reason: added mame4all/imame4all-libretro

    #101552
    Profile photo of Floob
    Floob
    Moderator

    This looks really useful thanks.
    Can you let us know what your criteria was when refining these in Romlister so we know whats been removed?

    RetroPie help guides --> https://goo.gl/Yfy8kj
    Please read this before asking for help --> http://goo.gl/eLErnl

    #101555
    Profile photo of dankcushions
    dankcushions
    Participant

    This looks really useful thanks.
    Can you let us know what your criteria was when refining these in Romlister so we know whats been removed?

    sure – here’s the two searches used:

    for all fba-lr, minus neogeo and clones:
    !'neogeo' & (!cloneof & !sample_name= & !emulation=preliminary)

    for all fba-lr neogeo minus homebrew and clones:
    'neogeo' & !'<manufacturer>homebrew</manufacturer>' & !'<year>????</year>' & (!cloneof & !sample_name= & !emulation=preliminary)
    (i also then had to manually delete some homebrew/prototypes from romlister, and also amend the .xml file to include the neogeo.zip bios file, as it seems to want to remove that, regardless of the search string)

    i’d like to eventually have .xml files for fba-lr and mame, which don’t cross over, as right now both systems have quite a bit of overlap. personal preference, but i feel like fba is for post-1990 releases, and mame for everything before.

    #101556
    Profile photo of windale
    windale
    Participant

    So do all Parent games work in lr-fba, otherwise this would be a pointless task. You can also filter clones and games that use Bios files etc. using ClrMAME without having to use Romlister (in the set information window settings).

    #101557
    Profile photo of dankcushions
    dankcushions
    Participant

    So do all Parent games work in lr-fba, otherwise this would be a pointless task. You can also filter clones and games that use Bios files etc. using ClrMAME without having to use Romlister (in the set information window settings).

    i think there might be a few rare games where the parent doesn’t work, and the clone does, but personally i’d sooner fix those as-and-when i find them, rather than deal with 100s of clones in my game lists! especially via emulationstation where it’s not always obvious what version you’re looking at.

    it might be possible to do this in clrmamepro but with the dat it’s preserved for future rebuilds, rather than having to remember and enter settings in clrmamepro.

    #101558
    Profile photo of Floob
    Floob
    Moderator

    Are you tempted to rip out the mahjong and pinball, lightgun games in Romlister as well?

    RetroPie help guides --> https://goo.gl/Yfy8kj
    Please read this before asking for help --> http://goo.gl/eLErnl

    #101561
    Profile photo of dankcushions
    dankcushions
    Participant

    Are you tempted to rip out the mahjong and pinball, lightgun games in Romlister as well?

    for sure! i suppose it gets into personal preference territory, so we could have a few datfile variants for each system. i would definitely be interested in making datfiles to strip out the following:
    – mahjong
    – porn games (although these are sort of hilarious)
    – hacks
    – quiz games
    – lightgun/trackball/etc stuff that is unplayable with a controller/stick

    all that should be possible in romlister

    #101587
    Profile photo of thatguyinfl
    thatguyinfl
    Participant

    Thanks for this.

    #101709
    Profile photo of dankcushions
    dankcushions
    Participant

    added gngeo (thanks to ekstreme)

    #101717
    Profile photo of dankcushions
    dankcushions
    Participant

    Added pifba (thanks to ekstreme)

    #101865
    Profile photo of dankcushions
    dankcushions
    Participant

    added no clone, no neogeo dat files for both mame4all cores. halves the game list for those!

    also added to the wiki https://github.com/RetroPie/RetroPie-Setup/wiki/Managing-ROMs

    still not quite sure how to split the fba and mame lists so they don’t have a crossover. I’m not sure there’s a smart way to do it.

    #102749
    Profile photo of momaw27
    momaw27
    Participant

    Thank you VERY MUCH for these! I appreciate all your hard work!

    #106930
    Profile photo of dankcushions
    dankcushions
    Participant

    i can’t seem to update my old post 🙁 but there’s a new one (added to the wiki)

    MAME:

    Emulator: lr-Mame2003
    Romset: MAME 0.78
    Datfile: http://s000.tinyupload.com/index.php?file_id=04618720736720226948
    Description: working, no clones
    No. Roms: 2256
    RomLister search used: (!cloneof & !sample_name= & !emulation=preliminary) & !(isbios=yes)

    Datfile: http://s000.tinyupload.com/index.php?file_id=05598248539957382925
    Description: ‘lite’ set: working, no clones, neogeo, PlayChoice (NES multiplay), no rotary/dial/trackball/lightgun controls, no casino/multiplay/quiz/mahjong/fruit_machines/rhythm/adult
    No. Roms: 1615
    RomLister search used: !(‘PlayChoice’ | ‘neogeo’) & !(rotary | dial | trackball | lightgun) & (!cloneof & !sample_name= & !emulation=preliminary) & !(isbios=yes | casino | multiplay | quiz | tabletop | Fruit_Machines | Rhythm | Mature)

    i’ve updated the wiki: https://github.com/RetroPie/RetroPie-Setup/wiki/Managing-ROMs

    • This reply was modified 1 year, 6 months ago by Profile photo of dankcushions dankcushions. Reason: corrected file - please re-download!
    • This reply was modified 1 year, 6 months ago by Profile photo of dankcushions dankcushions.
    #106933
    Profile photo of herbfargus
    herbfargus
    Moderator

    Thanks! Parents only Dats I think should be the default as most people probably dont care to slog through all the clones. I’ve been keeping a mirror github repo with all the dats here as well:

    https://github.com/HerbFargus/retropie-dat

    #107231
    Profile photo of dankcushions
    dankcushions
    Participant

    whoops, i messed up the mame2003 datfile! i think i buggered up the romlister stage, and it wasn’t the right parent datfile at all. i wondered why i was getting so many missing-file errors…

    i’ve uploaded and corrected the links to: http://s000.tinyupload.com/index.php?file_id=67593192132500562498

    @herbfargus – you’ll be wanting to re-download and correct your repo also, please 🙂

    sorry all!

    #107589
    Profile photo of dankcushions
    dankcushions
    Participant

    i’ve updated my previous post with just a ‘working’ mame2003 dat, with only working roms. i’ve also done a ‘lite’ set that strips out all non-working games, and clones, bad categories, etc. this is thread is getting messy so i’ll maybe re-do the whole thing once i’m done with the other emulators.

    #107623
    Profile photo of Floob
    Floob
    Moderator

    Thanks for all the work you put into this. Must be very time consuming!

    RetroPie help guides --> https://goo.gl/Yfy8kj
    Please read this before asking for help --> http://goo.gl/eLErnl

Viewing 17 posts - 1 through 17 (of 17 total)

Forums are currently read only - please visit the new RetroPie forums at https://retropie.org.uk/forums/

Skip to toolbar