Scrape without ES-Scrapper

This topic contains 30 replies, has 9 voices, and was last updated by Profile photo of epicrean epicrean 2 years, 9 months ago.

Viewing 31 posts - 1 through 31 (of 31 total)
  • Author
    Posts
  • #3103
    Profile photo of initmaster
    initmaster
    Participant

    Hi,

    I want to scrape my game on my PC, before send them to my RaspberryPi.

    Is there any soft for that ?

    Thanks in advance

    #3165
    Profile photo of lakeuk
    lakeuk
    Participant

    I’ve put together a little form app to help clear up the gamelist.xml file, it’s not a screen scrapper but helps clean up the lose ends that the ES Script hasn’t spotted or miss identified.

    Recommended steps:-
    Load roms on to RetroPie and run the ES-Scrapper script
    copy gamelist.xml and images for a console from Pi to windows PC
    launch RetroPie.exe in same directory on PC as gameslist.xml
    click ‘Get gamelist’
    scroll through list of rom titles, on right amend any titles, descriptions, image location.
    click Update to write back to gamelist.xml
    if image isn’t shown or is incorrect, use search box to search through browser, same images manually and reselect rom name to see it on the form
    once happy copy the gamelist.xml and images back to the Pi

    Image changes on the Pi should show but text changes will show after a reboot

    #3166
    Profile photo of lakeuk
    lakeuk
    Participant

    My noddy app for window pc –

    #3167
    Profile photo of lakeuk
    lakeuk
    Participant

    Not having much look with linking to my app, if it can’t be seen on this thread then anyone interested message me a contact email, happy to forward link

    #3170
    Profile photo of zoot
    zoot
    Participant

    Could you put it on github?

    #4586
    Profile photo of lakeuk
    lakeuk
    Participant

    It’s available on my link below, if it doesn’t launch then you may need a more recent version of the .net framework.

    Launch in windows folder with your gamelist.xml file

    #4865
    Profile photo of karloss
    karloss
    Participant

    can this be re-posted somewhere so people can download please?

    #4866
    Profile photo of crystalj
    crystalj
    Participant

    I am very interested in this as well, wonder why the links aren’t showing up for you?

    #4870
    Profile photo of cacophony555
    cacophony555
    Participant

    I ended up writing my own simple (~30 line) scripts.

    For MAME and FBA roms it utilizes local text files that have associations between the rom names and their actual names (ghouls -> Ghouls and Ghosts). It will create the gamelist.xml (just like ES-scraper), but does it entirely locally, so it only takes a couple seconds to process a couple thousand roms. It will even make image references in the xml file if you have png files in the rom directory with the same names as your roms.

    There are much faster ways to automatically fetch appropriately sized and named rom png files. I used the EmuMovies automated downloader, which is a Windows app. To grab the images you just point the program at your roms directory and it will take care of everything in a couple minutes. Then they just needed to be transferred to the appropriate rom directory on the pi.

    I also have a similar script for console games, but it doesn’t have the name association file so the roms need to be named in the same way you want them displayed. There are already lots of solutions for renaming rom files.

    The scripts don’t populate descriptions, but honestly I never felt the ES-scraper descriptions were of any real benefit.

    If there’s lots of interest I can post the scripts somewhere, but I’d want to clean them up and do some consolidation before they are shared.

    #4876
    Profile photo of lakeuk
    lakeuk
    Participant

    Plan Z 😉 I put the details in a Blog – RetroPie GameList Updater

    #4894
    Profile photo of karloss
    karloss
    Participant

    Thanks for re-posting mate 🙂

    #4903
    Profile photo of crystalj
    crystalj
    Participant

    Very cool! Great work.

    #4930
    Profile photo of karloss
    karloss
    Participant

    cacophony555 do you fancy sharing you console roms script?

    Thanks

    Karl

    #4943
    Profile photo of retrofredde
    retrofredde
    Participant

    It’s available on my link below, if it doesn’t launch then you may need a more recent version of the .net framework.

    Launch in windows folder with your gamelist.xml file

    Thank you! 😀 This is a fantastic little program! I managed to do a complete gamelist with images and descriptions in about half an hour, which can take almost a day doing it manually.

    #5328
    Profile photo of initmaster
    initmaster
    Participant

    Hi,

    Finally i made a little program to manage my roms. (inspired by “Game List Updater” by lakeuk)

    You can download it here : http://www.initial-team.com/initmaster/files/ES-Manager.rar

    You can put your own configuration in this file : ES-Manager.exe.config
    For example for SNES

    <add key="snes_name" value="SNES"/>
    <add key="snes_pc_path" value="P:\RetroPie\Roms\snes"/>
    <add key="snes_pi_path" value="/home/pi/RetroPie/roms/snes/"/>
    <add key="snes_cover_width" value="825"/>
    <add key="snes_cover_height" value="580"/>
    <add key="snes_cover_width_alternate" value="580"/>
    <add key="snes_cover_height_alternate" value="825"/>
    <add key="snes_rom_extension" value="smc,sfc,fig,swc"/>
    <add key="snes_gamesdb_machine_name" value="Super Nintendo (SNES)"/>

    If you don’t want a repertory to be scrapped, leave the “pc_path” parameter blank

    #5628
    Profile photo of initmaster
    initmaster
    Participant

    I did a small update on ES-Manager. (download link is still the same)

    – Better rom name cleaning
    – Possibility to do direct copy to pi smb share
    – Correct covers sizes for nes/snes/md/sms in the config file

    You can check for updates on my blog

    Attachments:
    #8254
    Profile photo of leosmeira
    Leonardo

    New Resources Required:
    – Download images without resizing.

    The problem is that the image is stretched or flattened when it resizes

    thanks for sharing! 🙂

    #9044
    Profile photo of zsprawl
    zSprawl

    I can’t seem to get it to run in Windows 8, it just crashes without error. I’ve tried all kinds of compatibility settings.

    #9078
    Profile photo of initmaster
    initmaster
    Participant

    It works on Windows 8 32 & 64.

    Be sure that the paths in the .config file are corrects (that’s usually the reason of crash).

    I hope to find time to make a new release before july.

    #9232
    Profile photo of zsprawl
    zSprawl

    ahh had one of the local paths wrong. Thanks

    #9234
    Profile photo of zsprawl
    zSprawl

    There is a bug with SNES default width. If I change it in the config file to 541 as the default, it won’t take. I need to change it manually each time in the gui.

    Also, is there a way to get it to do a full scrap first, then I can go in and micro manage?

    Love the tool. Thanks !

    #11736
    Profile photo of leosmeira
    leonardo

    how to set not to resize the image after download from google?

    #11950
    Profile photo of zsprawl
    zSprawl

    You need to edit the config file for your size options. Sadly it defaults to the first option everytime, and a bug appears to require it to be the default.

    Using this tool is very tedious and time consuming, but you get perfect results in the end. Using the built in scrapper is easy, but often has problems parsing special characters, thus mislabeling a game.

    #17122
    Profile photo of deaftolight
    deaftolight
    Participant

    ES-manager is a great tool! Is there a way to select all and let it scrape all the roms, or do you have to be there and do it one by one?

    #17237

    Haris

    Exe file doesnt work. When i run it it dows nothing. Do i miss smth? should i have opened my pi?

    #17269
    Profile photo of lakeuk
    lakeuk
    Participant

    You need to alter the directory locations in the ES Manager config file.

    Also if you using Retropie v2 then mine mini-app and the ES Manager won’t do the job as the update has moved the romart / gamelist.xml to different folders

    home\pi\.emulationstation\downloaded_images
    home\pi\.emulationstation\gamelists

    #17274
    Profile photo of lakeuk
    lakeuk
    Participant

    deaftolight – these tools were cobbled together help manually fill in the gaps and correct wrong info that the retropie main scrapper has picked up.

    In the Retropie v2 the scraper has moved to the main emulationstation GUI where you can do a scrape of everything.

    #17279

    Haris

    Lakeuk i agree but:

    1-internal es scraper canot find many rom’s thumbnails

    2-you have to do it one by one!! And all of us do have thousands of roms…

    so i guess that s not a solution:(

    #17373
    Profile photo of lakeuk
    lakeuk
    Participant

    I’ve updated my cleanup app to fit with the changes made under 2.X in directory locations and also added a few more sources for hunting out FanArt, new version can be found on my blog post and the link below:-

    Hint Hint – I like ininmaster’s app for the additions mine doesn’t have, hopefully he’ll update it in the near future 🙂

    #17374
    Profile photo of lakeuk
    lakeuk
    Participant

    Looks like is won’t post my link, you can find it in the previous post #4876 above

    #59760
    Profile photo of epicrean
    epicrean
    Participant

    Has a solution come up? If I use the ES scrapper it does a poor job. The only solution is going one game at a time, and honestly that doesn’t seem like a solution at all

    Epic

Viewing 31 posts - 1 through 31 (of 31 total)

Forums are currently read only - please visit the new RetroPie forums at https://retropie.org.uk/forums/

Skip to toolbar