ES Scraper Setup

RetroPie has a new website and forum. Please visit https://retropie.org.uk/ for the new site. The new forum is located at https://retropie.org.uk/forum/. This forum is left here as a read-only archive.

This topic contains 6 replies, has 3 voices, and was last updated by Profile photo of Floob Floob 2 years, 7 months ago.

Viewing 7 posts - 1 through 7 (of 7 total)
  • Author
    Posts
  • #30554

    Arnaud

    Hello,

    I have tons of roms and running the scraper from the emulation station seems to make the system crash (memory problems ?). I am forced to rerun the process many times and it takes forever. I’ve heard there is a way to run this from the command line ? Do you think that may help ? Or am I facing a memory problem if I do so also ?

    I am running the latest distro of Retro Pie but cannot find the ES Scraper from the Script Menu. Is there something to do to make it available ?

    Thanks a lot !

    #31656

    JayBi

    Same problem here. The scraper in the ES is working for a while, but more than 500 games and the ES GUI became extremely laggy and it freeze from time to time.
    I installed the PIL stuff (for resizing problem) and also searched for the solution described here
    but i didn’t manage to find the scraper.py. It’s not in /home/pi/RetroPie/supplementary/ES-scraper/scraper.py anymore.
    Any idea?

    #31855
    Profile photo of sfalexi
    sfalexi
    Participant

    If you’re using 2.x, a lot of the folders have moved from the HOME folder to the root folder and then /opt

    This is what I’m doing.

    Until I’m fully setup, I put the config file on the SD Boot in hdmi_safe=1

    This puts it (I believe) in 480i. I figure this should tax raspberry pi the least and give it its best shot to run for a while without crashing.

    I run the scraper one system at a time. So for SNES, I run it. I disable the user verification. And I have it plugged into an ethernet cable.

    I have the same problem, about 150 – 250 games it’ll crash or give an error about not enough memory. But you know what? Run it overnight. Wake up, restart it, and it’ll pick up where it left off and run it as you go off to work. Come back home, crashed again, but now it’s 400 games in. Run it again. It’ll crash or pause again, but as you go to bed and run it again, it’ll finish. 1000ish games in a day and a half.

    There will be missing games and errors cause it assumes the first hit is the right now. Go to the home folder and there should be a .emulationstation folder or something like that. There’ll be a gameslist folder, then my SNES folder, and then a gameslist.xml. Browse it and delete any errors. I just checked the game file name and checked what the new emulationstation title is. If they don’t match, delete all that data starting with <game> and ending with </game>

    At that point, I rerun the scraper, but make sure I try to verify each game. 70 games left, after manually scraping I can cut that down to about 35. And now the fun begins where I try to change the filenames to something that can be accurately scraped.

    Two days and I’m down to 35 games? Heck! I’m happy!

    #32100

    Jaybi

    Thanks,
    I’ve checked for scraper.py in all the possible directories also using
    sudo find * | grep scraper
    but i didn’t find it.
    Even when i manage to scrape (within the ES interface) more than 500 games, the system become laggy, not responsive and it crashes from time to time. This is, I guess, due to the size of the pictures it has to deal with.
    I would try to solve it resizing the pictures, but so far, i didn’t manage to find the proper way to resise them.

    #32686

    Arnaud

    I think it can be downloaded

    #32866
    Profile photo of boelze
    boelze
    Participant

    I have had the same problem. Here is what I did to solve it:
    I installed the old Version of RetroPie 1.1 where the scaper can still be accessed through the setup script, copied over my roms and ran the scraper (took around 5 hours for over 1,5k roms). Then I backupped the pictures and .xml files on my laptop, installed the new RetroPie image and copied them over again.
    Works like a charm. I don’t understand why the scraper was removed from the setup script though…

    If anyone can explain to me how to start the scraping script from console on the new RetroPie image I would highly appreciate it!

    #72062
    Profile photo of Floob
    Floob
    Moderator

    Looks like command line can be run from this approach?

    Section 5.3
    https://github.com/retropie/RetroPie-Setup/wiki/RetroPie-Arch-Linux-Flavor

    Section gamelist.xml
    https://github.com/Aloshi/EmulationStation

    RetroPie help guides --> https://goo.gl/Yfy8kj
    Please read this before asking for help --> http://goo.gl/eLErnl

Viewing 7 posts - 1 through 7 (of 7 total)

Forums are currently read only - please visit the new RetroPie forums at https://retropie.org.uk/forums/

Skip to toolbar