What exactly is SCRAPING?

Home Forums RetroPie Project New to RetroPie? Start Here! What exactly is SCRAPING?

RetroPie has a new website and forum. Please visit https://retropie.org.uk/ for the new site. The new forum is located at https://retropie.org.uk/forum/. This forum is left here as a read-only archive.

This topic contains 4 replies, has 3 voices, and was last updated by Profile photo of sselph sselph 1 year, 12 months ago.

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • #95947
    Profile photo of mavic19
    mavic19
    Participant

    Alight, so it’s probably a noob question (definitely a noob question), but what exactly is SCRAPING and its purpose? Thanks!

    #95949
    Profile photo of herbfargus
    herbfargus
    Moderator

    Scraping is pulling metadata from a database of sorts for your games- so they can pull boxart, descriptions, publishers, dates etc. Its mostly just an aesthetic thing to make your files and games look neat and organised. There are two main scrapers that are used- the built in es-scraper which is part of emulationstation:

    And sselphs command line scraper which many prefer as it is much faster than the built in emulation station scraper.:

    But its really up to you. Some people just do without. In order to scrape games you need an active internet connection as the data comes from thegamesdb.net.

    #95951
    Profile photo of mavic19
    mavic19
    Participant

    Thanks! I appreciate your fast response and that definitely answers my question.

    One more question: I notice that, after importing my games into the folders, I’m now seeing duplicates of each game. Now, when I transferred them into each of their respective emulator folders, I just transferred the games in their .zip form. Is the duplicate because it’s now showing the unzipped executable file + the original zip file?? If so, do I first need to unzip them all on my PC and then transferred the unzipped files? Thanks!

    #95954
    Profile photo of herbfargus
    herbfargus
    Moderator

    I always unzip my files on my computer (as some emulators dont like zipped files) and transfer the unzipped contents over (.smc, .gg, etc.) The only ROMs I don’t unzip are arcade based ROMs like mame, FBA, and neogeo. See each emulator page for which file types are accepted for each emulator:

    https://github.com/retropie/RetroPie-Setup/wiki

    Likelihood is you have both the zipped files and contents of the zipped files in the ROMs folder so you see both in emulationstation. You really only need the contents. Once you save games it will create a .SRm save state file or something like that but the save state files won’t show up in emulationstation

    #96329
    Profile photo of sselph
    sselph
    Participant

    A couple things can be going on:
    If you are using .zip files the contents are extracted when you start a game and deleted when you stop(i think). If you cut power to pi that could cause there to be a rom.zip and a rom.bin.
    ES removes anything in parens from the names when displayed so ‘rom (US).bin’ and ‘rom (EU).bin’ could show up as ‘rom’

    Auto-scraper: https://github.com/sselph/scraper

Viewing 5 posts - 1 through 5 (of 5 total)

Forums are currently read only - please visit the new RetroPie forums at https://retropie.org.uk/forums/

Skip to toolbar