Retropie Emulationstation Xml Problem

Home Forums RetroPie Project Everything else related to the RetroPie Project Retropie Emulationstation Xml Problem

RetroPie has a new website and forum. Please visit https://retropie.org.uk/ for the new site. The new forum is located at https://retropie.org.uk/forum/. This forum is left here as a read-only archive.

This topic contains 24 replies, has 7 voices, and was last updated by Profile photo of muffy muffy 1 year, 8 months ago.

Viewing 25 posts - 1 through 25 (of 25 total)
  • Author
    Posts
  • #102456
    Profile photo of muffy
    muffy
    Participant

    Hi,

    i have a problem to use my old xml files from my Retropie 2.6 Pi1.
    I have copied the xml files and the images to my new Retropie 3 Pi2.
    Emulationstation recognize the files and shows the Covors correct.

    But if i scroll down the covers i get instant an error and Emulationstation restarts.

    If i scrap the files new by Retropie, i get no errors.
    The xml files look the same and i see there no difference.

    So how can i fix this problem?

    #102457
    Profile photo of muffy
    muffy
    Participant

    here is the error image

    Attachments:
    #102461
    Profile photo of adcooney
    adcooney
    Participant

    Post the xml files if it is a problem there.

    #102462
    Profile photo of muffy
    muffy
    Participant

    This file is for the GameGear.
    But i have this error on every other console.

    • This reply was modified 1 year, 9 months ago by Profile photo of muffy muffy.
    Attachments:
    1. gamelist.rar
    #102483
    Profile photo of sselph
    sselph
    Participant

    Seems like maybe ES isn’t able to parse some unicode that it wrote in an older version. I ran the gamelist through my scraper to sanitize it a couple different ways. First was just parsing and rewriting(-clean) and the second had all unicode removed(-nounicode). If that works, I can tell you how to use it to clean up the other ones.

    Auto-scraper: https://github.com/sselph/scraper

    Attachments:
    1. gamelist.zip
    #102516
    Profile photo of muffy
    muffy
    Participant

    Both files work and i have no error !

    #102518
    Profile photo of machtroid
    machtroid
    Participant

    Hi sselph, can you tell me how to clean the XML file up without downloading new images? I copied some images over for my SNES and updated the gamelist.xml to use these new images and now it is restarting ES randomly when I start to scroll down the list.

    #102528
    Profile photo of smithers
    smithers
    Participant

    Did you make sure the permissions were set correctly when you copied the files over so user ‘Pi’ will be able to access them?

    That could be the problem.

    #102557
    Profile photo of sselph
    sselph
    Participant

    I had to fix a minor bug for parsing gamelists made by ES so that is uploading as v0.9.5-beta but my script has a -append flag that will only add things that are missing from the gamelist if the folder you run the scraper in doesn’t have anything in it it will just read the file then write the file.

    example:

    $ cd ~/.emulationstation/gamelists/nes/
    $ scraper -append

    If you haven’t seen my scraper before it is at:
    https://github.com/sselph/scraper
    On the release page are binaries for most systems, you could do this on windows or on the rpi itself. Simple rpi instructions are here:
    https://github.com/sselph/scraper#install-from-my-binaries

    Auto-scraper: https://github.com/sselph/scraper

    #102658
    Profile photo of nolageek
    nolageek
    Participant

    Hello sselph,

    I’ve used this scraper in the past and it’s always been great.. so much faster than any of the others.. I’m redoing my retropie from scratch and I can’t seem to get the scraper to work right.

    I followed the instructions and scraper is in my home dir (~/scraper)

    I go to the gbc directory and run ‘~/scraper’

    It does its thing and I watch as it goes though – most of the games seemed to be found, there’s a few that say hash not found, etc.. seems normal.

    When it’s done i check the gameslist.xml file and it looks like this:

    <?xml version="1.0"?>
    <gameList>
            <game>
                    <path>./10-Pin Bowling (U) [C][!].zip</path>
                    <name>10-Pin Bowling</name>
                    <playcount>1</playcount>
                    <lastplayed>20150722T215309</lastplayed>
            </game>
    </gameList>
    

    That’s it.

    I run it in the mame directory and it seems to work well, except I just get this:

    <?xml version="1.0"?>
    <gameList>
            <game>
                    <path>./jungleh.zip</path>
                    <name>Jungle Hunt</name>
                    <playcount>1</playcount>
                    <lastplayed>20150715T030042</lastplayed>
            </game>
    </gameList>
    

    I believe it worked in the atari and nes directories – but those already had some games listed so I’m not sure if scraper worked or if it just didn’t overwrite the previous ones.

    #102665
    Profile photo of sselph
    sselph
    Participant

    Odd those don’t look like xml created from my scraper and since they have last played and no information I suspect they were added by ES to track that information. I just realized my append flag is going to skip those so I’ll need to add something to try and tell if it was scraped or just played.

    Auto-scraper: https://github.com/sselph/scraper

    #102666
    Profile photo of muffy
    muffy
    Participant
    
    pi@retropie ~ $ cd ~/.emulationstation/gamelists/snes/
    pi@retropie ~/.emulationstation/gamelists/snes $ scraper -append
    2015/07/24 18:48:11 INFO: Checking for new hash.csv.
    2015/07/24 18:48:12 INFO: Upgrading hash.csv:  -> "702f1da40c52243050626173416d98ae".
    2015/07/24 18:48:18 ERR: Can't open gamelist.xml, creating new file. error "XML syntax error on line 73: invalid UTF-8"
    
    • This reply was modified 1 year, 9 months ago by Profile photo of muffy muffy.
    Attachments:
    1. gamelist1.rar
    #102670
    Profile photo of nolageek
    nolageek
    Participant

    sselph – will it create the gameslist.xml in the current directory or in ~/.emulationstation/gamelists/<console name>/ ?

    #102684
    Profile photo of sselph
    sselph
    Participant

    @muffy
    Looks like that xml was edited in windows at some point which did a number on it. I manually went in and fixed all the issues I could find.

    @nolageek
    The script be default looks for an xml file in the directory where it is run. So in your case the rom folder where you ran the scraper should have an xml. This is the first place ES looks for the xml. I told muffy to cd to the gamelist folder since I knew it wouldn’t have any roms in it so the -append would just read the xml see there weren’t any roms and write the file back out to disk.

    Auto-scraper: https://github.com/sselph/scraper

    Attachments:
    1. gamelist1.zip
    #102711
    Profile photo of muffy
    muffy
    Participant

    What do you mean with “did a number on it” ?

    I tried to edit my megadrive xml on my own, but i get an arror again.
    What did you change manually on the xml file?

    I replaced the first line with “<?xml version=”1.0″ encoding=”UTF-8″?>” and changed <game> with <game id=”” source=””>.
    Do i have something else to do ?

    #102719
    Profile photo of sselph
    sselph
    Participant

    XML requires things be encoded a specific way. When you edited it in a text editor one thing is that it used “smart” quotes. Where the opening and closing quote are different(“”) vs (“”) those get encoded using a special windows encoding which isn’t valid in XML. Another is the dash (—) vs hyphen(-). You could try and turn that off and just use a straight single quote.

    The id/source things are just some extra data I store in a way that doesn’t disrupt ES.

    Auto-scraper: https://github.com/sselph/scraper

    #102784
    Profile photo of muffy
    muffy
    Participant

    I tested something around and i think changing the code to <?xml version=”1.0″ encoding=”UTF-8″?> and <game id=”” source=””> is not significant.
    I removed it from your Snes xml you send me and everything is fine and works.

    But i can’t fix the errors of my megadrive xml. I tried to change the (“) and (-) but it doesn’t work. Maybe i do somehing wrong.

    Edit: gamelist2.rar and gamelist3.rar are a wrong upload

    • This reply was modified 1 year, 9 months ago by Profile photo of muffy muffy.
    • This reply was modified 1 year, 9 months ago by Profile photo of muffy muffy.
    #103235
    Profile photo of muffy
    muffy
    Participant

    If i want to restart ES, then i get this error. All these files i have manually copied.

    Attachments:
    #103331
    Profile photo of robertybob
    robertybob
    Participant

    If i want to restart ES, then i get this error. All these files i have manually copied.

    Check the permissions for those xml files 🙂

    #103372
    Profile photo of muffy
    muffy
    Participant

    All files that work have a permission of 644. So i thought that the other files work with 644 too. Anyways i changed it to 666 an i have no error now.

    Back to my megadrive xml problem:
    I figired out, that ES always crash at this point:

    	<game>
    		<path>./Action/Blaster Master 2.SMD</path>
    		<name>Blaster Master 2</name>
    		<desc>One day, four years after Jason, Eve, and Alex’s battles with the Plutonium Boss, a bolt of lightning struck Jason’s house in the land he had migrated to four years earlier and caused the ceiling to crumble, knocking him unconscious. When Jason woke up, SOPHIA 3rd NORA MA-02, which he had hidden in an abandoned barn, was in pieces and the parts he had collected four years ago from the Plutonium Boss’ mutant Underbosses were missing. Little did Jason know that there were more Lightning Beings, the kinfolk of the Plutonium Boss, and they were planning to destroy the Earth. These beings started digging towards the Earth’s core in order to shift the planet’s weight off its axis, leading to total annihilation while they sapped the underworld of plutonium. The parts from SOPHIA 3rd were being used to create a template for the robotic army to assist the Lightning Beings in accomplishing their mission. Jason knew he had to stop them, but he also knew he wouldn’t make it without the help of SOPHIA 3rd. Since he couldn’t recover the parts, he decided to redesign & reconstruct it with (assumedly) the help of Eve to better prepare himself for the ordeal facing him. After 29 days, SOPHIA was completely rebuilt and ready for battle again, now known as SOPHIA 4th.
    
    Following the Beings' footsteps, the group jumped in their new vehicle and headed towards the (presumably) Adirondack Mountains, where their quest began anew. In the end, Jason and company triumphed over the Lightning Beings' underworld invasion, vanquishing the Lightning being known only as the "Mutant Boss." SOPHIA 3rd's parts remained missing.</desc>
    		<image>~/.emulationstation/downloaded_images/megadrive/Blaster Master 2 # SMD-image.jpg</image>
    		<releasedate>19930101T000000</releasedate>
    		<developer></developer>
    		<publisher></publisher>
    		<genre>Action</genre>
    		<players></players>
    	</game>
    

    So where is the mistake?

    #103379
    Profile photo of sselph
    sselph
    Participant

    If I had to guess, In line 4 you have several non-straight apostrophes ’ instead of ' you could replace them with &apos; to be sure that they are correct. Or you could run these commands on the command line and they will remove some of the windows encoded values from https://en.wikipedia.org/wiki/Windows-1252

    $ perl -i~ -pe "s/\x91/'/" gamelist.xml
    $ perl -i~ -pe "s/\x92/'/" gamelist.xml
    $ perl -i~ -pe 's/\x93/"/' gamelist.xml
    $ perl -i~ -pe 's/\x94/"/' gamelist.xml
    $ perl -i~ -pe "s/\x95/*/" gamelist.xml
    $ perl -i~ -pe "s/\x96/-/" gamelist.xml
    $ perl -i~ -pe "s/\x97/-/" gamelist.xml

    Auto-scraper: https://github.com/sselph/scraper

    #103381
    Profile photo of muffy
    muffy
    Participant

    I think i fixed all errors! There was one ´ that i had to change too, but now everythink is fine.

    Thank you sselph !!!

    #103409
    Profile photo of sselph
    sselph
    Participant

    Glad I could help. I’m in the process of creating a tool to make it a little easier to manually edit the gamelist files since they have this issue of being picky about allowed characters and encoding, etc. But just taking much longer than expected.

    Auto-scraper: https://github.com/sselph/scraper

    #103429
    Profile photo of muffy
    muffy
    Participant

    there are maybe some problems with : and ; too

    #103536
    Profile photo of muffy
    muffy
    Participant

    <developer>Brøderbund Software</developer>

    makes an error too

Viewing 25 posts - 1 through 25 (of 25 total)

Forums are currently read only - please visit the new RetroPie forums at https://retropie.org.uk/forums/

Skip to toolbar