Taking a break from the game, this blog is currently on hiatus. 💤

GE HTML Offline Database

Repost of GE HTML database (offline, self-parsing) 🔗 by itsyous:

Long time passed since i released 1st version of my kit, but that`s a second one. Right now it`s parsing all major information about characters, stances, skills. That information is then store as one .HTML file, kinda big one (~1 MB for English, ~1.5 for Russian (coming soon)). Images is stored separately (storing the images in base64 would make the database 20+ mb in volume (maybe a bit less if images is converted into jpeg, but not so big difference).

It`s basically an alternative to Egado`s Excel database (which i really respect, actually half of data structures was made by looking on Egado`s way of presenting info). This theme will be updated from time to time to publish new versions of the kit. I will not publish new database after every update, only major updates and new char releases.

Differently from Egado`s database, this one IS NOT supposed to be updated by me . Instead i`m packing it into one "one-click" solution, so everyone can simply unpack it and parse the database from the actual game client.

Due to different client corruptions (which is happening way more often last few months, tbh), some images can be missing. On my game client that brought ~5 broken portraits and ~3 stances with no skill icons. That`s sometimes depends on GE`s inner files being incorrectly structured (for example - typo by devs in file names. Yah, happening sometimes).

Currently this one is in alpha-versions. Serious bugs can happen. I don`t know if i`m able to heavily develop it further (mainly because of my health issues), but i count that i`ll at least implement non-char information tabs (Achievements, items+recipes, etc). No ETA. No certain promises. Think of it as of it`s last update, any upcoming updates is not guaranteed. Maybe it`ll happen. Maybe it`s not.

Download:
https://mega.nz/#!eGBjUC6Q!zoFlI7Zc9...Dwc9czG5PV4AXA

Manual:
  1. Unpack .zip file into /ge/ folder of your client.
  2. Launch Prepare+Parse.bat.
  3. Launch EnglishDatabase.html
  4. Pick a character by pressing "Open" button on the page, and clicking the needed character in the list.
    • The DB will be put into EnglishDatabase.HTML file. It`s supposed to be shared with ./images/ folder, which this kit will also create. ./images/Misc/ currently will not be re-created if you delete it! ./images/Barrack/ and ./images/Skill/ will.
    • You can edit the page template in EnglishTemplate file. Do not touch 3 major keywords in this file - the ones in [[ ]] signs. You can edit the styles and mess with js variables a bit. That`s almost 100% enough to re-make this database into any Jupath-like , if you put enough efforts into CSS and images.
    • Use Parse(English).bat if you change the template. It`ll shorten the execution time to 10-15 sec, by skipping the IPF unpack and folder structure creation.
    • If you don`t use (and don`t plan to use) PHP on your PC, you can set .php files to be opened with /php/php.exe. This way double-clicking the .php file will launch it directly, so no .bat files needed.
  5. Click Prepare+Parse.bat again AFTER updating the game client to get the updated .html

Features:
  • As it`s purely HTML, it can be used on any device with web browser. It `s don`t use any modern (HTML5\CSS3) techniques , so it should be fine on any device.
  • Character list is searchable (you can filter the names. For example,entering "Cadet" will output all chars with that word, from Adriana to Romina.
  • Character table is sortable (you can sort the table to see the chars with the highest stats (like STR,INT), separate recruitable chars from rare\achievment ones,etc.
  • Totally cleared from all non-playable content (like GM versions of chars, Operator,etc). Basically equal to in-game info center.
  • Some of buffs\debuffs have it`s tooltips in skill table. Totally NOT every single ones, as devs is barely really putting correct info into the client.
  • Whole .html page is ~1.1mb, in .zip it can shrink to ~200kb. It can be reduced even further in future versions, if one`s is coming out.
  • The scripts is heavily optimized comparing to v1 kit. I basically rewrote everything from the scratch , abadoning the experience and code base that was shared with me by previous DB creator. Now the script itself (Parse part, as Prepare part is still related to ix3.exe and IZ\EZ toolkit from dneeht,and it`s run time can barely be shortened) is doing it`s job in ~10-15 seconds ( on underclocked R5 3600), and ram consumption is under 300mb (while old kit was 1GB+). Can become even better with the upcoming PHP versions (switching from PHP 5.4 to PHP 7 basically halved the RAM consumption)


The last time I tried this on USA (T3Fun) game client, I have encountered a problem that prepare.php gets stuck after extracting the .ipf files... Anyway, I managed to solve it by rewriting prepare.php for command line. If you have the same problem, you can use the command line alternative for non-Steam server below.

Since Steam server breaks each .ipf file into smaller files (ui00001.ipf, etc.), each smaller file must be extracted separately and then move the contents into a unified folder. Use the command line alternative for Steam server below.

Some non-buff skills display only Lv.1 details for some reasons...

Hmm. As it is currently written, the database uses the data tables from the root ies folder. It may be necessary to replace them with the localized data tables (when available) from ies\USA (or whatever regional code) subfolder instead to account for potential regional customization. Let me know if there's a problem with this issue. Update: Added 2 new lines to the commands.

Update: I have combined the Steam and Non-Steam version into one single file. To use regional data tables, simply remove REM from the 2 lines below :converties in the text.

COMMAND LINE ALTERNATIVE

  1. Copy and paste the following text into Notepad.
  2. Save it as preparenparse.cmd into C:\Program Files (x86)\Granado Espada\ge folder.
  3. Right-click on preparenparse.cmd and run as administrator.
    • This replaces Prepare+Parse.bat in the original package.
@echo off
net session >nul 2>&1
if not %errorlevel%==0 (
echo Right-click on %~nx0 and run as administrator.
pause
exit
cd %~dp0
md images\Barrack, images\Skills, xml, dictionary, ies, ui 2>nul
if exist ui00000.ipf (goto steam)

:nonsteam
iz.exe dictionary.ipf
ez.exe dictionary.zip
iz.exe ies.ipf
ez.exe ies.zip
iz.exe ui.ipf
ez.exe ui.zip
del dictionary.zip, ui.zip, ies.zip
goto :converties

:steam
for %%a in (dictionary*.ipf) do (
iz.exe %%a
ez.exe %%~na.zip
xcopy %%~na\* dictionary /s /y /i
rd /q /s %%~na
del %%~na.zip
)
for %%a in (ies*.ipf) do (
iz.exe %%a
ez.exe %%~na.zip
xcopy %%~na\* ies /s /y /i
rd /q /s %%~na
del %%~na.zip
)
for %%a in (ui*.ipf) do (
iz.exe %%a
ez.exe %%~na.zip
xcopy %%~na\* ui /s /y /i
rd /q /s %%~na
del %%~na.zip
)

:converties
REM for /f delims^=^"^ tokens^=2 %%a in ('type ..\release\client.xml^|find "ServiceNation"') do (set nation=%%a)
REM if exist ies\%nation%\datatable*.ies (copy /y ies\%nation%\datatable*.ies ies)
ix3.exe ies\datatable_item_*.ies
ix3.exe ies\datatable_job.ies
ix3.exe ies\datatable_stance.ies
ix3.exe ies\datatable_skill.ies
ix3.exe ies\datatable_npclist.ies
ix3.exe ies\datatable_stancecondition.ies
ix3.exe ies\datatable_expedition_tag.ies
ix3.exe ies\datatable_buff.ies
move /y ies\*.xml xml
php\php.exe English.php


Comments

  1. I cannot get it to work for JGE it actually compiles the data into the index.html it just won't do anything when I click the characters.

    ReplyDelete
  2. Hmm. It's hard for me to say since I don't have JGE client. I assume you are using the command line alternative for Steam server above already? Check if the xml folder has the datatable*.xml files.

    Anyway, I added 2 new lines (in yellow) to the commands above to account for regional customized files. See if it helps. In any case, since this database draws from the dictionary file, if used on JGE, the database should be in Japanese as well.

    ReplyDelete
  3. I am using English dictionary on my JGE client. I can send you my JGE files that you would need (including my English dictionary)


    UI.ipfs, IES.ipfs & Dictionary.ipf
    https://www.mediafire.com/file/jlfifm80yvw4nxo/JGE_Database_Stuff.zip/file

    ReplyDelete
  4. As an author of this kit, i`m kinda sad that i lost motivation for GE earlier then it was polished enough. I`ve tried to push myself to polish it for Steam servers when Prefix contacted me , but as it`s already ~ half a year since i last played GE, and i`m playing WoW atm...
    What is published there is a result of ~2-4 days of active development. I was planning to extend this scripts further (parse items& recipes, achieves and stuff), but dropped it half-way.
    I think i`ll rewrite it again soon to be compatible with Andromida`s customizations (as they use localize tricks to apply stat changes on client side, and since their changes is really important, like character stats and other stuff- it should be considered), and apply some changes to the original structure. But...No promises. Just hyped a bit with Andromida project vision.

    ReplyDelete
  5. Wait...You're using modified files? Uh, try the Steam command line with an unmodified client first. Your modded files seem to have issues with Parse(English).bat:

    PHP Warning: fopen(): Filename cannot be empty in C:\Games\TestData\ge\Function.xmls2Arrays.php on line 10
    PHP Warning: fgets() expects parameter 1 to be resource, bool given in C:\Games\TestData\ge\Function.xmls2Arrays.php on line 11
    PHP Warning: fclose() expects parameter 1 to be resource, bool given in C:\Games\TestData\ge\Function.xmls2Arrays.php on line 31
    PHP Notice: Undefined offset: 100 in C:\Games\TestData\ge\English.php on line 127
    etc.

    Restore unmodified ipf files first and try again and see if there's still a problem.

    ReplyDelete
  6. I restored the original dictionary.ipf file and it still has the same issue.

    ReplyDelete
  7. Ugh. I can't get it to work for Jap client. The list of characters show up but nothing happens when you click on the character name. I'm stumped. Sorry. Better let Roman/itsyuous take a look at it.

    ReplyDelete
  8. I`ve got a temp. fix.
    In short: ENG patch is messing with special characters. As i was a part of GEEU team(ENG=>RU translation), i adopted it mainly for well-written (technically, not in terms of typo) GEEU translation.
    JGE patch seems to be made by Google Translate from times to times, which breaks some special symbols. also, they use ' instead of ` sometimes, which is harmful for javascript (',", ` is used to separate stuff). Sooo..
    I made it work for at least majority of content (all that i tested in last 5-10 min, as i decided to look into in after Hand of Vecna comment).
    Fix:
    Open Function.xmls2Arrays.php
    Find:
    $a = trim($ArrayName) . trim($match[1]);
    global ${$a};

    Which is strings 23-24.
    add the following 3 lines below it:
    $match[2]=str_replace('"','',$match[2]);
    $match[2]=str_replace('\\','',$match[2]);
    $match[2]=str_replace('`','',$match[2]);

    After that run English.php again.

    Small addition for Ashardalon: please add datatable_item_* in ies list, it`s needed too.

    ReplyDelete
  9. Major update from what is posted here:
    https://granado-espada.to/index.php?threads/ge-database-toolkit.255/#post-2178
    Now moved to Andromida forums, as GEEU forums probably gonna shut down soon.

    ReplyDelete

Post a Comment

Comments are currently disabled.