Reading directly from the savefile

Mahrgell · Post by **Mahrgell** » Tue Feb 27, 2018 8:36 pm

Hi there,
for the purpose of some Discord Sim league, I was looking for a way how it would be possible to read data directly from the save file.
The idea is to write a Discord bot which can then automatically read the savefile and create various summaries and answer requests.

Data I would be interested in includes:
- standings of a league
- results of a league
- player/team stats
- player attributes
- player contracts
- potentially more, but that should do for the moment

Sadly I have only found information how to use the Data Editor, but for my purpose I would really need to read the data myself from the savegame.
Since the Editor as well as EHM assistant are obviously able to do so, I would love to know if there is some information (or even documentation

) on how the data is stored in the save file.

The necessary coding knowledge to write the bot with that information is available

Post by **archibalduk** » Tue Feb 27, 2018 9:17 pm

Unfortunately this would be extremely difficult to achieve. From your list:

- standings of a league - This might be possible with a lot of work
- results of a league - This might be possible with a lot of work
- player/team stats - Very difficult but this might be possible with a lot of work
- player attributes - Achievable (already accessible via the Editor)
- player contracts - Very difficult but this might be possible with a lot of work

The saved game is basically an expanded version of the database. In other words, the saved game consists of a large number of data tables. The Editor and the Assistant are very different. The Editor accesses the database/saved game and parses/decodes the data. The Assistant instead accesses the data loaded into the RAM by the game. I can't say for certain whether all of the game data is always in the RAM or whether some of it is buffered in temporary files (which I know the game does use) and is simply loaded to the RAM as and when needed. So accessing the data via the file *might* be the better method.

To access the saved game, you'll need to parse the saved game index and all related tables (i.e. the club, league and player tables) in order to make sense of the various data you want to export. The saved game (and the database) is a relational database which means that the tables contain lots of ID numbers linking to items in other databases. I can share the relevant club/league/player data structures with you if Riz agrees. However, that will only get you 50% of the way. Aside from the player attributes, all of the tables you mention above are tables specific to saved games and SI does not divulge the structure of saved game tables. So you'll need to figure this out yourself. And this is where it starts to get difficult. Not only is it a lot of trial and error, but certain tables can have variable length rows of data (i.e. because it contains arrays) - so it isn't always possible to identify patterns or indeed parse the data unless you can figure out where in the data the variable lengths are flagged and how the variable data is structured.

I have thought about trying to add some saved game exporting to my Editor because I can see how that could be of benefit to people playing the game and especially online leagues. However, it's not something I'd get to for a long time because I have to prioritise work which benefits the roster updates. That said, I can show you how to export the relevant raw data (in hex code) from the saved game using the Editor for you to look at - if you can figure out the patterns/structure then I could add this to the Editor sooner rather than later and this might speed things up. I also have some random notes on parts of the saved game I partially-decoded when I was experimenting with EHM 2007 (and there is an old thread on these forums that provides some basic structures too).

Mahrgell · Post by **Mahrgell** » Tue Feb 27, 2018 9:52 pm

Heya,
yeah, I kinda assumed that it was mostly saved in some sort of database structures.
If you could share the format of those data structures you know this would already be a great starting point. How to work with relational databases I do know. I would just need the format.

And for decoding the rest... Sounds like a daunting, yet interesting task. If my first attempts with the easily accessible data are successful, I could also give this a go.

Post by **archibalduk** » Thu Mar 01, 2018 9:23 pm

I have dropped Riz an email. I'll get back to you once I hear from him. He is generally extremely busy (he works on EHM in his spare time) - so it might take some time to come back to me.

Mahrgell · Post by **Mahrgell** » Thu Mar 15, 2018 7:40 am

Time has passed...

I'm still interested in it, has there been any response by Riz?

Post by **nino33** » Thu Mar 15, 2018 12:34 pm

Mahrgell wrote:Time has passed...

I'm still interested in it, has there been any response by Riz?

FYI - I’ve waited months before for replies; I think 2 weeks passing is basically no time at all

Post by **archibalduk** » Thu Mar 15, 2018 6:56 pm

Mahrgell wrote:Time has passed...

I'm still interested in it, has there been any response by Riz?

As it happens, he emailed this morning to say he is running this query by his colleagues at SI Games. I'll let you know once I hear anything further but, as I said, it takes him time to get back to me because of his workload. And like Nino said, two weeks is nothing - be prepared to receive a response any time between now and April/May.

Post by **archibalduk** » Sun Mar 25, 2018 10:13 am

I have had a response. Can you drop me an email with your email address please? Mine is ehmtheblueline AT gmail DOT com

pretzalcoatl · Post by **pretzalcoatl** » Tue Jun 12, 2018 9:13 pm

I'm trying to make a tool that will pull player season histories from saved games to create a viewable database for a league playthrough

Do you have any pointers as to what format the saved games are saved as?

(alternatively, it looks like EHM stores its in-game database in C:\Users\USERNAME\AppData\Local\Sports Interactive\EHM\Temporary and that could work too)

this is all SQLcipher-encrypted sqlite, or am I wrong?

Post by **archibalduk** » Wed Jun 13, 2018 9:23 pm

I have moved your post to the correct thread. No, it's not SQL. It's a custom binary structure.

yoye · Post by **yoye** » Thu Feb 14, 2019 4:33 pm

Does OP get a positive response ? I'm insterested too, I want to create an HTML exporter because those generate in game are not really good and even add wrong HTML structure.

Post by **archibalduk** » Thu Feb 14, 2019 7:55 pm

IIRC I gave him some initial details after getting Riz's approval but I don't think the OP took it any further as he never came back for further details. I'm not sure people realise how big of a task trying to process the saved game data is. It's a big task getting the database data within the saved game decoded (even with the full structure details) but trying to figure out the saved game structure with no details is really tough.

yoye · Post by **yoye** » Fri Feb 15, 2019 1:43 pm

archibalduk wrote: Thu Feb 14, 2019 7:55 pm IIRC I gave him some initial details after getting Riz's approval but I don't think the OP took it any further as he never came back for further details. I'm not sure people realise how big of a task trying to process the saved game data is. It's a big task getting the database data within the saved game decoded (even with the full structure details) but trying to figure out the saved game structure with no details is really tough.

could I have the same details in PM or I need to send you an email ?

Post by **archibalduk** » Sat Feb 16, 2019 11:41 am

You would need to ask Riz for his permission. I cannot share anything unless he tells me it is okay. Otherwise my Editor would be open source.