EHM Editor v2: Development thread

[archibalduk]

The latest alpha version of the EHM Editor v2 can be downloaded here: version 2.0.0-alpha.2 | 06-Mar-2022 | https://github.com/archibalduk/Eastside ... lpha.2.zip


IMPORTANT NOTES:
PLEASE READ ALL OF THE FOLLOWING BEFORE DOWNLOADING SO THAT YOU KNOW THE CURRENT LIMITATIONS OF THE EDITOR:

The Editor is very incomplete at this stage and consequently it is read only. This means it can load databases but IT CANNOT SAVE ANY DATA OR CHANGES. The purpose of the current alpha release is to allow people to try out the Editor and give feedback mostly on the user interface, ease of use, workflow and any crashes/issues. This alpha release has been released sooner than I planned as it has the ability to export filtered data to spreadsheets which Editor v1 does not and so this may be useful to database creators.

If you have a large screen (i.e. a high resolution with low/no DPI scaling then you might want to try enabling the "Large Screen" setting via the Settings screen (restart the Editor to apply the setting). This will display more editing fields on a single screen.

A very rough development plan is as follows:

• First draft of the user interface [95% COMPLETE]
• Add special filters from Editor v1 (e.g. the ability to filter a player by draft history, etc) [100% COMPLETE]
• Add the ability to save/load filters and custom spreadsheet layouts [Not started]
• Add all database validation functions from Editor v1 [80% COMPLETE]
• Add all mass editing functions from Editor v1 [95% COMPLETE]
• Add spreadsheet importing / research sheet exporting [95% COMPLETE]
• Add playable league & new EHM v1.5 editing screens (e.g. Finance Info, Rules & Structures, Ruling Bodies, Schedule Templates, Transfer Rules, Contract Details sub-screen, Historical Club Names, etc.) [Not started]
• Add UID validation [Not started]
• Add extra_config.cfg and records_config.cfg editing if possible [Not started]
• Dark mode [80% COMPLETE]

Colour scales: [Not started] | [>=1% COMPLETE] | [>= 50% COMPLETE] | [>= 75% COMPLETE] | [>= 90% COMPLETE]

Saved game viewing is not currently planned but I will revisit this once the Editor is further developed. Saved game editing will not be possible as the Editor does a lot of behind the scenes caching which is then re-imported into the database when saving or rebuilding the database - and this would corrupt any saved game.


KNOWN ISSUES:

• Database validation errors are not yet shown when loading/saving the database (i.e. you won't see the red or yellow error highlight until you edit or navigate away from that specific record);
• Mass editing players/non-players is slow (around 10,000 records per minute);and
• Screens without a navigation panel have a grey space to the left-hand side (e.g. settings screen and home screen).

INSTALLATION:

• You must first install Microsoft Visual Studio 2022 Redistributable 64-bit version (also known as vcredist_x64.exe): https://aka.ms/vs/17/release/VC_redist.x64.exe;
• Simply extract the EHM Editor v2 zip file to a new folder of your choice. Do NOT extract the EHM Editor v2 zip into your existing EHM Editor v1 folder as this may cause a conflict between .dll files as EHM Editor v2 is built using Qt v6.x whereas EHM Editor v1 is built using Qt v5.x;
• Double-click on 'EHM Editor 2.exe' to start the Editor.

CHANGELIST:
Given that the Editor is still in its early stages, I'm not proposing to post a changelist for now. I'm happy to post a copy of my GitHub commit notes on request however (these are very brief and fairly technical but give a flavour of the changes - for example: "8-July-2021: fix out of bounds issue with db_tool.cpp; added db_tool_staff_history_tidy_up.cpp").


FURTHER READING:
A bit a background about why EHM Editor v2 is in development can be found here: https://www.ehmtheblueline.com/forums/v ... 61#p244161

A brief overview of some of the new features can be found here: https://www.ehmtheblueline.com/forums/v ... 62#p244162

[archibalduk]

Back in January 2019 I started work on a complete re-write of the Eastside Hockey Manager Editor and have been working on it on and off for the past 2+ years. I had expected to have a partial demo released by now but I had to down tools last summer due to the issues with our webhosting which required me to transfer the site to another host as well as various work required with EHM Editor v1. Anyhow, I have made sufficient progress over the past three months that I have a few initial screenshots to share (see the post below).

Some people might be wondering why I'm putting so much time and effort in re-writing the Editor when we already have a perfectly decent Editor which could be developed further. As a self-taught amateur/hobby coder, the Editor is by far the most ambitious and complex project I have ever worked on. My EHM 2007 tools, such as the EHM Updater, were far more basic in comparison. It was a steep learning curve and, with the benefit of hindsight and experience, I would have designed things very differently. It wasn't helped that the playable league tables were so complex that I had to create the editing screens and then figure out what everything did. From a coding perspective, there are many areas of substantial inefficiencies which makes it very difficult to work with. Adding something as relatively simple as an additional filter is actually pretty cumbersome and time consuming. I had looked at whether the existing Editor could be re-designed one module at a time but it's too complex and needs a complete re-write.

Behind the scenes, the new Editor is so much easier for me to work with. The inefficiencies and code duplication of Editor v1 are a thing of the past with Editor v2. Rather than having to "roll-out" code for each table or editing screen, much of it is coded once and then automatically generated by the more generic code. For example, each of the editing screens for Editor v1 were coded by hand, so it was a pretty tedious job and it means that if, for instance, I wanted to change the way particular inputs work then I would have to manually change each input. In Editor v2, the editing screens are all automatically generated without any manual work by me. The only ones now designed by hand are those where I want to customise the layout slightly with multiple tabs (e.g. the Club and Staff editing screens) - and even then it's only a few lines of code. So the lines of code for the editing screens has been decreased from 2,000+ to around 100-200 (in each case excluding the playable league editing screens).

The improved design also allows me to add more advanced features very easily. For instance, the new clipboard and mini-editor functions were very quick and easy to create (the mini-editor is only 100 lines) and, at least in my opinion, are significant improvements over Editor v1.

There is still a lot of work to be done (and I haven't even started on the playable league screens yet) but I figured I'd start a thread now in case anybody had any suggestions/requests for features or improvements which could be added to Editor v2. In the meantime, I've posted a few screenshots in the post below.

[archibalduk]

New/Improved Features
I'm going to use this post to list out the new or improved features added to EHM Editor v2. I'll try to keep this post intermittently updated.

General User Interface Improvements
I have tidied up the UI a little and it looks a little more "native" with smaller tabs, although I will add an option to bring back the old style of tabs. The Editor has options for small screens and standard/large screens which allows multiple tabs of an editing screen to be shown on a single screen. For example, the player current ability, potential, etc fields are shown on a single screen with the attribute fields on standard/large screens. I'll post a screenshot if this when I'm next testing the Editor on my 22" monitor.

Various of the fields are now tidier too. For example, a thousands separator is now shown for numerals, there is better displaying/alignment of monetary values (e.g. salaries) and all of the drop-down/selection menus now allow text searching instead of using pop-up windows (they also have checkboxes rather than there being a "NONE" entry in each drop-down/selection menu). So instead of having to select a club from a pop-up window for a player's Club Contracted, you can just type it into the field like so:




Rebuilding the Database
The functions allowing for the rebuild/refresh of the database allow for additional database sorting or item deletion with far greater ease. This was a particularly complex aspect of Editor v1 which is why I hadn't yet implemented the ability to delete items from every single table. For instance, it is now possible to delete first names and second names. The Editor resetting each editor screen after importing spreadsheets is also generally no longer an issue. The Editor uses multi-threading for certain aspects of rebuilding/refreshing the database which speeds things up slightly.


Filtering
Pretty much any field can now be filtered other than things such as history fields (e.g. club history, player career history, etc). Filtering performance is much faster too. The screenshot below shows how filtering now works but note that I need to fix the funky alignments:




Find / Find & Replace
It is now possible to perform various types of searches as well as a search and replace (e.g. find all instances of "Anaheim Ducks" and change to "Calgary Flames"). This might be handy if you want to move all staff, club histories, etc from a defunct/duplicate club to another club or if you want to remove a superfluous city.




Copy & Paste
Every drop-down/selection field (e.g. club, competition, player role, city, etc fields) has a copy and paste button. So you could copy a player's primary nationality and then paste that nation into any other nation field in any screen. The clipboard is saved and re-loaded each time you load the Editor, so it carries across sessions which is mostly just for ease.


Mini Editor
Every drop-down/selection field has a button beside it which launches a Mini Editor in a separate window. For instance, if you click on the button beside the Club Contracted field on Alexander Ovechkin's profile, it will allow you to edit the Washington Capitals in a separate pop-up window. This lets you edit virtually everything other than things such as histories.




Status Panel / Validation
Every table can now be validated against set rules - e.g. if a club is set without a nation but has been assigned a league. Any validation warnings (flagged with a yellow warning icon) or errors (flagged with a red error icon) are shown in a panel at the bottom of the editing screen.

[ideawithoutamind]

If you need any additional testers, definitely let me know!

[nino33]

Amazing as always! Looks great! Thanks so much for your efforts

[archibalduk]

Thanks! I'll try and get a test version online at some point this year. If progress continues to go this well then I think it should be sooner rather than later this year. Note that it will initially be a "read only" test version as the playable league tables won't be added until I have completed everything else, which will be after the first test version.

[archibalduk]

Progress has been painstaking recently owing to the complexity of creating some new features for the first time, but nonetheless I am slowly getting there. A few of the notable new features recently added are:

1. The ability to add/delete any database record anywhere in the Editor by clicking on the Add/Delete buttons on each screen. It was proving difficult to implement this in Editor v1 which is why many of the "Delete" buttons don't work. The database is far better designed and so adding this to Editor v2 was actually very simple.

2. Easier deletion of rows in tables - see the first screenshot below. Mackecool93 recently asked for a Delete button to be added beside each row in order to make it easier to delete rows in Editor v1. Unfortunately there isn't an easy way to add this and in fact the sheer number of Delete buttons required would be a resource-hog. I have however been able to make deletion easier in Editor v2 by allowing you to ctrl+click on the rows you want to delete and then hitting the Delete button. E.g. in the first screenshot below you'll see that six rows have been highlighted in blue after I ctrl+clicked on them. If I click on the Delete button then it will delete those six rows. I'm going to see if I can somehow bring this over to Editor v1 but I'm not entirely sure if this will be possible simply because of the way in which Editor v1 works.

3. A new Extra Rules editor - see the second screenshot below. This was very fiddly to add because it was a complete re-write but it's pretty much up and running now, save that I need to incorporate a few final touches and tidy-up its appearance. My aim with redesigning this was to have a cleaner design (FYI the example in the screenshot isn't the finished article yet) and to make it easier to add and set rules. I've (hopefully) made things easier by including a selectable team/competition/nation when using a UID-related extra rule along with a searchable drop-down menu so that extra rules can hopefully be found more easily.

[archibalduk]

Question for xECK29x: When using the mass edit tool to make teams inactive where the assigned league is inactive, should the team's Last League setting be automatically set to the inactive league?

E.g. if we had a team set to:

League: British National League (being an active league)
Last League: British Division 1

When using the tool, the team would then be set to:

League: [None]
Last League: British National League

[xECK29x]

Question for xECK29x: When using the mass edit tool to make teams inactive where the assigned league is inactive, should the team's Last League setting be automatically set to the inactive league?

E.g. if we had a team set to:

League: British National League (being an active league)
Last League: British Division 1

When using the tool, the team would then be set to:

League: [None]
Last League: British National League

Yea that would be helpful and its what I usually do manually. This would be the preferred behavior for the 'Set as Inactive' button as well.

[archibalduk]

OK thanks. I'll add for v1 and v2 in that case.

[Mackecool93]

2. Easier deletion of rows in tables - see the first screenshot below. Mackecool93 recently asked for a Delete button to be added beside each row in order to make it easier to delete rows in Editor v1. Unfortunately there isn't an easy way to add this and in fact the sheer number of Delete buttons required would be a resource-hog. I have however been able to make deletion easier in Editor v2 by allowing you to ctrl+click on the rows you want to delete and then hitting the Delete button. E.g. in the first screenshot below you'll see that six rows have been highlighted in blue after I ctrl+clicked on them. If I click on the Delete button then it will delete those six rows. I'm going to see if I can somehow bring this over to Editor v1 but I'm not entirely sure if this will be possible simply because of the way in which Editor v1 works.

Awesome that is exactly what i asked for. Thanks for doing this change. Will Shift+click also work?

[archibalduk]

Yes shift+click works too

[archibalduk]

Progress has been bogged down with some fairly time-intensive items over the past month or so as I've been creating the new mass edit function structure and re-working all of the existing mass edit/tools functions to work with the new structure. All mass edit/tools functions now have three options: (i) Apply to the currently viewed record, (ii) Apply to all filtered records and (iii) Apply to all records. This will provide a bit more flexibility compared with Editor v1 as you can filter a range of players/clubs/etc exactly as you wish and then apply the mass edit just to those items. Here's a work in progress screenshot of how it is looking:



As I'm approaching the first beta/demo release, I've set up a rough to-do list setting out what needs to be complete before I release. The first release is going to be a "demo" in that it is purely there for anyone to give feedback on the UI and any of the functions and it won't be able to save databases. Nor will the data import function be complete (so it'll be disabled) and the playable leagues/contract details/historical club names haven't yet been implemented (I'm leaving these until once everything else is finished). I'm very optimistic that the demo will be released this year. Adding the playable league editing screens is going to be a big task as I want to completely re-design how it works in order to make it user friendly. I'm hoping that the first full release of the Editor v2 will be during 2022 but obviously that depends on how quickly I can progress the playable league editing screens. I'm approaching the three year mark in this project and so 2022 still feels fairly ambitious!...

[archibalduk]

I have made a great deal of progress over the past few days and I'm getting close to being able to release a demo version for feedback. A while back TBIF asked whether it would be possible to export a single attribute from the database to a spreadsheet. Whilst this isn't feasible for Editor v1, I have today added this to Editor v2 (see below). This will allow you to create custom spreadsheets of most data from the database (things such as staff histories, club histories, etc aren't yet possible but that's because I haven't yet added import/export for histories). The export works in conjunction with the filtering, so if you want to export certain data for all left-handed goalies born in New York then this is now possible...

[TBIF]

How do I super love something on the forum!

[laving39]

Will it be possible to create a draft to a league? In version 1 it's possible to create the draft, but I never managed to make that draft appear in-game. More specifically. Can I create a Scandinavian fictional league with a similar set of draft rules as the NHL, without actually having to replace the NHL?

And would it be possible to have a future draft pick manager in the editor, instead of having to do this in the config file? You know, an easier way to spread out all the Maple Leafs future draft picks to other teams.

[archibalduk]

Will it be possible to create a draft to a league? In version 1 it's possible to create the draft, but I never managed to make that draft appear in-game. More specifically. Can I create a Scandinavian fictional league with a similar set of draft rules as the NHL, without actually having to replace the NHL?

I'm afraid this is a game limitation rather than a limitation with the Editor. EHM Editor v1 will let you edit anything which is technically possible with the database. Unfortunately the game doesn't support extra drafts at present.

And would it be possible to have a future draft pick manager in the editor, instead of having to do this in the config file? You know, an easier way to spread out all the Maple Leafs future draft picks to other teams.

Extra_config editing is something I'd really like to bring into the Editor. It's not a simple thing to do, but it's something I'm definitely going to look at.

[archibalduk]

The first alpha release of EHM Editor v2 is now available to download: https://www.ehmtheblueline.com/forums/v ... 60#p244160

Do make sure that you read through the entirety of the above download post linked above as it contains a lot of important information about what state the Editor is currently in (e.g. it cannot currently save any changes). The first draft of the user interface is almost complete but there are some minor things I haven't yet added - e.g. (i) a sub-heading at the top of each screen to list out the player date of birth, etc. and (ii) I expect I'll merge some of the menus too (e.g. I think the Clubs and Club Competitions menus could be merged into a single menu).

The reason for releasing now is largely in order to enable DB authors such as xECK29x to take advantage of the filtered player/non-player data exports. Here's how to use filtered exports:

• Go to the chosen section of the Editor (e.g. People -> Staff);
• Click on the usual grey filter button in the bottom left corner of the menu to filter the list as you want;
• Click on the Export Data button at the bottom of the screen; and
• Select "Export as research sheet" and then click on the Export Current Filter button.

As mentioned in the first post of this thread, the research sheet is correctly set up for players/non-players but not for the other tables yet (but they will still export data, just not the correct column order).

EDIT: I accidentally uploaded with the Save buttons enabled (which would corrupt a database if you clicked on Save by mistake). I have re-uploaded with the Save buttons disabled. If you download the Editor after I have posted this edit then you're fine.

[Mgomrjsurf]

Thanks for update.

[archibalduk]

I've been making good progress with the spreadsheet import/export functions for the new Editor in recent weeks and should have the importer spreadsheets for most table types (e.g. clubs, affiliations, club competitions, players, etc) complete over the coming days. The only table type left to implement are histories/records as these are treated differently in Editor v2 compared with v1, so require a bit more work.

These new functions are substantially easier for me to work with in comparison to Editor v1. For Editor v1, I had to code every spreadsheet by hand which was extremely time-consuming and somewhat error prone. It also made it difficult to improve any aspect of importing (or exporting for that matter) as I'd have to go through the code for every spreadsheet and make that same change to each. The Editor v2 functions are completely centralised and so all I need to do for each spreadsheet is list out the what table columns are to appear in the spreadsheet (see example screenshot below) and the Editor will figure out the rest on the fly. The end product will look exactly the same as Editor v1 but the work involved behind the scenes is substantially less and will make it easier to improve or expand the spreadsheet import/export functions in the future if required.

[ideawithoutamind]

Let me know if you'd prefer me to post this in the regular editor forum.

Minor bug:
I've been playing around with this a bit mostly for the better player exporting options:
Found there's a few issue with exporting:


Staff with birthplaces with States/Provinces get the data shifted over a column, see screenshot above.

Secondly, if exporting from Editor v2, can we re-import using Editor v1, I noticed the date formats are different (m/d/yyyy) vs (d.m.yyyy)

Lastly, for the player filtering is definitely a great step in the right direction, a few notes:
Filtering by player rights id, I couldn't find the right range to find players. (I tried 0 to 999999 returns no results, I tried -999999 to 0, and all players showed up, even if they did not have rights). Ideally having a feature to filter "NHL Player Rights".

Club contracted and League contracted filter no longer is an option (as it was in Editor v1), would definitely like that put in.

Great work! I'm just starting to play around with this version, so I'm happy to see the progress!

[archibalduk]

Staff with birthplaces with States/Provinces get the data shifted over a column, see screenshot above.

Thanks - yes I'm aware and I'll fix this probably fairly soon as I'm currently working on improving the import and export functions. It's because of the way in which city names are formatted with commas between the city name and state abbreviation. CSV spreadsheets treat commas as a column separator. This is resolved in Editor v1 by using either colons or semi-colons instead of a comma (I can't remember which off the top of my head) but I might tackle it slightly differently in Editor v2 as quotation marks can be used to flag a comma being a literal comma rather than a column separator. The temporary work-around in the meantime is to export using XLSX instead of CSV.

Secondly, if exporting from Editor v2, can we re-import using Editor v1, I noticed the date formats are different (m/d/yyyy) vs (d.m.yyyy)

Yes this *should* be possible but do double-check that the columns for the Editor v2 spreadsheet match up precisely against the Editor v2 spreadsheet. I believe they do, but it's worth double-checking as the spreadsheets are a work in progress. The intention is to have the same arrangement of columns between Editor v1 and v2 importer spreadsheets unless there's a good reason for not doing so. You will probably find that things such as favourite/disliked staff might not export correctly at the moment (I think they might miss the DoB or are completely blank) but this is fixed in the work-in-progress version. I'll get another release out once the import process is more developed - this particular aspect has been rather slow.

Editor v2 will export according to your system date format whereas Editor v1 will do it according to what you set in your Editor v1 settings, so that'll be why you're seeing a difference. If you enter "mdy" in cell B1 of the spreadsheet then this should tell Editor v1 to treat any dates in m/d/yyyy format.

Lastly, for the player filtering is definitely a great step in the right direction, a few notes:
Filtering by player rights id, I couldn't find the right range to find players. (I tried 0 to 999999 returns no results, I tried -999999 to 0, and all players showed up, even if they did not have rights).

Thanks - I've made a note to take a look. That should filter by player rights Id and it sounds like it's not working if 0-999,999 doesn't bring anything up (unless you filtered by some other criteria in addition to this).

Ideally having a feature to filter "NHL Player Rights".

Club contracted and League contracted filter no longer is an option (as it was in Editor v1), would definitely like that put in.

Yes it's my plan to add all of the same filter categories from Editor v1 but I just haven't gotten to it yet. The Player Rights and Last Clubs filters in particular require some careful thought given the way in which the Editor now auto-generates filters/screens/spreadsheets/etc according to data structure rather than me doing things by hand.

Keep the suggestions coming!

[archibalduk]

xECK29x - I'm looking at simplifying the import settings for Editor v2 and was wondering what your thoughts are on the following:

1. Do you always use synonym files when importing spreadsheets? I'm thinking of just having a general Yes/No setting for synonyms as a whole rather than having individual settings for cities, clubs, leagues, etc.
2. Ignore Lists - Do you use these?
3. I don't see the point in the Use Nation Abbreviations as well as the Use Long Names / Use Short Names settings for nations, clubs and leagues - surely it makes sense to always use Long and Short Names as well as Nation Abbreviations.
4. Dates of birth settings: Use Year of Birth - is this used? If not, I'll remove other than where a person has a 1.2.1900 DOB.
5. Players/Non-Players/Officials: Do Not Create Duplicates - Is this of any use?
6. Players: Calculate Pass Tendency & Strength / Validate Positions - Do you use these?
7. Any additional settings you feel would be useful?

I don't want to remove any settings that are actually useful, so let me know what you think.

I've been spending the past two months working solidly on the import/export functions and it's been a real slog. Thankfully the core stuff is now 90% done and I'm about to start working on the synonyms and settings - hence the above questions!

[xECK29x]

xECK29x - I'm looking at simplifying the import settings for Editor v2 and was wondering what your thoughts are on the following:
Do you always use synonym files when importing spreadsheets? I'm thinking of just having a general Yes/No setting for synonyms as a whole rather than having individual settings for cities, clubs, leagues, etc.
Ignore Lists - Do you use these?
I don't see the point in the Use Nation Abbreviations as well as the Use Long Names / Use Short Names settings for nations, clubs and leagues - surely it makes sense to always use Long and Short Names as well as Nation Abbreviations.
Dates of birth settings: Use Year of Birth - is this used? If not, I'll remove other than where a person has a 1.2.1900 DOB.
Players/Non-Players/Officials: Do Not Create Duplicates - Is this of any use?
Players: Calculate Pass Tendency & Strength / Validate Positions - Do you use these?
Any additional settings you feel would be useful?
I don't want to remove any settings that are actually useful, so let me know what you think.

I've been spending the past two months working solidly on the import/export functions and it's been a real slog. Thankfully the core stuff is now 90% done and I'm about to start working on the synonyms and settings - hence the above questions!

1) Sometimes, depends on the context of the editing I'm doing. If I'm working on career/club history imports I may use it. I'm mainly working on a new 2000-01 project so I am heavily using it to map current to historical teams/comps.
2) Not that I am aware of
3) Agreed
4) No, I only put a complete DOB we never just use a year, if we don't have a month/day we will use 1.1.xxxx as a placeholder.
5) Yes, it has prevented duplicates
6) Yes, heavily
7) Not sure what you are asking here

[archibalduk]

Thanks - that's super helpful.

7) Not sure what you are asking here

I was just wondering if there's anything else you wish the Importer could do - e.g. like how we have auto-validation/calculation of Pass Tendency and Strength when importing players and career histories. I expect possibly not, given that we have the mass edit functions already but I thought I'd ask just in case.

I'm going to add the ability to filter out players/clubs/etc that have been edited via spreadsheets during that Editor session. So if you want to further work on players/etc after you've imported them, you can just select them via a filter in the Editor.