EHM Editor v2: Development thread

[xECK29x]

The auto-set of a player position based on role, if a player doesn't have any 20s set in any one position, would be helpful. We still have a lot of placeholder players in the DB that have no proper position set but have a role.

[xECK29x]

Oh one more thing, I'd LOVE to be able to have two editors open and not have to worry about writing over the last DB opened, if they can operate in two different places in memory that would be a huge help. I just used the Alpha to look at something for my 2000s project and ended up overwriting my Pivot master!

[archibalduk]

I just used the Alpha to look at something for my 2000s project and ended up overwriting my Pivot master!

I'll add all of the above to my task list. Different Editor sessions operate in different/segregated parts of the memory but the issue is that the Editor stores the file path to the last opened database in the registry. So when you hit Save it just pulls the path from the registry and saves to that path. This means that the last opened database will be the target path when hitting Save. I've already made a few tweaks to the Editor v2 in order to cache the file path away from the registry but it still needs a bit more work.

[archibalduk]

Do you mean being able to edit the synonym files whilst the DB is open or do you mean being able to select a custom location from which to load the synonym files? I can definitely do both for Editor v2. If it's straightforward then I can add one/both to Editor v1 too.

More so the later, I'd like a button to load a non-default synonym file and point to a different file of any type (Cities, Club, Comp) which would be used ONLY FOR THAT SESSION.

I haven't yet looked into this for Editor v1, but the above is now up and running for Editor v2. You can choose a session-specific synonym folder and optionally use the default folder path as a fall-back if a synonyms file cannot be found for a particular table type (but this can be disabled via a checkbox).

Synonyms now work for all table types other than staff/official names as they're handled differently in Editor v2 (they're cached which allows data in the first and second name tables to be deleted, etc) - and at a fraction of the code compared with Editor v1.

[xECK29x]

I haven't yet looked into this for Editor v1, but the above is now up and running for Editor v2. You can choose a session-specific synonym folder and optionally use the default folder path as a fall-back if a synonyms file cannot be found for a particular table type (but this can be disabled via a checkbox).

Synonyms now work for all table types other than staff/official names as they're handled differently in Editor v2 (they're cached which allows data in the first and second name tables to be deleted, etc) - and at a fraction of the code compared with Editor v1.

Awesome, looking forward to this! One MAJOR feedback item about V2 is if we can keep the layout as close to v1 as possible that would be greatly appreciated. You can't imagine how much muscle memory I've built up using the current editor, also having all the fields in that specific order and close together is very helpful.

[archibalduk]

Yeah I totally agree. I don't want to mess with the layout or ordering unless there's very good reason. Some things may have shifted around a little bit but that is largely because I haven't been back through to check everything off yet.

One exception is the Rules & Structures screen which is likely to see some fairly major redesign in order to make things easier to use.

[xECK29x]

One exception is the Rules & Structures screen which is likely to see some fairly major redesign in order to make things easier to use.

That's fine, it definitely is the hardest thing to understand, as long as you have a one to one match with the current labeling of items so we know what is what.

[TBIF]

Please have dark mode <3

[archibalduk]

Progress is still going really well on the import/export functions. All I've got left to do is to code the error spreadsheet output as well as add the prevent duplicates setting and the various post-import options (e.g. auto-calculate Pass Tendency, etc). I suspect there will be a few import bugs to iron out but I know what these are likely to be and they shouldn't be anything major. I'll then try to implement all of the missing filter options from Editor v1 (e.g. filtering staff by drafted club) and am hoping to have a new alpha with all of this done towards early March.

I don't know if I mentioned before, but the way the importer now works allows for custom import spreadsheet types. This doesn't mean that you can put two table types into one spreadsheet, but it does mean, for example, you could create your own types of staff import sheets if you'd rather have fewer spreadsheet columns than the default staff_import file. I haven't yet created the user interface to add/edit custom import types but that's all that is needed in order to enable this.

Please have dark mode <3

I'll do my best. The Qt framework which I use (which generates the Ui etc) doesn't have a native dark mode but there's third party css/styles which achieve just this: https://qdarkstylesheet.readthedocs.io/ ... shots.html

I guess there needs to be two options: (1) Automatically use dark mode when Windows is in dark mode or (2) Allow a user to select dark mode in the Editor even if Windows is in light mode.

[TBIF]

Any way to implement "markers" on profile pages?

I'm trying to think of the best way to word it but basically a permanent identifier on profile pages that would be saved when the db gets saved. This for me would mostly be used for editing within the editor itself so I could make off players strategically and maybe have a filter option. Even just a check box or something that highlights the profile as "completed" or "needs eyes on"

[xECK29x]

Any way to implement "markers" on profile pages?

I'm trying to think of the best way to word it but basically a permanent identifier on profile pages that would be saved when the db gets saved. This for me would mostly be used for editing within the editor itself so I could make off players strategically and maybe have a filter option. Even just a check box or something that highlights the profile as "completed" or "needs eyes on"

What you are describing are usually called labels or tags

[TBIF]

What you are describing are usually called labels or tags

I shall avoid TBL until I've finished my coffee next time!

But yeah tags in the editor would be amazing

[archibalduk]

Adding customisable (and filterable) tags is very easy to implement.

Enabling tags to persist between editing sessions would require some sort of sidecar file which would be saved alongside the db file. You'd need to ensure that the sidecar file is always in the same folder as your db file when editing the db. If you were to edit the db without the sidecar filen then it would go out of sync with the db and it wouldn't be possible to recover/re-sync the sidecar. Another option is to have the sidecar file bundled with the db file as a single file but you'd need to extract the db whenever you wanted to run it in EHM - so that's probably overkill.

The sidecar file would open up the possibility of having other data sat alongside the db in the Editor. E.g. you could add Elite Prospects ids to player profiles.

If this would be of use then let me know.

[archibalduk]

All of the importer settings are up and running and I've added all of the missing special filters from Editor v1 (e.g. filtering staff by past clubs played for). There's just a few UI bits I want to tidy-up together with some final custom export spreadsheet layouts to add for a few tables. All being well, I'll have the new alpha release online next weekend.

[archibalduk]

Please have dark mode <3

This is very much a work in progress (note the various white fields with white text!) and it won't make it into this coming release, but I am working on a dark mode:

[xECK29x]

All of the importer settings are up and running and I've added all of the missing special filters from Editor v1 (e.g. filtering staff by past clubs played for). There's just a few UI bits I want to tidy-up together with some final custom export spreadsheet layouts to add for a few tables. All being well, I'll have the new alpha release online next weekend.

Awesome!

[archibalduk]

Alpha-2 of the Editor is now available - see the download link here: https://www.ehmtheblueline.com/forums/v ... 60#p244160

IMPORTANT! The Editor now uses the Microsoft Visual Studio 2022 Redistributable 64-bit version. Prior versions of the Editor used the 2019 Redistributable. Future releases of Editor v1 will also use the 2022 Redistributable.

The vast majority of work over the last 4+ months has been on the spreadsheet import and export functions. Spreadsheet importing is 95% complete but isn't quite ready for wider testing yet. There's a few minor snags I need to finish off, namely:

1. Some spreadsheets are showing staff first name, second name and DOB across two columns instead of three (see e.g. the player career history spreadsheet);
2. I haven't tested deletion at all - it should work but you'll have to manually click on Database -> Rebuild Database afterwards otherwise the items might still remain present; and
3. I think there's a localisation issue with dates (e.g. dates of birth) which seems to vary across Windows 10 and 11 - see my request for help below about this.

If anyone does try either importing spreadsheets or exporting research sheets then be prepared for a few columns to appear slightly incorrectly (when exporting) or potentially crashing (when importing). I need to do some extensive testing of each spreadsheet and to fix any remaining snags. It'll then be ready for wider testing once I've completed this. There's also a lot of debug data which is shown in the log on the Home screen and this will substantially slow down any large imports. Nevertheless, this release should hopefully still be useful for exporting filtered data as it is now possible to export any table (e.g. player career history, club records, etc) - and this wasn't possible with Alpha-1.

Beyond the import/export, this release includes a large number of minor fixes and improvements as well as a few additional screens (e.g. the Database Tables and Flag Values - the latter of which will be editable in a future release).

KNOWN PERFORMANCE ISSUES

1. Mass editing players/non-players is slow (around 10,000 records per minute);
2. Exporting or filtering by certain fields on the Players & Non-Players screen is slow - e.g. filtering by clubs/leagues drafted by, club rights, etc - this is because the Editor generates these on the fly;
3. When importing data, certain fields take a while to cache (especially any fields relating to Staff names & DOBs - this is because the Editor generates a hashed list of staff to match against the spreadsheet;
4. Preparing data for csv export could be quicker.

Most of the above should be fixable using multi-threading which I'll look into at some point. The mass editing one is a bit more complex so might take more thought to improve.

HELP TESTING
It's probably a bit premature for any testing right now but there is one thing that I'd be really grateful for help with:

1. Open the Editor, click on File -> Open and select a copy of the Premier Pivot Rosters (any recent version should be fine);
2. Click on File -> Import;
3. At the bottom of the screen, click on the "Import Spreadsheet" button and select "staff_update_alpha2_test.xlsx" which is located in the EHM Editor v2 folder (it's included in the downloaded zip file). It'll take a short moment to import;
4. You'll probably encounter an error. It's fine if you do and you can just click on Abort;
5. Click on File -> Home and then locate the following in the log screen:

(A) Towards the top of the log, just below the initial detail shown when loading the database, you'll see a section that says this:




What I'm interested to see are the dates of birth which appear after "norair hovsepyan" (e.g. 27101994) and "mikkel aagaard" (e.g. 1995 10 18). I'm getting different date formats across my laptop which has Windows 11 and my desktop which has Windows 10. I'd be interested to see what other people are getting, especially those who use a system with a North American style date format (mm/dd/yyyy).


(B) Towards the end of the log, you'll see a section that says this:




The bit I'm interested in is the time it says for processing the spreadsheet (e.g. 8.07 seconds in the example above).

It would be great if you could post the following (I've completed it with my laptop's results):

CPU: Intel i7-7700HQ
OS: Windows 11
Usual Windows date format: dd/mm/yyyy
Norair Hovsepyan DOB: 27101994
Mikkel Aagaard DOB: 1995 10 18
Processing spreadsheet: 8.07 seconds

Any reports would be super helpful in narrowing down the date issue and getting an idea of import speed.

[archibalduk]

I've now added my Windows 10 desktop results to the below. It looks like a prior fix has now ensured that the date formats are consistent across the Windows 10 and 11. Now I just need to figure out why the two DOBs are in different formats as the Editor should be treating them the same (Norair's is ddmmyyyy whereas Mikkel's is yyyy mm dd). It's interesting how much quicker my desktop is at importing compared with my older laptop (8.07 secs v 2.3 secs) - hence why I'm interested to see what speeds are like on other CPUs such as an i3 or i5.

CPU: Intel i7-10700k
OS: Windows 10
Usual Windows date format: dd/mm/yyyy
Norair Hovsepyan DOB: 27101994
Mikkel Aagaard DOB: 1995 10 18
Processing spreadsheet: 2.3 seconds

CPU: Intel i7-7700HQ
OS: Windows 11
Usual Windows date format: dd/mm/yyyy
Norair Hovsepyan DOB: 27101994
Mikkel Aagaard DOB: 1995 10 18
Processing spreadsheet: 8.07 seconds

[xECK29x]

CPU: AMD Ryzen 9 3900X
OS: WIndows 11 (Insider Beta build)
Date format: MM/DD/YYYY
norair hovsepyan 27101994
Mikkel Aagaard: 1995 10 18
Processing Spreadsheet: 2.71 seconds

[archibalduk]

That's super helpful, thank you! Based on the dates being shown on your North American system, I believe I have since fixed the DOB mismatch issue.

[Spartacus70]

Why I cannot save my changes ?

[archibalduk]

I've been quiet in this thread for a while as I've been really busy IRL over the past few months. However, I have been making progress and have now finished implementing the ability to load and save the playable league structures in the database. This is a fairly big milestone as it is the most complex part of the database and involved a lot of work. The next big tasks on the playable leagues are to add all of the various functions which synchronise and validate the playable league records with the rest of the database and then to work on the editing screens - the latter is probably going to be the most complex and time-intensive aspect of coding the new Editor. There's a few other tasks I need to complete before I start tackling this aspect however (e.g. getting the ruling bodies, schedule templates, etc up and running in the Editor).

Why I cannot save my changes ?

Because this is still in beta and the ability to read/write the entire DB file is not yet implemented in full (as well as certain behind the scenes stuff which keeps records in sync when you make changes).

[archibalduk]

I'm still making slow but steady progress with the Editor. Essentially all of the editing screens other than the Playable Leagues (aka Rules & Structure) are up and running. There's still various mass editing to carry over from Editor v1 together with some behind the scenes work (e.g. auto-renumbering of UIDs and fixing a bug when renumbering IDs) but otherwise the Editor is in a good place.

I've improved general handling of files over the past few days. It is now possible to run multiple copies of the Editor at the same time without the risk of file conflicts/overwriting occurring. It is also possible to drag and drop database files and spreadsheets into the Editor rather than having to click on File -> Open etc. I've also added the ability to close a database file and then open a new file within the same editing session [edit: this is now up and running too].

[xECK29x]

I've improved general handling of files over the past few days. It is now possible to run multiple copies of the Editor at the same time without the risk of file conflicts/overwriting occurring.

This is huge! Is functional now wherein I can edit in v1 but have v2 open with no overwrite possiabilities?

[archibalduk]

So long as you close your Editor v2 session after your v1 session then it'll be fine. Editor v1 still has the limitation but it won't be impacted by v2 sessions so long as you stick to this order. Whilst I have tested this functionality, it's well worth you doing a quick test as well just to be sure.