More "faithful" transfers

[TheGag96]

First off, awesome project!! This is truly some amazing work from you guys.

That said, I'd like to bring up a concern I noticed looking at the page about how Pokemon are transferred. I see that you've chosen to act a lot like the 3DS Pokemon Bank in terms of how various attributes are calculated, using EXP for natures, randomizing a lot, etc. Although I think going this route is a very official-feeling move, it's perhaps not the ideal one. VGMoose has a great couple videos on this:

https://youtu.be/NwBbA8_KYSQ?si=tI-1XIlTlCpi_zoF
https://youtu.be/9ov9HrWOlH0?si=mgfPlr7P3gcs8CwM

He describes it as something like killing your dog and replacing him with a new dog that looks like the old one. Instead, even if it was ad-hoc, maybe a new system could be crafted that calculates attributes in a deterministic way that's more soulful. I would think this shouldn't have any legality concerns, since if random values were acceptable, then any particular deterministically-chosen one should be too, right?

Thanks for your hard work so far - please let me know what you think!

EDIT: Perhaps I spoke too quickly - looking again, I see that you guys improve on Pokemon Bank by taking DVs and PP Ups into account. Is there a reason EVs are reset? Are the other attributes treated more randomly because using DVs would tie these things too much to particular stat values?

[GearsProgress]

Thank you so much for the kind comments- I really appreciate it!!

I took a look at the videos and you definitely bring up a good point, these Pokemon should be as close to their Gen 1/2 counterparts as possible. Fortunately since that video released, GameFreak updated Bank's conversion method so that gender and shininess match gen 2's formula, so those to values will always be the same.

IVs are... tricky. Initially I had them based on the Pokemon's DV value. However, Gen 3's randomization formula means that certain IV, Nature, and Gender combinations are extremely rare and in some cases impossible. It also is possible to compare DV and IV values to determine if a Pokemon was legitimately created within Gen 3 or not (known as a PID IV Missmatch in PKHeX iirc). This means that, if I want to generate truly legal Pokemon, I need to conform to this. Now, technically it would be possible to find those very rare combinations, but since I have to follow the pseudo-random formula of Gen 3, it can take upwards of an hour to roll the correct combination of IVs, Natures, and Genders for all 30 Pokemon. I'd like to do more research into this to see if it's possible to reverse engineer, but as it stands right now it really isn't possible to have IVs un-randomized.

Stat Experience isn't as much of a problem, but the SE and EV systems are so different that it's almost impossible to convert one into the other. Moves also are a tricky part, since on one hand there's nothing stopping an illegal combination of mon/moves until the jump to Bank, but at the same time the goal is to make these Pokemon able to be sent all the way to HOME, and requiring move removal might be inconvenient for some people, depending on the game. If there's enough interest, I could definitely add it as an option in the future!

[TheGag96]

I see... And I'm sure the fact that you need to do these computations on the GBA's old CPU isn't doing you many favors either lol. Thanks for the response! I know this effort is in good hands.

I hardly know anything about the inner workings of PRNGs, let alone manipulating them like that, so I couldn't help there... At the very least, I'd love to test this project out when I get the chance.

[Aurareus]

Incredible project, the level of polish and attention to detail here is amazing, especially given all the very different pieces needed to put this puzzle together, super impressive.

Just wanted to say that, for me personally, finding a legit PID on the gen 3 side that matches everything as close to the gen 1/2 Pokemon as possible, including DVs to IVs, is the absolute priority and I'd be more than willing to wait multiple hours or however long it takes to maximize accuracy if that were offered as an option.
By converting DVs to IVs I mean x2 with 50% chance of +1, maybe even prioritize finding matching PIDs with +1 in 3 out of 6 stats to maximize how accurately the gen 3 IV spread represents the gen 1/2 DV spread, falling back to +1 for 4/2 out of 6 if none are found for 3, then 5/1 then 6/0, and if no matching PID exists at all for that combination of Nature, Gender, IVs, etc. then aim for one that is +/- 1 IV point away in any stat and so on.

Perhaps optional toggles for these various levels of accuracy with warnings about the amount of time they can add to the process.
Honestly any workaround, no matter what it entails, that allows me to fully maximize the accuracy of matching Pokemon to a legal counterpart in gen 3 that is as close a possible would make me happy.

Also, a toggle option to enable/disable removal of illegal moves would be greatly appreciated too.

[GearsProgress]

The biggest thing that's surprised me about releasing this program is how many people want the transfer process to be more accurate and less like how GameFreak did it for the Virtual Console- which is great!! I'll absolutely look into the IV calculation formula again to see how feasible it is (since in some cases it might be outright impossible to have 100% accurate IVs). Maintaining illegal movesets is also something I'm going to add an option for at some point, since a lot of people are interested in that (myself included haha)

[Deokishisu]

The biggest thing that's surprised me about releasing this program is how many people want the transfer process to be more accurate and less like how GameFreak did it for the Virtual Console- which is great!! I'll absolutely look into the IV calculation formula again to see how feasible it is (since in some cases it might be outright impossible to have 100% accurate IVs). Maintaining illegal movesets is also something I'm going to add an option for at some point, since a lot of people are interested in that (myself included haha)

I hope you're aware of the pret decompilations of the Gen 3 games. You can directly read the code and see how Pokemon are generated for each generation method without playing telephone through articles and such written years ago.

EDIT: I'm not sure how you set the met level, location, and game origin information (I haven't perused through the code yet), but if you set all hatchable Pokemon to a met level of 0 (indicating they were hatched) and set their met location to > 0xD6 && < 0xFD it will display as "met in a trade" in Gen 3's summary screen and be legal when Pal Parked up with Gen 4. The advantage to doing this is that there is much more freedom with the personality values and IVs for hatched Pokemon, as they can be generated on theoretically any frame and the IVs aren't 100% dependent on the state of the RNG when the personality was generated. Unhatchable Pokemon like Mewtwo would need to use the stationary generation method, but for the vast majority of Pokemon this could simplify generation. Though, the Pokemon will be technically illegal in Gen 3 because of its met location. Gen 4 discards the Gen 3 met location data, however, so it's not a huge deal.

[GearsProgress]

The biggest thing that's surprised me about releasing this program is how many people want the transfer process to be more accurate and less like how GameFreak did it for the Virtual Console- which is great!! I'll absolutely look into the IV calculation formula again to see how feasible it is (since in some cases it might be outright impossible to have 100% accurate IVs). Maintaining illegal movesets is also something I'm going to add an option for at some point, since a lot of people are interested in that (myself included haha)

I hope you're aware of the pret decompilations of the Gen 3 games. You can directly read the code and see how Pokemon are generated for each generation method without playing telephone through articles and such written years ago.

Oh of course!! Pret has made this process infinitely easier haha. Currently the IV generation code is the same as it is in the decomp, but that's because the Gen 3 games never have to generate specific IVs- that's when it becomes tricky due to the formula basically rolling random Pokémon until it fits the correct nature/gender/unown letter. Adding in the specific IVs causes it to be 31^6 times rarer to roll, even ignoring the nature, gender, and unown letter. I would need to find a way to reverse engineer the formula in order to make it feasible, although I'm not sure that's possible while maintaining PID/IV matching

[Deokishisu]

The biggest thing that's surprised me about releasing this program is how many people want the transfer process to be more accurate and less like how GameFreak did it for the Virtual Console- which is great!! I'll absolutely look into the IV calculation formula again to see how feasible it is (since in some cases it might be outright impossible to have 100% accurate IVs). Maintaining illegal movesets is also something I'm going to add an option for at some point, since a lot of people are interested in that (myself included haha)

I hope you're aware of the pret decompilations of the Gen 3 games. You can directly read the code and see how Pokemon are generated for each generation method without playing telephone through articles and such written years ago.

Oh of course!! Pret has made this process infinitely easier haha. Currently the IV generation code is the same as it is in the decomp, but that's because the Gen 3 games never have to generate specific IVs- that's when it becomes tricky due to the formula basically rolling random Pokémon until it fits the correct nature/gender/unown letter. Adding in the specific IVs causes it to be 31^6 times rarer to roll, even ignoring the nature, gender, and unown letter. I would need to find a way to reverse engineer the formula in order to make it feasible, although I'm not sure that's possible while maintaining PID/IV matching

There has been some talk of doing this in the pret Discord and how to make it efficient. You may want to search back or ask specifically in the #pokeemerald channel. Also, I edited my above post to add more thoughts. Just letting you know.

[GearsProgress]

I'll definitely take a look at that... and your edits are really interesting- that might just work! The legendary/unbreedable Pokemon already have some extra code iirc, so I can just modify that as well for those cases. Thanks for the suggestion, that's super helpful!!

[CharlieOwlie]

About finding a valid PID, I know that PKHeX can do it. At least the editor can generate legal PID for wild encounters and egg encounters after you provide the IVs, nature, ability and shininess. I'm not sure if they correspond to actual RNG-able PIDs, but at the very least they can pass the official legality checks. However the generation don't match the special method PIDs like legendaries and mythicals, so even if their code can run on GBA hardware you'll still have some work to do, but it could be a good place to start ?

[Lorenzooone]

To go from DV to IV while generating the correct PID (supports randomization):
https://github.com/Lorenzooone/Pokemon-Gen3-to-Gen-X/blob/main/include/pid_iv_tid.h
And the method making use of these: https://github.com/Lorenzooone/Pokemon-Gen3-to-Gen-X/blob/main/source/gen_converter.c#L694
Call init_unown_tsv when initializing the program.
Use generate_ot to generate the OT for the transfer Pokémon. It will output an OT which is guaranteed to work for all shiny Unowns, with any nature.

Worst case would be a box of 20 shiny Unowns (30 if the target is a Japanese game), taking 2 frames each to get the proper data. (Since Jirachi was not in Gen 1 or 2)

These methods were made with back and forth transfers in mind, so all the possible data is extracted. The idea is that a Pokémon converted like this to Gen 3 should be able to become the same Pokémon when taken back to Gen 2 (unless shinyness is involved, since the DV data is basically fixed in that case).

You will need to play with the headers and certain defaults a bit, but it will work.

[GearsProgress]

As always Lorenzooone, this is extremely helpful- thank you!! I'll work on implementing that as soon as I get the other languages working :)