Not totally erasing the testing validity, but very possibly changing the performance characteristics, positively or negatively. Additives respond differently in different base oils which can lead to surprises, some of which may not show up except in controlled engine tests. This is why the certifications are only valid in the specified base oils.
For example, changing from Group III to PAO can affect the solubility of some additives, and adding a polar base oil could compete with the effectiveness of polar additives. There is a lot of chemistry going on in an engine, influenced by fuel by-products, metallurgy, temperatures and loads. Bench tests attempt to isolate and measure specific properties under specific conditions such as wear or volatility or cleanliness, but a carefully controlled engine creates the total environment the oil will see and is better at catching interactions and formulation surprises. And surprises do happen, which is why there are millions of patents.
An experienced formulator will likely be able to predict the effects of changing a formulation, and be able to look for likely problems in appropriate bench tests, but there is still some risk and therefore still a matter of trust. The more experienced the formulator, the less the risk, but not zero. Even with certification the risk is not zero, but lower than not certified because the complete finished formulation has been through a variety of controlled and severe engine tests. Certification limits negative surprises, but also positive ones, so each individual needs to determine their personal balance.
I have altered formulations for my car and used oils altered by others because in those circumstances I knew enough about the changes to accept the risk. Without that knowledge I would have had to reassess my risk tolerance.