Let it go, you have "modern" oil in it, it can be pushed. If you wait 4 weeks you most likely will be up for it a lot more then you are now. Your bone will be set "enough" by then and you will be use to the cast. You will be looking for something to do about that time and just turn it into a 2 hour oil change, slow and rest the foot, slow and rest the foot.