It probably doesn't really matter when you are removing and replacing just the filter, but if it was put in dry, into a new metal housing, I could definitely see it binding up. Because there will be oil in it in the future, there may never be an issue again.
However, I do as demarpaint says...