I am with others, depends on the degree of difficulty. If just reaching your hand and removing a plug, then one at a time, then just as the go out. If a hard to reach or detailed removal, then just do them all.
Except for headlights, if one goes, the other gets changed as well (which should be done anyway since new lights are brighter than old).