Take the cover off and see if you have a silver box about 3 inches wide and 4 or 5 inches long.
If it is an older furnace it uses a long rod that goes into the firebox. When it get heated, it turns and turns on the fan. Reason being is if the fan came on as soon as the fire started, you would get cold air coming out of the vents. It may be stuck and if you turn it slightly, it may take care of your problem. If it keeps doing it, you can try adjusting it.
Make sure if you replace it that you get one with the correct length shaft.
If it is an older furnace it uses a long rod that goes into the firebox. When it get heated, it turns and turns on the fan. Reason being is if the fan came on as soon as the fire started, you would get cold air coming out of the vents. It may be stuck and if you turn it slightly, it may take care of your problem. If it keeps doing it, you can try adjusting it.
Make sure if you replace it that you get one with the correct length shaft.