So I think I got it figured out by now, but I'd still like a second opinion before I make my parts list ready, so far this forum's been the most helpful when it came to all my issues.
So it sounds kind of like a worn alternator bearing, or wind whooshing by you. but it's not that. It's most noticeable at speeds from 20-60 km/h, and more at higher RPM. It also completely stops when I press the clutch, and you can hear a slight "clunk" as I release the clutch and it catches the gear.
So from what I have researched I think the culprit is a wearing throw out bearing on the clutch, but someone else told me it's a bearing on the transmission input shaft, and ANOTHER guy affirms my suspicion of a throw-out bearing.
I mean in the end it's not too important which of these it is since the entire transmission is junk and I'll replace it this spring or summer hopefully, and with it comes a brand new clutch since the trans is coming out anyway.