K&N filters are not all bad. Alot of folks have had higher silica numbers in their oil due to dirt but as far as I can tell, the Holley would be about the same. I don't like their LS1 filter because it don't fit worth a dern. Seems they tried to make one filter fit two vehicles or they got some crappy engineers. You have to use a extra gasket to get it to seal to the airbox. I had to go out and source another gasket{weatherstripping} to get it sealed up. The Holley fits better and has a somewhat larger filter area.
If you are worried about the dirt, you could do a couple of things. Get paper one that would fit or find a thin loose open cell foam filter to use as a pre-filter for the K&N. You can find this stuff in the window ac area at your local hardware store. For instance, I found a open cell pre-filer that is used in Dry Vacs that fit the K&N cone filter I use on my Kia. I lightly oiled it with the K&N filter oil and slipped it over the filter. In your application, you could just lay the foam in the opening for the filter then put the filter in. Just make sure the prefilter fits well and there are no creases between the sealing surfaces for un-filtered air to go though.
BTW, if the filter is big enough the drop-in K&N wont really help power. I can swap the paper and the K&N at the track and I can't tell a difference in mph. In contrast,I can remove the filter completly and I consistantly see .5-1mph. I also had a K&N drop-in filter for a bike that actually flowed less air than a stock paper element.