In this paper, we propose the method for recognizing the degree of snowfall automatically even if most of backgrounds are covered with snow and the visibility is low by fog. When it snows heavily, fog often occurs simultaneously. Moreover, falling snow grains have low contrast to the background covered with white snow. In order to deal with these situations, the proposed method makes an input image clear by fog removal. We propose the novel fog removal method which can be applied to not only the usual scene but also the heavy snowy scene. This method can remove the influence of fog without depending on the grade of visibility because the degree of fog removal is changed dynamically. The degree of snowfall is estimated from the quantity of falling snow grains, which are extracted from the difference between the present defogged image and the background image created by the three-dimensional median. Experiments conducted under various degrees of snowfall have shown the effectiveness of the proposed method.