The simplest answer to your question - is to remember the point of supplements. Supplements are ingested to get additional calories/nutrients you are not getting from whole foods in accordance with your nutrition goals.
It is fine to take casein everyday. The people who do this, tend to be bodybuilders and/or those looking to gain weight. They take it before sleep, and this is a way for them to increase their caloric intake/daily protein total. Some people like to mix it in with their protein shake(s) - a combination of whey+casein, rather than just whey. Casein is a slow digesting protein. Some bodybuilders go so far as to claim that it is necessary to take it at night to prevent muscle breakdown. To me, that seems a bit extreme.
When I was looking to gain weight (and muscle), I ate cottage cheese OR took casein at night, before sleep. Both these foods were helpful in order to take in more calories, take in more protein and perhaps gain weight.
You shouldn't worry about consumption in the context of whether you went to the gym that day, but rather your daily nutrition (caloric intake AND distribution