I take a bath every day(it tells frequency, but it doesn't tell in which part of the day)Every morning tells both frequency and the part of the day.
I take a bath in the morning.(it tells the part of the day but not frequency).
I take a bath in the morning every day.(it tells both the part of the day and frequency)
I take a bath just after the dinner.(it tells only the time)I don't need to say the part of the day is a time too.
So, the sentence above tells both time and frequncy.
"I take a bath in the morning every day" would match.Because it tells both time and frequency.

"Every day" is good. "Every second" is also correct. So why would "every morning" be incorrect?
Let's think about context: we have an argument, a guy is about to loose his job. He wants to express and underline his devotion to his job - let's say he's working as consultant.
"I want to do my job good. So I'm always prepared. I always shave, I always iron my shirt. I take a bath every morning. Yes! Every morning, before I go to work."

No,lukaszpp, you misunderstood me. I have no objection for your sentence. It's correct.I wrote that explanation for CF's more natural sentences instead of your sentence.OK?
Take care.:)

Ok, thanks:)

A pleasure for me.