
Can this be interpreted literally?

According to Wikipedia, it's an English maxim, meaning "Don't get rid of something goos when discarding something bad," and it derives from an earlier German expression, "das Kind mit dem Bade ausschütten", a phrase from 'Narrenbeschwörung' (Appeal to Fools, 1512) by the satirist Thomas Murner. There is even an illustration admonishing people not to follow this unsavoury practice:
https://en.wikipedia.org/wiki/F...erung.kind.jpg

There are some expressions I understand idiomatically, and some that I only understand literally
This was one of the latter
I need to read more books :)