I believe baking soda is still an acceptable substitute for toothpaste, as suggested in the article on the left. It would have been very difficult, I'd say, to get the inside of that baby bottle clean. I wonder if people simply used soda bottles, or if baby bottles were actually made with such a small mouth. When I was raising orphan lambs and baby goats, I'd buy a nipple similar to the one in the picture and put it on a soda bottle.
Then I did some Googling and found this Clorox ad online: