I think a lot of atheists go through a process like the following. They apostatize from Christianity because the scientific evidence seems to indicate that it is false, and then they become an atheist. They still need a worldview (don’t we all?), so they latch on to contemporary science as a way of figuring out the nature of reality, knowledge, and morality.
I’m sympathetic to this form of atheism. After all, science is the most obviously successful knowledge gathering enterprise out there, so why wouldn’t we want to base all of our beliefs on contemporary science? Non-experimental methods of knowledge gathering must be a waste of time, or at best a source of aesthetic satisfaction.
However, science rests on a certain set of assumptions, like the assumption that there is an external world, that we are conscious and can gain knowledge, that falsifying experimental data is morally wrong, and so forth. These assumptions are not themselves based on experiments. They are perfectly reputable, of course, and in no way do they provide an opening for religion, but they are not based on controlled experimental inquiry.
Where do these assumptions come from? Presumably, at some point in the early modern era, a group of people noticed that conclusions that were based on controlled experimental inquiry were generally much more reliable than results that weren’t based on experiments (perhaps Francis Bacon was one of these people). They didn’t do an experiment to verify this, because there is no way to perform such an experiment. They just reflected on their experience with a bunch of different experimental and non-experimental generalizations and the relative utility that all of the generalizations in question turned out to have.
This sort of reflection is philosophical reflection. It is not based on experimentation, but it is based on powerful inductive arguments from life experience. Philosophical reflection can help tell scientists what areas of research they should look into next and what methodologies they should use. For example, biologists know from philosophical reflection that trying to prove Intelligent Design theory is not a useful way of spending their research time, because it violates a philosophical assumption of science, namely naturalism. Again, philosophical reflection on the nature of free will has helped researchers in psychology decide how to construct experiments investigating what laypeople believe about free will.
So, I think atheists with an interest in science should also look into philosophy and the philosophical assumptions of the things we do in science, as well as in everyday life. Philosophy is a valuable way of learning about the world in addition to science.