The traditional hypothesis testing I learned in school involves this:

1. Stating with null hypothesis, which assumes no effects, and
2. Alternative hypothesis, which state there is an effect.

So we can make the null hypothesis that “Earth is flat”, and the alternative hypothesis that Earth is not flat.

For humans, we can explore the physical world and design experiments to disapprove the null hypothesis. But for a AI, assuming these is no way for the AI to interact with the world through physics. How we can we truly test the hypothesis?

One way to go about this is find a general consensus amongst human knowledge. For example, using Google search result to see whether human agree on the hypothesis. This of course doesn’t mean that human consensus is the truth. In the early times, the consensus was that “Earth is flat”, and the consensus only changed recently if we consider the whole span of human consensus. It is very likely that the human consensus will change over time.

So what then? Do we believe that if AI cannot interact with the physical world, then it cannot prove or disapprove a hypothesis? Is there any alternative? Is there even an approximation? Is it possible that we can deduce from logic and evidence, that upon the collection of enough evidence and applying proper logic that we can disapprove the null hypothesis without physically interact with the world? Sometimes theoretical physics prove the existence of certain things before experiment verification.