Scientific discovery and AI

There might be a path forward for scientific discovery and AI. Copying from the now classic Abstraction and the acquisition of complex ideas

Interpreted as a rational reconstruction of the discovery process, the classical generalization idea fares no better. The laws of motion could not even in principle have been discovered by measuring a large number of instances and then looking for commonalities. If such a bizarre procedure were to be carried out, no commonalities would be found. Different objects do not, as a matter of empirical fact, have the same acceleration in free fall; even a single comparison between a stone and a feather should be sufficient to remind us of this obvious fact. On the Aristotelian view of higher-order knowledge, Newton’s celebrated law of gravitation is false. Interpreted as an empirical generalization, it contradicts experience. Nevertheless, the law is universally adopted throughout science. Its truth, if truth it has, must be of a different type than that championed by Aristotle and modern empiricists.

Effectively, a regime where you train using tons of data is not how humans get our concepts. Given what we can do with machine learning, a possible viable path forward would be to generate as may abstractions as possible a-priori (by thinking hard!), do some meta-learning to understand how these abstractions map to sensory output, and then go back via some compositionality search and map the abstractions to the real data.

A third alternative is to seek the origin of initial abstractions in discourse. People talk to each other (as well as to little children) about objects and events that are not in the listener’s current perceptual field (e.g., “the shovel is in the basement”, “your birthday is next week”). To understand such communications the listener must construct their meaning without support from current perceptual input. Perhaps, then, the ultimate origin of abstraction can be found in those mechanisms of language understanding that make it possible to understand out-of-context discourse.