Dr. Stephen Omohundro, founder and president of Self-Aware Systems, spoke about self-improving systems using the science of microeconomic theory. Using space, time, matter, and free energy resources, AI, like humans, will seek self-improvement and converge on rational agents, a small area of mind space.
While evolution lead to rational agents, it could only act through many generations and existing situations. A self-improving agent can not only adapt to existing situations but also think about situations it has not yet faced. Omohundro provided one possible definition of self-improving systems: devices for converting resources into expected utility.
How can these self-improving agents continue improving? Through efficiency gains, self-preservation, acquisition of resources, and creativity, any one of these can be considered the drive of self-improving agents.