Richard Yonck - Toward a Standard Metric of Machine Intelligence (2012)

Context

We would like to have two things:

• A way to measure machine intelligence
• A way to measure intelligence in general (animals, humans, computers, etc.)

Learned in this study

Things to explore

• Are benchmarks (PCMark, 3DMark, PassMark) of any interest as intelligence metric? They do give us an idea of the performance of the machine, but do they give us any idea about the "potential" intelligence of the machine?
  • Is this type of metric applicable to human? (faster to reason? bigger memory?)
• Is it possible to measure intelligence as being the processing ability of a machine (animal, human, computer), measured in bits (similar to Shannon's theory of communication)
  • Intelligence = number of bits processed per second (does not evaluate quality/correctness of the answer)

Overview

• There is currently no way for us to evaluate if technology of 30 years ago is more intelligent than current technology
• What does it mean to have achieved the processing equivalent of a rat's cortical column or having emulated a cat's brain?
• One of the major flaws in many machine intelligence tests is their exclusive focus on identifying systems that achieve human-equivalence
• Suggested intelligence tests:
  • C-test: A series of abduction and prediction problems, similar to those in standard IQ tests. Questions are based on Kolmogorov complexity. Formatted and presented as a series of increasing complexity
  • Text compression
  • Universal Intelligence Test: The agent is challenged in a fully interactive environment using reinforcement learning methods, its actions generating observations and rewards. This agent works to maximize its reward
• Use a vector-based system where each intelligence factor is assigned a dimension

See also

References