My path to AGI
Created: October 22, 2015 / Updated: February 6, 2021 / Status: in progress / 18 min read (~3490 words)
If you are interested in a more digestible path through the same content I've experienced, I suggest you look up my suggested path to AGI article. It basically contains most of the stuff I've gone through myself, but it is organized in a constructive manner to a new comer (at least, in my opinion).
If you have comments and suggestions, feel free to let me know through the comments!
Note: The dates in parenthesis indicate when I started and finished reading a given book/paper, not their publication date.
Stuart Russell, Peter Norvig
Only reading the content without doing any exercises is about 60-65h of reading.
Douglas Engelbart
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
YKY
Ray Kurzweil
Andrew Ng
Alan M. Turing, B. Jack Copeland
- On Computable Numbers, with an Application to the Entscheidungsproblem (2015-06 - ?)
- Intelligent Machinery (2015-10)
- Computing Machinery and Intelligence (2015-10)
- Can Digital Computer Think? (2015-10)
- Solvable and Unsolvable Problems (2015-10)
- Systems of Logic Based on Ordinals (2016-01 - ?)
- Can Automatic Calculating Machines Be Said to Think?
- Lecture on the Automated Computing Engine
- The Chemical Basis of Morphogenesis
- Chess
Tom Murphy VII
Paul R. Halmos
Claude E. Shannon
Matt Mahoney
Marcus Hutter
Shane Legg
Douglas R. Hofstadter
Marvin Minsky
John von Neumann
- Tensorflow 2016-01
See PHP-Brain
For about two weeks, I spent my time trying to find ways to do efficient string matching and document learning. The reason I wanted to do this was to attempt to learn language through examples. Provided with enough (hopefully valid) documents, it would be possible to assess if a given sentence is valid by matching it with previously seen sentences. It would also allow me to feed it a bunch of my chatlogs so that it may learn my vocabulary behavior and potentially predict what I would type next (basically autocompletion).
With a program that could do not much more than receive texts and match them against previously seen documents, I wanted to figure out a way to match generic data instead of words/sentences.
However, it wasn't only about matching generic data. It was also about being able to match data of different types with one another. In other words, if an image contained a sentence within one of its region, I'd like to be able to reuse that region to store the sentence as well as this region of the image. Thus, using only one run of the data I have two different informations: a sentence and a part of an image.
Furthermore, the idea is also to be able to relate concepts of varying degree of abstraction with one another.
In other to get an idea of how this could be accomplished, I started to reflect on the architecture of the human body and how the brain processes signal/information.
I began by designing a model-based brain, in other words, something that would look like a brain but from the inside would be something different. The main idea was to have a loop that would run forever, which would select tasks to execute. Input from the environment as well as actions to undertake (output) were considered to be tasks. The brain would have to process an input task, which would consist of a stream of data to be parsed and looked for patterns. An output task would be the emission of a stream of data (a stored pattern we expect to produce the desired result) to a target actuator. In other words, the brain would have to learn how to use its outputs through pattern emission, like it would try to learn to decipher the input streams it would receive.
After playing with the idea for a while, I came across an issue: what should I do when a task may take several seconds, even maybe minutes or hours? It doesn't make sense to stop processing everything else until said task is complete. That is not a real time approach. Thus, it meant we'd have to implement some sort of system to do task preemption. PHP being a single threaded language, this would prove to be more complicated than I wanted to deal with. Thus, I decided to move to a language that would support multithreading, C#.
See Sharp-Brain
The first thing I did when moving to C# was to build a complete body in an object oriented fashion. Body, head, brain, eyes, ears, nerves and so on. The head had eyes, which were attached to their own optic nerve, which would then connect to the brain. The brain was connected to a set of nerves, such as the optic nerve, which could be used either to receive or transmit data/signal.
Since I was now using a multithreaded language, I made each input a thread of its own. Thus, there would be 2 threads for the eyes, one per eye, 2 threads for the ears, one per ear, a thread for the brain, and so on. This would be the model of a human.
However, it was possible to generalize the idea to a robot. If we replaced the eyes with a single webcam, then the architecture would be similar in the sense that the eye, now a webcam, would take a frame and transmit it over the optic nerve to the brain. The brain would receive this array of bytes and process it (whatever that would mean, at this point no processing was done within the brain other than receive the data).
The problem at this point though was that the brain would now build its own network of understanding/patterns, which would make it more opaque and difficult to understand compared to a system of tasks.
Ben Goertzel, Cassio Pennachin
Pei Wang
Ben Goertzel
Peter Voss
Hugo de Garis
Patrick Hammer, Tony Lofthouse, Pei Wang
Jürgen Schmidhuber
Jürgen Schmidhuber
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis
Łukasz Kaiser
Vladimir G. Red'ko
Alex Graves, Greg Wayne, Ivo Danihelka
Keith A. Hoyes
José Luis Bermúdez
Breden Lake
Douglas Lenat
Douglas Lenat
Douglas Lenat
Ken Haase
G.D. Ritchie, F.K. Hanna
Allen Newell, John Clifford Shaw, Herbert A. Simon
Allen Newell, Herbert A. Simon
Douglas Hofstadter, Melanie Mitchell
J.R. Lucas
Began research related to automated programming. The main goal is to determine what is doable at the moment (based on current knowledge of the field).
Began research related to automated refactoring. The main goal is to determine a list of refactoring tools that can be developed to simplify the job of refactoring of programmers.
Alan Biermann
Francesco Zanoni
George Pólya
Allen Newell
Mark F. Bear, Barry W. Connors, Michael A. Paradiso
Patrick J. Hurley
Cordell Green, David Barstow
Peter Flach
Jeff Hawkins
John McCarthy
Tom Schaul, Dan Horgan, Karol Gregor, David Silver
Charles Rich, Richard C. Waters
Carl E. Hewitt, Brian Smith
David Marr
Began research related to automated research. The main goal is to determine a basic research flow and automate most of the process (researching papers, establishing important papers, determine a reading order, create a wordcloud of the most important terms, define which papers cite which papers, etc.).
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman
John von Neumann
Roman V. Yampolskiy
John Storrs Hall
John Storrs Hall
Richard Yonck
Masafumi Oizumi, Larissa Albantakis, Giulio Tononi
Tai-Hung Chen, Chun-Han Tseng, Chia-Ping Chen
Sam Abrahams, Danijar Hafner, Erik Erwitt, Ariel Scarpinelli
E. Mark Gold
J. Roland Olsson
Marcus Hutter
Marcus Hutter
Began research related to automated test generator. The main goal of this project is to use the knowledge I've acquired thinking about automated programming and refactoring and attempt to apply it to test generation. One of the goals is to develop a tool which will automate most of the process of testing PHP OOP code, specifically MVC-oriented websites using frameworks such as Laravel or Symfony.
My research into static analysis is related to my research on automated programming, automated refactoring and automated test generator. The main goal is to produce a piece of software that will be able to reason as a programmer would reason about a program in his head.
Patrice Godefroid, Peli de Halleux, Aditya Nori, Sriram Rajamani, Wolfram Schulte, Nikolai Tillmann, Michael Y. Levin
Paul Jorgensen
Thomas Fahringer, Bernhard Scholz
Flemming Nielson, Hanne R. Nielson, Chris Hankin
Toby Segaran, Colin Evans, Jamie Taylor
Ian Goodfellow, Yoshua Bengio, Aaron Courville
David Silver
Marek Rosa, Jan Feyereisl, The GoodAI Collective
I wrote a polymer/javascript-based application which dealt with triples in order to manage knowledge. Everything is considered to be a node and nodes can be associated to one another through anonymous.
See Knowledge base.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston
Joe Z. Tsien
Kun Xie, Grace E. Fox, Jun Liu, Cheng Lyu, Jason C. Lee, Hui Kuang, Stephanie Jacobs, Meng Li, Tianming Liu, Sen Song, Joe Z. Tsien
Lion Kimbro
J.R. Quinlan
Began experimenting with the problem of handwriting recognition. The main goal is to be able to feed a page through a camera/webcam/scanner and have its text content extracted.
Hugo Larochelle
Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, Pushmeet Kohli
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Alex Graves, Jürgen Schmidhuber
Diederik P. Kingma, Max Welling
Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, Rif A. Saurous
Kedar Tatwawadi
Théodore Bluche, Jérôme Louradour, Ronaldo Messina
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow
Barret Zoph, Quoc V. Le
Miltiadis Allamanis, Marc Brockschmidt
Bradly C. Stadie, Pieter Abbeel, Ilya Sutskever
Thomas M. Cover, Joy A. Thomas
Sercan O. Arik, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi
Sercan Arik, Gregory Diamos, Andrew Gibiansky, John Miller, Kainan Peng, Wei Ping, Jonathan Raiman, Yanqi Zhou
Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka, Tom M. Mitchell
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis
Vicent Wiegel
Alex Graves, Santiago Fernandez, Juergen Schmidhuber
Nal Kalchbrenner, Ivo Danihelka, Alex Graves
Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit
Ken Kansky, Tom Silver, David A. Mély, Mohamed Eldawy, Miguel Lázaro-Gredilla, Xinghua Lou, Nimrod Dorfman, Szymon Sidor, Scott Phoenix, Dileep George
David Silver, Hado van Hasselt, Matteo Hessel, Tom Schaul, Arthur Guez, Tim Harley, Gabriel Dulac-Arnold, David Reichert, Neil Rabinowitz, André Barreto, Thomas Degris
Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, Koray Kavukcuoglu
Manuel Lopes, Pierre-Yves Oudeyer
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, Yoshua Bengio
Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, Dario Amodei
Pedro Domingos
Brian Christian, Tom Griffiths
Misha Denil, Sergio Gomez Colmenarejo, Serkan Cabi, David Saxton, Nando de Freitas
David Sterratt, Bruce Graham, Andrew Gillies, David Willshaw
Chris Eliasmith
Scott Aaronson
Nick Bostrom
David Eagleman
Yoshua Bengio
Jakob N. Foerster, Richard Y. Chen, Maruan Al-Shedivat, Shimon Whiteson, Pieter Abbeel, Igor Mordatch
Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy P. Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, Rodney Tsing
Max Tegmark
John Johnston
Scott Gilbert
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas R Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy P. Lillicrap, Fan Xin Hui, Laurent Sifre, George van den Driessche, Thore Graepel, Demis Hassabis
Thomas Anthony, Zheng Tian, David Barber
Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Daniel Horgan, Bilal Piot, Mohammad Gheshlaghi Azar, David Silver
Carlos Florensa, David Held, Markus Wulfmeier, Pieter Abbeel
Yoshua Bengio
Wei Ping, Kainan Peng, Andrew Gibiansky, Sercan Ömer Arik, Ajay Kannan, Sharan Narang, Jonathan Raiman, John Miller
David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis
Levente Kocsis, Csaba Szepesvári
Ronen I. Brafman, Moshe Tennenholtz
Max Tegmark
Irving John Good
Yoav Freund, Robert E. Schapire
I have recorded a ton of audio of myself while I use my computer at home. I wanted to be able to find the slices of time where I speak and say something intelligible in order to extract those regions and then potentially send them to a service such as Google Voice to have it converted into text.
Burr Settles, Brendan Meeder
Hans Moravec
Shane Legg
Karl Friston
Yingfei Xiong, Jie Wang, Runfa Yan, Jiachen Zhang, Shi Han, Gang Huang, Lu Zhang
Satinder P. Singh, Andrew G. Barto, Nuttapong Chentanez
Xuan-Bach D. Le, David Lo, Claire Le Goues
Neil C. Rabinowitz, Frank Perbet, H. Francis Song, Chiyuan Zhang, S. M. Ali Eslami, Matthew M Botvinick
Jurgen Schmidhuber
Jurgen Schmidhuber