Question

#

Should I solve my problem with AI?

Answer

#

First start with a non-AI solution, then look into AI if what you have is unsatisfactory.

In this day and age, we want to make AI solve all kinds of problems, even those that don’t require AI.

When you build a product, you generally have a problem you are trying to solve. With AI, companies are looking at existing problems and trying to find ways to turn them into AI problems. It is not a problem in itself to attempt the exercise, but it is a mistake to implement an AI solution where a non-AI solution would’ve been more than adequate. There is still a lot of work that needs to be done in the field of automation that does not require AI, but simpler statistical approaches.

A lot of research in the field of machine learning and deep learning like to point to Occam’s razor in order to create simple models, but they sometimes seem to forget to apply the same principle to the whole solution, that is, do I need AI for this or would something simpler be as good?

A problem with the current wave of AI companies is their relation to AI itself. They corner themselves into doing AI only, while AI still highly relies on programming and IT, which are still highly technical but a lot less glamorous. It is definitely exciting to sell products that have AI in them, but starting with the tool and not the problem that needs to be solved is similar to trying to find all the problems a hammer can solve instead of knowing that you need a hammer when you want to nail something.

Question

#

Is there value in writing everything down?

Answer

#

I had this question when I started my research journey towards AGI. At one point I realized I wouldn’t be able to hold everything in my brain and that I needed to write down what I was thinking about so that I would be able to come back later, re-read what I wrote and continue making progress on what I was working on, whether it had been a day, a few weeks, or many months.

There is definite value in writing everything down. It frees your mind, letting you delegate the task of remembering to the computer. A thought I might have had more than five years ago can still be accessible to me if I wrote it down, but it’s unlikely I’ll remember exactly what I was thinking five years ago by “thinking really hard”. Nowadays I have trouble even remembering what I was working on a few hours prior due to the high volume of interruption I face in my work. Writing things down allows me to keep a record of what I’m doing.

One of the main concerns of writing everything down is that it can be seen as a waste of time. We often write and never read what we wrote again. As I write more and more, it becomes less and less likely that I will read what I’ve written in the past because the volume of content increases requiring more time to read. For the same amount of time spent reading, I cover the same constant amount of content, but the amount of uncovered content keeps growing, assuming I review prior content more than once.

I don’t view writing that is not read again to be wasteful. Sometimes writing down our thoughts allows us to figure out an issue that is blocking us. Sometimes expressing what we feel makes us realize that the feelings themselves are very hard to explain. Sometimes we just need a buffer where we can jot things down and get rid of it once we’re done.

Writing (and sharing) has the benefit that your thoughts can be reused by others. They can also be reused by your future self if you end up facing the same situation again.

References

#

• How often should I review my personal notes?
• Research methodology and organization

Question

#

What is the most complex sequence that can be made from a n-long binary string?

Answer

#

0….0 and 1….1 are easily compressible, thus what is the most unpredictable sequence?

Let’s proceed by induction.

1-long binary string

#

0 and 1 have a 50% to be selected in a 1-long binary string, thus they are both the most complex examples of a 1-long binary string.

2-long binary string

#

With two bits, we introduce two patterns: repeating and alternating. Repeating means that the bits are repeated for the complete string. Alternating means that the bits are alternating between 0 and 1 given a certain periodicity.

From our initial observation we suggested that the repeating pattern was likely to be the simplest because it consists of defining the symbol (0 or 1) that is repeated and the number of repetitions. We can say that this function has two parameters: length and symbol.

The same can be said about the alternating pattern. If you provide the initial symbol and the length of the pattern, you’ve defined its parameters. We can say that this function also has two parameters: length and starting symbol.

Using those two descriptions, we can observe that the alternating pattern is in fact a special case of the repeating pattern, where the symbol is 2 bits instead of one.

00 = repeating(2, 0) 01 = alternating(2, 0) = repeating(1, 01) 10 = alternating(2, 1) = repeating(1, 10) 11 = repeating(2, 1)

3-long binary string

#

001, 011, 100, 110 can be described as alternating and repeating, which is more complex than the previous examples as it requires the composition of two operations.

4 complex examples (001, 011, 100, 110) have been introduced at this stage.

4-long binary string

#

Here we first observe the presence of the alternating pattern with a periodicity of 2. We also observe the first case of a mirror pattern with 0110 and 1001. It could be claimed that 010 and 101 is the uneven equivalent of the mirror pattern for 3-long binary strings. The remaining “complex” examples are again displaying alternating and repeating patterns at the lower level.

One observation is that the complex patterns appear to be the variants where 1 bit is the opposite of the remaining bits, the only special case so far being for the 3-long binary string where the pattern is considered alternating. Instead of considering it a complex pattern, it could be described as the operation “fill X bits with 0/1, then flip by Y”.

From those few inductive iterations, we can see that the first and last string are always the repeating pattern. We also observe that the first half of the strings exhibit the same pattern of the second half of the strings, which is expected due to the binary nature of the string.

At this point there is still not enough information to be able to derive an answer to the original question, given that the strings can be described using 4 patterns: repeating, alternating, mirror, and fill & flip.

8 complex examples are found at this stage.

5-long binary string

#

The second half of this table was not written down due to the previous observation that the patterns are the same, simply inverted.

At this stage, the complex patterns that emerge are difficult to express using the prior language. Composition needs to be used, e.g. 00101 can be described as 00 then 101.

14 complex examples are found at this stage.

At this point in my study, I would summarize as follows:

• Repeating, alternating and fill & flip are low complexity. For a n-long binary string, this implies 2 repeating patterns, 2n fill & flip patterns, 2n alternating patterns of periodicity=1.
• Complexity appears to emerge from alternating patterns with varying periodicity. As we’re working with longer and longer strings, the periodic patterns will start to become periodic themselves (I believe this is leading to fractal).
• The fill & flip pattern appears to be able to cover most of the complex cases that appear as the length of the binary string increases.
• Some of the patterns can be expressed as a few alternatives, for example 01110 as alternating, periodicity=1,3,1, but also as 0 and fill & flip (or even an “odd” mirror). My intuition would lead me to think that those cases that can be expressed in multiple ways are less complex.
• It wasn’t done during this analysis, but it may be possible to apply operations overlayed on one another, such as two alternating patterns, one with a periodicity of 2 and one with a periodicity of 3, and use logic operators such as AND/OR/XOR (and their negative variants) to recreate the pattern
• The sequence of complex examples seems to be as follows: 0, 4, 8, 14, …
• It would be interesting to look at those strings from a logic circuit point of view. For example, 00000 can be produced from a single signal (0) routed to 5 exits, 00001 would be a single signal (0) routed to 4 exits and to one not gate which is then routed to an exit. Assuming the input signal is always 0, producing 11111 would require 1 not gate.

Question

#

What can cause us to learn inefficiently?

Answer

#

There are many reasons which can cause someone to learn at a slower pace than what should be possible for them. This can be caused by many factors, amongst them:

• Learning the same material under different wordings: You are going through material which you’ve already read, thus this “new” material is not providing you with any fresh information.
• Learning outdated material: You are learning material that has already been replaced with better material. It may be useful to learn outdated material for the historical purpose, but you should always aim to know what is the latest material.
• Learning incorrect material: You are learning material that is effectively not providing you with the information you want to learn. It may appear like you are learning something, but learning the wrong facts is a waste of your time.
• Learning from a source that makes the material difficult to understand: Learning efficiently is about learning from the best sources. If the material you have makes it difficult to understand what you are trying to learn, you should look for content elsewhere.
• Learning material for which we do not know the prerequisites: Learning a topic when we do not have the prerequisite knowledge will generally lead to confusion and make it difficult, if not impossible to learn.
• Learning from multiple sources with contradictory information: Learning from sources that disagree on facts makes it difficult to learn because you have to either accept that both answers are true, which can be complicated when they contradict each other, or that you need to establish which answer is the correct one, which requires additional effort.
• Learning while not being completely focused on the task: When learning, it is imperative to dedicate our focus to the task. Not doing so is likely to lead to information falling through the cracks of focus, leading to gaps in understanding, leading to requiring to go through the material again.

What is important to do while learning is to recognize when certain behaviors or circumstances make learning less efficient than desired and writing those behaviors or circumstances down so that you may have your own list of causes for learning less efficiently. As you discover those causes, make sure to put in place some contingency plans that will help you reduce their impact on your learning abilities.

Question

#

How often should I review my personal notes?

Answer

#

The simple answer is to do it as often as possible.

The more often you can go through your notes and review them, the more they will be fresh in your mind.

However, as you write more and more notes, it will be difficult to go through them all. At one point you will need to start prioritizing what you want to review and at what frequency. I see writing the same way I see programming: certain parts of the code work well and don’t need to be touched for a while, so they are left alone. However, sometimes certain parts have become neglected with time and need to be looked at again.

All my notes on this blog, as well as my private project and personal notes are committed to git automatically every time I switch out of VS Code. This means that I have a very granular history of the last time a file was touched. This allows me to look back at the list of files I have and to find the files that haven’t been touched in a while. Sometimes I’ll read them again, think about them a bit, then close them if I have nothing to add. Other times I’ll write about something that I’ve learned since then. Doing this feels a lot like improving some part of a codebase after you’ve learned how a piece of code would be used.

Another way you can schedule reviewing your notes would be with a spaced repetition system like Anki. It wouldn’t be too much work to have a system that automatically creates an Anki note/card with a link to the file itself. You would click on the link, which would open the file in your text editor of choice, read the note, change it if needed, then close it. You would then go back to Anki and pick between hard/good/easy to decide when is the next time you would review the note.

I’d suggest dedicating a small amount of your week going through what you wrote and refreshing your memory. It is definitely useful to simply write things to get them out of your mind and to attempt to express your thoughts clearly. It’s also useful to review what you thought about because your subconscious might have kept working on those thoughts well after you wrote them down and there might be more to contribute on a topic.

Question

#

What is the point of asking ourselves many questions when we only answer a few of them?

Answer

#

The purpose of asking yourself questions is to prompt your subconscious to look for the answer.

Asking yourself questions is a way to determine if you know or do not know the answer. If you don’t know the answer, then nothing will come on the top of your head quickly. On the other hand, if you know an acceptable answer, it will show itself rather quickly.

Asking yourself questions is a way to explore the domain of your knowledge and look for areas where you might need to do additional research.

If you ask yourself a lot of questions on a certain topic you’re studying and you don’t know the answers, then it can let you know that your knowledge is lacking and that you need to study more.

In some cases you might ask yourself questions that you don’t really want to know the answer to.

One of the main reason a lot of questions remain unanswered is because asking questions is easy, but answering is hard.

References

#

• Answering your own questions

Question

#

What is belief traceability?

Answer

#

In software development, is it very common to want to have traceability between things. We want to know from where a requirement came from, who created the entry, who approved that it should be supported by the software, the issues in the issue tracker that will implement the requirement, who implemented the requirement, who reviewed the implementation, etc. The goal is to be able to retell the story of this requirement from beginning to end if necessary.

In the case of belief traceability, we want to be able to trace how we’ve come to believe in something. Being able to trace back to the source of our beliefs allows us to determine if we still want to believe in them. Being able to also determine how a belief affects or generates other beliefs can be quite critical when we are re-evaluating our beliefs at some points in our lives.

Belief traceability starts with making a list of all the beliefs you have. Once you have made this list, you can try to determine if certain beliefs lead to other beliefs. You can also try to figure out if you have conflicting beliefs. You can attempt to associate beliefs with people, either because you think they are the ones that led you to believe something, or because they believe in that belief themselves. You may also have changed beliefs at certain points in your life. As such, it is interesting to create a timeline of your beliefs, where said beliefs are seen as being replaced or evolving over time.

As you get familiar with the process of tracking your beliefs, you will want to update your belief tracking system as you acquire new beliefs. This will allow you to reflect explicitly on what you believe and help you make a thorough analysis of those beliefs.

Question

#

What is learning according to machine learning?

Answer

#

It is (for supervised learning) looking at numerous samples, decomposing them into input variables and their associated target variable, and deriving according to an algorithm how to predict the target variable given input variables.

It is the (potentially lossy) compression of the observed samples, where the learning algorithm describes the compression/decompression algorithm. The compressed data is the information necessary for the algorithm to make predictions (decompression).

It is the creation of some “memory” of the observed samples. Whereas an untrained model has no memory of the dataset since it hasn’t seen the data, a trained model has some form of memory. A simple model such as sklearn’s DummyRegressor will learn and memorize the mean of the target variable. It may not have learned and memorized much, but it has built its internal model of the data.

It is to imitate as closely as possible the source of data it is trained on. This means that given input variables, it should produce target values that are as close as possible to those observed during training (learning).

References

#

• Learning

Question

#

How does one determine the best material to learn a topic?

Answer

#

My approach has always been based on the wisdom of the crowd.

If I want to read a book on a given topic, I will first look for the books that are available and create a list. From that list I will then look at websites like goodreads to gather people’s opinion on the book. I look for two factors: how many people read the book, and the overall rating of the book. The book should have been read by as many people as possible and have the highest rating. I will also read some of the low rating comments to get a sense of the negative feedback that has been provided on the book.

If I already have some knowledge about the topic I am studying, I might be looking for specific sub-topics to be covered. I would then inspect the table of contents of the books to determine if those sub-topics are covered. I may end up reading only a chapter of a book because it addresses something I’m interested in. The topic may also only be covered in this book.

In some cases, I will ask people that are more knowledgeable than I am to give me references. I use my knowledge of their expertise to determine the topics on which they can and cannot offer expertise and suggestions.

If there are multiple options of equally good books, then I will skim a few of them, maybe read a few passages to get a personal impression of the book. I prefer concise and non-repetitive content.

With my chosen book in hand, I will start to read through it from beginning to end. If at some point the content is less than ideal, I don’t mind skipping it and looking for additional sources (e.g., books or websites) to fill those gaps.

Question

#

How can one identify the areas he needs to improve?

Answer

#

Through introspection.

As we do things through our daily lives, there are events which we wished had unfolded better. As we realize those areas of weaknesses, it is important to write them down so that we can build a list of those areas we may want to spend time improving in the future.

Another way one can identify the areas he needs to improve is to use the Feynman technique.

1. Choose a concept
2. Pretend you are teaching it to someone else
3. Identify gaps in your explanation, go back and learn some more, then try explaining some more
4. Review and simplify

This technique can also be applied to the skills you lack. Instead of trying to teach someone else, you basically want to be able to explain the skill to someone else as accurately as possible. As you identify gaps in the description of the skill to perform, you are basically documenting what you need to work on.

For example, you might want to determine which one of your programming skills are lacking. You might start by asking yourself how you would design and implement things you use in your daily life: how is a text editor implemented? what happens when I type a URL and press enter in a browser? how are files read from a disk? how is this special effect rendered in a video game?

Make sure that you ask yourself questions that are relevant to areas you want to improve. If you don’t know how networking works, but it has no incidence on you, then you do not need to address this weakness. Spend time improving skills which will be useful to you in the future.

Look for things that create friction in your life, those are generally places where you’ll find potential for improvement.

Reference

#

• Feynman technique
  • https://fs.blog/2012/04/feynman-technique/
  • https://doist.com/blog/feynman-technique/