When to stop working on a task
History / Edit / PDF / EPUB / BIB / 3 min read (~420 words)Given that you define a return on investment (ROI) on a task, when should you stop working on a task and abandon it given its cost?
Let's assume that your task has an estimate of effort (in hours) and an estimate of value (in dollars). The ROI is defined as the value divided by the effort ($/h).
Before starting any task, you should define what is the minimal ROI a task should have to be considered (i.e., your ROI threshold). If you value your time at 50 $/h and a task has an estimated ROI of 25 \$/h, then it is probably not worth your time to work on this task. Any task with a ROI superior to 50 $/h is considered worthwhile.
A task should be reviewed at half the estimated effort duration. If you estimated the task will take 8h, then at 4h you should reevaluate whether you think you will finish the task in the remaining 4 hours. If 8h still stands, then you can proceed. If you believe you can accomplish the task in less time, then update your effort estimate. This should by the same act increase its ROI. If on the other hand you think that the task requires more time, then update the effort estimate as well. If you have increased your effort estimate, then I suggest reevaluating the task at the midpoint of the remaining effort (e.g., an 8h task updated to 12h would be first evaluated at 4h, then at 8h).
Since you are updating your ROI estimate, you can also update your estimate of the value of the task you are working on. If it is a large task, maybe after having prototyped the idea you realize it will not provide as much value as anticipated. You can then review its value down. If on the other hand you get excited by what it brings to the table, by all means, increase its estimated value.
Once you have adjusted the estimated effort and value, take a look at your estimated ROI. If it has gone below your threshold, consider dropping it. Investing additional time in it may provide less value than working on a new, more valuable task. If on the other hand, the ROI is still superior to your threshold, then continue working on the task.
Determining a model usefulness
History / Edit / PDF / EPUB / BIB / 3 min read (~460 words)How do you determine whether you have a useful model?
In machine learning, one way you can determine that you have a useful model is to compare it against baseline models. In a field such as time series, one can create models that are based on previous values, such as lag 0, which predicts that the next value will be equal to the current value, or a moving average, which takes the X last values and averages them and returns this average as the next predicted value. In this field, we expect that a model that can predict more accurately than those baseline models may prove to be useful.
In other cases, we may already have an existing model from which we can generate predictions. This model may also serve as a baseline which other models will have to beat in order to replace it.
There is however a case where the answer isn't clear: what happens when out of various models, one of the baseline models is the best? Then it becomes a question of whether the prediction interval produced by the model satisfies your needs for your problem.
Note that even when you beat the baseline models, if the best model still does not satisfy the error requirements that have been defined on the metrics you care about, the model may still not be useful. For example, an OCR model that has 95% accuracy at the character level will still produce 5 errors every 100 characters, which may be too high for the requirements of the system to be produced.
Another factor to keep in mind when comparing models is whether the improvements on the metric you are to optimize for are significant. The mean absolute error (MAE) may be lower for a model compared to another, but if its confidence interval is larger compared to the other model and their intervals overlap, then you cannot claim that one model is really better than the other. There may even be cases where you will prefer a model with higher MAE simply because its confidence interval is smaller than the other model with lower MAE but larger confidence interval.
There might be other attributes of the model that you may need to take into account. A model that takes days to train may be useless when you need it to be up to date every hour. A faster to train but less accurate model may be more useful in this case.
- Can the model under evaluation beat baseline models?
- Does the model under evaluation satisfy error requirements?
- Is the model under evaluation significantly better than existing models?
- Does the model under evaluation satisfy requirements such as time to train (e.g., less than 6 hours) or necessary resources (cpu, gpu, ram)?
Reducing our size for space exploration
History / Edit / PDF / EPUB / BIB / 2 min read (~397 words)Is reducing our size a viable alternative/approach to space exploration?
I don't have enough knowledge of physics to determine all the impacts that scaling ourselves down would have. There are a few things I would consider, such as:
- the impact on being launched in space
- do we need less energy?
- are there other means to get us in space if we're smaller?
- our ability to accelerate/decelerate in space
- what kind of technology can we use?
- our ability to send communication signals back on earth
- given our small size, how can we produce enough power for the signal to be received?
- are there physical phenomenons that we need to take into account due to our reduced size?
When I asked myself this question, I was reading Accelerando, where a crew is sent far in space in a "can". I always perceived it as being the size of a soft drink can and I thought that was an interesting idea. In the book they are not shipping human beings, but rather a simulation of their consciousness. In other words, they're sending an AI in a can™. I've also thought of this as explaining why we've never been visited by aliens. Maybe one of the steps in evolution or progress is that we are able to scale ourselves down so that we are not as visible as we currently are. Imagine being an ant colony instead of the human society, our footprint would be a lot less noticeable from outer space. Furthermore, if you were to travel in a can, it would be a lot more difficult to detect you than if you were travelling in an extremely large ship.
The benefit I could see from being able to reduce the size of our satellites or vessels is that less mass is required to be turned into a spacecraft, which could allow us to create many more spacecrafts for the same amount of materials.
This question is also reminiscent of movies like Honey, I shrunk the Kids, Downsizing, Ant-Man or even a recent Rick & Morty episode. They explore the differences between full and miniature scales.
$$ \left| 0 \right> = \begin{pmatrix} 1\\ 0 \end{pmatrix}$$
$$ \left| 1 \right> = \begin{pmatrix} 0\\ 1 \end{pmatrix}$$
- Quantum computers only use reversible operations
- Identity and Negation are reversible
- Constant-0 and Constant-1 aare not reversible
$$ \begin{pmatrix} x_0\\x_1 \end{pmatrix} \otimes \begin{pmatrix} y_0\\y_1 \end{pmatrix} = \begin{pmatrix} x_0 \begin{pmatrix} y_0\\y_1 \end{pmatrix}\\x_1 \begin{pmatrix} y_0\\y_1 \end{pmatrix} \end{pmatrix} = \begin{pmatrix} x_0y_0\\ x_0y_1\\ x_1y_0\\ x_1y_1 \end{pmatrix}$$
$$ \begin{pmatrix} 1\\2 \end{pmatrix} \otimes \begin{pmatrix} 3\\4 \end{pmatrix} = \begin{pmatrix} 3\\ 4\\ 6\\ 8 \end{pmatrix}$$
- This tensored representation is called the product state
$$ \left| 00 \right> = \begin{pmatrix} 1\\ 0 \end{pmatrix} \otimes \begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 1\\ 0\\ 0\\ 0\\ \end{pmatrix}$$
$$ \left| 01 \right> = \begin{pmatrix} 1\\ 0 \end{pmatrix} \otimes \begin{pmatrix} 0\\ 1 \end{pmatrix} = \begin{pmatrix} 0\\ 1\\ 0\\ 0\\ \end{pmatrix}$$
$$ \left| 10 \right> = \begin{pmatrix} 0\\ 1 \end{pmatrix} \otimes \begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 0\\ 0\\ 1\\ 0\\ \end{pmatrix}$$
$$ \left| 11 \right> = \begin{pmatrix} 0\\ 1 \end{pmatrix} \otimes \begin{pmatrix} 0\\ 1 \end{pmatrix} = \begin{pmatrix} 0\\ 0\\ 0\\ 1\\ \end{pmatrix}$$
$$ \left| 4 \right> = \left| 100 \right> = \begin{pmatrix} 0\\ 1 \end{pmatrix} \otimes \begin{pmatrix} 1\\ 0 \end{pmatrix} \otimes \begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 0\\ 0\\ 0\\ 0\\ 1\\ 0\\ 0\\ 0\\ \end{pmatrix}$$
$$ C = \begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 1 & 0\\ \end{pmatrix}$$
$$ C\left| 10 \right> = C \begin{pmatrix} \begin{pmatrix} 0\\ 1\\ \end{pmatrix} \otimes \begin{pmatrix} 1\\ 0\\ \end{pmatrix} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 1 & 0\\ \end{pmatrix} \begin{pmatrix} 0\\ 0\\ 1\\ 0\\ \end{pmatrix} = \begin{pmatrix} 0\\ 0\\ 0\\ 1\\ \end{pmatrix} = \begin{pmatrix} 0\\ 1\\ \end{pmatrix} \otimes \begin{pmatrix} 0\\ 1\\ \end{pmatrix} = \left| 11 \right>$$
Run your program with python -m cProfile -o profile.cprofile my-script.py
Install snakeviz (pip install snakeviz) to visualize the generated profile.
snakeviz profile.cprofile
Alternative approach
Install pyprof2calltree to convert the cprofile to a kcachegrind compatible profile.
pyprof2calltree -i profile.cprofile -o callgrind.profile.cprofile