MonthJanuary 2011

Optimizing Sleeping Time

Many people make generalizations about the amount of sleep they need each day.  I hear many people say, “I only need six hours of sleep”, but there is more to it than just how many hours to sleep for.  Someone can sleep for 10 hours and still feel tired.  Some say that sleeping too much can cause someone to be tired.  The idea here isn’t only about how many hours to sleep for, it’s also about when to sleep.

I won’t try to quantify exactly how much sleep one needs since it differs for each person, but rather explain some of the factors and what people can do in order to optimize their sleeping pattern.

The REM Cycle

During sleep, the body goes through stages of light and deep sleep.  Generally speaking, these types are known as rapid eye movement (REM) sleep , and non-rapid eye movement (NREM) sleep.  The NREM type consists of three different stages, or cycles of sleep known as N1, N2, and N3.  The names of each type are not really important in the context of this article, but the body moves between these stages during sleep and each stage serves a different function.  The image below shows the body moving through the different phases of sleep over time.

On average, each sleep cycle last 90 to 110 minutes.  When one wakes up in between a sleep cycle, they will feel tired when they wake up.  A lot of times, the sleep cycle gets interrupted when the alarm clock rings in the morning.  But when one wakes up at the end of the cycle, they will feel very refreshed. 

The time to optimize here is plan the sleeping time in accordance with the sleeping cycle to ensure that one will wake up at the end of a sleep cycle.  Of course, this will take some experimentation to gage exactly how long each sleeping cycle is.

My method involves leaving some extra time to wake up.  I overestimate the time it takes me to fall asleep and the time it takes for one sleep cycle, I will wake up feeling refreshed before my alarm clock rings.  However, having too much extra time will cause the body to start its next cycle before waking up, but that depends on the person and their sleeping pattern.  Again, this takes experimentation.

Another issue is how long to sleep for.  To me, this doesn’t really matter as long one is able to wake up naturally.  This way, the body is able to communicate that it has had enough sleep.  Although, there are times where people wake up prematurely from sleep and feel that they need more sleep.  Another way would be to see if someone can get through the day without feeling tired or falling asleep.

Circadian Rhythms

Circadian rhythms are why we should not sleep whenever we want.  This is just a fancy term for explaining that each person’s body maintains the same pattern everyday.  Below is a picture from Wikipedia of what happens throughout the day for the average person.

A lot of these events don’t really have to do with sleep except for a few.  For example at 4:30 AM, our body should be at its lowest body temperature if the above pattern is followed.  Wikipedia states the following:

The optimal amount of sleep is not a meaningful concept unless the timing of that sleep is seen in relation to an individual’s circadian rhythms. A person’s major sleep episode is relatively inefficient and inadequate when it occurs at the "wrong" time of day; one should be asleep at least six hours before the lowest body temperature.  The timing is correct when the following two circadian markers occur after the middle of the sleep episode and before awakening:

  • maximum concentration of the hormone melatonin, and
  • minimum core body temperature.

Six hours before the lowest body temperature would mean that the optimal time to go to sleep for the average human is at 10:30 PM.  I experienced this first hand.  I used to sleep at 4 AM until 2 PM and always felt that my sleep felt super restless, but now that I sleep from 8:00 PM to 4:00 AM (I chose to sleep at this time because this way I get to talk to people in Vancouver while I’m abroad), I feel super refreshed waking and most of the time, I don’t need an alarm clock to wake me up.  I only need the clock tell me what time it is.


This is what I follow when I plan my sleeping pattern.  I used to have a really erratic sleep schedule, where I woke up early one day and late the next because I’d always wake up in time to go to my first class of the day.  If the start time differed everyday, my sleeping time would change too.  As a result, I felt tired everyday and tried to adjust my sleeping pattern.  After changing up my habits, I no longer feel the need to take naps everyday, and I don’t feel fatigue anymore.



An Awesome Day

Day 17

As usual, I eat breakfast every morning, but this morning’s breakfast was extra awesome.  Here is why:

The bacon looks a little raw, but I cooked it for too long so it became crunchy.

Then later, after trekking for about 20 minutes in the cold (-4 degrees Celsius), I bought this from the supermarket that actually sold some real Asian stuff:

Since coming here, I found no food (debatable whether this can be called food) from Asian brands that I’m familiar with.  But not anymore!  I know where to buy my cup noodles and Pocky substitute!  The Pocky is still from the same company in Japan, but for some reason it’s not called Pocky, although it looks the same.  I also like the fact that the labels are in German.  I know it’s supposed to be like that, but I’m not used to seeing the labels in English.

Numerical Methods for Non-Linear Optimization


  • Introduction
  • Motivation
  • Newton’s Method
  • Steepest Descent Method
  • Limitation
  • Sources


In many sciences, the problem of non-linear optimization appears quite often.  What happens if they’re non-linear?  Sometimes, the system of equations remains easy to solve even given non-linearity, but other times, the possibility of solving the problem analytically may not exist if the algebra gets too complicated.  The solution lies in using numerical methods (approximations).  Here, I present two common methods: Newton’s Method, and the Steepest Descent Method. 

Each method has its own limitations and the choice of which method to use depends on the problem.  These methods involve using iterative computation in order to reach a solution.  Luckily, computers can prove useful in providing solutions very quickly due to its quick processing power for numerical computation.

In order to understand some of the concepts and terms used in this article, the reader has knowledge in multivariable calculus.  Also, equations in picture form made the line spacing look funny, so I’m going to do my best write the equations as text.  I should also point out that I couldn’t write the vector arrow on top of the variables in text form so vectors are written in bold.


In calculus, the idea of optimization involves taking the derivative of the function and equating it to zero.  In multivariable calculus, the same idea applies except, we set all the partial derivatives to zero and solve the system.

So in general, if we are given a function:

ƒ(x), where x = [x1, x2, … xn]T

The maximum or minimum can be found by taking a partial derivative with respect to the vector x.  This gives the following equations:

ƒ/dx1 = 0

ƒ/dx2 = 0

ƒ/dxn = 0

A short-hand way of expressing the above equations is by using the gradient vector, giving∇ƒ(x) = 0.  By definition, the gradient is defined by:

∇ƒ(x) = [ƒ/dx1, ∂ƒ/dx2, … , ƒ/dxn].

An analytical solution is not always possible, hence the motivation for approximating the solution using numerical methods.

Newton’s Method

In general, this method is written below.  Explanation of each step will follow.

  1. Given the equations to solve, derive equations for the approximating line or plane, depending on whether the problem is 2D, 3D, nD, etc.
  2. Using the linearized equations, solve the system of equations
  3. Using the results from step 2, generate new equations for a new approximating line or plane, and repeat step 2.

This method also involves starting with a guess, denoted by a variable with a subscript of 0.  A guess is required, because this is the starting point for this method.  Using this guess, each successive solution should converge (more on that later).

Step 1

In order to solve the equations ∇ƒ(x) = 0 (see Motivation for more information), we can apply multivariate Newton’s method to approximate the solution.  Like the single-variate (two-dimensional) version, a linear approximation is required.  However, instead of having an approximation in the form of a line, the approximation is a plane instead.

2D (line): ƒ(x) ≈ ƒ(x0) + ƒ’(x0)(x – x0)

3D (plane): ƒ(x, y) ≈ ƒ(x0, y0) + ƒx(x0, y0)(x – x0) + ƒy(x0, y0)(y – y0)

nD (nD plane): ƒ(x) ≈ ƒ(x0) + ∇ƒ(x0)(xx0)

Since we are trying to optimize, we have the partial derivatives of the function we are trying to optimize, but in order to apply Newton’s Method, we need a linear approximation of the partial derivatives.  This means that in order to approximate the partial derivatives, we need to take another derivative in order to get the slopes of the approximating planes.  If that description didn’t make sense, try the following.  For an equation with two independent variables, there will be two equations for the partial derivatives, one for x and one for y.  Taking partial derivatives from each of those equations again will give the slopes of the plane that approximate the first partial derivative.  So if we have two independent variables, we end up getting four equations: ƒxx, ƒxyyy, and ƒyxxy and ƒyx should be the same).  In general, for n partial derivatives, we would have equations if we took another set of derivatives.  The approximations for both of the partial derivatives would be:

ƒx(x, y) ≈ ƒx(x0, y0) + ƒxx(x0, y0)(x – x0) + ƒxy(x0, y0)(y – y0)

ƒy(x, y) ≈ ƒy(x0, y0) + ƒyy(x0, y0)(x – x0) + ƒyx(x0, y0)(y – y0)

Plug in the values for the second derivatives, and the values from the initial guess (the x y values and what function value you get using those values).

Step 2

Although the equations generated from step 1 is yet another system of equations, the main difference is that the equations from step 1 are linear.  This means that the system can be solved analytically.  Solve for the solution of the system using row reduction methods, or other methods.  The solution from solving the system will be the values used for the next iteration and will replace the guessed values.  The solution may or may not be close to the actual solution.  More iterations need to be done to see if the answer changes with each iteration.  Usually if the guess is really far away, then it takes a lot of iterations to get it really close.  However, sometimes, a bad guess may cause the solution to diverge, meaning that the solution will be further and further off.

Step 3

Generate new equations for the approximating planes using the solution from step 2.  Using these equations, go back to step 2 and solve it again.  This method is super tedious if done on paper.  Matlab can help, but then again, Matlab can do the entire approximation with just a few commands, but not everyone has access to it unfortunately.

Hopefully after a few iterations, the solution stops changing.  When that happens, you can substitute the final solution back into the original equation to see if it works!

Steepest Descent Method

Here is the general method:

  1. Find the direction of the maximum or minimum directional derivative
  2. Follow it until a maximum or minimum is reached

Newton’s Method is sufficient in a lot of cases, but sometimes it may be too hard to go beyond first derivatives.  This main idea of this method is to use the first derivative and follow the direction in which the function gets larger/smaller.  A fundamental concept used in this method is the directional derivative.  Since in 3D (or higher dimensions), we are not restricted to movement along the x axis.  This means we could move in any random direction, and in each direction the rate of change is going to be different.  To calculate the max or min, we need to calculate where the directional derivative is largest or smallest.  Just like a boulder rolling down the steepest slope on the hill.

Starting with a guess, we can find out the value of the function at the point and the direction to travel in.  Take a step in that direction and see how the value of the function changed.  Keep doing that until a max or min is reached.

Step 1

The definition of the directional derivative is the dot product of the gradient and the unit vector specifying the direction.  Here is the definition of the directional derivative:

Duƒ = ∇ƒ(x)·u

Since the definition involves a dot product, it is true that if the two vectors are orthogonal, the resulting dot product is zero.  The direction that this derivative is in is called the level direction.  But if the two vectors are parallel, we get the highest value of the directional derivative.  For a vector that is parallel in the same direction, the directional derivative will be a maximum, but if the vector is antiparallel (parallel but both pointing opposite ways), the directional derivative will be a minimum.

Since the maximum value of the directional derivative is parallel to the gradient, we can calculate the unit vector of the gradient and use it as the unit vector u.  In order to get the minimum, simply flip the direction of the vector by multiplying it by –1.

u = ∇ƒ(x)/|∇ƒ(x)|

Although not necessarily required, substituting this value into the definition of the directional derivative, and doing some simplification with algebra, the maximum and minimum directional derivative is given by:

Max Duƒ = |∇ƒ(x)|

Min Duƒ = -|∇ƒ(x)|

Using the initial guess, calculate the direction of to travel in at that point.

Step 2

Knowing the direction to travel choose a desired step size “h”.  This means that if the equation will be traversed in a particular direction, we need to know exactly how far to go.  Choosing smaller step sizes gives better computational accuracy but increases computation time.

To get to the maximum, use the direction for the maximum directional derivative.  To get the minimum, use the direction for the smallest directional derivative.

Starting with a guess, calculate the function value at that point.  Then take a step in the direction given by Step 1 using the step size.  The variable “u” here is a component of the unit vector

xn+1 = xn + uxh

yn+1 = yn + uyh

At this new value, calculate the function value again.  Keep repeating this until the maximum or minimum is reached.


The approximation is generally only as good as the initial guess.  Sometimes, a bad guess can lead to a solution that never converges.  In that case, a different value must be used in order to find a converging solution.  For example, if we have a fourth order polynomial function, a local extreme will exist.  As x approaches –∞ or ∞, y will do the same.  If I use the steepest descent and give a bad guess, my solution will go to ∞ instead of the local extreme.


  1. Calculus by Gilbert Strang
  2. Applied Regression Analysis by Norman Draper and Harry Smith

uTorrent Feature: Prioritize by File Order

A feature that probably has been around that I just discovered on uTorrent: Prioritize by File Order.  Apparently if the files in a torrent have sequential names like episode numbers for seasons of tv shows, that feature will automatically set the priorities for the files with emphasis on the first couple of episodes so that they get finished first.  I used to mess with the priorities manually for that, but this feature makes it so much easier.  To get to it, click on the torrent that’s being downloaded and go to the files tab.  Select all the files, right click and hit Prioritize by File Order.

German Keyboards

This is the layout of a North America keyboard:

This is the layout of a German keyboard:

When I first saw this, I thought, “Oh cool, it supports the ä, ö, ü, ß and the euro characters!”, and then I realized, that because of it, many of the other keys are in different places.  The most annoying change of all, is the switching of the z and y characters.  During work, I kept typing my name wrong.  As a result, I changed it to the North American keyboard and all was well again.

© 2018 Henry Poon's Blog

Theme by Anders NorénUp ↑