machine-learning-course | Machine Learning and Deep Learning Course | Machine Learning library

While the exercise itself assumes 5000 observations, a much smaller number of observations is used to evaluate your code for submission.

You hardcoded the number 5000 in your code, rather than using a variable which captures the number of observations (presumably m in the code). Therefore in your for loop, once you've gone past 16, which presumably is the number of observations used in the submission context, octave complains that you're trying to access an index that is undefined for that particular array.

Long story short you should have used for i = 1:m rather than for i = 1:5000, to make your code generalise to sample sizes different to 5000.

If you recall from regularization, you do not regularize the bias coefficient. Not only do you set the gradient to zero when performing gradient descent but you do not include this in the cost function. You have a slight mistake where you are including this as part of the sum (see cell #18 on your notebook that you linked - the sum should start from j = 1 but you have it as j = 0). Therefore, you need to sum from the second element to the end for your theta, not the first. You can verify this on Page 9 of the ex2.pdf PDF assignment that is seen on your Github repo. This explains the inflated cost as you are including the bias unit as part of the regularization.

Therefore, when computing regularization in reg, index theta so that you start from the second element and onwards: