Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use https for github pages and indeed dot com #23

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

use https for github pages and indeed dot com #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions _posts/2015-07-12-basic-python-network.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -641,11 +641,11 @@ var ind_q = 'Machine Learning';
var ind_l = '';
var ind_chnl = 'none';
var ind_n = 15;
var ind_d = 'http://www.indeed.com';
var ind_d = 'https://www.indeed.com';
var ind_t = 40;
var ind_c = 30;
</script>
<script type='text/javascript' src='http://www.indeed.com/ads/jobroll-widget-v3.js'></script>
<script type='text/javascript' src='https://www.indeed.com/ads/jobroll-widget-v3.js'></script>
<div id='indeed_widget_wrapper' style=''>
<div id='indeed_widget_header'>Machine Learning Jobs</div>

Expand All @@ -662,7 +662,7 @@ formInputs[i].value = '';
}
}
</script>
<form onsubmit='clearDefaults();' method='get' action='http://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<form onsubmit='clearDefaults();' method='get' action='https://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<div id="qc"><label>What:</label><input type='text' onfocus='this.value=""' value='title, keywords' name='q' id='q'></div>
<div id="lc"><label>Where:</label><input type='text' onfocus='this.value=""' value='city, state, or zip' name='l' id='l'></div>
<div id='indeed_search_footer'>
Expand All @@ -673,7 +673,7 @@ formInputs[i].value = '';
</div>

<div id='indeed_link'>
<a title="Job Search" href="http://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='http://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
<a title="Job Search" href="https://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='https://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
</a>
</div>
</div>
Expand Down
10 changes: 5 additions & 5 deletions _posts/2015-07-27-python-network-part2.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ for j in xrange(60000):

<h2 class="section-heading">Part 1: Optimization</h2>

<p>In <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/">Part 1</a>, I laid out the basis for backpropagation in a simple neural network. Backpropagation allowed us to measure how each weight in the network contributed to the overall error. This ultimately allowed us to change these weights using a different algorithm, <b>Gradient Descent</b>.
<p>In <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/">Part 1</a>, I laid out the basis for backpropagation in a simple neural network. Backpropagation allowed us to measure how each weight in the network contributed to the overall error. This ultimately allowed us to change these weights using a different algorithm, <b>Gradient Descent</b>.

<p>The takeaway here is that <b>backpropagation doesn't optimize</b>! It moves the error information from the end of the network to all the weights inside the network so that a different algorithm can optimize those weights to fit our data. We actually have a plethora of different <b>nonlinear optimization methods</b> that we could use with backpropagation:</p>
<p style="margin-left:40px">
Expand Down Expand Up @@ -896,11 +896,11 @@ var ind_q = 'Machine Learning';
var ind_l = '';
var ind_chnl = 'none';
var ind_n = 15;
var ind_d = 'http://www.indeed.com';
var ind_d = 'https://www.indeed.com';
var ind_t = 40;
var ind_c = 30;
</script>
<script type='text/javascript' src='http://www.indeed.com/ads/jobroll-widget-v3.js'></script>
<script type='text/javascript' src='https://www.indeed.com/ads/jobroll-widget-v3.js'></script>

<div id='indeed_widget_wrapper' style=''>
<div id='indeed_widget_header'>Machine Learning Jobs</div>
Expand All @@ -918,7 +918,7 @@ formInputs[i].value = '';
}
}
</script>
<form onsubmit='clearDefaults();' method='get' action='http://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<form onsubmit='clearDefaults();' method='get' action='https://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<div id="qc"><label>What:</label><input type='text' onfocus='this.value=""' value='title, keywords' name='q' id='q'></div>
<div id="lc"><label>Where:</label><input type='text' onfocus='this.value=""' value='city, state, or zip' name='l' id='l'></div>
<div id='indeed_search_footer'>
Expand All @@ -929,7 +929,7 @@ formInputs[i].value = '';
</div>

<div id='indeed_link'>
<a title="Job Search" href="http://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='http://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
<a title="Job Search" href="https://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='https://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
</a>
</div>
</div>
Expand Down
12 changes: 6 additions & 6 deletions _posts/2015-07-28-dropout.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ for j in xrange(60000):

<h2 class="section-heading">Part 1: What is Dropout?</h2>

<p>As discovered in the <a href="http://iamtrask.github.io/2015/07/27/python-network-part2/">previous post</a>, a neural network is a glorified search problem. Each node in the neural network is searching for correlation between the input data and the correct output data.</p>
<p>As discovered in the <a href="https://iamtrask.github.io/2015/07/27/python-network-part2/">previous post</a>, a neural network is a glorified search problem. Each node in the neural network is searching for correlation between the input data and the correct output data.</p>

<img class="img-responsive" width="100%" src="{{ site.baseurl }}/img/sgd_randomness_ensemble.png" alt="">

Expand Down Expand Up @@ -63,7 +63,7 @@ Consider the graphic above from the previous post. The line represents the error
<!--
<h3 class="section-heading">Part 3: Why does Dropout Work?</h3>

<i> (This section builds upon concepts laid out in the <a href="http://iamtrask.github.io/2015/07/27/python-network-part2/">previous post</a>.)</i>
<i> (This section builds upon concepts laid out in the <a href="https://iamtrask.github.io/2015/07/27/python-network-part2/">previous post</a>.)</i>

<p>Imagine that you had two identical values in your hidden layer. By "idential", I mean that their input weights were the same, so they turned on and off at exactly the same time. Given what we learned in Section 1 of this post, we know that this is a very real possibility.</p>

Expand Down Expand Up @@ -108,11 +108,11 @@ var ind_q = 'Machine Learning';
var ind_l = '';
var ind_chnl = 'none';
var ind_n = 15;
var ind_d = 'http://www.indeed.com';
var ind_d = 'https://www.indeed.com';
var ind_t = 40;
var ind_c = 30;
</script>
<script type='text/javascript' src='http://www.indeed.com/ads/jobroll-widget-v3.js'></script>
<script type='text/javascript' src='https://www.indeed.com/ads/jobroll-widget-v3.js'></script>

<div id='indeed_widget_wrapper' style=''>
<div id='indeed_widget_header'>Machine Learning Jobs</div>
Expand All @@ -130,7 +130,7 @@ formInputs[i].value = '';
}
}
</script>
<form onsubmit='clearDefaults();' method='get' action='http://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<form onsubmit='clearDefaults();' method='get' action='https://www.indeed.com/jobs' id='indeed_jobform' target="_new">
<div id="qc"><label>What:</label><input type='text' onfocus='this.value=""' value='title, keywords' name='q' id='q'></div>
<div id="lc"><label>Where:</label><input type='text' onfocus='this.value=""' value='city, state, or zip' name='l' id='l'></div>
<div id='indeed_search_footer'>
Expand All @@ -141,7 +141,7 @@ formInputs[i].value = '';
</div>

<div id='indeed_link'>
<a title="Job Search" href="http://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='http://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
<a title="Job Search" href="https://www.indeed.com/" target="_new">jobs by <img alt="Indeed" src='https://www.indeed.com/p/jobsearch.gif' style='border: 0;vertical-align: bottom;'>
</a>
</div>
</div>
Expand Down
6 changes: 3 additions & 3 deletions _posts/2015-08-07-anyone-can-code-lstm.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ Why the hidden layer? Well, we could technically do this.

<h2 class="section-heading">Part 2: RNN - Neural Network Memory</h2>

<p>Now that we have the intuition, let's dive down a layer (ba dum bump...). As described in the <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/"><u>backpropagation post</u></a>, our input layer to the neural network is determined by our input dataset. Each row of input data is used to generate the hidden layer (via forward propagation). Each hidden layer is then used to populate the output layer (assuming only 1 hidden layer). As we just saw, memory means that the hidden layer is a combination of the input data and the previous hidden layer. How is this done? Well, much like every other propagation in neural networks, it's done with a matrix. This matrix defines the relationship between the previous hidden layer and the current one.</p>
<p>Now that we have the intuition, let's dive down a layer (ba dum bump...). As described in the <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/"><u>backpropagation post</u></a>, our input layer to the neural network is determined by our input dataset. Each row of input data is used to generate the hidden layer (via forward propagation). Each hidden layer is then used to populate the output layer (assuming only 1 hidden layer). As we just saw, memory means that the hidden layer is a combination of the input data and the previous hidden layer. How is this done? Well, much like every other propagation in neural networks, it's done with a matrix. This matrix defines the relationship between the previous hidden layer and the current one.</p>

<img class="img-responsive" width="100%" src="{{ site.baseurl }}/img/basic_recurrence_singleton.png" alt="">

Expand Down Expand Up @@ -304,7 +304,7 @@ Why the hidden layer? Well, we could technically do this.

<p><b>Lines 0-2:</b> Importing our dependencies and seeding the random number generator. We will only use numpy and copy. Numpy is for matrix algebra. Copy is to copy things.</p>

<p><b>Lines 4-11:</b> Our nonlinearity and derivative. For details, please read this <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/"> Neural Network Tutorial</a></p>
<p><b>Lines 4-11:</b> Our nonlinearity and derivative. For details, please read this <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/"> Neural Network Tutorial</a></p>

<p><b>Line 15:</b> We're going to create a lookup table that maps from an integer to its binary representation. The binary representations will be our input and output data for each math problem we try to get the network to solve. This lookup table will be very helpful in converting from integers to bit strings.</p>

Expand Down Expand Up @@ -382,7 +382,7 @@ Why the hidden layer? Well, we could technically do this.
<p><b>Line 94:</b> Selecting the previous hidden layer from the list</p>
<p><b>Line 97:</b> Selecting the current output error from the list</p>
<p><b>Line 99:</b> this computes the current hidden layer error given the error at the hidden layer from the future and the error at the current output layer.</p>
<p><b>Line 102-104:</b> Now that we have the derivatives backpropagated at this current time step, we can construct our weight updates (but not actually update the weights just yet). We don't actually update our weight matrices until after we've fully backpropagated everything. Why? Well, we use the weight matrices for the backpropagation. Thus, we don't want to go changing them yet until the actual backprop is done. See the <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/">backprop blog post</a> for more details.</p>
<p><b>Line 102-104:</b> Now that we have the derivatives backpropagated at this current time step, we can construct our weight updates (but not actually update the weights just yet). We don't actually update our weight matrices until after we've fully backpropagated everything. Why? Well, we use the weight matrices for the backpropagation. Thus, we don't want to go changing them yet until the actual backprop is done. See the <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/">backprop blog post</a> for more details.</p>

<p><b>Line 109 - 115</b> Now that we've backpropped everything and created our weight updates. It's time to update our weights (and empty the update variables).</p>

Expand Down
6 changes: 3 additions & 3 deletions _posts/2016-02-25-deepminds-neural-stack-machine.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ A close eye might ask, "We learn things with neural networks. What is there to l

<center><p><b>How will a neural network learn when to push and pop?</b></p></center>

<p>A neural network will learn to push and pop using backpropgation. Certainly a pre-requisite to this blogpost is an intuitive understanding of neural networks and backpropagation in general. Everything in <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/"> this blogpost </a>will be enough.</p>
<p>A neural network will learn to push and pop using backpropgation. Certainly a pre-requisite to this blogpost is an intuitive understanding of neural networks and backpropagation in general. Everything in <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/"> this blogpost </a>will be enough.</p>

<p>So, how will a neural network learn when to push and pop? To answer this question, we need to understand what a "correct sequence" of pushing and popping would look like? And that's right... it's a "sequence" of pushing and popping. So, that means that our <b>input data</b> and our correct <b>output data</b> will both be sequences. So, what kinds of sequences are stacks good at modeling?</p>

Expand Down Expand Up @@ -362,7 +362,7 @@ def r_t(t):

<h3>Let's Revisit How The Neural Stack Will Learn</h3>

<p>In Part 2, we discussed how the neural stack is unique in that we can backpropgate error from the output of the stack back through to the input. The reason we can do this is that the stack is fully differentiable. (For background on determining whether a function is differentiable, please see <a href="https://www.youtube.com/watch?v=pj1VvscsV0s"> Khan Academy </a>. For more on derivatives and differentiability, see the rest of that tutorial.) Why do we care that the stack (as a function) is differentiable? Well, we used the "derivative" of the function to move the error around (more specifically... to backpropagate). For more on this, please see the <a href="http://iamtrask.github.io/2015/07/12/basic-python-network/">Tutorial I Wrote on Basic Neural Networks</a>, <a href="https://iamtrask.github.io/2015/07/27/python-network-part2/"> Gradient Descent</a>, and <a href="https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/">Recurrent Neural Networks</a>. I particularly recommend the last one because it demontrates backpropgating through somewhat more arbitrary vector operations... kindof like what we're going to do here. :)</p>
<p>In Part 2, we discussed how the neural stack is unique in that we can backpropgate error from the output of the stack back through to the input. The reason we can do this is that the stack is fully differentiable. (For background on determining whether a function is differentiable, please see <a href="https://www.youtube.com/watch?v=pj1VvscsV0s"> Khan Academy </a>. For more on derivatives and differentiability, see the rest of that tutorial.) Why do we care that the stack (as a function) is differentiable? Well, we used the "derivative" of the function to move the error around (more specifically... to backpropagate). For more on this, please see the <a href="https://iamtrask.github.io/2015/07/12/basic-python-network/">Tutorial I Wrote on Basic Neural Networks</a>, <a href="https://iamtrask.github.io/2015/07/27/python-network-part2/"> Gradient Descent</a>, and <a href="https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/">Recurrent Neural Networks</a>. I particularly recommend the last one because it demontrates backpropgating through somewhat more arbitrary vector operations... kindof like what we're going to do here. :)</p>

<p>Perhaps you might say, "Hey Andrew, pre-requisite information is all nice and good... but I'd like a little bit more intuition on why this stack is differentiable". Let me try to simplify most of it to a few easy to use rules. Backpropagation is really about credit attribution. It's about the neural network saying "ok... I messed up. I was supposed to predict 1.0 but i predicted 0.9. What parts of the function caused me to do this? What parts can I change to better predict 1.0 next time?". Consider this problem.</p>

Expand Down Expand Up @@ -492,7 +492,7 @@ u_delta = list() # pop strengths
<iframe width="700px" height="440px" src="https://www.youtube.com/embed/J6Zaugshvqs" frameborder="0" allowfullscreen></iframe>
<br />

<p>We did it! At this point, we have built a neural stack and all of its components. However, there's more to do to get it to learn arbitrary sequences. So, for a little while we're going to return to some of our fundamentals of neural networks. The next phase is to control v_t, u_t, and d_t with an external neural network called a "Controller". This network will be a Recurrent Neural Network (because we're still modeling sequences). Thus, the knowledge of RNNs contained in the previous blogpost on <a href="http://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/">Recurrent Neural Networks</a> will be considered a pre-requisite.</p>
<p>We did it! At this point, we have built a neural stack and all of its components. However, there's more to do to get it to learn arbitrary sequences. So, for a little while we're going to return to some of our fundamentals of neural networks. The next phase is to control v_t, u_t, and d_t with an external neural network called a "Controller". This network will be a Recurrent Neural Network (because we're still modeling sequences). Thus, the knowledge of RNNs contained in the previous blogpost on <a href="https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/">Recurrent Neural Networks</a> will be considered a pre-requisite.</p>

<p>To determine what kind of neural network we will use to control these various operations, let's take a look back at the formulas describing it in the paper.</p>

Expand Down
Loading