Deep Learning with Python, TensorFlow, and Keras tutorial

Deep Learning with Python, TensorFlow, and Keras tutorial

November 16, 2019 100 By Stanley Isaacs


What is going on everybody and welcome to a much-needed Update to the deep learning and Python with tensorflow as well as now chaos tutorial it’s been a bit over two years since I did just a basic deep learning video in Python and Since then a lot has changed. It’s now much simpler to both like get into it But then also just to work with deep learning models So if you want to get into the more nitty gritty details in the lower-level Tensorflow code you can still check out the older video But if you’re just trying to get started with deep learning that’s not necessary anymore because we have these nice high-level api’s like chaos that sit on top of tensorflow and Make it super super simple. So anybody can follow along if you don’t know anything about deep learning that’s totally fine We’re going to do a quick run-through of neural networks. Also, you’re gonna want Python 3.6 at least as of the release of this video hopefully very very soon Tensorflow will be supported on three seven and later versions of Python just happens to be the case right now it isn’t I think it’s something to do with the async Changes, I’m not really sure anyways Let’s get into it starting with an overview of how neural networks just work Alright to begin we need to have some sort of balance between treating neural networks like a total black box that we just don’t understand at all and Understanding every single detail to them. So I’m gonna show you guys what I think is just the kind of bare essential to understanding What’s going on? So a neural network is going to consist of the following things. Like what’s the goal of any machine learning model? Well, you’ve got some input So let’s say X 1 X 2 X 3 and you’re just trying to map those inputs to some sort of output Let’s say that output is determining whether something is a dog or that something is a cat So the output is going to be two neurons in this case. So it’s just boom two neurons Now what we want to do is is figure out how are we going to map to those things? We could use a single hidden layer. Let’s say we’re going to do some neurons here and That’s our first Hidden lair now What’s gonna happen is each of these X 1 X 2 and X 3 these are gonna map to that hidden lair each of the input data x’ gets Connected to each of the neurons in that first hidden layer. And each of those connections has its own Unique weight now from here that first hidden layer could then map and connect to that output layer the problem is if you did this the relationship between x1 and dog or cat and All the other ones those relationships would only be linear relationships so if we’re looking to map and nonlinear relationships Which is probably going to be the case in a complex question. You need to have two or more one Hidden layer means you just have a neural network two or more hidden layers means you have a quote-unquote deep neural network So we’ll add one more layer and then we’re gonna fully connect that one two And then once that’s fully connected again all unique weights, each of those blue lines has a unique weight associated with it and then that gets mapped to The output and again each blue line has a unique weight associated with it so now what we’re gonna do is talk about what’s happening on an individual Neuron level. So again that neuron has certain inputs coming to it It might be the input layer X values or it could be inputs coming from the other neurons So we’re gonna again we’re gonna call the inputs x1 x2 and x3 But just keep in mind this could actually not be it might not be your input data It might be data coming from another neuron But regardless that data’s gonna come in and we’re just gonna get the sum of that data So it’s gonna come in and be summed all together But remember we also have those weights each of the inputs has a unique weight that gets put you know multiplied Against the input data and then we sum it together finally and this is kind of where the artificial neural network comes into play we have an activation function and this activation function is kind of meant to Simulate a neuron actually firing or not So you can think of the activation function like on a graph, you know? You got your X and your Y and then a really basic activation function would be like a step or function So if X is than a certain value boom we step up and we have a value. So let’s say here This is zero here. The value is one So let’s say this is our x-axis 1 2 3 so if X, you know after being all the inputs are multiplied by their weights sum together if that value is let’s say greater than 3 well, then this activation function returns a 1 but today we tend to use more of a sigmoid activation function so it’s not going to be a 0 or 1 it’s going to be some sort of value between 0 and a 1 so instead we might actually return like a point seven nine or something like that So coming back to this neural network here that we’ve been drawing Let’s say here on this final output layer. You’ve got dog and cat well, this output layer is almost certain to have just a sigmoid activation function and What’s gonna say is maybe dog is a point seven nine and cat is a point two one these two values are gonna add up to a perfect 1.0 but we’re gonna go with whatever the Largest value is so in this case The neural network is you could say 79 percent confident that it’s a dog 21 percent confidence a cat We’re gonna say we’re gonna take the Arg max basically and we’re gonna say hmm. We think it’s a dog All right. Now that we’re all experts on the concepts of neural networks. Let’s go ahead and build one. You’re gonna need tensorflow So do a pip install – – upgrade tensorflow you should be on tensorflow version 1.1 or greater. So one thing you can do is import tensorflow and then Actually touch flow as TF and then TF dot version will give you your current version so mine is 1.10 Now let’s go ahead and get started. So the first thing we’re going to do is import a data set. We’re going to use em nacelle of Data sets with machine learning. It is a dataset that consists of 28 by 28 sized images So it’s like the resolution images of handwritten Digits 0 through 9. So it’ll be like a 0 1 2 3 and so on and it’s a handwritten kind of unique image so it’s actually a Picture we can graph it soon enough so you can see it’s actually an image and the idea is to feed through the pixel values to the neural network and Then have the neural network output Which number it actually thinks that image is So that’s our data set, and now what we want to do is Unpack that data set to training and testing variables So this is a far more complex Operation when it’s actually a data set that you’re kind of bringing in or that you built or whatever For the sake of this tutorial we want to use something real basic like M inist so we’re gonna unpack it to X train Y train and then we’re going to do X test Y test and that’s gonna equal m n– Astana score data, so that’s gonna unpack it into there now Just to show you guys what this is We’re gonna use Matt plot Lib you can pip install or just look at it with me, but we’re gonna import matplotlib Pipe lot as a PLT. And what we’re gonna do is peel TM show and we’re gonna do X train And we’ll do the zero width So one thing we could do just just for the record Let me just print so we can you can see what we’re talking about here. So this is just going to be an array It’ll be a multi-dimensional array which is all a tensor is by the way So this is this is here’s your tensor right Okay, so that’s the the actual data that we’re gonna attempt to pass through our neural network and just to show you if we were To actually graph it and then do a peel t touch show. It’s gonna be the number and you can just excuse the color It’s definitely black and white. It’s a single color. It’s a binary So one thing we could say is the color map is equal to P LTCM for color map Binary Reap lot it and there you go. It’s a it’s not a color image So anyways back to our actual code up here Once we have the data one thing we want to do is is normalize that data so again, if I print it out, you can see it’s data that seems to vary from 0 to Looks like we have as high as 253. It’s 0 to 255 4 pixel data So what we want to do is scale this data or normalize it but really what we’re doing in this normalization is scaling it So we’re going to just redefine X train and X testing but it’s gonna be TF caras dot utils dot Normalize and we’re gonna pass X Train and it’ll be on axis 1 and then we’re gonna copy paste and we’re gonna do the exact same thing for X test and All this does let’s just run that and then we’ll run this again and you can see how the 5 has changed a little bit looks like I got a little lighter and Then we come down here and we can see the values here are now Scaled between 0 and 1 and that just makes it easier for a network to learn we don’t have to do this But at the end of this only probably won’t have time But if you want on, you know, comment those lines out and see how it effects the network. It’s it’s pretty significant Ok. So the next thing we’re gonna do now is actually build the model So the model itself is gonna start as TF karosta model’s dot and then it’s going to be the sequential type of model There’s two types of models The sequential is your your most common one. It’s a feed-forward like the image we drew So we’re gonna use this sequential model and then from here we can use this like model dot add syntax so the first layer is gonna be the input layer and now right now our images are 28 by 28 in this like Multi-dimensional array we don’t want that We want them to be just like flat if we were doing like a confident or something like that We might not want it to be flat but in this case we definitely want to flatten it so we could use that we could use like numpy and reshape or We can actually use one of the layers that’s built into chaos, which is flattened. So So we’re gonna do ad and what we’re gonna add is TF. Chaos layers dot flatten so one of the reasons why you you want this to actually be a layer type is like when you have a Convolutional neural network a lot of times at the end of the convolutional neural network. There’ll be just like a densely connected Layer, and so you need to flatten it before that layer. So it’s it’s it’s used for more than then the input layer We’re just use it for the input layer Just to make our lives easier. So once we’ve got that That’s our input layer. Now. We want to do our hidden layers again We’re going to go with I think just two hidden layers. This isn’t a complex problem to solve So again, we’re going to use the model set up model dot add syntax and we’re gonna add and in fact I think what I’m gonna do is copy paste and then rather than a flattened layer it’s a dense layer in the dense layer We’re gonna pass a couple parameters here. So the first one is gonna be how many units in the layer. So we’re gonna use 128 units or 128 neurons in the layer, then we’re gonna pass the activation function This is like the function. Like I said like a stepper function or a sigmoid function What is gonna make that neuron fire or sort of fire whatever so we’re gonna use TF tenon Lu for rectified linear, this is kind of the default go-to Activation function just use it as your default and then later you can tweak it to see if you can get better results But it’s a pretty good one to always fall back on So we’re gonna add the second one just by copying and pasting Again, I’m gonna add a final one and this is our output layer, which is still going to be a dense layer It’s not a hundred and twenty eight your output layer will always if it’s in the case of classification Anyways, it’ll have your number of classifications in our case. That’s ten and the activation function we don’t want it to be rel you because we actually this is like a Probability distribution so we want to use softmax for a probability distribution. So That is our entire model. We’re done with defining, you know, the the architecture I guess of our model Now what we need to define is some parameters for the training of the model So here we’re going to say model dot compile and in here We’re gonna pass some we’re gonna pass the optimizer that we want to use We’re gonna pass the loss metric which loss you don’t know we haven’t really talked about it loss is the degree of error Basically, it’s it’s it’s it’s what you got wrong. So a normal network doesn’t actually attempt to optimize for accuracy It doesn’t try to maximize accuracy. It’s always trying to minimize loss So the way that you calculate loss can make a huge impact because it’s it’s what’s the losses? relationship to your to your accuracy op timer Optimizer Okay, so the optimizer that we’re going to use is going to be the Adam optimizer you could use something this is basically it’s just your it’s your It’s the this is like the most complex part of the entire neural network So if you’re familiar with gradient descent, you could pass something like stochastic gradient descent But the Adam optimizer kind of like the rectified linear is the you know Just kind of the default go-to optimizer you can use others There’s lots of them not lots, but I don’t know ten or so in Karis anyways, so anyways There’s other ones to go with Adam seems to be the one that you should start with for loss again There’s many ways to calculate loss probably the most popular one is a categorical Cross and true P or some version of that and that in this case, we’re gonna use sparse You can also use binary like in the case of cats versus dogs. You probably use binary in that case But you could you could just kind of blanket categorical cross entropy everything Anyways, then finally what what are the metrics we want to track like as we go and we’re going to just do accuracy accuracy Okay. So once we have all this we’re actually ready to to train the model So to Train it’s just model that fit and then you’re gonna pass your what do you want to fit? so X train X test, I’m sorry y Ok X train white rain and then epochs three total brain for it, okay, let’s go ahead and run that and We should start to get some training Hopefully it doesn’t crash me as I’m recording but okay looks good Let’s zoom out just slightly So it looks a little better and we can see actually our accuracy is always already quite well Our loss is also still dropping so our accuracy should still be improving and sure enough it is Awesome, ok, so we did pretty good. We got a 97% accuracy after only three epochs which is pretty good so once we have this we can This was in sample. So this might this is always gonna really excite you But what’s really important to remember is neural networks are great at fitting the question is did they over fit? So the idea or the hope is that your model actually generalized, right it learned Patterns and in actual attributes to the toe to what makes an eight ten what makes a for rather than memorizing every single sample you passed and you’d be surprised how easily a Model can just memorize all the samples that you passed to do very well so the next thing we want to always do is calculate the validation loss in the validation Accuracy, and that is just model dot evaluate evaluate X tests Y test and then we’ll go ahead and just print that loss and Val accuracy and We can see here the loss is almost point 11 and the accuracy is at ninety six point five so a little less than the one that we ended on and the loss is quite a bit higher relatively, but You should expect that You should expect your out-of-sample accuracy to be slightly lower and your loss to be slightly higher What you definitely don’t want to see is either too close or it too much of a delta if there’s a huge Delta Chances are you probably already have over fit and you’d want to like kind of dial it back a little bit. So That’s basically everything as far as the basics of Karos and all that the only other things that I wouldn’t mind Covering here is like if you want to save a model and load a model It’s just model dot save and we can save this as epoch num reader model and Then if you want to reload that model We’ll save it as it will call the new underscore model that’s going to be equal to TF. Caras models dot load model And it’s this exact model name Whoops. There we go So that’s our new model And then finally if we wanted to make a prediction we could say predictions equals new model dot Dict and keep in mind predict always takes a list. This will get you a few times for sure But anyways, they’ll take a list and we’ll do X underscore test and Then if we just print predictions, it’s probably not gonna look too friendly It’s a little messy. So What’s going on here? These are all one hot arrays and it’s it. These are our probability distributions So what do we do with these so I’m gonna use numpy you can also use TF to arc max, but it’s an abstract It’s a tensor and we have to pull it down. We need a session and all that It’s just easier to import numpy for at least for this tutorial here import numpy as NP and then print for example NP arg max or max Let’s do predictions And let’s just do the zero width prediction. Okay It says it’s a seven So the prediction for X test zero like the 0th index is it’s a seven So gee if only we had a way to draw it. Okay, we can definitely do this so we can do PLT I’m sure and we’re gonna do X test zero with and Then PLT dot show. Oh Look at that. It’s a seven Okay So I think that’s basically all the things I would want to show you guys as far as like just a quick in you know QuickStart with Deep learning and Python and caris and tensorflow this just barely scratches the surface. There’s so many things for us to do I Definitely plan to have at least one more follow-up video Covering things like loading and outside datasets and definitely some tensor board reading the model understanding what’s going on and also what’s going wrong because that’s Eventually, you know, it’s really fun when we’re doing tutorials and problems or like already solved and we know the answer It’s very exciting. But in reality a lot of times you have to dig to find the model that works with your data. So Anyways, that that’s definitely something we have to cover or at least that you’re gonna have to learn somehow or other Anyway, that is all for now. If you got questions comments concerns, whatever. Feel free to leave them below Definitely check out red comm slash are slash machine learning the learned machine learning subreddit You can come join our discord If you’ve got questions, that’s just discord dot G G slash cent decks will get you there. Also special. Thanks to my most recent Channel members Daniel Jeffrey KB AB Ajit H nur Newcastle geek fubá 44 Jason and eight counts Thank you guys so much for your support without you guys. I couldn’t do stuff like this. So really, thank you Anyways, that’s it for now more to come at some point until next time