Accuracy Decreasing with higher epochs
up vote
0
down vote
favorite
I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ?
Here's my code -
model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)
python machine-learning keras deep-learning classification
asked Nov 10 at 19:58
anirudh
112
add a comment |
up vote
0
down vote
favorite
I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ?
Here's my code -
model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)
python machine-learning keras deep-learning classification
asked Nov 10 at 19:58
anirudh
112
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
2
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ?
Here's my code -
model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)
python machine-learning keras deep-learning classification
asked Nov 10 at 19:58
anirudh
112
I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ?
Here's my code -
model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)
python machine-learning keras deep-learning classification
python machine-learning keras deep-learning classification
asked Nov 10 at 19:58
anirudh
112
asked Nov 10 at 19:58
anirudh
112
asked Nov 10 at 19:58
anirudh
112
asked Nov 10 at 19:58
anirudh
112
asked Nov 10 at 19:58
anirudh
112
112
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
2
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07
add a comment |
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
2
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
2
2
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
According to me, you can take into consideration the following factors.
Reduce your learning rate to a very small number like 0.001 or even 0.0001.- Provide more data.
- Set Dropout rates to a number like 0.2. Keep them uniform across the network.
- Try decreasing the batch size.
- Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.
If any of the above factors work for you, please let me know about it in the comments section.
answered Nov 11 at 7:14
Shubham Panchal
16418
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
According to me, you can take into consideration the following factors.
Reduce your learning rate to a very small number like 0.001 or even 0.0001.- Provide more data.
- Set Dropout rates to a number like 0.2. Keep them uniform across the network.
- Try decreasing the batch size.
- Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.
If any of the above factors work for you, please let me know about it in the comments section.
answered Nov 11 at 7:14
Shubham Panchal
16418
add a comment |
up vote
0
down vote
According to me, you can take into consideration the following factors.
Reduce your learning rate to a very small number like 0.001 or even 0.0001.- Provide more data.
- Set Dropout rates to a number like 0.2. Keep them uniform across the network.
- Try decreasing the batch size.
- Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.
If any of the above factors work for you, please let me know about it in the comments section.
answered Nov 11 at 7:14
Shubham Panchal
16418
add a comment |
up vote
0
down vote
up vote
0
down vote
According to me, you can take into consideration the following factors.
Reduce your learning rate to a very small number like 0.001 or even 0.0001.- Provide more data.
- Set Dropout rates to a number like 0.2. Keep them uniform across the network.
- Try decreasing the batch size.
- Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.
If any of the above factors work for you, please let me know about it in the comments section.
answered Nov 11 at 7:14
Shubham Panchal
16418
According to me, you can take into consideration the following factors.
Reduce your learning rate to a very small number like 0.001 or even 0.0001.- Provide more data.
- Set Dropout rates to a number like 0.2. Keep them uniform across the network.
- Try decreasing the batch size.
- Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.
If any of the above factors work for you, please let me know about it in the comments section.
answered Nov 11 at 7:14
Shubham Panchal
16418
answered Nov 11 at 7:14
Shubham Panchal
16418
answered Nov 11 at 7:14
Shubham Panchal
16418
answered Nov 11 at 7:14
Shubham Panchal
16418
16418
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53242875%2faccuracy-decreasing-with-higher-epochs%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
How is you loss doing? also increasing?
– Or Dinari
Nov 10 at 20:02
2
One word: Overfitting
– Matias Valdenegro
Nov 10 at 20:03
As Matias mentioned above, it's the case of overfitting: the more data you provide to the model, the more noise it generates and subsequently fails to maintain accuracy. Because each layer is initialized with randomized weights, each time you run the model from clean state, it produces different results. If you added the input data sample to the question, it might help us dig a bit deeper, because it's hard right now to judge from the code alone (except that it's quite simple) what could go wrong.
– rm-
Nov 10 at 20:17
The loss always goes down to around 0.6. Would increasing the amount of input data avoid overfitting ? Each data sample is a 2D array of floating point numbers of the form (6,50)
– anirudh
Nov 11 at 6:07