Pandas : Key error while slicing Pandas Dataframe
3
I am trying to split the Pandas Dataframe into Train and Test.
Example:
Y = final_dataset['Species']
X = final_dataset[X_best_features]
X_train = X[0:120]
X_test = X[120:150]
y_train = Y[0:120]
y_test = Y[120:150]
Now I am using Hyperopt to get the best classifier.
estimator = hpsklearn.HyperoptEstimator(
preprocessing=hpsklearn.components.any_preprocessing('pp'),
classifier=hpsklearn.components.any_classifier('clf'),
algo=hyperopt.tpe.suggest,
trial_timeout=15.0, # seconds
max_evals=15,
)
fit_iterator = estimator.fit_iter(X_train,y_train)
fit_iterator.__next__()
plot_helper = hpsklearn.demo_support.PlotHelper(estimator,
mintodate_ylim=(-.01, .05))
while len(estimator.trials.trials) < estimator.max_evals:
fit_iterator.send(1) # -- try one more model
plot_helper.post_iter()
plot_helper.post_loop()
# -- Model selection was done on a subset of the training data.
# -- Now that we've picked a model, train on all training data.
estimator.retrain_best_model_on_full_data(X_train, y_train)
print('Best preprocessing pipeline:')
print('')
for pp in estimator._best_preprocs:
print(pp)
print('')
print('Best classifier:n', estimator._best_learner)
test_predictions = estimator.predict(X_test)
acc_in_percent = 100 * np.mean(test_predictions == y_test)
print('')
print('Prediction accuracy in generalisation is %.1f%%' % acc_in_percent)
After running the above code I am getting this error:
KeyError: '[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49n 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74n 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95] not in index'
Can anyone help me with this error??
Thanks
python pandas hyperopt
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
add a comment |
3
I am trying to split the Pandas Dataframe into Train and Test.
Example:
Y = final_dataset['Species']
X = final_dataset[X_best_features]
X_train = X[0:120]
X_test = X[120:150]
y_train = Y[0:120]
y_test = Y[120:150]
Now I am using Hyperopt to get the best classifier.
estimator = hpsklearn.HyperoptEstimator(
preprocessing=hpsklearn.components.any_preprocessing('pp'),
classifier=hpsklearn.components.any_classifier('clf'),
algo=hyperopt.tpe.suggest,
trial_timeout=15.0, # seconds
max_evals=15,
)
fit_iterator = estimator.fit_iter(X_train,y_train)
fit_iterator.__next__()
plot_helper = hpsklearn.demo_support.PlotHelper(estimator,
mintodate_ylim=(-.01, .05))
while len(estimator.trials.trials) < estimator.max_evals:
fit_iterator.send(1) # -- try one more model
plot_helper.post_iter()
plot_helper.post_loop()
# -- Model selection was done on a subset of the training data.
# -- Now that we've picked a model, train on all training data.
estimator.retrain_best_model_on_full_data(X_train, y_train)
print('Best preprocessing pipeline:')
print('')
for pp in estimator._best_preprocs:
print(pp)
print('')
print('Best classifier:n', estimator._best_learner)
test_predictions = estimator.predict(X_test)
acc_in_percent = 100 * np.mean(test_predictions == y_test)
print('')
print('Prediction accuracy in generalisation is %.1f%%' % acc_in_percent)
After running the above code I am getting this error:
KeyError: '[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49n 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74n 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95] not in index'
Can anyone help me with this error??
Thanks
python pandas hyperopt
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32
add a comment |
3
3
3
I am trying to split the Pandas Dataframe into Train and Test.
Example:
Y = final_dataset['Species']
X = final_dataset[X_best_features]
X_train = X[0:120]
X_test = X[120:150]
y_train = Y[0:120]
y_test = Y[120:150]
Now I am using Hyperopt to get the best classifier.
estimator = hpsklearn.HyperoptEstimator(
preprocessing=hpsklearn.components.any_preprocessing('pp'),
classifier=hpsklearn.components.any_classifier('clf'),
algo=hyperopt.tpe.suggest,
trial_timeout=15.0, # seconds
max_evals=15,
)
fit_iterator = estimator.fit_iter(X_train,y_train)
fit_iterator.__next__()
plot_helper = hpsklearn.demo_support.PlotHelper(estimator,
mintodate_ylim=(-.01, .05))
while len(estimator.trials.trials) < estimator.max_evals:
fit_iterator.send(1) # -- try one more model
plot_helper.post_iter()
plot_helper.post_loop()
# -- Model selection was done on a subset of the training data.
# -- Now that we've picked a model, train on all training data.
estimator.retrain_best_model_on_full_data(X_train, y_train)
print('Best preprocessing pipeline:')
print('')
for pp in estimator._best_preprocs:
print(pp)
print('')
print('Best classifier:n', estimator._best_learner)
test_predictions = estimator.predict(X_test)
acc_in_percent = 100 * np.mean(test_predictions == y_test)
print('')
print('Prediction accuracy in generalisation is %.1f%%' % acc_in_percent)
After running the above code I am getting this error:
KeyError: '[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49n 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74n 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95] not in index'
Can anyone help me with this error??
Thanks
python pandas hyperopt
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
I am trying to split the Pandas Dataframe into Train and Test.
Example:
Y = final_dataset['Species']
X = final_dataset[X_best_features]
X_train = X[0:120]
X_test = X[120:150]
y_train = Y[0:120]
y_test = Y[120:150]
Now I am using Hyperopt to get the best classifier.
estimator = hpsklearn.HyperoptEstimator(
preprocessing=hpsklearn.components.any_preprocessing('pp'),
classifier=hpsklearn.components.any_classifier('clf'),
algo=hyperopt.tpe.suggest,
trial_timeout=15.0, # seconds
max_evals=15,
)
fit_iterator = estimator.fit_iter(X_train,y_train)
fit_iterator.__next__()
plot_helper = hpsklearn.demo_support.PlotHelper(estimator,
mintodate_ylim=(-.01, .05))
while len(estimator.trials.trials) < estimator.max_evals:
fit_iterator.send(1) # -- try one more model
plot_helper.post_iter()
plot_helper.post_loop()
# -- Model selection was done on a subset of the training data.
# -- Now that we've picked a model, train on all training data.
estimator.retrain_best_model_on_full_data(X_train, y_train)
print('Best preprocessing pipeline:')
print('')
for pp in estimator._best_preprocs:
print(pp)
print('')
print('Best classifier:n', estimator._best_learner)
test_predictions = estimator.predict(X_test)
acc_in_percent = 100 * np.mean(test_predictions == y_test)
print('')
print('Prediction accuracy in generalisation is %.1f%%' % acc_in_percent)
After running the above code I am getting this error:
KeyError: '[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49n 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74n 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95] not in index'
Can anyone help me with this error??
Thanks
python pandas hyperopt
python pandas hyperopt
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
edited Dec 19 '18 at 11:31
Vivek Kumar
16.8k42156
16.8k42156
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
asked Nov 16 '18 at 5:01
shreyas nanawareshreyas nanaware
255
255
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32
add a comment |
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32
add a comment |
1 Answer
1
active
oldest
votes
1
I have solved the above problem by converting X and Y into Numpy array.
It seems that Hyperopt only takes numpy array as input and not Pandas Dataframe.
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331723%2fpandas-key-error-while-slicing-pandas-dataframe%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
I have solved the above problem by converting X and Y into Numpy array.
It seems that Hyperopt only takes numpy array as input and not Pandas Dataframe.
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
add a comment |
1
I have solved the above problem by converting X and Y into Numpy array.
It seems that Hyperopt only takes numpy array as input and not Pandas Dataframe.
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
add a comment |
1
1
1
I have solved the above problem by converting X and Y into Numpy array.
It seems that Hyperopt only takes numpy array as input and not Pandas Dataframe.
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
I have solved the above problem by converting X and Y into Numpy array.
It seems that Hyperopt only takes numpy array as input and not Pandas Dataframe.
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
answered Nov 16 '18 at 5:37
shreyas nanawareshreyas nanaware
255
255
add a comment |
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331723%2fpandas-key-error-while-slicing-pandas-dataframe%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Your data set doesn't have the indices mentioned in the error. So it's raising an error when you try to access them.
– Mohit Motwani
Nov 16 '18 at 5:21
Can you provide the line number on which the error is raised?
– ojas mohril
Nov 16 '18 at 5:32