Insert documents into collection unless they are already inserted?
up vote
3
down vote
favorite
I get a list of objects from an API:
let sold = [
{ objId: 3240747,
soldDate: '2018-09-27',
soldPrice: 4610000,
apartmentNumber: '1202',
soldPriceSource: 'bid',
},
{ objId: 3234263,
soldDate: '2018-09-24',
soldPrice: 2580000,
soldPriceSource: 'bid',
}
...
]
I store these in a collection:
soldCollection.insertMany(sold)
Some of the objects have been retrieved before, and I only want to store the once that are not already in the database.
dbo.collection("sold").createIndex({ "objId": 1 }, { unique: true })
What would be an efficient way of doing this? Should I ask for each object before storing it or is there method for dealing with this?
javascript node.js mongodb ecmascript-6
asked Nov 10 at 20:11
Himmators
5,4812684171
add a comment |
up vote
3
down vote
favorite
I get a list of objects from an API:
let sold = [
{ objId: 3240747,
soldDate: '2018-09-27',
soldPrice: 4610000,
apartmentNumber: '1202',
soldPriceSource: 'bid',
},
{ objId: 3234263,
soldDate: '2018-09-24',
soldPrice: 2580000,
soldPriceSource: 'bid',
}
...
]
I store these in a collection:
soldCollection.insertMany(sold)
Some of the objects have been retrieved before, and I only want to store the once that are not already in the database.
dbo.collection("sold").createIndex({ "objId": 1 }, { unique: true })
What would be an efficient way of doing this? Should I ask for each object before storing it or is there method for dealing with this?
javascript node.js mongodb ecmascript-6
asked Nov 10 at 20:11
Himmators
5,4812684171
You can just insert into the database since you are adding auniquekey. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.
– Get Off My Lawn
Nov 10 at 20:19
And if memory serves me correctly there is an option ininsertManyon how to handle errors as in stop or ignore
– charlietfl
Nov 10 at 20:20
add a comment |
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I get a list of objects from an API:
let sold = [
{ objId: 3240747,
soldDate: '2018-09-27',
soldPrice: 4610000,
apartmentNumber: '1202',
soldPriceSource: 'bid',
},
{ objId: 3234263,
soldDate: '2018-09-24',
soldPrice: 2580000,
soldPriceSource: 'bid',
}
...
]
I store these in a collection:
soldCollection.insertMany(sold)
Some of the objects have been retrieved before, and I only want to store the once that are not already in the database.
dbo.collection("sold").createIndex({ "objId": 1 }, { unique: true })
What would be an efficient way of doing this? Should I ask for each object before storing it or is there method for dealing with this?
javascript node.js mongodb ecmascript-6
asked Nov 10 at 20:11
Himmators
5,4812684171
I get a list of objects from an API:
let sold = [
{ objId: 3240747,
soldDate: '2018-09-27',
soldPrice: 4610000,
apartmentNumber: '1202',
soldPriceSource: 'bid',
},
{ objId: 3234263,
soldDate: '2018-09-24',
soldPrice: 2580000,
soldPriceSource: 'bid',
}
...
]
I store these in a collection:
soldCollection.insertMany(sold)
Some of the objects have been retrieved before, and I only want to store the once that are not already in the database.
dbo.collection("sold").createIndex({ "objId": 1 }, { unique: true })
What would be an efficient way of doing this? Should I ask for each object before storing it or is there method for dealing with this?
javascript node.js mongodb ecmascript-6
javascript node.js mongodb ecmascript-6
asked Nov 10 at 20:11
Himmators
5,4812684171
asked Nov 10 at 20:11
Himmators
5,4812684171
asked Nov 10 at 20:11
Himmators
5,4812684171
asked Nov 10 at 20:11
Himmators
5,4812684171
asked Nov 10 at 20:11
Himmators
5,4812684171
5,4812684171
You can just insert into the database since you are adding auniquekey. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.
– Get Off My Lawn
Nov 10 at 20:19
And if memory serves me correctly there is an option ininsertManyon how to handle errors as in stop or ignore
– charlietfl
Nov 10 at 20:20
add a comment |
You can just insert into the database since you are adding auniquekey. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.
– Get Off My Lawn
Nov 10 at 20:19
And if memory serves me correctly there is an option ininsertManyon how to handle errors as in stop or ignore
– charlietfl
Nov 10 at 20:20
You can just insert into the database since you are adding a
unique key. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.– Get Off My Lawn
Nov 10 at 20:19
You can just insert into the database since you are adding a
unique key. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.– Get Off My Lawn
Nov 10 at 20:19
And if memory serves me correctly there is an option in
insertMany on how to handle errors as in stop or ignore– charlietfl
Nov 10 at 20:20
And if memory serves me correctly there is an option in
insertMany on how to handle errors as in stop or ignore– charlietfl
Nov 10 at 20:20
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
answered Nov 10 at 20:32
mickl
10.1k51435
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's noINSERT IF NOT EXISTSin MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully
– mickl
Nov 10 at 20:36
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
answered Nov 10 at 20:32
mickl
10.1k51435
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's noINSERT IF NOT EXISTSin MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully
– mickl
Nov 10 at 20:36
add a comment |
up vote
1
down vote
accepted
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
answered Nov 10 at 20:32
mickl
10.1k51435
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's noINSERT IF NOT EXISTSin MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully
– mickl
Nov 10 at 20:36
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
answered Nov 10 at 20:32
mickl
10.1k51435
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
answered Nov 10 at 20:32
mickl
10.1k51435
answered Nov 10 at 20:32
mickl
10.1k51435
answered Nov 10 at 20:32
mickl
10.1k51435
answered Nov 10 at 20:32
mickl
10.1k51435
10.1k51435
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's noINSERT IF NOT EXISTSin MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully
– mickl
Nov 10 at 20:36
add a comment |
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's noINSERT IF NOT EXISTSin MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully
– mickl
Nov 10 at 20:36
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
cool, I hadn't even noticed it worked at all. Is there some way of disabling this specific error? Either for this operation of in general.
– Himmators
Nov 10 at 20:34
Unfortunately there's no
INSERT IF NOT EXISTS in MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully– mickl
Nov 10 at 20:36
Unfortunately there's no
INSERT IF NOT EXISTS in MongoDB however with unordered I think this can be considered more like a warning than error, it only informs you that some inserts failed however entire "batch" is processed succesfully– mickl
Nov 10 at 20:36
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53243000%2finsert-documents-into-collection-unless-they-are-already-inserted%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
You can just insert into the database since you are adding a
uniquekey. Mongodb will not insert if it already exists. Check for errors if you want to do something extra if it already exists, otherwise just ignore the error.– Get Off My Lawn
Nov 10 at 20:19
And if memory serves me correctly there is an option in
insertManyon how to handle errors as in stop or ignore– charlietfl
Nov 10 at 20:20