manipulating (nested) JSON keys and there values, using nifi
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I am currently facing an issue where I have to read a JSON file that has mostly the same structure, has about 10k+ lines, and is nested.
I thought about creating my own custom processor which reads the JSON and replaces several matching key/values to the ones needed. As I am trying to use NiFi I assume that there should be a more comfortable way as the JSON-structure itself is mostly consistent.
I already tried using the ReplaceText processor as well as the JoltTransformJson processor, but I could not figure out. How can I transform both keys and values, if needed? For example: if there is something like this:
{
"id": "test"
},
{
"id": "14"
}
It might be necessary to turn the "id" into "Number" and map "test" to "3", as I am using different keys/values in my jsonfiles/database, so they need to fit those. Is there a way of doing so without having to create my own processor?
Regards,
Steve
json apache-nifi
asked Nov 16 '18 at 10:29
SteveSteve
83
add a comment |
I am currently facing an issue where I have to read a JSON file that has mostly the same structure, has about 10k+ lines, and is nested.
I thought about creating my own custom processor which reads the JSON and replaces several matching key/values to the ones needed. As I am trying to use NiFi I assume that there should be a more comfortable way as the JSON-structure itself is mostly consistent.
I already tried using the ReplaceText processor as well as the JoltTransformJson processor, but I could not figure out. How can I transform both keys and values, if needed? For example: if there is something like this:
{
"id": "test"
},
{
"id": "14"
}
It might be necessary to turn the "id" into "Number" and map "test" to "3", as I am using different keys/values in my jsonfiles/database, so they need to fit those. Is there a way of doing so without having to create my own processor?
Regards,
Steve
json apache-nifi
asked Nov 16 '18 at 10:29
SteveSteve
83
it's not clear why"test"should be mapped to"3". please edit your question and provide larger input and output examples.
– daggett
Nov 16 '18 at 19:32
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55
add a comment |
I am currently facing an issue where I have to read a JSON file that has mostly the same structure, has about 10k+ lines, and is nested.
I thought about creating my own custom processor which reads the JSON and replaces several matching key/values to the ones needed. As I am trying to use NiFi I assume that there should be a more comfortable way as the JSON-structure itself is mostly consistent.
I already tried using the ReplaceText processor as well as the JoltTransformJson processor, but I could not figure out. How can I transform both keys and values, if needed? For example: if there is something like this:
{
"id": "test"
},
{
"id": "14"
}
It might be necessary to turn the "id" into "Number" and map "test" to "3", as I am using different keys/values in my jsonfiles/database, so they need to fit those. Is there a way of doing so without having to create my own processor?
Regards,
Steve
json apache-nifi
asked Nov 16 '18 at 10:29
SteveSteve
83
I am currently facing an issue where I have to read a JSON file that has mostly the same structure, has about 10k+ lines, and is nested.
I thought about creating my own custom processor which reads the JSON and replaces several matching key/values to the ones needed. As I am trying to use NiFi I assume that there should be a more comfortable way as the JSON-structure itself is mostly consistent.
I already tried using the ReplaceText processor as well as the JoltTransformJson processor, but I could not figure out. How can I transform both keys and values, if needed? For example: if there is something like this:
{
"id": "test"
},
{
"id": "14"
}
It might be necessary to turn the "id" into "Number" and map "test" to "3", as I am using different keys/values in my jsonfiles/database, so they need to fit those. Is there a way of doing so without having to create my own processor?
Regards,
Steve
json apache-nifi
json apache-nifi
asked Nov 16 '18 at 10:29
SteveSteve
83
asked Nov 16 '18 at 10:29
SteveSteve
83
edited Nov 16 '18 at 22:56
Steve
asked Nov 16 '18 at 10:29
SteveSteve
83
asked Nov 16 '18 at 10:29
SteveSteve
83
asked Nov 16 '18 at 10:29
SteveSteve
83
83
it's not clear why"test"should be mapped to"3". please edit your question and provide larger input and output examples.
– daggett
Nov 16 '18 at 19:32
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55
add a comment |
it's not clear why"test"should be mapped to"3". please edit your question and provide larger input and output examples.
– daggett
Nov 16 '18 at 19:32
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55
it's not clear why
"test" should be mapped to "3". please edit your question and provide larger input and output examples.– daggett
Nov 16 '18 at 19:32
it's not clear why
"test" should be mapped to "3". please edit your question and provide larger input and output examples.– daggett
Nov 16 '18 at 19:32
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335977%2fmanipulating-nested-json-keys-and-there-values-using-nifi%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335977%2fmanipulating-nested-json-keys-and-there-values-using-nifi%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
it's not clear why
"test"should be mapped to"3". please edit your question and provide larger input and output examples.– daggett
Nov 16 '18 at 19:32
The keys and values I am receiving differ from those, that I am using (for my json/db later on). So for example the json-file uses as key "name" while I am using "companyname" or the value "DE" needs to be mapped to "276" (as of ISO 3166). It is pretty hard to be more accurate about this, but I hope you are getting my point.
– Steve
Nov 16 '18 at 22:55