How to create PySpark job as a Web Service in AWS
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
0
I have few PySpark jobs runs on EMR and i would like create them as a web service. Any guidance or help in this case or any other better approach ?
Thanks.
amazon-web-services web-services apache-spark pyspark amazon-emr
asked Nov 16 '18 at 11:41
RK.RK.
517
add a comment |
0
I have few PySpark jobs runs on EMR and i would like create them as a web service. Any guidance or help in this case or any other better approach ?
Thanks.
amazon-web-services web-services apache-spark pyspark amazon-emr
asked Nov 16 '18 at 11:41
RK.RK.
517
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10
add a comment |
0
0
0
I have few PySpark jobs runs on EMR and i would like create them as a web service. Any guidance or help in this case or any other better approach ?
Thanks.
amazon-web-services web-services apache-spark pyspark amazon-emr
asked Nov 16 '18 at 11:41
RK.RK.
517
I have few PySpark jobs runs on EMR and i would like create them as a web service. Any guidance or help in this case or any other better approach ?
Thanks.
amazon-web-services web-services apache-spark pyspark amazon-emr
amazon-web-services web-services apache-spark pyspark amazon-emr
asked Nov 16 '18 at 11:41
RK.RK.
517
asked Nov 16 '18 at 11:41
RK.RK.
517
edited Dec 3 '18 at 13:29
RK.
asked Nov 16 '18 at 11:41
RK.RK.
517
asked Nov 16 '18 at 11:41
RK.RK.
517
asked Nov 16 '18 at 11:41
RK.RK.
517
517
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10
add a comment |
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10
add a comment |
1 Answer
1
active
oldest
votes
0
As of now using "Amazon API Gateway" to achieve above my requirement.
answered Dec 6 '18 at 9:56
RK.RK.
517
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337173%2fhow-to-create-pyspark-job-as-a-web-service-in-aws%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
As of now using "Amazon API Gateway" to achieve above my requirement.
answered Dec 6 '18 at 9:56
RK.RK.
517
add a comment |
0
As of now using "Amazon API Gateway" to achieve above my requirement.
answered Dec 6 '18 at 9:56
RK.RK.
517
add a comment |
0
0
0
As of now using "Amazon API Gateway" to achieve above my requirement.
answered Dec 6 '18 at 9:56
RK.RK.
517
As of now using "Amazon API Gateway" to achieve above my requirement.
answered Dec 6 '18 at 9:56
RK.RK.
517
answered Dec 6 '18 at 9:56
RK.RK.
517
answered Dec 6 '18 at 9:56
RK.RK.
517
answered Dec 6 '18 at 9:56
RK.RK.
517
517
add a comment |
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337173%2fhow-to-create-pyspark-job-as-a-web-service-in-aws%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Started looking into this, if any updates i'll post here.
– RK.
Nov 16 '18 at 11:45
Sparkjob-server : REST job server for Apache Spark github.com/spark-jobserver/spark-jobserver
– user238607
Nov 19 '18 at 13:08
Whether the above link supports PySpark and YARN as well ? Because i'm seeing all scala code only.
– RK.
Nov 28 '18 at 13:10