Login from popup Get data from profile
up vote
0
down vote
favorite
I am trying to log into investing.com to go to my portfolio so I can get the current information on the page.
I am having an issue with the Login because it is not on the main page, but a popup page called "loginPopupform"
Here is my code that I have tried.
String baseUrl = "https://www.investing.com";
String sChartLink = "https://www.investing.com/portfolio";
String strLogin = "username";
String strPassword = "password" ;
String ua = ""Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0"";
Response res = null;
Document docRes = null;
res = Jsoup
.connect(baseUrl)
.timeout(30000)
.method(Connection.Method.POST)
.data("loginFormUser_email", strLogin, "loginForm_password", strPassword)
.userAgent(ua)
.execute();
docRes = res.parse();
Map<String, String> cookies = res.cookies();
doc1 = Jsoup.connect(sChartLink).userAgent(ua).cookies(cookies).get();
sDate = doc1.select("input[id="curDate"]").text();
The sDate is empty and when I look into the doc1, there isn't any information that is in my profile.
java authentication jsoup
add a comment |
up vote
0
down vote
favorite
I am trying to log into investing.com to go to my portfolio so I can get the current information on the page.
I am having an issue with the Login because it is not on the main page, but a popup page called "loginPopupform"
Here is my code that I have tried.
String baseUrl = "https://www.investing.com";
String sChartLink = "https://www.investing.com/portfolio";
String strLogin = "username";
String strPassword = "password" ;
String ua = ""Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0"";
Response res = null;
Document docRes = null;
res = Jsoup
.connect(baseUrl)
.timeout(30000)
.method(Connection.Method.POST)
.data("loginFormUser_email", strLogin, "loginForm_password", strPassword)
.userAgent(ua)
.execute();
docRes = res.parse();
Map<String, String> cookies = res.cookies();
doc1 = Jsoup.connect(sChartLink).userAgent(ua).cookies(cookies).get();
sDate = doc1.select("input[id="curDate"]").text();
The sDate is empty and when I look into the doc1, there isn't any information that is in my profile.
java authentication jsoup
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am trying to log into investing.com to go to my portfolio so I can get the current information on the page.
I am having an issue with the Login because it is not on the main page, but a popup page called "loginPopupform"
Here is my code that I have tried.
String baseUrl = "https://www.investing.com";
String sChartLink = "https://www.investing.com/portfolio";
String strLogin = "username";
String strPassword = "password" ;
String ua = ""Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0"";
Response res = null;
Document docRes = null;
res = Jsoup
.connect(baseUrl)
.timeout(30000)
.method(Connection.Method.POST)
.data("loginFormUser_email", strLogin, "loginForm_password", strPassword)
.userAgent(ua)
.execute();
docRes = res.parse();
Map<String, String> cookies = res.cookies();
doc1 = Jsoup.connect(sChartLink).userAgent(ua).cookies(cookies).get();
sDate = doc1.select("input[id="curDate"]").text();
The sDate is empty and when I look into the doc1, there isn't any information that is in my profile.
java authentication jsoup
I am trying to log into investing.com to go to my portfolio so I can get the current information on the page.
I am having an issue with the Login because it is not on the main page, but a popup page called "loginPopupform"
Here is my code that I have tried.
String baseUrl = "https://www.investing.com";
String sChartLink = "https://www.investing.com/portfolio";
String strLogin = "username";
String strPassword = "password" ;
String ua = ""Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0"";
Response res = null;
Document docRes = null;
res = Jsoup
.connect(baseUrl)
.timeout(30000)
.method(Connection.Method.POST)
.data("loginFormUser_email", strLogin, "loginForm_password", strPassword)
.userAgent(ua)
.execute();
docRes = res.parse();
Map<String, String> cookies = res.cookies();
doc1 = Jsoup.connect(sChartLink).userAgent(ua).cookies(cookies).get();
sDate = doc1.select("input[id="curDate"]").text();
The sDate is empty and when I look into the doc1, there isn't any information that is in my profile.
java authentication jsoup
java authentication jsoup
edited Nov 11 at 17:04
Flimzy
37k96496
37k96496
asked Nov 11 at 17:03
EddiRae
307
307
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
To make that you need to force the JS of the website because the pop-up modal is not static content so Jsoup does not retrieve it. So you need to use Selenium to force the JS and retrieve the dynamic content.
If it's a pop-up modal I guess you need to click on a button to load it, the way to do that using Selenium for Java is as it follows:
WebDriver webDriver = new FirefoxDriver();
JavascriptExecutor js = (JavascriptExecutor)webDriver;
webDriver.get(URL);
WebDriverWait webDriverWait = new WebDriverWait(webDriver, 15); // wait 15 seconds
WebElement button = webDriver.findElement(By.id("id of the button that triggers the modal"));
js.executeScript("arguments[0].click();", button);
webDriverWait.until(ExpectedConditions.visibilityOfElementLocated(By.name("loginPopupform")));
String fullHtml = webDriver.getPageSource();
Hope it helped you! Feel free to ask me anything else!
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53251083%2flogin-from-popup-get-data-from-profile%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
To make that you need to force the JS of the website because the pop-up modal is not static content so Jsoup does not retrieve it. So you need to use Selenium to force the JS and retrieve the dynamic content.
If it's a pop-up modal I guess you need to click on a button to load it, the way to do that using Selenium for Java is as it follows:
WebDriver webDriver = new FirefoxDriver();
JavascriptExecutor js = (JavascriptExecutor)webDriver;
webDriver.get(URL);
WebDriverWait webDriverWait = new WebDriverWait(webDriver, 15); // wait 15 seconds
WebElement button = webDriver.findElement(By.id("id of the button that triggers the modal"));
js.executeScript("arguments[0].click();", button);
webDriverWait.until(ExpectedConditions.visibilityOfElementLocated(By.name("loginPopupform")));
String fullHtml = webDriver.getPageSource();
Hope it helped you! Feel free to ask me anything else!
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
add a comment |
up vote
0
down vote
To make that you need to force the JS of the website because the pop-up modal is not static content so Jsoup does not retrieve it. So you need to use Selenium to force the JS and retrieve the dynamic content.
If it's a pop-up modal I guess you need to click on a button to load it, the way to do that using Selenium for Java is as it follows:
WebDriver webDriver = new FirefoxDriver();
JavascriptExecutor js = (JavascriptExecutor)webDriver;
webDriver.get(URL);
WebDriverWait webDriverWait = new WebDriverWait(webDriver, 15); // wait 15 seconds
WebElement button = webDriver.findElement(By.id("id of the button that triggers the modal"));
js.executeScript("arguments[0].click();", button);
webDriverWait.until(ExpectedConditions.visibilityOfElementLocated(By.name("loginPopupform")));
String fullHtml = webDriver.getPageSource();
Hope it helped you! Feel free to ask me anything else!
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
add a comment |
up vote
0
down vote
up vote
0
down vote
To make that you need to force the JS of the website because the pop-up modal is not static content so Jsoup does not retrieve it. So you need to use Selenium to force the JS and retrieve the dynamic content.
If it's a pop-up modal I guess you need to click on a button to load it, the way to do that using Selenium for Java is as it follows:
WebDriver webDriver = new FirefoxDriver();
JavascriptExecutor js = (JavascriptExecutor)webDriver;
webDriver.get(URL);
WebDriverWait webDriverWait = new WebDriverWait(webDriver, 15); // wait 15 seconds
WebElement button = webDriver.findElement(By.id("id of the button that triggers the modal"));
js.executeScript("arguments[0].click();", button);
webDriverWait.until(ExpectedConditions.visibilityOfElementLocated(By.name("loginPopupform")));
String fullHtml = webDriver.getPageSource();
Hope it helped you! Feel free to ask me anything else!
To make that you need to force the JS of the website because the pop-up modal is not static content so Jsoup does not retrieve it. So you need to use Selenium to force the JS and retrieve the dynamic content.
If it's a pop-up modal I guess you need to click on a button to load it, the way to do that using Selenium for Java is as it follows:
WebDriver webDriver = new FirefoxDriver();
JavascriptExecutor js = (JavascriptExecutor)webDriver;
webDriver.get(URL);
WebDriverWait webDriverWait = new WebDriverWait(webDriver, 15); // wait 15 seconds
WebElement button = webDriver.findElement(By.id("id of the button that triggers the modal"));
js.executeScript("arguments[0].click();", button);
webDriverWait.until(ExpectedConditions.visibilityOfElementLocated(By.name("loginPopupform")));
String fullHtml = webDriver.getPageSource();
Hope it helped you! Feel free to ask me anything else!
answered Nov 13 at 12:58
alvarobartt
12418
12418
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
add a comment |
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
Thanks for the info ... There is so much on the Selenium link, what should I be downloading?
– EddiRae
Nov 14 at 3:47
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
I found what I needed to download ... thanks
– EddiRae
Nov 14 at 3:58
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
Also, looking at the web page, what you press is a link for an onclick. Here is the entire <a> ... <a onclick="overlay.overlayLogin();" href="javascript:void(0);" class="login bold">Sign In</a> What would I use for the button id?
– EddiRae
Nov 14 at 4:00
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
@EddiRae check the Selenium docs or this post examples.javacodegeeks.com/enterprise-java/selenium/…, what you need is to learn how to use the By selector. Basically you need to change By.id() to By + the css selector you want to look for, for example: By.className(), By.name(), By.id, etc. In this particular case you need to use: By.class("login bold") or By.class("login"). Hope it helped you! +1 the answer please!
– alvarobartt
Nov 14 at 8:31
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53251083%2flogin-from-popup-get-data-from-profile%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown