Cannot start dask cluster over SSH
up vote
0
down vote
favorite
I'm trying to start a dask cluster over SSH, but I am encountering a strange errors like these:
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/localuser/miniconda3/lib/python3.6/site-packages/distributed/deploy/ssh.py", line 57, in async_ssh
banner_timeout=20) # Helps prevent timeouts when many concurrent ssh connections are opened.
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 329, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 200, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
File "/home/localuser/miniconda3/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
I'm starting the cluster like this:
$ dask-ssh --ssh-private-key ~/.ssh/cluster_id_rsa
--hostfile ~/dask-hosts.txt
--remote-python "~/miniconda3/bin/python3.6"
My dask-hosts.txt looks like this:
localuser@127.0.0.1
remoteuser@10.10.4.200
...
remoteuser@10.10.4.207
I get the same error with/without the localhost line.
I have checked the ssh setup, I can login to all the nodes using a public key setup (the key is unencrypted, to avoid decryption prompts). What am I missing?
dask dask-distributed
asked Nov 8 at 12:39
suvayu
1,8681422
add a comment |
up vote
0
down vote
favorite
I'm trying to start a dask cluster over SSH, but I am encountering a strange errors like these:
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/localuser/miniconda3/lib/python3.6/site-packages/distributed/deploy/ssh.py", line 57, in async_ssh
banner_timeout=20) # Helps prevent timeouts when many concurrent ssh connections are opened.
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 329, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 200, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
File "/home/localuser/miniconda3/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
I'm starting the cluster like this:
$ dask-ssh --ssh-private-key ~/.ssh/cluster_id_rsa
--hostfile ~/dask-hosts.txt
--remote-python "~/miniconda3/bin/python3.6"
My dask-hosts.txt looks like this:
localuser@127.0.0.1
remoteuser@10.10.4.200
...
remoteuser@10.10.4.207
I get the same error with/without the localhost line.
I have checked the ssh setup, I can login to all the nodes using a public key setup (the key is unencrypted, to avoid decryption prompts). What am I missing?
dask dask-distributed
asked Nov 8 at 12:39
suvayu
1,8681422
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to start a dask cluster over SSH, but I am encountering a strange errors like these:
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/localuser/miniconda3/lib/python3.6/site-packages/distributed/deploy/ssh.py", line 57, in async_ssh
banner_timeout=20) # Helps prevent timeouts when many concurrent ssh connections are opened.
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 329, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 200, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
File "/home/localuser/miniconda3/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
I'm starting the cluster like this:
$ dask-ssh --ssh-private-key ~/.ssh/cluster_id_rsa
--hostfile ~/dask-hosts.txt
--remote-python "~/miniconda3/bin/python3.6"
My dask-hosts.txt looks like this:
localuser@127.0.0.1
remoteuser@10.10.4.200
...
remoteuser@10.10.4.207
I get the same error with/without the localhost line.
I have checked the ssh setup, I can login to all the nodes using a public key setup (the key is unencrypted, to avoid decryption prompts). What am I missing?
dask dask-distributed
asked Nov 8 at 12:39
suvayu
1,8681422
I'm trying to start a dask cluster over SSH, but I am encountering a strange errors like these:
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/localuser/miniconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/localuser/miniconda3/lib/python3.6/site-packages/distributed/deploy/ssh.py", line 57, in async_ssh
banner_timeout=20) # Helps prevent timeouts when many concurrent ssh connections are opened.
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 329, in connect
to_try = list(self._families_and_addresses(hostname, port))
File "/home/localuser/miniconda3/lib/python3.6/site-packages/paramiko/client.py", line 200, in _families_and_addresses
hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
File "/home/localuser/miniconda3/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
I'm starting the cluster like this:
$ dask-ssh --ssh-private-key ~/.ssh/cluster_id_rsa
--hostfile ~/dask-hosts.txt
--remote-python "~/miniconda3/bin/python3.6"
My dask-hosts.txt looks like this:
localuser@127.0.0.1
remoteuser@10.10.4.200
...
remoteuser@10.10.4.207
I get the same error with/without the localhost line.
I have checked the ssh setup, I can login to all the nodes using a public key setup (the key is unencrypted, to avoid decryption prompts). What am I missing?
dask dask-distributed
dask dask-distributed
asked Nov 8 at 12:39
suvayu
1,8681422
asked Nov 8 at 12:39
suvayu
1,8681422
asked Nov 8 at 12:39
suvayu
1,8681422
asked Nov 8 at 12:39
suvayu
1,8681422
asked Nov 8 at 12:39
suvayu
1,8681422
1,8681422
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:
–hostfile PATH Textfile with hostnames/IP addresses
You can use --ssh-username to set a username (although only a single one).
answered Nov 11 at 20:57
fuglede
6,61911239
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53207918%2fcannot-start-dask-cluster-over-ssh%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:
–hostfile PATH Textfile with hostnames/IP addresses
You can use --ssh-username to set a username (although only a single one).
answered Nov 11 at 20:57
fuglede
6,61911239
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
add a comment |
up vote
1
down vote
accepted
The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:
–hostfile PATH Textfile with hostnames/IP addresses
You can use --ssh-username to set a username (although only a single one).
answered Nov 11 at 20:57
fuglede
6,61911239
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:
–hostfile PATH Textfile with hostnames/IP addresses
You can use --ssh-username to set a username (although only a single one).
answered Nov 11 at 20:57
fuglede
6,61911239
The error indicates that name resolution is the culprit. Most likely this is happening because of the inclusion of usernames in your dask-hosts.txt. According to its documentation, the host file should contain only hostnames/IP addresses:
–hostfile PATH Textfile with hostnames/IP addresses
You can use --ssh-username to set a username (although only a single one).
answered Nov 11 at 20:57
fuglede
6,61911239
answered Nov 11 at 20:57
fuglede
6,61911239
answered Nov 11 at 20:57
fuglede
6,61911239
answered Nov 11 at 20:57
fuglede
6,61911239
6,61911239
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
add a comment |
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
That was it! I thought I already tried that, I guess not. I wish there was a way to specify different user names.
– suvayu
Nov 12 at 7:00
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Yep, seems like a reasonable feature if it isn't already possible to work around the limitation; you could try submitting an issue about it on github.com/dask/distributed/issues.
– fuglede
Nov 12 at 7:30
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
Done: github.com/dask/distributed/issues/2350 :)
– suvayu
Nov 12 at 7:42
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53207918%2fcannot-start-dask-cluster-over-ssh%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
