Installing parquet-tools
I am trying to install parquet tools on a FreeBSD machine.
I cloned this repo: git clone https://github.com/apache/parquet-mr
Then I did cd parquet-mr/parquet-tools
Then I did `mvn clean package -Plocal
As specified here: https://github.com/apache/parquet-mr/tree/master/parquet-tools
This is what I got:
Why is this dependency error here? How do I get around it?
java maven freebsd parquet parquet-mr
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
add a comment |
I am trying to install parquet tools on a FreeBSD machine.
I cloned this repo: git clone https://github.com/apache/parquet-mr
Then I did cd parquet-mr/parquet-tools
Then I did `mvn clean package -Plocal
As specified here: https://github.com/apache/parquet-mr/tree/master/parquet-tools
This is what I got:
Why is this dependency error here? How do I get around it?
java maven freebsd parquet parquet-mr
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
2
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09
add a comment |
I am trying to install parquet tools on a FreeBSD machine.
I cloned this repo: git clone https://github.com/apache/parquet-mr
Then I did cd parquet-mr/parquet-tools
Then I did `mvn clean package -Plocal
As specified here: https://github.com/apache/parquet-mr/tree/master/parquet-tools
This is what I got:
Why is this dependency error here? How do I get around it?
java maven freebsd parquet parquet-mr
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
I am trying to install parquet tools on a FreeBSD machine.
I cloned this repo: git clone https://github.com/apache/parquet-mr
Then I did cd parquet-mr/parquet-tools
Then I did `mvn clean package -Plocal
As specified here: https://github.com/apache/parquet-mr/tree/master/parquet-tools
This is what I got:
Why is this dependency error here? How do I get around it?
java maven freebsd parquet parquet-mr
java maven freebsd parquet parquet-mr
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
asked Nov 14 '18 at 18:05
user3685285user3685285
1,59741840
1,59741840
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
2
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09
add a comment |
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
2
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
2
2
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09
add a comment |
1 Answer
1
active
oldest
votes
parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
I've been trying to compile parquet-tools only but keep getting the following error:[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type.I'm not able to find any solution or workaround for this situation. I've already installed Thrift.Thrift version 1.0.0Any ideas?
– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
Now I got a different error message. it says the following/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not existI´ve already opened the java file, and it does include the following packages:import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;
– Zombraz
Jan 9 at 23:55
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53306327%2finstalling-parquet-tools%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
I've been trying to compile parquet-tools only but keep getting the following error:[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type.I'm not able to find any solution or workaround for this situation. I've already installed Thrift.Thrift version 1.0.0Any ideas?
– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
Now I got a different error message. it says the following/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not existI´ve already opened the java file, and it does include the following packages:import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;
– Zombraz
Jan 9 at 23:55
add a comment |
parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
I've been trying to compile parquet-tools only but keep getting the following error:[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type.I'm not able to find any solution or workaround for this situation. I've already installed Thrift.Thrift version 1.0.0Any ideas?
– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
Now I got a different error message. it says the following/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not existI´ve already opened the java file, and it does include the following packages:import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;
– Zombraz
Jan 9 at 23:55
add a comment |
parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
edited Nov 15 '18 at 12:54
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
answered Nov 15 '18 at 12:44
ZoltanZoltan
1,302414
1,302414
I've been trying to compile parquet-tools only but keep getting the following error:[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type.I'm not able to find any solution or workaround for this situation. I've already installed Thrift.Thrift version 1.0.0Any ideas?
– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
Now I got a different error message. it says the following/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not existI´ve already opened the java file, and it does include the following packages:import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;
– Zombraz
Jan 9 at 23:55
add a comment |
I've been trying to compile parquet-tools only but keep getting the following error:[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type.I'm not able to find any solution or workaround for this situation. I've already installed Thrift.Thrift version 1.0.0Any ideas?
– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
Now I got a different error message. it says the following/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not existI´ve already opened the java file, and it does include the following packages:import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;
– Zombraz
Jan 9 at 23:55
I've been trying to compile parquet-tools only but keep getting the following error:
[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type. I'm not able to find any solution or workaround for this situation. I've already installed Thrift. Thrift version 1.0.0 Any ideas?– Zombraz
Jan 9 at 20:09
I've been trying to compile parquet-tools only but keep getting the following error:
[ERROR] thrift failed output: [WARNING:/home/user/parquet-mr/parquet-format-structures/target/parquet-format-thrift/parquet.thrift:295] The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the signedness of this type. I'm not able to find any solution or workaround for this situation. I've already installed Thrift. Thrift version 1.0.0 Any ideas?– Zombraz
Jan 9 at 20:09
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
That's strange, it's just a warning, it shouldn't fail the build. Nevertheless, try Thrift 0.9.3, that is the one that parquet-mr needs and it won't have this issue. See also issues.apache.org/jira/browse/PARQUET-1425
– Zoltan
Jan 9 at 21:45
1
1
Now I got a different error message. it says the following
/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not exist I´ve already opened the java file, and it does include the following packages: import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;– Zombraz
Jan 9 at 23:55
Now I got a different error message. it says the following
/home/edwinalejandro/parquet-mr/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexFilterUtils.java:[30,34] package it.unimi.dsi.fastutil.ints does not exist I´ve already opened the java file, and it does include the following packages: import it.unimi.dsi.fastutil.ints.IntArrayList; import it.unimi.dsi.fastutil.ints.IntList;– Zombraz
Jan 9 at 23:55
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53306327%2finstalling-parquet-tools%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The error seems to be fairly self-explanatory; the artifact you're looking for isn't in Jitpack's repository. Is it in Maven Central?
– Makoto
Nov 14 '18 at 18:08
Ah, turns out I just needed to checkout the latest stable release tag, not the master branch.
– user3685285
Nov 14 '18 at 18:33
2
Do not post images of code or errors! Images and screenshots can be a nice addition to a post, but please make sure the post is still clear and useful without them. If you post images of code or error messages make sure you also copy and paste or type the actual code/message into the post directly.
– Rob
Nov 15 '18 at 14:09