Mapping memory of one vector to another
I have a vector of binary data used by some middleware
std::vector<uint> data
The data inside this vector maps to a class Foo basically like that:
|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |
Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>
But if I operate on the second vector the size of the first one is not updated.
Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).
Any ideas for an elegant solution?
c++ vector mapping
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
|
show 2 more comments
I have a vector of binary data used by some middleware
std::vector<uint> data
The data inside this vector maps to a class Foo basically like that:
|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |
Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>
But if I operate on the second vector the size of the first one is not updated.
Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).
Any ideas for an elegant solution?
c++ vector mapping
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does astd::vectorgive you that makes you want to jump through these hoops in order to use it?
– Paul Sanders
Nov 15 '18 at 0:57
What operations do you need to do onvector<Foo>? I'd do something simple likeauto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size());Then usestarttofinishas a range. Write utility functions for things likepush_back.
– Filipp
Nov 15 '18 at 1:01
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08
|
show 2 more comments
I have a vector of binary data used by some middleware
std::vector<uint> data
The data inside this vector maps to a class Foo basically like that:
|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |
Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>
But if I operate on the second vector the size of the first one is not updated.
Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).
Any ideas for an elegant solution?
c++ vector mapping
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
I have a vector of binary data used by some middleware
std::vector<uint> data
The data inside this vector maps to a class Foo basically like that:
|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |
Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>
But if I operate on the second vector the size of the first one is not updated.
Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).
Any ideas for an elegant solution?
c++ vector mapping
c++ vector mapping
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
asked Nov 15 '18 at 0:39
Andreas PasternakAndreas Pasternak
562314
562314
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does astd::vectorgive you that makes you want to jump through these hoops in order to use it?
– Paul Sanders
Nov 15 '18 at 0:57
What operations do you need to do onvector<Foo>? I'd do something simple likeauto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size());Then usestarttofinishas a range. Write utility functions for things likepush_back.
– Filipp
Nov 15 '18 at 1:01
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08
|
show 2 more comments
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does astd::vectorgive you that makes you want to jump through these hoops in order to use it?
– Paul Sanders
Nov 15 '18 at 0:57
What operations do you need to do onvector<Foo>? I'd do something simple likeauto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size());Then usestarttofinishas a range. Write utility functions for things likepush_back.
– Filipp
Nov 15 '18 at 1:01
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a
std::vector give you that makes you want to jump through these hoops in order to use it?– Paul Sanders
Nov 15 '18 at 0:57
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a
std::vector give you that makes you want to jump through these hoops in order to use it?– Paul Sanders
Nov 15 '18 at 0:57
What operations do you need to do on
vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.– Filipp
Nov 15 '18 at 1:01
What operations do you need to do on
vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.– Filipp
Nov 15 '18 at 1:01
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08
|
show 2 more comments
1 Answer
1
active
oldest
votes
The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.
This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.
A free function example
template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)
{
auto it = vec.insert(vec.end(), sizeof(Foo), {});
new (&*it) Foo{std::forward<T>(t)};
}
And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.
A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53310858%2fmapping-memory-of-one-vector-to-another%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.
This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.
A free function example
template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)
{
auto it = vec.insert(vec.end(), sizeof(Foo), {});
new (&*it) Foo{std::forward<T>(t)};
}
And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.
A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
add a comment |
The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.
This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.
A free function example
template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)
{
auto it = vec.insert(vec.end(), sizeof(Foo), {});
new (&*it) Foo{std::forward<T>(t)};
}
And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.
A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
add a comment |
The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.
This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.
A free function example
template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)
{
auto it = vec.insert(vec.end(), sizeof(Foo), {});
new (&*it) Foo{std::forward<T>(t)};
}
And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.
A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.
This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.
A free function example
template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)
{
auto it = vec.insert(vec.end(), sizeof(Foo), {});
new (&*it) Foo{std::forward<T>(t)};
}
And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.
A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
answered Nov 15 '18 at 8:09
Passer ByPasser By
9,92432558
9,92432558
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53310858%2fmapping-memory-of-one-vector-to-another%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?
– Aconcagua
Nov 15 '18 at 0:55
Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a
std::vectorgive you that makes you want to jump through these hoops in order to use it?– Paul Sanders
Nov 15 '18 at 0:57
What operations do you need to do on
vector<Foo>? I'd do something simple likeauto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size());Then usestarttofinishas a range. Write utility functions for things likepush_back.– Filipp
Nov 15 '18 at 1:01
@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...
– Aconcagua
Nov 15 '18 at 1:05
@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.
– Andreas Pasternak
Nov 15 '18 at 2:08