Specialize std::copy for custom iterators
From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T>
class and implemented a random access iterator for this class. As is the std::copy()
algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy()
on internal pointers.
My question is that even possible? I know how I would create an overload of std::copy()
as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)
c++ iterator template-specialization
|
show 3 more comments
From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T>
class and implemented a random access iterator for this class. As is the std::copy()
algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy()
on internal pointers.
My question is that even possible? I know how I would create an overload of std::copy()
as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)
c++ iterator template-specialization
It seems thatCircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespacestd
.
– Jarod42
Nov 15 '18 at 12:25
2
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
1
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
1
Just to clarify: yourCircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar tostd::deque
?
– Caleth
Nov 15 '18 at 12:56
|
show 3 more comments
From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T>
class and implemented a random access iterator for this class. As is the std::copy()
algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy()
on internal pointers.
My question is that even possible? I know how I would create an overload of std::copy()
as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)
c++ iterator template-specialization
From what I understand one can write template specializations for function template in the std namespace. I have written a CircularBuffer<T>
class and implemented a random access iterator for this class. As is the std::copy()
algorithm works with my custom iterators but it is not optimal. It iterates through the range and copies elements one by one. I could write a more optimal implementation for trivially copiable types by using std::memcpy()
on internal pointers.
My question is that even possible? I know how I would create an overload of std::copy()
as a template with output iterator being the template parameter. But that cannot be done since you can only write template specializations of function templates in the std namespace. Any help in pointing me in the right direction would be helpful. From what I could gather through Google, this can't be done, but I would love to be proven wrong :)
c++ iterator template-specialization
c++ iterator template-specialization
edited Nov 15 '18 at 12:30
rozina
asked Nov 15 '18 at 12:22
rozinarozina
2,2151232
2,2151232
It seems thatCircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespacestd
.
– Jarod42
Nov 15 '18 at 12:25
2
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
1
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
1
Just to clarify: yourCircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar tostd::deque
?
– Caleth
Nov 15 '18 at 12:56
|
show 3 more comments
It seems thatCircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespacestd
.
– Jarod42
Nov 15 '18 at 12:25
2
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
1
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
1
Just to clarify: yourCircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar tostd::deque
?
– Caleth
Nov 15 '18 at 12:56
It seems that
CircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std
.– Jarod42
Nov 15 '18 at 12:25
It seems that
CircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespace std
.– Jarod42
Nov 15 '18 at 12:25
2
2
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
1
1
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
1
1
Just to clarify: your
CircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque
?– Caleth
Nov 15 '18 at 12:56
Just to clarify: your
CircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar to std::deque
?– Caleth
Nov 15 '18 at 12:56
|
show 3 more comments
1 Answer
1
active
oldest
votes
Maybe your question should be: should I do it ?
Users who make use of your class should already be aware of what std::copy
is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy
guarantees that N assignments are made; according to the standard:
Exactly
(last - first)
assignments
Also, it is not uncommon that when std::copy
is used, it is also used with back_inserter
s or other manipulators, that will probably not play well with optimization.
However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data
.
My question is that even possible?
One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.
https://en.cppreference.com/w/cpp/named_req/ContiguousIterator
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since forint
s,memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.
– HolyBlackCat
Nov 15 '18 at 14:08
I want to specializestd::copy
for myCircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization forstd::copy
.
– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to astd::deque
than astd::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator
– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data fromCircularBuffer
that uses a maximum of 2 calls ofstd::copy()
, which for integers translates tostd::memcpy()
. I would love ifstd::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overloadstd::copy()
, my question is, if that is possible and how.
– rozina
Nov 16 '18 at 13:24
|
show 3 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319423%2fspecialize-stdcopy-for-custom-iterators%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Maybe your question should be: should I do it ?
Users who make use of your class should already be aware of what std::copy
is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy
guarantees that N assignments are made; according to the standard:
Exactly
(last - first)
assignments
Also, it is not uncommon that when std::copy
is used, it is also used with back_inserter
s or other manipulators, that will probably not play well with optimization.
However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data
.
My question is that even possible?
One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.
https://en.cppreference.com/w/cpp/named_req/ContiguousIterator
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since forint
s,memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.
– HolyBlackCat
Nov 15 '18 at 14:08
I want to specializestd::copy
for myCircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization forstd::copy
.
– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to astd::deque
than astd::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator
– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data fromCircularBuffer
that uses a maximum of 2 calls ofstd::copy()
, which for integers translates tostd::memcpy()
. I would love ifstd::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overloadstd::copy()
, my question is, if that is possible and how.
– rozina
Nov 16 '18 at 13:24
|
show 3 more comments
Maybe your question should be: should I do it ?
Users who make use of your class should already be aware of what std::copy
is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy
guarantees that N assignments are made; according to the standard:
Exactly
(last - first)
assignments
Also, it is not uncommon that when std::copy
is used, it is also used with back_inserter
s or other manipulators, that will probably not play well with optimization.
However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data
.
My question is that even possible?
One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.
https://en.cppreference.com/w/cpp/named_req/ContiguousIterator
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since forint
s,memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.
– HolyBlackCat
Nov 15 '18 at 14:08
I want to specializestd::copy
for myCircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization forstd::copy
.
– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to astd::deque
than astd::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator
– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data fromCircularBuffer
that uses a maximum of 2 calls ofstd::copy()
, which for integers translates tostd::memcpy()
. I would love ifstd::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overloadstd::copy()
, my question is, if that is possible and how.
– rozina
Nov 16 '18 at 13:24
|
show 3 more comments
Maybe your question should be: should I do it ?
Users who make use of your class should already be aware of what std::copy
is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy
guarantees that N assignments are made; according to the standard:
Exactly
(last - first)
assignments
Also, it is not uncommon that when std::copy
is used, it is also used with back_inserter
s or other manipulators, that will probably not play well with optimization.
However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data
.
My question is that even possible?
One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.
https://en.cppreference.com/w/cpp/named_req/ContiguousIterator
Maybe your question should be: should I do it ?
Users who make use of your class should already be aware of what std::copy
is and how it works and inherently the performance implications. So providing a specialization could make matters worse. std::copy
guarantees that N assignments are made; according to the standard:
Exactly
(last - first)
assignments
Also, it is not uncommon that when std::copy
is used, it is also used with back_inserter
s or other manipulators, that will probably not play well with optimization.
However, you could, for instance, choose to provide a direct access to the buffer like in std::vector::data
.
My question is that even possible?
One way around this issue that you seem to have is to export this knowledge - in a sense - to the users of your class. Just add an extra level of indirection. So instead of having iterators to the elements directly you will return iterators to the blocks of memory. Then you will be able to have Contiguous iterators.
https://en.cppreference.com/w/cpp/named_req/ContiguousIterator
edited Nov 16 '18 at 12:17
answered Nov 15 '18 at 13:45
darunedarune
1,556516
1,556516
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since forint
s,memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.
– HolyBlackCat
Nov 15 '18 at 14:08
I want to specializestd::copy
for myCircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization forstd::copy
.
– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to astd::deque
than astd::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator
– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data fromCircularBuffer
that uses a maximum of 2 calls ofstd::copy()
, which for integers translates tostd::memcpy()
. I would love ifstd::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overloadstd::copy()
, my question is, if that is possible and how.
– rozina
Nov 16 '18 at 13:24
|
show 3 more comments
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since forint
s,memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.
– HolyBlackCat
Nov 15 '18 at 14:08
I want to specializestd::copy
for myCircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization forstd::copy
.
– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to astd::deque
than astd::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator
– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data fromCircularBuffer
that uses a maximum of 2 calls ofstd::copy()
, which for integers translates tostd::memcpy()
. I would love ifstd::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overloadstd::copy()
, my question is, if that is possible and how.
– rozina
Nov 16 '18 at 13:24
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
From what I understand using std::copy() on std::vector<int> invokes std::memcpy() or std::memmove(). Are you saying that is incorrect behavior?
– rozina
Nov 15 '18 at 13:50
@rozina Since for
int
s, memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.– HolyBlackCat
Nov 15 '18 at 14:08
@rozina Since for
int
s, memcpy
is indistinguishable from assignments, the 'as-if rule' allows the compiler to use one instead of another.– HolyBlackCat
Nov 15 '18 at 14:08
I want to specialize
std::copy
for my CircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy
.– rozina
Nov 15 '18 at 14:11
I want to specialize
std::copy
for my CircularBuffer<int>
. which I have stated in my question as trivially copy-able types. So hopefully that answers your question that I should write my optimization for std::copy
.– rozina
Nov 15 '18 at 14:11
@rozina It's based on the iterator properties as I see it. You have something more akin to a
std::deque
than a std::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator– darune
Nov 16 '18 at 12:12
@rozina It's based on the iterator properties as I see it. You have something more akin to a
std::deque
than a std::vector
. So if this is essential to your class - then I guess you are left with exporting this knowledge to the user so that your iterators can become contiguois in type; see en.cppreference.com/w/cpp/named_req/ContiguousIterator– darune
Nov 16 '18 at 12:12
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from
CircularBuffer
that uses a maximum of 2 calls of std::copy()
, which for integers translates to std::memcpy()
. I would love if std::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy()
, my question is, if that is possible and how.– rozina
Nov 16 '18 at 13:24
I don't see how my iterators could become ContiguousIterators, because the range that two iterators represent is not contiguous in memory. However I have an algorithm for copying data from
CircularBuffer
that uses a maximum of 2 calls of std::copy()
, which for integers translates to std::memcpy()
. I would love if std::copy()
could somehow use my algorithm instead of its own. But since I am not supposed to overload std::copy()
, my question is, if that is possible and how.– rozina
Nov 16 '18 at 13:24
|
show 3 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319423%2fspecialize-stdcopy-for-custom-iterators%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It seems that
CircularBuffer
is template, and we cannot partial specialize function, so the only solution is overload... But you cannot add it in namespacestd
.– Jarod42
Nov 15 '18 at 12:25
2
I wouldn't write any function specializations in new code.
– StoryTeller
Nov 15 '18 at 12:25
@StoryTeller it also states that it is fine as long as there are at least one program defined type involved.
– darune
Nov 15 '18 at 12:31
1
@darune - It also says C++20 removed the allowance for function templates, making it okay only for class templates.
– StoryTeller
Nov 15 '18 at 12:32
1
Just to clarify: your
CircularBuffer::iterator
is RandomAccessIterator but not ContiguousIterator, but there are contiguous sections, similar tostd::deque
?– Caleth
Nov 15 '18 at 12:56