why use cell object in python closure implemention?











up vote
2
down vote

favorite
1












def outer():
n = 1
def inner():
return n
n = 2
return inner

inner = outer()
print innner() # output 2


I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.



python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.



The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.



I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?










share|improve this question
























  • Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
    – timgeb
    Nov 11 at 12:23












  • what's the problem? inner function is a local, look like you assign outer() to any variable like A!
    – RaminNietzsche
    Nov 11 at 12:44















up vote
2
down vote

favorite
1












def outer():
n = 1
def inner():
return n
n = 2
return inner

inner = outer()
print innner() # output 2


I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.



python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.



The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.



I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?










share|improve this question
























  • Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
    – timgeb
    Nov 11 at 12:23












  • what's the problem? inner function is a local, look like you assign outer() to any variable like A!
    – RaminNietzsche
    Nov 11 at 12:44













up vote
2
down vote

favorite
1









up vote
2
down vote

favorite
1






1





def outer():
n = 1
def inner():
return n
n = 2
return inner

inner = outer()
print innner() # output 2


I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.



python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.



The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.



I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?










share|improve this question















def outer():
n = 1
def inner():
return n
n = 2
return inner

inner = outer()
print innner() # output 2


I know it well that how CPython implement the closure, my question is not why output is 2, but why Python design it to output 2.



python use cell object in the closure implementiton, which indirectly ref the exactly PyObject which we want to capture. PythonVM create exactly one cell object for one freevar, in this example, in the outer scope, cell object first ref to 1, then ref to 2. when we call inner function, freevar always load the newest value in the outer function, so output 2.



The "cell object" is the additional abstract level in closure implemention. Actually I modifyed a few lines of the CPython code about the STORE_DEREF and LOAD_DEREF opcode process, remove the "cell object" level, save the real object in the inner's closure. Then the example will output 1. Everything runs ok except a simple traceback in standard library, some code assume cell is hashable. But i think it's not a big matter.



I think output "1" is intuitive sense. So my question is why python make a "cell object" level in the closure implemention ?I know the implemention clearly, but why python design like this ?







python closures cell






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 11 at 14:37

























asked Nov 11 at 12:21









codinglh

112




112












  • Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
    – timgeb
    Nov 11 at 12:23












  • what's the problem? inner function is a local, look like you assign outer() to any variable like A!
    – RaminNietzsche
    Nov 11 at 12:44


















  • Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
    – timgeb
    Nov 11 at 12:23












  • what's the problem? inner function is a local, look like you assign outer() to any variable like A!
    – RaminNietzsche
    Nov 11 at 12:44
















Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
– timgeb
Nov 11 at 12:23






Aaah, it's the same gotcha as funcs = [(lambda: x) for x in range(3)] (known as late binding).
– timgeb
Nov 11 at 12:23














what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44




what's the problem? inner function is a local, look like you assign outer() to any variable like A!
– RaminNietzsche
Nov 11 at 12:44












1 Answer
1






active

oldest

votes

















up vote
0
down vote













The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.



This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.



It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.






share|improve this answer





















  • Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
    – codinglh
    Nov 13 at 6:34










  • @codinglh: In Python 3 with nonlocal it need not be immutable.
    – Davis Herring
    Nov 13 at 14:30











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53248701%2fwhy-use-cell-object-in-python-closure-implemention%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.



This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.



It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.






share|improve this answer





















  • Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
    – codinglh
    Nov 13 at 6:34










  • @codinglh: In Python 3 with nonlocal it need not be immutable.
    – Davis Herring
    Nov 13 at 14:30















up vote
0
down vote













The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.



This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.



It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.






share|improve this answer





















  • Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
    – codinglh
    Nov 13 at 6:34










  • @codinglh: In Python 3 with nonlocal it need not be immutable.
    – Davis Herring
    Nov 13 at 14:30













up vote
0
down vote










up vote
0
down vote









The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.



This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.



It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.






share|improve this answer












The mental model is that the nested function (including a lambda, which is purely a syntactic difference) uses the same variable as the outer function and therefore observes changes to its value even after the function is created. This can be useful: a nested function is always up-to-date with assignments in a long function (that contains and calls it). This also, however, gives the famous issue with lambdas created in a loop: they all share the one loop variable.



This model is no more or less powerful than one where the function captures a value: to emulate that mode, you just create another variable just for the nested function’s use (which means another function call if a loop is involved), while to emulate Python’s behavior with value capture you just capture a container whose (single) element can be mutated.



It’s a matter of philosophy and language consistency as to which behavior is favored by the syntax. The decision here is to have all reads be of a variable; C++, by contrast, supports both behaviors even within one lambda, and even allows (with mutable) updating copies of captured values.







share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 12 at 3:08









Davis Herring

7,3771634




7,3771634












  • Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
    – codinglh
    Nov 13 at 6:34










  • @codinglh: In Python 3 with nonlocal it need not be immutable.
    – Davis Herring
    Nov 13 at 14:30


















  • Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
    – codinglh
    Nov 13 at 6:34










  • @codinglh: In Python 3 with nonlocal it need not be immutable.
    – Davis Herring
    Nov 13 at 14:30
















Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34




Thanks very much. I considered about that, in my view, capture value is a litte more consistent. Current model here makes a inmutable object looks like mutable, also many people make a mistake in the "lambda creates in loop" issue. eh... maybe from a different perspective.
– codinglh
Nov 13 at 6:34












@codinglh: In Python 3 with nonlocal it need not be immutable.
– Davis Herring
Nov 13 at 14:30




@codinglh: In Python 3 with nonlocal it need not be immutable.
– Davis Herring
Nov 13 at 14:30


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53248701%2fwhy-use-cell-object-in-python-closure-implemention%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Florida Star v. B. J. F.

Danny Elfman

Lugert, Oklahoma