Xcom pull returns a NameError on `ti`
0
I'm trying to push into xcom a value with key last_date in last_date_task and then pull it from xcom in second task ga_wh_task. when I test this dag by running python dag.py it returns an error at this line like:
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
NameError: name 'ti' is not defined
What am I doing wrong here? I've tried using task_instance instead of ti and context['ti'] instead of kwargs['ti']
Here's the dag file:
default_args = {
'owner': 'me',
'start_date': dt.datetime(2017, 10, 30),
'retries': 1,
'retry_delay': dt.timedelta(minutes=10),
'provide_context' : True,
}
def get_last_date(**kwargs):
kwargs['ti'].xcom_push(key='last_date', value='2018-11-15')
return True
with DAG('ga_mysql_dag2',
default_args=default_args,
schedule_interval=None,
catchup=False,
) as dag:
last_date_task = PythonOperator(task_id='last_date_task', python_callable=get_last_date, provide_context=True)
ga_wh_task = GoogleAnalyticsReportingToMySqlOperator(task_id='ga_wh_task', google_analytics_conn_id='google_analytics', key_file=key_file,
view_id=view_id, until=until, dimensions=dimensions, metrics=metrics, database=database,
table = table, mysql_conn_id = mysql_conn_id,
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
sleep = BashOperator(task_id='sleep', bash_command='sleep 10')
# Dependencies
last_date_task >> ga_warehouse_task >> sleep
python-3.x airflow airflow-scheduler apache-airflow-xcom
edited Nov 16 '18 at 10:23
kaxil
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
add a comment |
0
I'm trying to push into xcom a value with key last_date in last_date_task and then pull it from xcom in second task ga_wh_task. when I test this dag by running python dag.py it returns an error at this line like:
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
NameError: name 'ti' is not defined
What am I doing wrong here? I've tried using task_instance instead of ti and context['ti'] instead of kwargs['ti']
Here's the dag file:
default_args = {
'owner': 'me',
'start_date': dt.datetime(2017, 10, 30),
'retries': 1,
'retry_delay': dt.timedelta(minutes=10),
'provide_context' : True,
}
def get_last_date(**kwargs):
kwargs['ti'].xcom_push(key='last_date', value='2018-11-15')
return True
with DAG('ga_mysql_dag2',
default_args=default_args,
schedule_interval=None,
catchup=False,
) as dag:
last_date_task = PythonOperator(task_id='last_date_task', python_callable=get_last_date, provide_context=True)
ga_wh_task = GoogleAnalyticsReportingToMySqlOperator(task_id='ga_wh_task', google_analytics_conn_id='google_analytics', key_file=key_file,
view_id=view_id, until=until, dimensions=dimensions, metrics=metrics, database=database,
table = table, mysql_conn_id = mysql_conn_id,
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
sleep = BashOperator(task_id='sleep', bash_command='sleep 10')
# Dependencies
last_date_task >> ga_warehouse_task >> sleep
python-3.x airflow airflow-scheduler apache-airflow-xcom
edited Nov 16 '18 at 10:23
kaxil
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
add a comment |
0
0
0
I'm trying to push into xcom a value with key last_date in last_date_task and then pull it from xcom in second task ga_wh_task. when I test this dag by running python dag.py it returns an error at this line like:
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
NameError: name 'ti' is not defined
What am I doing wrong here? I've tried using task_instance instead of ti and context['ti'] instead of kwargs['ti']
Here's the dag file:
default_args = {
'owner': 'me',
'start_date': dt.datetime(2017, 10, 30),
'retries': 1,
'retry_delay': dt.timedelta(minutes=10),
'provide_context' : True,
}
def get_last_date(**kwargs):
kwargs['ti'].xcom_push(key='last_date', value='2018-11-15')
return True
with DAG('ga_mysql_dag2',
default_args=default_args,
schedule_interval=None,
catchup=False,
) as dag:
last_date_task = PythonOperator(task_id='last_date_task', python_callable=get_last_date, provide_context=True)
ga_wh_task = GoogleAnalyticsReportingToMySqlOperator(task_id='ga_wh_task', google_analytics_conn_id='google_analytics', key_file=key_file,
view_id=view_id, until=until, dimensions=dimensions, metrics=metrics, database=database,
table = table, mysql_conn_id = mysql_conn_id,
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
sleep = BashOperator(task_id='sleep', bash_command='sleep 10')
# Dependencies
last_date_task >> ga_warehouse_task >> sleep
python-3.x airflow airflow-scheduler apache-airflow-xcom
edited Nov 16 '18 at 10:23
kaxil
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
I'm trying to push into xcom a value with key last_date in last_date_task and then pull it from xcom in second task ga_wh_task. when I test this dag by running python dag.py it returns an error at this line like:
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
NameError: name 'ti' is not defined
What am I doing wrong here? I've tried using task_instance instead of ti and context['ti'] instead of kwargs['ti']
Here's the dag file:
default_args = {
'owner': 'me',
'start_date': dt.datetime(2017, 10, 30),
'retries': 1,
'retry_delay': dt.timedelta(minutes=10),
'provide_context' : True,
}
def get_last_date(**kwargs):
kwargs['ti'].xcom_push(key='last_date', value='2018-11-15')
return True
with DAG('ga_mysql_dag2',
default_args=default_args,
schedule_interval=None,
catchup=False,
) as dag:
last_date_task = PythonOperator(task_id='last_date_task', python_callable=get_last_date, provide_context=True)
ga_wh_task = GoogleAnalyticsReportingToMySqlOperator(task_id='ga_wh_task', google_analytics_conn_id='google_analytics', key_file=key_file,
view_id=view_id, until=until, dimensions=dimensions, metrics=metrics, database=database,
table = table, mysql_conn_id = mysql_conn_id,
provide_context=True, since={{ti.xcom_pull(task_ids="last_date_task", key='last_date')}})
sleep = BashOperator(task_id='sleep', bash_command='sleep 10')
# Dependencies
last_date_task >> ga_warehouse_task >> sleep
python-3.x airflow airflow-scheduler apache-airflow-xcom
python-3.x airflow airflow-scheduler apache-airflow-xcom
edited Nov 16 '18 at 10:23
kaxil
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
edited Nov 16 '18 at 10:23
kaxil
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
edited Nov 16 '18 at 10:23
kaxil
3,723928
edited Nov 16 '18 at 10:23
kaxil
3,723928
edited Nov 16 '18 at 10:23
kaxil
3,723928
3,723928
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
asked Nov 16 '18 at 6:42
Nihal SangeethNihal Sangeeth
620414
620414
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
0
Looks like GoogleAnalyticsReportingToMySqlOperator is an operator you created.
since param should be string. So change it to since="{{ti.xcom_pull(task_ids='last_date_task', key='last_date')}}"
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
Tried that. Now I have a NameError onlast_date....NameError: name 'last_date' is not defined
– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try withsince="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}". This should work
– kaxil
Nov 16 '18 at 18:12
add a comment |
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53332704%2fxcom-pull-returns-a-nameerror-on-ti%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
Looks like GoogleAnalyticsReportingToMySqlOperator is an operator you created.
since param should be string. So change it to since="{{ti.xcom_pull(task_ids='last_date_task', key='last_date')}}"
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
Tried that. Now I have a NameError onlast_date....NameError: name 'last_date' is not defined
– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try withsince="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}". This should work
– kaxil
Nov 16 '18 at 18:12
add a comment |
0
Looks like GoogleAnalyticsReportingToMySqlOperator is an operator you created.
since param should be string. So change it to since="{{ti.xcom_pull(task_ids='last_date_task', key='last_date')}}"
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
Tried that. Now I have a NameError onlast_date....NameError: name 'last_date' is not defined
– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try withsince="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}". This should work
– kaxil
Nov 16 '18 at 18:12
add a comment |
0
0
0
Looks like GoogleAnalyticsReportingToMySqlOperator is an operator you created.
since param should be string. So change it to since="{{ti.xcom_pull(task_ids='last_date_task', key='last_date')}}"
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
Looks like GoogleAnalyticsReportingToMySqlOperator is an operator you created.
since param should be string. So change it to since="{{ti.xcom_pull(task_ids='last_date_task', key='last_date')}}"
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
answered Nov 16 '18 at 9:49
kaxilkaxil
3,723928
3,723928
Tried that. Now I have a NameError onlast_date....NameError: name 'last_date' is not defined
– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try withsince="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}". This should work
– kaxil
Nov 16 '18 at 18:12
add a comment |
Tried that. Now I have a NameError onlast_date....NameError: name 'last_date' is not defined
– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try withsince="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}". This should work
– kaxil
Nov 16 '18 at 18:12
Tried that. Now I have a NameError on
last_date....NameError: name 'last_date' is not defined– Nihal Sangeeth
Nov 16 '18 at 11:26
Tried that. Now I have a NameError on
last_date....NameError: name 'last_date' is not defined– Nihal Sangeeth
Nov 16 '18 at 11:26
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
GoogleAnalyticsReportingToMySqlOperator is an operator I created, yes.
– Nihal Sangeeth
Nov 16 '18 at 11:38
Try with
since="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}" . This should work– kaxil
Nov 16 '18 at 18:12
Try with
since="{{ti.xcom_pull(task_ids='last_date_task', key=None)}}" . This should work– kaxil
Nov 16 '18 at 18:12
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53332704%2fxcom-pull-returns-a-nameerror-on-ti%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
