WSO2 Siddhi Complex Event Processor question
Environment: WSO2 Stream Processor 4.3.0
Let's say I have two very simple streams:
Stream where newly created requests (unfulfilled) are being delivered in real time (t1)
RequestStream(requestId)
Stream where requestsIds appear when the request has been fulfilled in real time (t2)
FulfilmentStream(requestId)
It's guaranteed that t2 is always > t1
How can I implement a SiddhiQL statement to identify requestIds that appear at RequestStream (event1) and haven't appeard in FulfilmentStream (event2) after 5 minutes have been elapsed since event1?
Working Siddhi App based on Tishan answer:
@App:name('FailedToFulfillInAmountOfTime')
@source(
type="kafka",
topic.list="some_topic",
threading.option="single.thread",
group.id="some_group",
bootstrap.servers="xxx.xxx.xxx.xxx:6667",
@Map(type="json", @attributes(request_id = '$.alarm_id', severity = '$.severity', managed_object = '$.ManagedObject')))
define stream OrigAlarmStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this execution request')
define stream RequestStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this fulfillment confirmation:')
define stream FulfillmentStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='This fulfillment was not done within 1 min:')
define stream AlertStream(request_id int);
@info(name='getExpiredRequests')
from every e1=RequestStream -> not FulfillmentStream[e1.request_id == request_id] for 1 min
select e1.request_id
insert into AlertStream;
@info(name='CopyFulfillments')
from OrigAlarmStream[severity == 'Clear']
select request_id, severity, managed_object
insert into FulfillmentStream;
@info(name='CopyRequests')
from OrigAlarmStream[severity != 'Clear']
select request_id, severity, managed_object
insert into RequestStream;
wso2 siddhi wso2sp
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
add a comment |
Environment: WSO2 Stream Processor 4.3.0
Let's say I have two very simple streams:
Stream where newly created requests (unfulfilled) are being delivered in real time (t1)
RequestStream(requestId)
Stream where requestsIds appear when the request has been fulfilled in real time (t2)
FulfilmentStream(requestId)
It's guaranteed that t2 is always > t1
How can I implement a SiddhiQL statement to identify requestIds that appear at RequestStream (event1) and haven't appeard in FulfilmentStream (event2) after 5 minutes have been elapsed since event1?
Working Siddhi App based on Tishan answer:
@App:name('FailedToFulfillInAmountOfTime')
@source(
type="kafka",
topic.list="some_topic",
threading.option="single.thread",
group.id="some_group",
bootstrap.servers="xxx.xxx.xxx.xxx:6667",
@Map(type="json", @attributes(request_id = '$.alarm_id', severity = '$.severity', managed_object = '$.ManagedObject')))
define stream OrigAlarmStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this execution request')
define stream RequestStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this fulfillment confirmation:')
define stream FulfillmentStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='This fulfillment was not done within 1 min:')
define stream AlertStream(request_id int);
@info(name='getExpiredRequests')
from every e1=RequestStream -> not FulfillmentStream[e1.request_id == request_id] for 1 min
select e1.request_id
insert into AlertStream;
@info(name='CopyFulfillments')
from OrigAlarmStream[severity == 'Clear']
select request_id, severity, managed_object
insert into FulfillmentStream;
@info(name='CopyRequests')
from OrigAlarmStream[severity != 'Clear']
select request_id, severity, managed_object
insert into RequestStream;
wso2 siddhi wso2sp
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
add a comment |
Environment: WSO2 Stream Processor 4.3.0
Let's say I have two very simple streams:
Stream where newly created requests (unfulfilled) are being delivered in real time (t1)
RequestStream(requestId)
Stream where requestsIds appear when the request has been fulfilled in real time (t2)
FulfilmentStream(requestId)
It's guaranteed that t2 is always > t1
How can I implement a SiddhiQL statement to identify requestIds that appear at RequestStream (event1) and haven't appeard in FulfilmentStream (event2) after 5 minutes have been elapsed since event1?
Working Siddhi App based on Tishan answer:
@App:name('FailedToFulfillInAmountOfTime')
@source(
type="kafka",
topic.list="some_topic",
threading.option="single.thread",
group.id="some_group",
bootstrap.servers="xxx.xxx.xxx.xxx:6667",
@Map(type="json", @attributes(request_id = '$.alarm_id', severity = '$.severity', managed_object = '$.ManagedObject')))
define stream OrigAlarmStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this execution request')
define stream RequestStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this fulfillment confirmation:')
define stream FulfillmentStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='This fulfillment was not done within 1 min:')
define stream AlertStream(request_id int);
@info(name='getExpiredRequests')
from every e1=RequestStream -> not FulfillmentStream[e1.request_id == request_id] for 1 min
select e1.request_id
insert into AlertStream;
@info(name='CopyFulfillments')
from OrigAlarmStream[severity == 'Clear']
select request_id, severity, managed_object
insert into FulfillmentStream;
@info(name='CopyRequests')
from OrigAlarmStream[severity != 'Clear']
select request_id, severity, managed_object
insert into RequestStream;
wso2 siddhi wso2sp
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
Environment: WSO2 Stream Processor 4.3.0
Let's say I have two very simple streams:
Stream where newly created requests (unfulfilled) are being delivered in real time (t1)
RequestStream(requestId)
Stream where requestsIds appear when the request has been fulfilled in real time (t2)
FulfilmentStream(requestId)
It's guaranteed that t2 is always > t1
How can I implement a SiddhiQL statement to identify requestIds that appear at RequestStream (event1) and haven't appeard in FulfilmentStream (event2) after 5 minutes have been elapsed since event1?
Working Siddhi App based on Tishan answer:
@App:name('FailedToFulfillInAmountOfTime')
@source(
type="kafka",
topic.list="some_topic",
threading.option="single.thread",
group.id="some_group",
bootstrap.servers="xxx.xxx.xxx.xxx:6667",
@Map(type="json", @attributes(request_id = '$.alarm_id', severity = '$.severity', managed_object = '$.ManagedObject')))
define stream OrigAlarmStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this execution request')
define stream RequestStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='Got this fulfillment confirmation:')
define stream FulfillmentStream (request_id int, severity string, managed_object string);
@sink(type='log', prefix='This fulfillment was not done within 1 min:')
define stream AlertStream(request_id int);
@info(name='getExpiredRequests')
from every e1=RequestStream -> not FulfillmentStream[e1.request_id == request_id] for 1 min
select e1.request_id
insert into AlertStream;
@info(name='CopyFulfillments')
from OrigAlarmStream[severity == 'Clear']
select request_id, severity, managed_object
insert into FulfillmentStream;
@info(name='CopyRequests')
from OrigAlarmStream[severity != 'Clear']
select request_id, severity, managed_object
insert into RequestStream;
wso2 siddhi wso2sp
wso2 siddhi wso2sp
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
edited Nov 13 '18 at 14:23
Alexandre Juma
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
asked Nov 12 '18 at 19:03
Alexandre JumaAlexandre Juma
536214
536214
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You can use logical patterns to achieve your requirement. Please refer below query.
from e1=RequestStream -> not e2=FulfilmentStream[e1.requestId == e2.requestId] for '5 min'
select e1.requestId as requestId
insert into AlertStream;
Here we have defined a pattern with not condition. This will be triggered when an event in RequestStream comes and within 5 minutes no event comes into FulfilmentStream within 5 minutes. Please refer logical patterns for more information.
answered Nov 13 '18 at 5:06
TishanTishan
64547
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53268512%2fwso2-siddhi-complex-event-processor-question%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can use logical patterns to achieve your requirement. Please refer below query.
from e1=RequestStream -> not e2=FulfilmentStream[e1.requestId == e2.requestId] for '5 min'
select e1.requestId as requestId
insert into AlertStream;
Here we have defined a pattern with not condition. This will be triggered when an event in RequestStream comes and within 5 minutes no event comes into FulfilmentStream within 5 minutes. Please refer logical patterns for more information.
answered Nov 13 '18 at 5:06
TishanTishan
64547
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
add a comment |
You can use logical patterns to achieve your requirement. Please refer below query.
from e1=RequestStream -> not e2=FulfilmentStream[e1.requestId == e2.requestId] for '5 min'
select e1.requestId as requestId
insert into AlertStream;
Here we have defined a pattern with not condition. This will be triggered when an event in RequestStream comes and within 5 minutes no event comes into FulfilmentStream within 5 minutes. Please refer logical patterns for more information.
answered Nov 13 '18 at 5:06
TishanTishan
64547
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
add a comment |
You can use logical patterns to achieve your requirement. Please refer below query.
from e1=RequestStream -> not e2=FulfilmentStream[e1.requestId == e2.requestId] for '5 min'
select e1.requestId as requestId
insert into AlertStream;
Here we have defined a pattern with not condition. This will be triggered when an event in RequestStream comes and within 5 minutes no event comes into FulfilmentStream within 5 minutes. Please refer logical patterns for more information.
answered Nov 13 '18 at 5:06
TishanTishan
64547
You can use logical patterns to achieve your requirement. Please refer below query.
from e1=RequestStream -> not e2=FulfilmentStream[e1.requestId == e2.requestId] for '5 min'
select e1.requestId as requestId
insert into AlertStream;
Here we have defined a pattern with not condition. This will be triggered when an event in RequestStream comes and within 5 minutes no event comes into FulfilmentStream within 5 minutes. Please refer logical patterns for more information.
answered Nov 13 '18 at 5:06
TishanTishan
64547
answered Nov 13 '18 at 5:06
TishanTishan
64547
answered Nov 13 '18 at 5:06
TishanTishan
64547
answered Nov 13 '18 at 5:06
TishanTishan
64547
64547
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
add a comment |
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Thanks! It works great. The only difference is that I just couldn't compile the code in wso2 stream processor studio with the '5 min', had to take out the single quotes.
– Alexandre Juma
Nov 13 '18 at 10:43
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
Actually couldn't get it to work properly (although I got an initial behaviour that appear to be correct). Basically it works for the first case, then ceases to work. I've updated the code and behaviour in the initial request.
– Alexandre Juma
Nov 13 '18 at 12:44
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
So, in the end only the every keyword was missing and the single quotes in the for clause didn't work. After these changes, it works perfect.
– Alexandre Juma
Nov 13 '18 at 14:20
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
Sorry about the quotes and every keyword. I just changed pattern query that is there in the documentation and modified it a bit to match your scenario. But did not do any explicit tests. That's why mention to refer it rather than use it. Anyway glad you got it working.
– Tishan
Nov 14 '18 at 13:30
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53268512%2fwso2-siddhi-complex-event-processor-question%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
