I have a dataset with time series and for some of them (test sample) dates are known (maybe from zero to several pieces per time series), which we will call target events, the occurrence of which I want to predict.
Also, for each rserie, several heuristics have been calculated that catch some kind of local change in the serie (different types of drops in the serie level) and fix in themselves whether a change has occurred on such and such a date or not.
I want to take advantage of these heuristics and so, to learn how to predict the occurrence of targeted events.
Three metrics are of interest: forecast accuracy, forecast completeness (recall), and forecast of interval between heuristics date and target event (for example, that a target event will occur n days after a set of heuristics is triggered).
For each date, I can calculate how many target events will occur on the test sample after each of the heuristics is triggered for n days. By this calculation, I want to evaluate the usefulness of each of the heuristics.
But there are several questions:
How to relate the triggering of the heuristic on date n with the desired event, and for example, not with the next one? This is necessary to assess that the accuracy of the heuristics.
is it possible to somehow generalize the triggering of heuristics to predict the occurrence of a target event exactly after the calculated number of days?
maybe there is some way to predict the class of the event in n days (whether it will be target or not) or the probability that it will be target. This is necessary in order to ultimately generalize the triggering of heuristics and additional known time series features.