7bulls_predictions.md 4.88 KB
Newer Older
Anna Warno's avatar
Anna Warno committed
1
2
3
4
5
6
7
8
9
10
11
# Time series prediction - results

1. FCR - average response time
  * fcr_2020-10-29 to 2020-10-30
    * time series 
    ![Alt text](images/fcr_2020-10-29 to 2020-10-30_ts.png?raw=true "Title")
    Average spike length: 151 seconds
    Median spike length: 120.0 seconds (2 points)
    * prediction (tft)
    ![Alt text](images/fcr_2020-10-29 to 2020-10-30_1_preds.png?raw=true "Title")
    * prediction with fake added points (2s)(tft)
12
    ![Alt text](images/fcr_fake_points.png?raw=true "Title")       
Anna Warno's avatar
Anna Warno committed
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

  * fcr_2020-11-27 to 2020-11-27
    * time series 
    ![Alt text](images/fcr_2020-11-27 to 2020-11-27_1.png?raw=true "Title")
    Average spike length: 377 seconds
    Median spike length: 180.0 seconds
    * prediction (tft)
    ![Alt text](images/fcr_2020-11-27 to 2020-11-27_1_preds_res.png?raw=true "Title")  
      
  * fcr_2020-12-09 to 2020-12-09
    * time series 
    ![Alt text](images/fcr_2020-12-09 to 2020-12-09_1.png?raw=true "Title")
    Average peak length is 10.57 seconds
    Median peak length is 10 seconds (2 points)
    * prediction
    ![Alt text](images/fcr_2020-12-09 to 2020-12-09_1_preds.png?raw=true "Title")
  * fcr_2020-12-15 to 2020-12-15
    * time series 
    ![Alt text](images/fcr_2020-12-15 to 2020-12-15_1.png?raw=true "Title")
    Average peak length is 10.67 seconds
    Median peak length is 10 seconds (2 points)
      
    ![Alt text](images/fcr_2020-12-15 to 2020-12-15_1_preds.png?raw=true "Title")


  * fcr_2021-01-18 to 2021-01-19
    * TODO
  * comments
    * predictions are saved with format:  predictions_{METHOD_NAME}.csv 
    * fcr average repsonse time - forecast with 1 min horizon
    * fcr average response time series is aggregated over 60 seconds  
    * TODO
    

2. FCR - requests
  * fcr_2020-10-29 to 2020-10-30 
    * prediction (tft)
      * 1 minute
      ![Alt text](images/fcr_2020-10-29 to 2020-10-30_1_preds_0.png?raw=true "Title")
      * 2 minutes  
      ![Alt text](images/fcr_2020-10-29 to 2020-10-30_1_preds_res.png?raw=true "Title")
      * 3 minutes   
      ![Alt text](images/fcr_2020-10-29 to 2020-10-30_1_preds_2.png?raw=true "Title")
    * prediction (n-beats)
      * 1 minute
      ![Alt text](images/nbeats_pred_0.png?raw=true "Title")
      * 2 minutes  
      ![Alt text](images/nbeats_pred_1.png?raw=true "Title")  
      * 3 minutes         
      ![Alt text](images/nbeats_pred_2.png?raw=true "Title") 
63
64
65
66
67
68
69
70
71
        
    * experiments with ensembles
      25 models (n-betas, tft) with different: predictions length, context length,
      loss functions (rmse, mae, dilate, quantileloss), model architecture
      * simple lasso regression (averaged 5 minutes prediction)
      ![Alt text](images/preds_ens_lasso.png?raw=true "Title")  
      * lgbm (overfitting)  (averaged 5 minutes prediction)    
      ![Alt text](images/preds_ens_lgbm.png?raw=true "Title") 

Anna Warno's avatar
Anna Warno committed
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
    * prediction (es-rnn)
      TODO     
  * comments
    * predictions for longer horizonts (egz. 10 minutes are also avaliable)
    * Requests were aggregated over 60 seconds
    * predictions are saved with format:  predictions_{METHOD_NAME}{STEP}.csv 
    * TODO

3. Genome
    * 2020-12-02 to 2020-12-02
        *  MinimumCoresContext prediction (tft)
           * 1 minute
          ![Alt text](images/genome_preds_0.png?raw=true "Title")
           * 2 minutes  
          ![Alt text](images/genome_preds_1.png?raw=true "Title")
           * 3 minutes          
          ![Alt text](images/genome_preds_2.png?raw=true "Title")
           * 4 minutes          
          ![Alt text](images/genome_preds_3.png?raw=true "Title")
           * 5 minutes            
          ![Alt text](images/genome_preds_4.png?raw=true "Title")
            
        *  MinimumCoresContext prediction (n-beats)
95
           * TODO
Anna Warno's avatar
Anna Warno committed
96
        *  MinimumCoresContext prediction (es-rnn)
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
           * TODO 
        
        * feature importance
          * ETPercentile
            
            ![Alt text](images/etpercentile_features.png?raw=true "Title")
          * MinimumCoresContext
            ![Alt text](images/minimumcorescontext_features.png?raw=true "Title")
        * Experiments with ensembles
          15 models with different: predictions length (5-20 min), context length,
          loss functions (rmse, mae, dilate, quantileloss), model architecture and metrics, 
          scores for single models are avaliable on neptune  
            * ETPercentile (averaged 5 minutes prediction)
              * simple lasso regression
                ![Alt text](images/ETPercentile_preds_ens_lasso.png?raw=true "Title")
              * lgbm (overfitting)
                ![Alt text](images/ETPercentile_preds_ens_lgbm.png?raw=true "Title")
            * MinimumCoresContext (averaged 5 minutes prediction)
115
116
117
              * simple lasso regression
                ![Alt text](images/preds_ens_lasso_mini.png?raw=true "Title")
    
Anna Warno's avatar
Anna Warno committed
118
119


120
121
  * comments
    * predictions are saved with format:  {METRIC_NAME}_{METHOD_NAME}_preds{STEP}.csv
Anna Warno's avatar
Anna Warno committed
122
123
    * TODO