Multivariate LSTM RMSE value is getting very high

machine_learning
time_series
deep_learning
python
lstm

#1

I want to predict a time series with multiple variables. I am using Keras’s LSTM class.

Here is my data set description :

dataset

I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , var5(t-1) , var6(t-1) and var7(t-1).

And this is how my model configuration looks like :

{
	"data": {
		
		"columns": [            
			"var1(t-1)",
			"var3(t-1)",
			"var4(t-1)",          
			"var5(t-1)",
			"var6(t-1)",
			"var7(t-1)" 
		],
		"sequence_length": 5,
		"train_test_split": 0.80,
		"normalise": false
	},
	"training": {
		"epochs": 100,
		"batch_size":10 
	},
	"model": {
		"loss": "mse",
		"optimizer": "nadam",
		"save_dir": "saved_models_multi",
		"layers": [
			{
				"type": "lstm",
				"neurons": 200,
				"input_timesteps": 5,
				"input_dim": 5,
				"return_seq": true
			},
            {
              "type":"batch_norm"  
                
            },  
			{
				"type": "dropout",
				"rate": 0.4
			},
            
            {
				"type": "lstm",
				"neurons": 200,
				"return_seq": true
			},
            {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.4
			},
			{
				"type": "dense",
				"neurons": 50,
				"activation": "sigmoid"
			},
            
             {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.2
			},
            
            {
				"type": "lstm",
				"neurons": 200,
				"return_seq": false
			},
            {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.4
			},
            {
				"type": "dense",
				"neurons": 25,
				"activation": "sigmoid"
			},
            
             {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.2
			},

			{
				"type": "dense",
				"neurons": 1,
				"activation": "linear"
			}
		]
	}
}

This is number of parameters in each layer and output shape :

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_1 (LSTM)                (None, 5, 200)            164800    
_________________________________________________________________
batch_normalization_1 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_1 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 5, 200)            320800    
_________________________________________________________________
batch_normalization_2 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_2 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
dense_1 (Dense)              (None, 5, 50)             10050     
_________________________________________________________________
batch_normalization_3 (Batch (None, 5, 50)             200       
_________________________________________________________________
dropout_3 (Dropout)          (None, 5, 50)             0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 200)               200800    
_________________________________________________________________
batch_normalization_4 (Batch (None, 200)               800       
_________________________________________________________________
dropout_4 (Dropout)          (None, 200)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 25)                5025      
_________________________________________________________________
batch_normalization_5 (Batch (None, 25)                100       
_________________________________________________________________
dropout_5 (Dropout)          (None, 25)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 26        
=================================================================
Total params: 704,201
Trainable params: 702,851
Non-trainable params: 1,350
_________________________________________________________________
Time taken: 0:00:01.914407

But , I am seeing RMSE value for test data is getting very high.
This is my train and validation loss graph.

What am I doing wrong here ? Is my model suffering from overfitting problem ?

Please help .

enter image description here


#2

How you find the lag length as 1. i.e…,for all variable you include only one step before value. Please use some lag selection criteria for more precise answer https://www.researchgate.net/post/How_do_you_choose_the_optimal_laglength_in_a_time_series