TShopping

 找回密碼
 註冊
搜索
查看: 760|回復: 0

[教學] 利用Keras建構LSTM模型,以Stock Prediction 為例

[複製鏈接]
發表於 2021-6-29 21:19:56 | 顯示全部樓層 |閱讀模式
 
Push to Facebook
LSTM介紹

機器學習當中最常利用多層感知器(Multi-Layer Perceptron, MLP)來訓練模型,如下圖所示


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




Multi-layer Perceptron

而利用MLP的方式並非能處理所有問題,因為他沒辦法處理時序性的問題,例如:當輸入為[1, 2, 3] 希望輸出4 ,而當輸入[3, 2, 1] 時希望輸出0 ,對於MLP來說,[1, 2, 3] 和 [3, 2, 1] 是相同的,因此無法得到預期的結果。

因此有了遞歸神經網絡(Recurrent Neural Network, RNN)的出現設計如下圖所示。


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction



Recurrent Neural Network

主要概念是將前面輸入得到的權重(Weight)加入下一層,這樣就可以完成時序性的概念。

而長短期記憶(Long Short-Term Memory, LSTM)是RNN的一種,而其不相同之處在於有了更多的控制單元input gate、output gate、forget gate 示意圖如下。


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




瞭解更多:李弘毅 — ML Lecture 21–1: Recurrent Neural Network (Part I)

Stock Prediction為例

SPY dataset: Yahoo SPDR S&P 500 ETF (SPY)


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




SPY.csv

目標:利用過去的資料預測未來幾天的Adj Close 。

資料建置匯入套件

將pandas、numpy、keras、matplotlib 匯入

  1. import pandas as pd
  2. import numpy as np
  3. from keras.models import Sequential
  4. from keras.layers import Dense, Dropout, Activation, Flatten, LSTM, TimeDistributed, RepeatVector
  5. from keras.layers.normalization import BatchNormalization
  6. from keras.optimizers import Adam
  7. from keras.callbacks import EarlyStopping, ModelCheckpoint
  8. import matplotlib.pyplot as plt
  9. %matplotlib inline
複製代碼


讀取資料
  1. def readTrain():
  2.   train = pd.read_csv("SPY.csv")
  3.   return train
複製代碼

Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction


Augment Features

除了基本資料提供的Features(Open, High, Low, Close, Adj Close, Volume)以外,還可自己增加Features,例如星期幾、幾月、幾號等等。

  1. def augFeatures(train):
  2.   train["Date"] = pd.to_datetime(train["Date"])
  3.   train["year"] = train["Date"].dt.year
  4.   train["month"] = train["Date"].dt.month
  5.   train["date"] = train["Date"].dt.day
  6.   train["day"] = train["Date"].dt.dayofweek
  7.   return train
複製代碼

Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




After Augment FeaturesNormalization

將所有資料做正規化,而由於Date 是字串非數字,因此先將它drop掉

  1. def normalize(train):
  2.   train = train.drop(["Date"], axis=1)
  3.   train_norm = train.apply(lambda x: (x - np.mean(x)) / (np.max(x) - np.min(x)))
  4.   return train_norm
複製代碼

Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




Build Training Data

輸入X_train: 利用前30天的Open, High, Low, Close, Adj Close, Volume, month, year, date, day作為Features,shape為(30, 10)

輸出Y_train: 利用未來5天的Adj Close作為Features,shape為(5,1)

我們須將資料做位移的展開作為Training Data,如圖(1)所示。


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction

將資料做位移展開,其中N為整份SPY.csv的資料樣本數

  1. def buildTrain(train, pastDay=30, futureDay=5):
  2.   X_train, Y_train = [], []
  3.   for i in range(train.shape[0]-futureDay-pastDay):
  4.     X_train.append(np.array(train.iloc[i:i+pastDay]))
  5.     Y_train.append(np.array(train.iloc[i+pastDay:i+pastDay+futureDay]["Adj Close"]))
  6.   return np.array(X_train), np.array(Y_train)
  7. view rawbuildTrain.py hosted with ❤ by GitHub
複製代碼

資料亂序

將資料打散,而非照日期排序

  1. def shuffle(X,Y):
  2. np.random.seed(10)
  3. randomList = np.arange(X.shape[0])
  4. np.random.shuffle(randomList)
  5. return X[randomList], Y[randomList]
複製代碼


Training data & Validation data

將Training Data取一部份當作Validation Data

  1. def splitData(X,Y,rate):
  2.   X_train = X[int(X.shape[0]*rate):]
  3.   Y_train = Y[int(Y.shape[0]*rate):]
  4.   X_val = X[:int(X.shape[0]*rate)]
  5.   Y_val = Y[:int(Y.shape[0]*rate)]
  6.   return X_train, Y_train, X_val, Y_val
複製代碼

因此最後將輸出合併為

  1. # read SPY.csv
  2. train = readTrain()

  3. # Augment the features (year, month, date, day)
  4. train_Aug = augFeatures(train)

  5. # Normalization
  6. train_norm = normalize(train_Aug)

  7. # build Data, use last 30 days to predict next 5 days
  8. X_train, Y_train = buildTrain(train_norm, 30, 5)

  9. # shuffle the data, and random seed is 10
  10. X_train, Y_train = shuffle(X_train, Y_train)

  11. # split training data and validation data
  12. X_train, Y_train, X_val, Y_val = splitData(X_train, Y_train, 0.1)
  13. # X_trian: (5710, 30, 10)
  14. # Y_train: (5710, 5, 1)
  15. # X_val: (634, 30, 10)
  16. # Y_val: (634, 5, 1)
複製代碼


模型建置

Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




Multiple models

比較many to one以及many to many


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




一對一模型

由於是一對一模型,因此return_sequences 也可設為False ,但Y_train 以及Y_val的維度需改為二維(5710,1)以及(634,1) 。


  1. def buildOneToOneModel(shape):
  2.   model = Sequential()
  3.   model.add(LSTM(10, input_length=shape[1], input_dim=shape[2],return_sequences=True))
  4.   # output shape: (1, 1)
  5.   model.add(TimeDistributed(Dense(1)))    # or use model.add(Dense(1))
  6.   model.compile(loss="mse", optimizer="adam")
  7.   model.summary()
  8.   return model
複製代碼

將過去的天數pastDay設為1,預測的天數futureDay也設為1

  1. train = readTrain()
  2. train_Aug = augFeatures(train)
  3. train_norm = normalize(train_Aug)
  4. # change the last day and next day
  5. X_train, Y_train = buildTrain(train_norm, 1, 1)
  6. X_train, Y_train = shuffle(X_train, Y_train)
  7. X_train, Y_train, X_val, Y_val = splitData(X_train, Y_train, 0.1)

  8. # from 2 dimmension to 3 dimension
  9. Y_train = Y_train[:,np.newaxis]
  10. Y_val = Y_val[:,np.newaxis]

  11. model = buildOneToOneModel(X_train.shape)
  12. callback = EarlyStopping(monitor="loss", patience=10, verbose=1, mode="auto")
  13. model.fit(X_train, Y_train, epochs=1000, batch_size=128, validation_data=(X_val, Y_val), callbacks=[callback])
複製代碼



由下圖可見變數的使用量


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction


最後val_loss: 2.2902e-05 停在第164個Epoch


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




多對一模型

LSTM參數return_sequences=False ,未設定時default也為False,而且不可使用TimeDistribution

  1. def buildManyToOneModel(shape):
  2.   model = Sequential()
  3.   model.add(LSTM(10, input_length=shape[1], input_dim=shape[2]))
  4.   # output shape: (1, 1)
  5.   model.add(Dense(1))
  6.   model.compile(loss="mse", optimizer="adam")
  7.   model.summary()
  8.   return model
複製代碼

需要設定的有pastDay=30、future=1 ,且注意Y_train 的維度需為二維


  1. train = readTrain()
  2. train_Aug = augFeatures(train)
  3. train_norm = normalize(train_Aug)
  4. # change the last day and next day
  5. X_train, Y_train = buildTrain(train_norm, 30, 1)
  6. X_train, Y_train = shuffle(X_train, Y_train)
  7. # because no return sequence, Y_train and Y_val shape must be 2 dimension
  8. X_train, Y_train, X_val, Y_val = splitData(X_train, Y_train, 0.1)

  9. model = buildManyToOneModel(X_train.shape)
  10. callback = EarlyStopping(monitor="loss", patience=10, verbose=1, mode="auto")
  11. model.fit(X_train, Y_train, epochs=1000, batch_size=128, validation_data=(X_val, Y_val), callbacks=[callback])
複製代碼

由下圖可見變數的使用量


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction


最後val_loss: 3.9465e-05 停在第113個Epoch


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




一對多模型

因為是一對多模型Timesteps只有1,因此return_sequences=False 才可執行

  1. def buildOneToManyModel(shape):
  2.   model = Sequential()
  3.   model.add(LSTM(10, input_length=shape[1], input_dim=shape[2]))
  4.   # output shape: (5, 1)
  5.   model.add(Dense(1))
  6.   model.add(RepeatVector(5))
  7.   model.compile(loss="mse", optimizer="adam")
  8.   model.summary()
  9.   return model
複製代碼


將pastDay 設為1, futureDay 設為5

  1. train = readTrain()
  2. train_Aug = augFeatures(train)
  3. train_norm = normalize(train_Aug)
  4. # change the last day and next day
  5. X_train, Y_train = buildTrain(train_norm, 1, 5)
  6. X_train, Y_train = shuffle(X_train, Y_train)
  7. X_train, Y_train, X_val, Y_val = splitData(X_train, Y_train, 0.1)

  8. # from 2 dimmension to 3 dimension
  9. Y_train = Y_train[:,:,np.newaxis]
  10. Y_val = Y_val[:,:,np.newaxis]

  11. model = buildOneToManyModel(X_train.shape)
  12. callback = EarlyStopping(monitor="loss", patience=10, verbose=1, mode="auto")
  13. model.fit(X_train, Y_train, epochs=1000, batch_size=128, validation_data=(X_val, Y_val), callbacks=[callback])
複製代碼

由下圖可見變數的使用量


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction



最後val_loss: 5.6081e-05 停在第163個Epoch


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction




多對多模型 (輸入與輸出相同長度)

將return_sequences 設為True ,再用TimeDistributed(Dense(1)) 將輸出調整為(5,1)

  1. def buildManyToManyModel(shape):
  2.   model = Sequential()
  3.   model.add(LSTM(10, input_length=shape[1], input_dim=shape[2], return_sequences=True))
  4.   # output shape: (5, 1)
  5.   model.add(TimeDistributed(Dense(1)))
  6.   model.compile(loss="mse", optimizer="adam")
  7.   model.summary()
  8.   return model
複製代碼


將pastDay 以及futureDay 設為相同長度5



  1. train = readTrain()
  2. train_Aug = augFeatures(train)
  3. train_norm = normalize(train_Aug)
  4. # change the last day and next day
  5. X_train, Y_train = buildTrain(train_norm, 5, 5)
  6. X_train, Y_train = shuffle(X_train, Y_train)
  7. X_train, Y_train, X_val, Y_val = splitData(X_train, Y_train, 0.1)

  8. # from 2 dimmension to 3 dimension
  9. Y_train = Y_train[:,:,np.newaxis]
  10. Y_val = Y_val[:,:,np.newaxis]

  11. model = buildManyToManyModel(X_train.shape)
  12. callback = EarlyStopping(monitor="loss", patience=10, verbose=1, mode="auto")
  13. model.fit(X_train, Y_train, epochs=1000, batch_size=128, validation_data=(X_val, Y_val), callbacks=[callback])
複製代碼

由下圖可見變數的使用量


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction


最後val_loss: 9.3788e-05 停在第169個Epoch


Keras,LSTM,python,Stock,Prediction

Keras,LSTM,python,Stock,Prediction



文章出處


 

臉書網友討論
您需要登錄後才可以回帖 登錄 | 註冊 |

本版積分規則



Archiver|手機版|小黑屋|免責聲明|TShopping

GMT+8, 2021-11-29 10:57 , Processed in 5.106801 second(s), 25 queries .

本論壇言論純屬發表者個人意見,與 TShopping綜合論壇 立場無關 如有意見侵犯了您的權益 請寫信聯絡我們。

Powered by Discuz! X3.2

© 2001-2013 Comsenz Inc.

快速回復 返回頂部 返回列表