Tensorflow

0. Infos

libs

• pandas
• matplotlib
• numpy
• seaborn
• sklearn
• tensorflow

activation functions

• Sigmoid: R -> [0, 1]
• Tanh: R -> [-1, 1]
• Relu: R -> [0, inf]
• Leaky Relu: R -> [-inf, inf]
• Softmax: R -> [0, inf]
• Swish: R -> [min, inf] ![[Pasted image 20240706102353.png]]

Begriffe

• ANN, NN: artificial neural network (allgemein)
• MLP: multilayer perceptron (Struktur mit mehreren Lagen)
  • input layer
  • hidden layer
  • output layer
• FC: fully connected layer (dense layer)
• loss function L: Bewertungsfunktion
  • liefert unterschiedliche Werte, wenn Daten normalisiert wurden
• $\eta$: learning rate: wie schnell wird Neues übernommen?
• batch size: back propagation not after each sample but after each batch
• epoch: number of repetitions with all samples
• CLR: Cyclical Learning Rate learning rate toggles between low and high rate

Neural Networks

1. read & pre-process data using pandas

energy_data = pd.read_csv('input/energydata_complete.csv', delimiter=",", decimal=".", )

• type conversion
• extract date, time, etc.
• convert categorial to one-hot coded: pd.get_dummies()
• handle outlayers
• optional: show correlation matrix

2. prepare training data

• split test and training data: train_test_split() (sklearn)
• scale data (X and Y): sklearn.preprocessing.StandardScaler()
  • calculate mean and std: scaler.fit()
  • transform: `scaler.transform()
• determine number of features for later re-shaping: num_features = X_train.shape[1]

3. define model

• setup model: model = tf.keras.Sequential()
• number of layers (Sequential, Dense, BatchNormalization, Dense, BatchNormalization, Dense)
• pro Layer: model.add() (activation function, number of neurons)
  • layers.Dense()
  • layers.BatchNormalization()
  • layers.Conv2D()
  • layers.MaxPooling2D()
  • layers.Flatten()
• define loss function: 'mean_squared_error' / negative log likelihood ``
• define optimizer (can include scheduler): 'adam' (adaptive => learning_rate) or 'SGD' (stochastic gradient descent)
• compile model: model.compile()
• optional: print model parameters: model.summary()

4. apply model

• additional functions for improved learning:
  • ReduceLROnPlateau(): learning rate is reduced when no further improvement is seen
  • EarlyStopping() (=> restore_best_weights): stops when no further improvement is seen to prevent overfitting
  • Cyclical Learning Rate (CLR)
• train model: model.fit()
  • validation_split: monitoring of the progress during the training (e.g. recognition of overfitting)
• predict values: model.predict()
• visualize result quality: true value vs. predicted value, residual value vs. frequency, predicted value vs. residual

Convolutional Neurol Networks

Terms & Hyperparameters

• CNN explainer
• convolution operation => scalar product => similarity between input data and kernal
• pooling layer:
  • reduce data size by boosting either max values (Max Pooling) or average values (Average Pooling)
• flatten layer:
  • transform n-D layer to 1-D layer => connect to FC layers for further evaluation
• padding:
  • input data is extended so that output data size is increased, can be used to make output data have the same size as input data
  • if padding = 0, the output data size will become smaller and smaller => limited number of steps possible
• kernal size: size of the "search pattern"
• stride:
  • number of pixels to shift the kernal from step to step
  • reduces output data size
• dilation