Deep Learning2019. 12. 18. 22:28
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)



1. 딥러닝의 기본 개념

 

2. 딥러닝을 위한 '학습'의 불가능성 주장 by Marvin Minsky

 - XOR 문제풀이 불가

 

3. 딥러닝을 위한 '학습'의 가능성 주장 = Backpropagation by Paul Werbos, Hinton

 

4. Convolutional Neural Networks (알파고) by LeCun

 

5. 복잡한 문제(많은 레이어)의 경우 학습 불가

 - neormal neural nets가 성능 향상이 되지 않는 문제가 발생

 

6. 적절한 W 초기값을 통해 학습 가능 주장 by Hinton and Bengio

 - 기존의 neural network를 deep nets, deep learning으로 리브랜딩

 - 딥러닝은 복잡한 문제를 해결하기 위한 효과적인 수단

 

 

반응형
Posted by CCIBOMB
Deep Learning2019. 12. 12. 19:22
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)

 

1. Learning rate

 - 어느 정도의 크기로 기울기가 줄어드는 지점으로 이동하겠는가를 나타내는 지표

 - cost function을 최소화시키기 위해 사용하는 'Gradient descent algorithm'에서 cost 값의 미분한 값 앞 알파값

  c.f) Gradient descent algorithm : 기울기 크기가 줄어드는 쪽으로 가면 cost function이 최소가 되는 지점을 찾아간다.

 - overshooting : learning rate의 값이 너무 커서 최소값에 도달하지 않고 오히려 그래프를 벗어나는 경우

 - small learning rate : step의 간격이 매우 작아서 학습 속도가 매우 느리게 됨

 - cost 값을 확인하여, 적절한 learning rate을 정하는 것이 중요함 : data와 환경에 따라 다름

 

2. Preprocessing for gradient descent algorithm (전처리, 선처리)

 - 두 개의 입력 값(x1, x2)이 적절한 범위 차이를 갖는 경우 : 

 - 두 개의 입력 값(x1, x2)이 매우 큰 차이를 갖는 경우 :

 

 - Preprocessing 필요 

 

 - 여러 normalization (표준화) 중 standardization

 

3. Overfitting

 - 학습 데이터에 딱 맞는 모델을 만드는 경우, 다른 실제 데이터와는 안 맞는 경우가 생김

 - 학습 데이터(training data)가 많을 수록, feature(입력으로 들어오는 변수)의 개수를 줄일 수록, Regularization(일반화)할 수록 Overfitting을 방지할 수 있음

 

 - Regularization : weight에 너무 큰 가중치를 두지 말 것. weight의 값이 커지면 그래프 형태가 구부러진 형태가 되고, weight의 크기가 작으면 그래프 형태가 선형을 이룬다. 이를 위해 cost function 뒤에 아래와 같은 식을 추가한다.

 - Regularization의 Python 구현 : cost에 다음 변수 l2reg을 더함. cost 함수가 틀렸을 때 높은 비용이 발생할 수 있도록 벌점(penalty)을 부과하는 것처럼 W에 대한 값이 클 경우에 penalty를 부여함. λ 값을 사용하여 얼마나 penalty를 부여할 것인지 결정함

                                           l2reg = 0.001 * tf.reduce_sum(tf.square(W))

 

반응형
Posted by CCIBOMB
Deep Learning2019. 12. 11. 19:16
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)



1.  Logistic regression (= binary classfication)

 

2. Multinomial classification

 

3. Softmax classification

 - softmax : ① sigmoid와 마찬가지로 0과 1사이의 값으로 변환, ② 변환된 결과에 대한 합계가 1이 되도록 해줌(≒ 확률)
                (tensorflow의 softmax 함수 이용)

 

 - one-hot encoding : softmax로 구한 값 중에서 가장 큰 값을 1로, 나머지를 0으로 만듦
                             (tensorflow의 argmax 함수 이용)

 

4. Cost function (=cross-entropy)

 - S(Y) = sotmax가 예측한 값

 - L = 실제 Y의 값

 - Cost function = (sotmax가 예측한 값)과 (실제 Y의 값)의 차이를 계산 = Distance(S, L)

 

 - Loss(=cost=error) = D(S,L)의 평균

 

 - Gradient descent 알고리즘 = 미분을 통해 최소비용 찾기

 

5. Logistic cost VS cross entropy

 - Logistic regression의 cost 함수 = Multinomial classification의 cross-entropy cost 함수

 

6. TensorFlow로 Softmax Classification의 구현

 - hypothesis = tf.nn.softmax(tf.matmul(X,W)+b)

 - cost = tf.reduce_mean(-tf.reduce_sum(Y*tf.log(hypothesis),axis=1))

 - optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

 

 6-1. code 구현

import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
 
x_data = [[1, 2, 1, 1],
[2, 1, 3, 2],
[3, 1, 3, 4],
[4, 1, 5, 5],
[1, 7, 5, 5],
[1, 2, 5, 6],
[1, 6, 6, 6],
[1, 7, 7, 7]]
y_data = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
 
X = tf.placeholder("float", [None, 4])     # x_data와 같은 크기의 열 가짐. 행 크기는 모름.
Y = tf.placeholder("float", [None, 3])
nb_classes = 3
 
W = tf.Variable(tf.random_normal([4, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
 
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
 
# Cross entropy cost/loss
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
 
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
 
# Launch graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
 
for step in range(2001):
_, cost_val = sess.run([optimizer, cost], feed_dict={X: x_data, Y: y_data})
 
if step % 200 == 0:
print(step, cost_val)
 
print('--------------')
# Testing & One-hot encoding
a = sess.run(hypothesis, feed_dict={X: [[1, 11, 7, 9]]})
print(a, sess.run(tf.argmax(a, 1)))
 
print('--------------')
b = sess.run(hypothesis, feed_dict={X: [[1, 3, 4, 3]]})
print(b, sess.run(tf.argmax(b, 1)))
 
print('--------------')
c = sess.run(hypothesis, feed_dict={X: [[1, 1, 0, 1]]})
print(c, sess.run(tf.argmax(c, 1)))
 
print('--------------')
all = sess.run(hypothesis, feed_dict={X: [[1, 11, 7, 9], [1, 3, 4, 3], [1, 1, 0, 1]]})
print(all, sess.run(tf.argmax(all, 1)))

 

 6-2. 결과값

(0, 6.926112)
(200, 0.60050154)
(400, 0.47295803)
(600, 0.37342972)
(800, 0.2801836)
(1000, 0.23280506)
(1200, 0.21065345)
(1400, 0.19229901)
(1600, 0.17682335)
(1800, 0.16359548)
(2000, 0.15216163)
--------------
a = (array([[1.3890485e-03, 9.9860197e-01, 9.0612575e-06]], dtype=float32), array([1]))
--------------
b = (array([[0.9311919 , 0.06290215, 0.00590591]], dtype=float32), array([0]))
--------------
c = (array([[1.2732816e-08, 3.3411387e-04, 9.9966586e-01]], dtype=float32), array([2]))
--------------
all = (array([[1.3890457e-03, 9.9860197e-01, 9.0612402e-06],
       [9.3119192e-01, 6.2902153e-02, 5.9059057e-03],
       [1.2732816e-08, 3.3411387e-04, 9.9966586e-01]], dtype=float32), array([1, 0, 2]))

 

7. TensorFlow로 Fancy Softmax Classifier 구현 (cross_entropy, one_hot, reshape)

 7-1. softmax_cross_entropy_with_logits

 - logits(=score) = tf.matmul(X, W) + b

 - hypothesis = tf.nn.softmax(logits)

 

 7-2. tensorflow 구현 (tensorflow 내장함수 이용)

# Cross entropy cost/loss

   cost = tf.reduce_mean(-tf.reduce_sum(Y*tf.log(hypothesis), axis=1)

# Cross entropy cost/loss

   cost_i = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y_one_hot)

   cost = tf.reduce_mean(cost_i)

 

 7-3. 예제 : 동물 분류(Animal classification)

 - tf.one_hot : one_hot을 사용하게 되면 하나의 차원의 수를 높여준다. 예를들어 [0, 3]의 행렬을 [[1000000], [0001000]]의 식으로 만들어 주게 된다.

 - tf.reshape : 늘어난 차원의 수를 다시 줄이기 위해, reshape 함수를 이용한다.

 

 - zoo.data : https://archive.ics.uci.edu/ml/machine-learning-databases/zoo/zoo.data

불러오는 중입니다...

 

 - 전체 소스코드

import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
 
# Predicting animal type based on various features
xy = np.loadtxt('data-04-zoo.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
 
print(x_data.shape, y_data.shape)
 
'''
(101, 16) (101, 1)
'''
 
nb_classes = 7 # 0 ~ 6
 
X = tf.placeholder(tf.float32, [None, 16]) # x_data(동물 형질 항목)의 개수
Y = tf.placeholder(tf.int32, [None, 1]) # 0 ~ 6
 
Y_one_hot = tf.one_hot(Y, nb_classes) # one hot, 차원의 수가 1차원 증가함
print("one_hot:", Y_one_hot)
Y_one_hot = tf.reshape(Y_one_hot, [-1, nb_classes])
print("reshape one_hot:", Y_one_hot)
 
'''
one_hot: Tensor("one_hot:0", shape=(?, 1, 7), dtype=float32)
reshape one_hot: Tensor("Reshape:0", shape=(?, 7), dtype=float32)
'''
 
W = tf.Variable(tf.random_normal([16, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
 
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
logits = tf.matmul(X, W) + b
hypothesis = tf.nn.softmax(logits)
 
# Cross entropy cost/loss
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits,
labels=tf.stop_gradient([Y_one_hot])))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
 
prediction = tf.argmax(hypothesis, 1)  # probability -> 0~6
correct_prediction = tf.equal(prediction, tf.argmax(Y_one_hot, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
 
# Launch graph (학습)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
 
for step in range(2001):
_, cost_val, acc_val = sess.run([optimizer, cost, accuracy], feed_dict={X: x_data, Y: y_data})
 
if step % 100 == 0:
print("Step: {:5}\tCost: {:.3f}\tAcc: {:.2%}".format(step, cost_val, acc_val))
 
# Let's see if we can predict
pred = sess.run(prediction, feed_dict={X: x_data})
# y_data: (N,1) = flatten => (N, ) matches pred.shape
for p, y in zip(pred, y_data.flatten()):
print("[{}] Prediction: {} True Y: {}".format(p == int(y), p, int(y)))

 

 - 결과값

Step:     0 Cost: 5.480 Acc: 37.62%
Step:   100 Cost: 0.806 Acc: 79.21%
Step:   200 Cost: 0.488 Acc: 88.12%
Step:   300 Cost: 0.350 Acc: 90.10%
Step:   400 Cost: 0.272 Acc: 94.06%
Step:   500 Cost: 0.222 Acc: 95.05%
Step:   600 Cost: 0.187 Acc: 97.03%
Step:   700 Cost: 0.161 Acc: 97.03%
Step:   800 Cost: 0.141 Acc: 97.03%
Step:   900 Cost: 0.124 Acc: 97.03%
Step:  1000 Cost: 0.111 Acc: 97.03%
Step:  1100 Cost: 0.101 Acc: 99.01%
Step:  1200 Cost: 0.092 Acc: 100.00%
Step:  1300 Cost: 0.084 Acc: 100.00%
Step:  1400 Cost: 0.078 Acc: 100.00%
Step:  1500 Cost: 0.072 Acc: 100.00%
Step:  1600 Cost: 0.068 Acc: 100.00%
Step:  1700 Cost: 0.064 Acc: 100.00%
Step:  1800 Cost: 0.060 Acc: 100.00%
Step:  1900 Cost: 0.057 Acc: 100.00%
Step:  2000 Cost: 0.054 Acc: 100.00%
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 3 True Y: 3
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 3 True Y: 3
[True] Prediction: 6 True Y: 6
[True] Prediction: 6 True Y: 6
[True] Prediction: 6 True Y: 6
[True] Prediction: 1 True Y: 1
[True] Prediction: 0 True Y: 0
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 1 True Y: 1
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 5 True Y: 5
[True] Prediction: 4 True Y: 4
[True] Prediction: 4 True Y: 4
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 5 True Y: 5
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 3 True Y: 3
[True] Prediction: 5 True Y: 5
[True] Prediction: 5 True Y: 5
[True] Prediction: 1 True Y: 1
[True] Prediction: 5 True Y: 5
[True] Prediction: 1 True Y: 1
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 6 True Y: 6
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 5 True Y: 5
[True] Prediction: 4 True Y: 4
[True] Prediction: 6 True Y: 6
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 1 True Y: 1
[True] Prediction: 1 True Y: 1
[True] Prediction: 1 True Y: 1
[True] Prediction: 3 True Y: 3
[True] Prediction: 3 True Y: 3
[True] Prediction: 2 True Y: 2
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 6 True Y: 6
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 2 True Y: 2
[True] Prediction: 6 True Y: 6
[True] Prediction: 1 True Y: 1
[True] Prediction: 1 True Y: 1
[True] Prediction: 2 True Y: 2
[True] Prediction: 6 True Y: 6
[True] Prediction: 3 True Y: 3
[True] Prediction: 1 True Y: 1
[True] Prediction: 0 True Y: 0
[True] Prediction: 6 True Y: 6
[True] Prediction: 3 True Y: 3
[True] Prediction: 1 True Y: 1
[True] Prediction: 5 True Y: 5
[True] Prediction: 4 True Y: 4
[True] Prediction: 2 True Y: 2
[True] Prediction: 2 True Y: 2
[True] Prediction: 3 True Y: 3
[True] Prediction: 0 True Y: 0
[True] Prediction: 0 True Y: 0
[True] Prediction: 1 True Y: 1
[True] Prediction: 0 True Y: 0
[True] Prediction: 5 True Y: 5
[True] Prediction: 0 True Y: 0
[True] Prediction: 6 True Y: 6
[True] Prediction: 1 True Y: 1

반응형
Posted by CCIBOMB
Deep Learning2019. 12. 4. 20:08
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)



1. Linear Regression

 

2. (binary) classification -> 0, 1 encoding

 - Spam E-mail Detection: Spam(0) or Ham(1)

 - Facebook feed: show(0) or hide(1)

 - Credit Card Fraudulent Transaction detection: legitimate(0) or fraud(1)

 

3. Logistic Hypothesis

 

4. Logistic Regression의 새로운 cost 함수

 

5. cost 함수의 최소화 - Gradient decent algorithm

  -> tensorflow

# cost function

cost = -tf.reduce_mean(-tf.reduce_sum(Y*tf.log(hypothesis) + (1-Y)*tf.log(1-hypothesis)))

 

# Minimize

a = tf.Variable(0.1) # Learnign rate, alpha

optimizer = tf.train.GradientDescentOptimizer(a)

train = optimizer.miminize(cost)

 

6. Tensorflow로 Logistic (regression) classifier 구현하기

 (1) Training Data

x_data = [[1, 2], [2, 3], [3, 1], [4, 3], [5, 3],[6, 2]]
y_data = [[0], [0], [0], [1], [1], [1]]
 
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 2])
Y = tf.placeholder(tf.float32, shape=[None, 1])

 

 (2) tensorflow로 hypothesis 구현

W = tf.Variable(tf.random_normal([2, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)

 

 (3) tensorflow로 cost/loss function 구현

# cost function

cost = -tf.reduce_mean(-tf.reduce_sum(Y*tf.log(hypothesis) + (1-Y)*tf.log(1-hypothesis)))

 

 (4) tensorflow로 cost 최소화 구현

# Minimize

a = tf.Variable(0.01) # Learnign rate, alpha

optimizer = tf.train.GradientDescentOptimizer(a)

train = optimizer.miminize(cost)

 

 (5) 예측 정확도 계산

# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))

 

 (6) Training the model

# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
 
for step in range(10001):
cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, cost_val)

 

 (7) 예측 정확도 출력

# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)

 

 (8) 실행결과 : 정확도 100% !!

 

7. 실제 데이터로 테스트

 (1) diabetes.csv

https://github.com/hunkim/DeepLearningZeroToAll/blob/master/data-03-diabetes.csv

 

hunkim/DeepLearningZeroToAll

TensorFlow Basic Tutorial Labs. Contribute to hunkim/DeepLearningZeroToAll development by creating an account on GitHub.

github.com

 

 (2) tensorflow 구현

# Lab 5 Logistic Regression Classifier
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
 
xy = np.loadtxt('data-03-diabetes.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
 
print(x_data.shape, y_data.shape)
 
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 8])
Y = tf.placeholder(tf.float32, shape=[None, 1])
 
W = tf.Variable(tf.random_normal([8, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(-tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
 
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
 
train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(cost)
 
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
 
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
 
for step in range(10001):
cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, cost_val)
 
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)

 

 (3) 실행결과

step, cost :

(0, 0.82793975)
(200, 0.75518084)
(400, 0.7263554)
(600, 0.70517904)
(800, 0.6866306)
(1000, 0.669853)

...

(9000, 0.49424884)
(9200, 0.49348038)
(9400, 0.49275032)
(9600, 0.49205625)
(9800, 0.49139577)
(10000, 0.4907668)

 

hypothesis :

[0.4434849 ],
[0.9153646 ],
[0.22591159],
[0.93583125],
[0.3376363 ],
[0.70926887],
[0.94409144],

...

 

correct(Y) :

[0.],
[1.],
[0.],
[1.],
[0.],
[1.],
[1.],

...

 

Accuracy :

0.7628459

반응형
Posted by CCIBOMB
Deep Learning2019. 12. 2. 22:04
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)

 



1. Loading data from file

import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
 
xy = np.loadtxt('data-01-test-score.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
 
# Make sure the shape and data are OK
print(x_data, "\nx_data shape:", x_data.shape)
print(y_data, "\ny_data shape:", y_data.shape)
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
 
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
# Hypothesis
hypothesis = tf.matmul(X, W) + b
 
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
 
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(2001):
cost_val, hy_val, _ = sess.run([cost, hypothesis, train],
feed_dict={X: x_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost:", cost_val, "\nPrediction:\n", hy_val)

1950, Cost: 2.8077145

Prediction:

array([[154.30186],
       [183.31505],
       [181.97646],
       [194.59978],
       [142.33385],
       [ 99.34767]], dtype=float32))

1960, Cost: 2.7977974

Prediction:

array([[154.296  ],
       [183.31776],
       [181.97401],
       [194.59859],
       [142.33716],
       [ 99.35353]], dtype=float32))

1970, Cost: 2.787885

Prediction:

array([[154.29016],
       [183.32051],
       [181.97154],
       [194.5974 ],
       [142.34042],
       [ 99.35938]], dtype=float32))

1980, Cost: 2.778064

Prediction:

array([[154.28435],
       [183.32324],
       [181.9691 ],
       [194.59624],
       [142.3437 ],
       [ 99.3652 ]], dtype=float32))

1990, Cost: 2.7683241

Prediction:

array([[154.27856],
       [183.32594],
       [181.96667],
       [194.59506],
       [142.34695],
       [ 99.37102]], dtype=float32))

2000, Cost: 2.7586195

Prediction:

array([[154.27278 ],
       [183.32866 ],
       [181.96426 ],
       [194.5939  ],
       [142.35019 ],
       [ 99.376816]], dtype=float32))

 

2. Loading data from multi-file

import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
 
filename_queue = tf.train.string_input_producer(
['data-01-test-score.csv'], shuffle=False, name='filename_queue')
 
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
 
# Default values, in case of empty columns. Also specifies the type of the
# decoded result.
record_defaults = [[0.], [0.], [0.], [0.]]
xy = tf.decode_csv(value, record_defaults=record_defaults)
 
# collect batches of csv in
train_x_batch, train_y_batch = \
tf.train.batch([xy[0:-1], xy[-1:]], batch_size=10)
 
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
 
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
# Hypothesis
hypothesis = tf.matmul(X, W) + b
 
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
 
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
# Start populating the filename queue.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
 
for step in range(2001):
x_batch, y_batch = sess.run([train_x_batch, train_y_batch])
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_batch, Y: y_batch})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
 
coord.request_stop()
coord.join(threads)
 
# Ask my score
print("Your score will be ",
sess.run(hypothesis, feed_dict={X: [[100, 70, 101]]}))
 
print("Other scores will be ",
sess.run(hypothesis, feed_dict={X: [[60, 70, 110], [90, 100, 80]]}))

1980, Cost: 2.2382462

Prediction:

array([[152.35132],
       [183.37514],
       [180.53424],
       [197.20535],
       [139.35315],
       [103.52445],
       [152.35132],
       [183.37514],
       [180.53424],
       [197.20535]], dtype=float32))

1990, Cost: 3.407795

Prediction:

array([[139.34067],
       [103.51443],
       [152.33455],
       [183.35727],
       [180.5155 ],
       [197.18425],
       [139.34067],
       [103.51443],
       [152.33455],
       [183.35727]], dtype=float32))

2000, Cost: 3.3214183

Prediction:

array([[180.62273],
       [197.30028],
       [139.42564],
       [103.57615],
       [152.42416],
       [183.46718],
       [180.62273],
       [197.30028],
       [139.42564],
       [103.57615]], dtype=float32))

'Your score will be ', array([[182.8681]], dtype=float32))
'Other scores will be ', array([[169.80573], [177.92252]], dtype=float32))

반응형
Posted by CCIBOMB
Deep Learning2019. 11. 29. 19:47
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)



1. Hypothesis using matrix

 - H(x1, x2, x3) = x1w1 + x2w2 + x3w3

 -> H(X) = XW

 

2. tensorflow 구현

 2-1. 기존 방법

import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
 
x1_data = [73., 93., 89., 96., 73.]
x2_data = [80., 88., 91., 98., 66.]
x3_data = [75., 93., 90., 100., 70.]
 
y_data = [152., 185., 180., 196., 142.]
 
# placeholders for a tensor that will be always fed.
x1 = tf.placeholder(tf.float32)
x2 = tf.placeholder(tf.float32)
x3 = tf.placeholder(tf.float32)
 
Y = tf.placeholder(tf.float32)
 
w1 = tf.Variable(tf.random_normal([1]), name='weight1')
w2 = tf.Variable(tf.random_normal([1]), name='weight2')
w3 = tf.Variable(tf.random_normal([1]), name='weight3')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
hypothesis = x1 * w1 + x2 * w2 + x3 * w3 + b
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize. Need a very small learning rate for this data set
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
 
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(2001):
cost_val, hy_val, _ = sess.run([cost, hypothesis, train],
feed_dict={x1: x1_data, x2: x2_data, x3: x3_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)

 

 2-2. matrix 이용

import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
 
x_data = [[73., 80., 75.],
[93., 88., 93.],
[89., 91., 90.],
[96., 98., 100.],
[73., 66., 70.]]
y_data = [[152.],
[185.],
[180.],
[196.],
[142.]]
 
 
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
 
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
 
# Hypothesis
hypothesis = tf.matmul(X, W) + b
 
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
 
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(2001):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)

 

 

반응형
Posted by CCIBOMB
Deep Learning2019. 11. 21. 19:53
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://holehouse.org/mlclass/ (note)

 

1. Hypothesis and Cost

 - 기존 Hypothesis and Cost

 

 - 단순화한 Hypothesis and Cost

 

2. Gradient descent algorithm

 - cost 최소화 등 많은 최소화 문제 해결에 활용되는 알고리즘

 - cost(W,b)의 경우, cost를 최소화하는 W, b 값을 찾아냄

 - 미분을 하여 기울기가 최소가 되는 점이 cost가 최소가 되는 점

 - linear regression을 사용할 때, cost function이 convex function(볼록함수)이 되는지 확인해야 함

   -> 그래야 정확한 값 획득 가능

 

3. Linear Regression의 cost 최소화의 Tensorflow 구현

import tensorflow as tf
import matplotlib.pyplot as plt // "pip install matplotlib" 명령어 실행을 통해 사전 설치 필요 (그래프 그려주는 라이브러리)
 
X = [1, 2, 3]
Y = [1, 2, 3]
 
W = tf.placeholder(tf.float32)
 
# Our hypothesis for linear model X * W
hypothesis = X * W
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Variables for plotting cost function
W_history = []
cost_history = []
 
# Launch the graph in a session.
with tf.Session() as sess:
for i in range(-30, 50):
curr_W = i * 0.1
curr_cost = sess.run(cost, feed_dict={W: curr_W})
 
W_history.append(curr_W)
cost_history.append(curr_cost)
 
# Show the cost function
plt.plot(W_history, cost_history)
plt.show()

 

4. Gradient descent algorithm 적용

 (1) 미분을 이용한 알고리즘 수동 구현

# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative
learning_rate = 0.1 // 알파 값 
gradient = tf.reduce_mean((W * X - Y) * X)
descent = W - learning_rate * gradient
update = W.assign(descent)

 

  - full code

import tensorflow as tf
 
tf.set_random_seed(777) # for reproducibility
 
x_data = [1, 2, 3]
y_data = [1, 2, 3]
 
# Try to find values for W and b to compute y_data = W * x_data
# We know that W should be 1
# But let's use TensorFlow to figure it out
W = tf.Variable(tf.random_normal([1]), name="weight")
 
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
 
# Our hypothesis for linear model X * W
hypothesis = X * W
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative
learning_rate = 0.1
gradient = tf.reduce_mean((W * X - Y) * X)
descent = W - learning_rate * gradient
update = W.assign(descent)
 
# Launch the graph in a session.
with tf.Session() as sess:
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(21):
_, cost_val, W_val = sess.run(
[update, cost, W], feed_dict={X: x_data, Y: y_data}
)
print(step, cost_val, W_val)

 

 (2) 텐서플로우의 Optimizer를 이용한 구현 (굳이 우리가 직접 미분하지 않아도 됨)

#Minimize: Gradient Descent Magic
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
train = optimizer.minimize(cost)

 

  - full code


import tensorflow as tf
tf.set_random_seed(777) # for reproducibility

# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]
W = tf.Variable(tf.random_normal([1], name="weight"))

# Linear model
hypothesis = X * W

# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))

# Minimize: Gradient Descent Optimizer
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

# Launch the graph in a session.
sess = tf.Session()

# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())

for step in range(101):
    _, W_val = sess.run([train, W])
    print(step, W_val)

 

 

  - W 초기값을 5로 주었을 때,

import tensorflow as tf
 
# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]
 
# Set wrong model weights
W = tf.Variable(5.0)
 
# Linear model
hypothesis = X * W
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize: Gradient Descent Optimizer
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
 
# Launch the graph in a session.
with tf.Session() as sess:
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(101):
_, W_val = sess.run([train, W])
print(step, W_val)

 

5. (Optional) Tensoflow의 Gradient 계산 및 적용

import tensorflow as tf
 
# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]
 
# Set wrong model weights
W = tf.Variable(5.)
 
# Linear model
hypothesis = X * W
 
# Manual gradient
gradient = tf.reduce_mean((W * X - Y) * X) * 2
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
 
# Minimize: Gradient Descent Optimizer
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
 
# Get gradients
gvs = optimizer.compute_gradients(cost) // optimizer의 gradients 계산
 
# Optional: modify gradient if necessary
# gvs = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gvs]
 
# Apply gradients
apply_gradients = optimizer.apply_gradients(gvs) // optimizer의 gradients 적용
 
# Launch the graph in a session.
with tf.Session() as sess:
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
for step in range(101):
gradient_val, gvs_val, _ = sess.run([gradient, gvs, apply_gradients])
print(step, gradient_val, gvs_val)

반응형
Posted by CCIBOMB
Deep Learning2019. 11. 21. 18:50
반응형

 김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

  2) http://www.holehouse.org/mlcass/ (note)



1. (Linear) Hypothesis and cost function

  * Hypothesis : H(x) = Wx + b

  * Cost function(W,b) = ( H(x) - y ) ^ 2  // How fit the line to our (training) data

  * Goal = Minimize cost

 

  2. How to minimize cost

  * 학습 : W,b 값을 조정하여 cost 값을 최소화 하는 과정

 

  (1) 그래프 생성

import tensorflow as tf
 
# X and Y data
x_train = [1, 2, 3]
y_train = [1, 2, 3]
 
# Try to find values for W and b to compute y_data = x_data * W + b
# We know that W should be 1 and b should be 0
# But let TensorFlow figure it out
W = tf.Variable(tf.random_normal([1]), name="weight")   // Variable은 다른 프로그래밍 언어의 변수와는 달리, Tensorflow가 트레이닝을 위해 사용하는 변수임
b = tf.Variable(tf.random_normal([1]), name="bias")
 
# Our hypothesis XW+b
hypothesis = x_train * W + b
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - y_train))

 

   * GradientDescent : 학습

# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)

 

  (2) 세션 실행 : 데이터 입력 및 그래프 실행

# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
# Fit the line
for step in range(2001):
  sess.run(train)
  if step % 20 == 0:
    print(step, sess.run(cost), sess.run(W), sess.run(b))

 

  (3) 그래프 업데이트 및 결과값 반환 : 학습에 의해 cost를 최소화하는 W, b 값 추론

...

(0, 3.5240757, array([2.1286771], dtype=float32), array([-0.8523567], dtype=float32))
(20, 0.19749945, array([1.533928], dtype=float32), array([-1.0505961], dtype=float32))
(40, 0.15214379, array([1.4572546], dtype=float32), array([-1.0239124], dtype=float32))
(60, 0.1379325, array([1.4308538], dtype=float32), array([-0.9779527], dtype=float32))
(80, 0.12527025, array([1.4101374], dtype=float32), array([-0.93219817], dtype=float32))
(100, 0.11377233, array([1.3908179], dtype=float32), array([-0.8884077], dtype=float32))
(120, 0.10332986, array([1.3724468], dtype=float32), array([-0.8466577], dtype=float32))
(140, 0.093845844, array([1.3549428], dtype=float32), array([-0.80686814], dtype=float32))
(160, 0.08523229, array([1.3382617], dtype=float32), array([-0.7689483], dtype=float32))
(180, 0.07740932, array([1.3223647], dtype=float32), array([-0.73281056], dtype=float32))
(200, 0.07030439, array([1.3072149], dtype=float32), array([-0.6983712], dtype=float32))
(220, 0.06385162, array([1.2927768], dtype=float32), array([-0.6655505], dtype=float32))
(240, 0.05799109, array([1.2790174], dtype=float32), array([-0.63427216], dtype=float32))
(260, 0.05266844, array([1.2659047], dtype=float32), array([-0.6044637], dtype=float32))
(280, 0.047834318, array([1.2534081], dtype=float32), array([-0.57605624], dtype=float32))
(300, 0.043443877, array([1.2414987], dtype=float32), array([-0.5489836], dtype=float32))
(320, 0.0394564, array([1.2301493], dtype=float32), array([-0.5231833], dtype=float32))
(340, 0.035834935, array([1.2193329], dtype=float32), array([-0.49859545], dtype=float32))
(360, 0.032545824, array([1.2090251], dtype=float32), array([-0.47516325], dtype=float32))
(380, 0.029558638, array([1.1992016], dtype=float32), array([-0.45283225], dtype=float32))
(400, 0.026845641, array([1.18984], dtype=float32), array([-0.4315508], dtype=float32))
(420, 0.024381675, array([1.1809182], dtype=float32), array([-0.41126958], dtype=float32))
(440, 0.02214382, array([1.1724157], dtype=float32), array([-0.39194146], dtype=float32))
(460, 0.020111356, array([1.1643128], dtype=float32), array([-0.37352163], dtype=float32))
(480, 0.018265454, array([1.1565907], dtype=float32), array([-0.35596743], dtype=float32))
(500, 0.016588978, array([1.1492316], dtype=float32), array([-0.33923826], dtype=float32))
(520, 0.015066384, array([1.1422179], dtype=float32), array([-0.3232953], dtype=float32))
(540, 0.01368351, array([1.1355343], dtype=float32), array([-0.30810148], dtype=float32))
(560, 0.012427575, array([1.1291647], dtype=float32), array([-0.29362184], dtype=float32))
(580, 0.011286932, array([1.1230947], dtype=float32), array([-0.2798227], dtype=float32))
(600, 0.010250964, array([1.1173096], dtype=float32), array([-0.26667204], dtype=float32))
(620, 0.009310094, array([1.1117964], dtype=float32), array([-0.25413945], dtype=float32))
(640, 0.008455581, array([1.1065423], dtype=float32), array([-0.24219586], dtype=float32))
(660, 0.0076795053, array([1.1015354], dtype=float32), array([-0.23081362], dtype=float32))
(680, 0.006974643, array([1.0967635], dtype=float32), array([-0.21996623], dtype=float32))
(700, 0.0063344706, array([1.0922159], dtype=float32), array([-0.20962858], dtype=float32))
(720, 0.0057530706, array([1.0878822], dtype=float32), array([-0.19977672], dtype=float32))
(740, 0.0052250377, array([1.0837522], dtype=float32), array([-0.19038804], dtype=float32))
(760, 0.004745458, array([1.0798159], dtype=float32), array([-0.18144041], dtype=float32))
(780, 0.004309906, array([1.076065], dtype=float32), array([-0.17291337], dtype=float32))
(800, 0.003914324, array([1.0724902], dtype=float32), array([-0.16478711], dtype=float32))
(820, 0.0035550483, array([1.0690835], dtype=float32), array([-0.1570428], dtype=float32))
(840, 0.0032287557, array([1.0658368], dtype=float32), array([-0.14966238], dtype=float32))
(860, 0.0029324207, array([1.0627428], dtype=float32), array([-0.14262886], dtype=float32))
(880, 0.0026632652, array([1.059794], dtype=float32), array([-0.13592596], dtype=float32))
(900, 0.0024188235, array([1.056984], dtype=float32), array([-0.12953788], dtype=float32))
(920, 0.0021968128, array([1.0543059], dtype=float32), array([-0.12345006], dtype=float32))
(940, 0.001995178, array([1.0517538], dtype=float32), array([-0.11764836], dtype=float32))
(960, 0.0018120449, array([1.0493214], dtype=float32), array([-0.11211928], dtype=float32))
(980, 0.0016457299, array([1.0470035], dtype=float32), array([-0.10685005], dtype=float32))
(1000, 0.0014946823, array([1.0447946], dtype=float32), array([-0.10182849], dtype=float32))
(1020, 0.0013574976, array([1.0426894], dtype=float32), array([-0.09704296], dtype=float32))
(1040, 0.001232898, array([1.0406833], dtype=float32), array([-0.09248237], dtype=float32))
(1060, 0.0011197334, array([1.038771], dtype=float32), array([-0.08813594], dtype=float32))
(1080, 0.0010169626, array([1.0369489], dtype=float32), array([-0.08399385], dtype=float32))
(1100, 0.0009236224, array([1.0352125], dtype=float32), array([-0.08004645], dtype=float32))
(1120, 0.0008388485, array([1.0335577], dtype=float32), array([-0.07628451], dtype=float32))
(1140, 0.0007618535, array([1.0319806], dtype=float32), array([-0.07269943], dtype=float32))
(1160, 0.0006919258, array([1.0304775], dtype=float32), array([-0.06928282], dtype=float32))
(1180, 0.00062842044, array([1.0290452], dtype=float32), array([-0.06602671], dtype=float32))
(1200, 0.0005707396, array([1.0276802], dtype=float32), array([-0.06292368], dtype=float32))
(1220, 0.00051835255, array([1.0263793], dtype=float32), array([-0.05996648], dtype=float32))
(1240, 0.00047077626, array([1.0251396], dtype=float32), array([-0.05714824], dtype=float32))
(1260, 0.00042756708, array([1.0239582], dtype=float32), array([-0.0544625], dtype=float32))
(1280, 0.00038832307, array([1.0228322], dtype=float32), array([-0.05190301], dtype=float32))
(1300, 0.00035268333, array([1.0217593], dtype=float32), array([-0.04946378], dtype=float32))
(1320, 0.0003203152, array([1.0207369], dtype=float32), array([-0.04713925], dtype=float32))
(1340, 0.0002909189, array([1.0197623], dtype=float32), array([-0.0449241], dtype=float32))
(1360, 0.00026421514, array([1.0188333], dtype=float32), array([-0.04281275], dtype=float32))
(1380, 0.0002399599, array([1.0179482], dtype=float32), array([-0.04080062], dtype=float32))
(1400, 0.00021793543, array([1.0171047], dtype=float32), array([-0.03888312], dtype=float32))
(1420, 0.00019793434, array([1.0163009], dtype=float32), array([-0.03705578], dtype=float32))
(1440, 0.00017976768, array([1.0155348], dtype=float32), array([-0.03531429], dtype=float32))
(1460, 0.00016326748, array([1.0148047], dtype=float32), array([-0.03365463], dtype=float32))
(1480, 0.00014828023, array([1.0141089], dtype=float32), array([-0.03207294], dtype=float32))
(1500, 0.00013467176, array([1.0134459], dtype=float32), array([-0.03056567], dtype=float32))
(1520, 0.00012231102, array([1.0128139], dtype=float32), array([-0.02912918], dtype=float32))
(1540, 0.0001110848, array([1.0122118], dtype=float32), array([-0.0277602], dtype=float32))
(1560, 0.000100889745, array([1.0116379], dtype=float32), array([-0.02645557], dtype=float32))
(1580, 9.162913e-05, array([1.011091], dtype=float32), array([-0.02521228], dtype=float32))
(1600, 8.322027e-05, array([1.0105698], dtype=float32), array([-0.02402747], dtype=float32))
(1620, 7.5580865e-05, array([1.0100728], dtype=float32), array([-0.02289824], dtype=float32))
(1640, 6.8643785e-05, array([1.0095996], dtype=float32), array([-0.02182201], dtype=float32))
(1660, 6.234206e-05, array([1.0091484], dtype=float32), array([-0.02079643], dtype=float32))
(1680, 5.662038e-05, array([1.0087185], dtype=float32), array([-0.01981908], dtype=float32))
(1700, 5.142322e-05, array([1.0083088], dtype=float32), array([-0.01888768], dtype=float32))
(1720, 4.6704197e-05, array([1.0079182], dtype=float32), array([-0.01800001], dtype=float32))
(1740, 4.2417145e-05, array([1.0075461], dtype=float32), array([-0.01715406], dtype=float32))
(1760, 3.852436e-05, array([1.0071915], dtype=float32), array([-0.01634789], dtype=float32))
(1780, 3.4988276e-05, array([1.0068535], dtype=float32), array([-0.01557961], dtype=float32))
(1800, 3.1776715e-05, array([1.0065314], dtype=float32), array([-0.01484741], dtype=float32))
(1820, 2.8859866e-05, array([1.0062244], dtype=float32), array([-0.0141496], dtype=float32))
(1840, 2.621177e-05, array([1.005932], dtype=float32), array([-0.01348464], dtype=float32))
(1860, 2.380544e-05, array([1.0056531], dtype=float32), array([-0.01285094], dtype=float32))
(1880, 2.1620841e-05, array([1.0053875], dtype=float32), array([-0.012247], dtype=float32))
(1900, 1.9636196e-05, array([1.0051342], dtype=float32), array([-0.01167146], dtype=float32))
(1920, 1.7834054e-05, array([1.004893], dtype=float32), array([-0.01112291], dtype=float32))
(1940, 1.6197106e-05, array([1.0046631], dtype=float32), array([-0.01060018], dtype=float32))
(1960, 1.4711059e-05, array([1.004444], dtype=float32), array([-0.01010205], dtype=float32))
(1980, 1.3360998e-05, array([1.0042351], dtype=float32), array([-0.00962736], dtype=float32))
(2000, 1.21343355e-05, array([1.0040361], dtype=float32), array([-0.00917497], dtype=float32))


  3. How to minimize cost (placeholder 이용) // 에러..

import tensorflow as tf
W = tf.Variable(tf.random_normal([1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
X= tf.placeholder(tf.float32, shape=[None])
Y= tf.placeholder(tf.float32, shape=[None])
 
# Our hypothesis XW+b
hypothesis = X * W + b
 
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y)
 
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)
 
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
 
# Fit the line with new training data
for step in range(2001):
  cost_val, W_val, b_val, _ = sess.run([cost, W, b, train], feed_dict={X: [1, 2, 3, 4, 5], Y: [2.1, 3.1, 4.1, 5.1, 6.1])
  if step % 20 == 0:
    print(step, cost_val, W_val, b_val)

(0, 1.2035878, array([1.0696986], dtype=float32), array([0.01276637], dtype=float32))
(20, 0.16904518, array([1.2650416], dtype=float32), array([0.13934135], dtype=float32))
(40, 0.14761032, array([1.2485868], dtype=float32), array([0.20250577], dtype=float32))
(60, 0.1289092, array([1.2323107], dtype=float32), array([0.26128453], dtype=float32))
(80, 0.112577364, array([1.2170966], dtype=float32), array([0.3162127], dtype=float32))
(100, 0.09831471, array([1.2028787], dtype=float32), array([0.36754355], dtype=float32))
(120, 0.08585897, array([1.189592], dtype=float32), array([0.41551268], dtype=float32))
(140, 0.07498121, array([1.1771754], dtype=float32), array([0.46034035], dtype=float32))
(160, 0.0654817, array([1.165572], dtype=float32), array([0.5022322], dtype=float32))
(180, 0.05718561, array([1.1547288], dtype=float32), array([0.54138047], dtype=float32))
(200, 0.049940635, array([1.1445953], dtype=float32), array([0.5779649], dtype=float32))
(220, 0.043613486, array([1.1351256], dtype=float32), array([0.6121535], dtype=float32))
(240, 0.038087945, array([1.1262761], dtype=float32), array([0.64410305], dtype=float32))
(260, 0.033262506, array([1.1180062], dtype=float32), array([0.6739601], dtype=float32))
(280, 0.029048424, array([1.1102779], dtype=float32), array([0.7018617], dtype=float32))
(300, 0.025368208, array([1.1030556], dtype=float32), array([0.7279361], dtype=float32))
(320, 0.022154227, array([1.0963064], dtype=float32), array([0.7523028], dtype=float32))
(340, 0.019347461, array([1.0899993], dtype=float32), array([0.7750737], dtype=float32))
(360, 0.016896311, array([1.0841053], dtype=float32), array([0.7963533], dtype=float32))
(380, 0.014755693, array([1.0785972], dtype=float32), array([0.8162392], dtype=float32))
(400, 0.012886246, array([1.0734499], dtype=float32), array([0.83482295], dtype=float32))
(420, 0.011253643, array([1.0686395], dtype=float32), array([0.85218966], dtype=float32))
(440, 0.009827888, array([1.0641443], dtype=float32), array([0.868419], dtype=float32))
(460, 0.008582776, array([1.0599433], dtype=float32), array([0.88358533], dtype=float32))
(480, 0.0074953884, array([1.0560175], dtype=float32), array([0.89775866], dtype=float32))
(500, 0.006545782, array([1.0523489], dtype=float32), array([0.9110037], dtype=float32))
(520, 0.005716468, array([1.0489205], dtype=float32), array([0.9233812], dtype=float32))
(540, 0.0049922303, array([1.0457168], dtype=float32), array([0.93494815], dtype=float32))
(560, 0.004359761, array([1.0427227], dtype=float32), array([0.94575745], dtype=float32))
(580, 0.0038074062, array([1.0399247], dtype=float32), array([0.95585895], dtype=float32))
(600, 0.0033250246, array([1.0373099], dtype=float32), array([0.96529907], dtype=float32))
(620, 0.0029037776, array([1.0348666], dtype=float32), array([0.9741207], dtype=float32))
(640, 0.0025359015, array([1.0325832], dtype=float32), array([0.9823645], dtype=float32))
(660, 0.002214623, array([1.0304493], dtype=float32), array([0.99006844], dtype=float32))
(680, 0.0019340345, array([1.028455], dtype=float32), array([0.99726814], dtype=float32))
(700, 0.00168901, array([1.0265915], dtype=float32), array([1.0039961], dtype=float32))
(720, 0.0014750187, array([1.02485], dtype=float32), array([1.0102835], dtype=float32))
(740, 0.0012881459, array([1.0232226], dtype=float32), array([1.0161589], dtype=float32))
(760, 0.0011249502, array([1.0217017], dtype=float32), array([1.0216497], dtype=float32))
(780, 0.0009824366, array([1.0202806], dtype=float32), array([1.026781], dtype=float32))
(800, 0.00085795636, array([1.0189523], dtype=float32), array([1.0315762], dtype=float32))
(820, 0.00074926845, array([1.017711], dtype=float32), array([1.0360574], dtype=float32))
(840, 0.0006543383, array([1.0165511], dtype=float32), array([1.0402449], dtype=float32))
(860, 0.00057143776, array([1.0154672], dtype=float32), array([1.0441583], dtype=float32))
(880, 0.00049904286, array([1.0144542], dtype=float32), array([1.0478154], dtype=float32))
(900, 0.0004358191, array([1.0135076], dtype=float32), array([1.0512332], dtype=float32))
(920, 0.00038059853, array([1.0126231], dtype=float32), array([1.0544269], dtype=float32))
(940, 0.00033238466, array([1.0117964], dtype=float32), array([1.0574113], dtype=float32))
(960, 0.0002902703, array([1.0110238], dtype=float32), array([1.0602009], dtype=float32))
(980, 0.00025349384, array([1.0103018], dtype=float32), array([1.0628073], dtype=float32))
(1000, 0.00022137808, array([1.009627], dtype=float32), array([1.0652432], dtype=float32))
(1020, 0.00019332914, array([1.0089965], dtype=float32), array([1.0675194], dtype=float32))
(1040, 0.00016882908, array([1.0084072], dtype=float32), array([1.0696473], dtype=float32))
(1060, 0.00014743926, array([1.0078566], dtype=float32), array([1.0716351], dtype=float32))
(1080, 0.00012875989, array([1.007342], dtype=float32), array([1.0734928], dtype=float32))
(1100, 0.00011244613, array([1.0068612], dtype=float32), array([1.0752288], dtype=float32))
(1120, 9.8200355e-05, array([1.0064118], dtype=float32), array([1.0768511], dtype=float32))
(1140, 8.5755724e-05, array([1.0059919], dtype=float32), array([1.0783674], dtype=float32))
(1160, 7.489431e-05, array([1.0055996], dtype=float32), array([1.0797837], dtype=float32))
(1180, 6.5406595e-05, array([1.0052328], dtype=float32), array([1.0811077], dtype=float32))
(1200, 5.7120622e-05, array([1.0048901], dtype=float32), array([1.0823449], dtype=float32))
(1220, 4.9882394e-05, array([1.0045699], dtype=float32), array([1.0835012], dtype=float32))
(1240, 4.3564207e-05, array([1.0042707], dtype=float32), array([1.0845816], dtype=float32))
(1260, 3.804614e-05, array([1.003991], dtype=float32), array([1.0855912], dtype=float32))
(1280, 3.3225275e-05, array([1.0037296], dtype=float32), array([1.0865349], dtype=float32))
(1300, 2.901571e-05, array([1.0034853], dtype=float32), array([1.0874166], dtype=float32))
(1320, 2.5340463e-05, array([1.003257], dtype=float32), array([1.0882409], dtype=float32))
(1340, 2.2129901e-05, array([1.0030438], dtype=float32), array([1.089011], dtype=float32))
(1360, 1.9328054e-05, array([1.0028446], dtype=float32), array([1.0897301], dtype=float32))
(1380, 1.6878726e-05, array([1.0026582], dtype=float32), array([1.0904027], dtype=float32))
(1400, 1.4740454e-05, array([1.0024842], dtype=float32), array([1.0910312], dtype=float32))
(1420, 1.2873619e-05, array([1.0023215], dtype=float32), array([1.0916185], dtype=float32))
(1440, 1.1241735e-05, array([1.0021695], dtype=float32), array([1.0921675], dtype=float32))
(1460, 9.818069e-06, array([1.0020274], dtype=float32), array([1.0926803], dtype=float32))
(1480, 8.574677e-06, array([1.0018947], dtype=float32), array([1.0931597], dtype=float32))
(1500, 7.4886166e-06, array([1.0017706], dtype=float32), array([1.0936075], dtype=float32))
(1520, 6.539272e-06, array([1.0016547], dtype=float32), array([1.0940262], dtype=float32))
(1540, 5.711003e-06, array([1.0015464], dtype=float32), array([1.0944173], dtype=float32))
(1560, 4.9874334e-06, array([1.001445], dtype=float32), array([1.094783], dtype=float32))
(1580, 4.3559958e-06, array([1.0013504], dtype=float32), array([1.0951246], dtype=float32))
(1600, 3.804345e-06, array([1.0012621], dtype=float32), array([1.0954438], dtype=float32))
(1620, 3.322312e-06, array([1.0011792], dtype=float32), array([1.0957422], dtype=float32))
(1640, 2.9007756e-06, array([1.0011021], dtype=float32), array([1.0960212], dtype=float32))
(1660, 2.5334934e-06, array([1.00103], dtype=float32), array([1.0962818], dtype=float32))
(1680, 2.2123513e-06, array([1.0009624], dtype=float32), array([1.0965253], dtype=float32))
(1700, 1.9319202e-06, array([1.0008993], dtype=float32), array([1.096753], dtype=float32))
(1720, 1.6872369e-06, array([1.0008405], dtype=float32), array([1.0969656], dtype=float32))
(1740, 1.4738443e-06, array([1.0007855], dtype=float32), array([1.0971642], dtype=float32))
(1760, 1.2871467e-06, array([1.0007341], dtype=float32), array([1.0973498], dtype=float32))
(1780, 1.12424e-06, array([1.0006859], dtype=float32), array([1.0975232], dtype=float32))
(1800, 9.815564e-07, array([1.0006411], dtype=float32), array([1.0976855], dtype=float32))
(1820, 8.573661e-07, array([1.0005993], dtype=float32), array([1.0978369], dtype=float32))
(1840, 7.4871434e-07, array([1.00056], dtype=float32), array([1.0979784], dtype=float32))
(1860, 6.5427787e-07, array([1.0005234], dtype=float32), array([1.0981107], dtype=float32))
(1880, 5.712507e-07, array([1.0004891], dtype=float32), array([1.0982342], dtype=float32))
(1900, 4.989224e-07, array([1.0004572], dtype=float32), array([1.0983498], dtype=float32))
(1920, 4.358085e-07, array([1.0004272], dtype=float32), array([1.0984578], dtype=float32))
(1940, 3.8070743e-07, array([1.0003992], dtype=float32), array([1.0985587], dtype=float32))
(1960, 3.3239553e-07, array([1.000373], dtype=float32), array([1.098653], dtype=float32))
(1980, 2.9042917e-07, array([1.0003488], dtype=float32), array([1.0987409], dtype=float32))
(2000, 2.5372992e-07, array([1.000326], dtype=float32), array([1.0988233], dtype=float32))

반응형
Posted by CCIBOMB
Deep Learning2019. 11. 19. 23:24
반응형

김성훈 교수님의 [모두를 위한 딥러닝] 강의 정리

 - https://www.youtube.com/watch?reload=9&v=BS6O0zOGX4E&feature=youtu.be&list=PLlMkM4tgfjnLSOjrEJN31gZATbcj_MpUm&fbclid=IwAR07UnOxQEOxSKkH6bQ8PzYj2vDop_J0Pbzkg3IVQeQ_zTKcXdNOwaSf_k0

 - 참고자료 : Andrew Ng's ML class

  1) https://class.coursera.org/ml-003/lecture

 

Coursera

 

class.coursera.org

  2) http://holehouse.org/mlclass/ (note)

 

Machine Learning - complete course notes

Stanford Machine Learning The following notes represent a complete, stand alone interpretation of Stanford's machine learning course presented by Professor Andrew Ng and originally posted on the ml-class.org website during the fall 2011 semester. The topic

holehouse.org

 

 

1. Tensorflow

 - '데이터 플로우 그래프(data flow graphs)'를 이용한 '수치 계산(numerical computation)'을 위한 오픈소스 라이브러리

 

2. Tensorflow 설치 : pip install --upgrade tensorflow

 

3. Tensorflow 버전확인 : tensorflow.__version__

 

4. Deep Learning 관련 소스코드

 - https://github.com/hunkim/DeepLearningZeroToAll/   (2017년경 다수 commit, 최근에는 활동이 적음)

 

Build software better, together

GitHub is where people build software. More than 40 million people use GitHub to discover, fork, and contribute to over 100 million projects.

github.com

 

5. Hello, Tensorflow

 - Input : 

# Create a constant op

# This op is added as a node to the default graph // "Hello, TensorFlow!"라는 노드 생성

hello = tf.constant("Hello, TensorFlow!")

 

# start a TF session // 다른 프로그래밍 언어와 달리, 명령어 실행을 하려면 Session 생성이 필요

sess = tf.Session()

 

# run the op and get result

print(sess.run(hello))

 

 - Output : 

Hello, TensorFlow!

 

6. 계산 그래프

 - Input : 

# 노드 1, 2, 3 생성

node1 = tf.constant(3.0, tf.float32)

node2 = tf.constant(4.0) // tf.float32 생략

node3 = tf.add(node1, node2)

 

# 계산결과 출력

sess = tf.Session()

print("sess.run(node1, node2): ", sess.run([node1, node2]))

print("sess.run(node3): ", sess.run(node3))

 

 - Output : 

 

7. TensorFlow 메커니즘

 (1) 그래프 생성

 (2) 세션 실행 : 데이터 입력 및 그래프 실행

 (3) 그래프 업데이트 및 결과값 반환

※ 그림출처 : mathwarehouse.com

 

8. Placeholder // 처음에는 값이 없지만 입력 값을 받을 수 있는 노드

 - Input : 

a = tf.placeholder(tf.float32)

b = tf.placeholder(tf.float32)

adder_node = a + b

 

print(sess.run(adder_node, feed_dict={a: 3, b: 4.5}))

print(sess.run(adder_node, feed_dict={a: [1,3], b: [2, 4]}))

 

 - Output : 

반응형
Posted by CCIBOMB