728x90

SGD 1

[Deep Learning]Optimizer ๋งค๊ฐœ๋ณ€์ˆ˜ ๊ฐฑ์‹ 

๋งค๊ฐœ๋ณ€์ˆ˜ ๊ฐฑ์‹  ์‹ ๊ฒฝ๋ง ํ•™์Šต์˜ ๋ชฉ์ ์€ ์†์‹ค ํ•จ์ˆ˜์˜ ๊ฐ’์„ ๊ฐ€๋Šฅํ•œ ํ•œ ๋‚ฎ์ถ”๋Š” ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ ์ฐพ๋Š” ๊ฒƒ ์ด๊ฒƒ์€ ๋งค๊ฐœ๋ณ€์ˆ˜์˜ ์ตœ์ ๊ฐ’์„ ์ฐพ๋Š” ๋ฌธ์ œ์ด๋ฉฐ, ์ด๋Ÿฌํ•œ ๋ฌธ์ œ๋ฅผ ํ‘ธ๋Š” ๊ฒƒ์„ ์ตœ์ ํ™”๋ผ ํ•จ. ํ™•๋ฅ ์  ๊ฒฝ์‚ฌ ํ•˜๊ฐ•๋ฒ•(SGD) class SGD: def __init__(self, lr = 0.01): self.lr = lr def update(self, params, grads): for key in params.keys(): params[key] -= self.lr * grads[key] optimizer๋Š” '์ตœ์ ํ™”๋ฅผ ํ–‰ํ•˜๋Š” ์ž'๋ผ๋Š” ๋œป ๋งค๊ฐœ๋ณ€์ˆ˜ ๊ฐฑ์‹ ์€ optimizer๊ฐ€ ์ฑ…์ž„์ง€๊ณ  ์ˆ˜ํ–‰ํ•˜๋‹ˆ optimizer์— ๋งค๊ฐœ๋ณ€์ˆ˜์™€ ๊ธฐ์šธ๊ธฐ ์ •๋ณด๋งŒ ๋„˜๊ฒจ์ฃผ๋ฉด ๋จ. SGD์˜ ๋‹จ์  ํ•จ์ˆ˜์˜ ๊ทธ๋ž˜ํ”„์™€ ๋“ฑ๊ณ ์„  [SGD์— ์˜ํ•œ ์ตœ์ ํ™” ๊ฐฑ์‹  ๊ฒฝ๋กœ: ์ตœ์†Ÿ๊ฐ’..

Deep Learning 2022.11.13
728x90