.. _sec_single_variable_calculus:
Cálculo de Variável Única
=========================
Em :numref:`sec_calculus`, vimos os elementos básicos do cálculo
diferencial. Esta seção dá um mergulho mais profundo nos fundamentos do
cálculo e como podemos entendê-lo e aplicá-lo no contexto do aprendizado
de máquina.
Cálculo diferencial
-------------------
O cálculo diferencial é fundamentalmente o estudo de como as funções se
comportam sob pequenas mudanças. Para ver por que isso é tão importante
para o\ *deep learning*, vamos considerar um exemplo.
Suponha que temos uma rede neural profunda onde os pesos são, por
conveniência, concatenados em um único vetor
:math:`\mathbf{w} = (w_1, \ldots, w_n)`. Dado um conjunto de dados de
treinamento, consideramos a perda de nossa rede neural neste conjunto de
dados, que escreveremos como :math:`\mathcal{L}(\mathbf{w})`.
Esta função é extraordinariamente complexa, codificando o desempenho de
todos os modelos possíveis da arquitetura dada neste conjunto de dados,
então é quase impossível dizer qual conjunto de pesos :math:`\mathbf{w}`
irá minimizar a perda. Assim, na prática, geralmente começamos
inicializando nossos pesos *aleatoriamente* e, em seguida, damos passos
pequenos iterativamente na direção que faz com que a perda diminua o
mais rápido possível.
A questão então se torna algo que superficialmente não é mais fácil:
como encontramos a direção que faz com que os pesos diminuam o mais
rápido possível? Para nos aprofundarmos nisso, vamos primeiro examinar o
caso com apenas um único peso: :math:`L(\mathbf{w}) = L(x)` para um
único valor real :math:`x`.
Vamos pegar :math:`x` e tentar entender o que acontece quando o
alteramos por uma pequena quantia para :math:`x + \epsilon`. Se você
deseja ser concreto, pense em um número como
:math:`\epsilon = 0.0000001`. Para nos ajudar a visualizar o que
acontece, vamos representar graficamente uma função de exemplo,
:math:`f(x) = \sin(x^x)`, sobre :math:`[0, 3]`.
.. raw:: html
.. raw:: html
.. code:: python
%matplotlib inline
from IPython import display
from mxnet import np, npx
from d2l import mxnet as d2l
npx.set_np()
# Plot a function in a normal range
x_big = np.arange(0.01, 3.01, 0.01)
ys = np.sin(x_big**x_big)
d2l.plot(x_big, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_3_0.svg
.. raw:: html
.. raw:: html
.. code:: python
%matplotlib inline
import torch
from IPython import display
from d2l import torch as d2l
torch.pi = torch.acos(torch.zeros(1)).item() * 2 # Define pi in torch
# Plot a function in a normal range
x_big = torch.arange(0.01, 3.01, 0.01)
ys = torch.sin(x_big**x_big)
d2l.plot(x_big, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_6_0.svg
.. raw:: html
.. raw:: html
.. code:: python
%matplotlib inline
import tensorflow as tf
from IPython import display
from d2l import tensorflow as d2l
tf.pi = tf.acos(tf.zeros(1)).numpy() * 2 # Define pi in TensorFlow
# Plot a function in a normal range
x_big = tf.range(0.01, 3.01, 0.01)
ys = tf.sin(x_big**x_big)
d2l.plot(x_big, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_9_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_med = np.arange(1.75, 2.25, 0.001)
ys = np.sin(x_med**x_med)
d2l.plot(x_med, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_15_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_med = torch.arange(1.75, 2.25, 0.001)
ys = torch.sin(x_med**x_med)
d2l.plot(x_med, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_18_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_med = tf.range(1.75, 2.25, 0.001)
ys = tf.sin(x_med**x_med)
d2l.plot(x_med, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_21_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_small = np.arange(2.0, 2.01, 0.0001)
ys = np.sin(x_small**x_small)
d2l.plot(x_small, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_27_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_small = torch.arange(2.0, 2.01, 0.0001)
ys = torch.sin(x_small**x_small)
d2l.plot(x_small, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_30_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Plot a the same function in a tiny range
x_small = tf.range(2.0, 2.01, 0.0001)
ys = tf.sin(x_small**x_small)
d2l.plot(x_small, ys, 'x', 'f(x)')
.. figure:: output_single-variable-calculus_47f3dd_33_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Define our function
def L(x):
return x**2 + 1701*(x-4)**3
# Print the difference divided by epsilon for several epsilon
for epsilon in [0.1, 0.001, 0.0001, 0.00001]:
print(f'epsilon = {epsilon:.5f} -> {(L(4+epsilon) - L(4)) / epsilon:.5f}')
.. parsed-literal::
:class: output
epsilon = 0.10000 -> 25.11000
epsilon = 0.00100 -> 8.00270
epsilon = 0.00010 -> 8.00012
epsilon = 0.00001 -> 8.00001
.. raw:: html
.. raw:: html
.. code:: python
# Define our function
def L(x):
return x**2 + 1701*(x-4)**3
# Print the difference divided by epsilon for several epsilon
for epsilon in [0.1, 0.001, 0.0001, 0.00001]:
print(f'epsilon = {epsilon:.5f} -> {(L(4+epsilon) - L(4)) / epsilon:.5f}')
.. parsed-literal::
:class: output
epsilon = 0.10000 -> 25.11000
epsilon = 0.00100 -> 8.00270
epsilon = 0.00010 -> 8.00012
epsilon = 0.00001 -> 8.00001
.. raw:: html
.. raw:: html
.. code:: python
# Define our function
def L(x):
return x**2 + 1701*(x-4)**3
# Print the difference divided by epsilon for several epsilon
for epsilon in [0.1, 0.001, 0.0001, 0.00001]:
print(f'epsilon = {epsilon:.5f} -> {(L(4+epsilon) - L(4)) / epsilon:.5f}')
.. parsed-literal::
:class: output
epsilon = 0.10000 -> 25.11000
epsilon = 0.00100 -> 8.00270
epsilon = 0.00010 -> 8.00012
epsilon = 0.00001 -> 8.00001
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = np.arange(-np.pi, np.pi, 0.01)
plots = [np.sin(xs)]
# Compute some linear approximations. Use d(sin(x)) / dx = cos(x)
for x0 in [-1.5, 0, 2]:
plots.append(np.sin(x0) + (xs - x0) * np.cos(x0))
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_51_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = torch.arange(-torch.pi, torch.pi, 0.01)
plots = [torch.sin(xs)]
# Compute some linear approximations. Use d(sin(x))/dx = cos(x)
for x0 in [-1.5, 0.0, 2.0]:
plots.append(torch.sin(torch.tensor(x0)) + (xs - x0) *
torch.cos(torch.tensor(x0)))
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_54_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = tf.range(-tf.pi, tf.pi, 0.01)
plots = [tf.sin(xs)]
# Compute some linear approximations. Use d(sin(x))/dx = cos(x)
for x0 in [-1.5, 0.0, 2.0]:
plots.append(tf.sin(tf.constant(x0)) + (xs - x0) *
tf.cos(tf.constant(x0)))
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_57_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = np.arange(-np.pi, np.pi, 0.01)
plots = [np.sin(xs)]
# Compute some quadratic approximations. Use d(sin(x)) / dx = cos(x)
for x0 in [-1.5, 0, 2]:
plots.append(np.sin(x0) + (xs - x0) * np.cos(x0) -
(xs - x0)**2 * np.sin(x0) / 2)
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_63_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = torch.arange(-torch.pi, torch.pi, 0.01)
plots = [torch.sin(xs)]
# Compute some quadratic approximations. Use d(sin(x)) / dx = cos(x)
for x0 in [-1.5, 0.0, 2.0]:
plots.append(torch.sin(torch.tensor(x0)) + (xs - x0) *
torch.cos(torch.tensor(x0)) - (xs - x0)**2 *
torch.sin(torch.tensor(x0)) / 2)
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_66_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute sin
xs = tf.range(-tf.pi, tf.pi, 0.01)
plots = [tf.sin(xs)]
# Compute some quadratic approximations. Use d(sin(x)) / dx = cos(x)
for x0 in [-1.5, 0.0, 2.0]:
plots.append(tf.sin(tf.constant(x0)) + (xs - x0) *
tf.cos(tf.constant(x0)) - (xs - x0)**2 *
tf.sin(tf.constant(x0)) / 2)
d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])
.. figure:: output_single-variable-calculus_47f3dd_69_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
.. code:: python
# Compute the exponential function
xs = np.arange(0, 3, 0.01)
ys = np.exp(xs)
# Compute a few Taylor series approximations
P1 = 1 + xs
P2 = 1 + xs + xs**2 / 2
P5 = 1 + xs + xs**2 / 2 + xs**3 / 6 + xs**4 / 24 + xs**5 / 120
d2l.plot(xs, [ys, P1, P2, P5], 'x', 'f(x)', legend=[
"Exponential", "Degree 1 Taylor Series", "Degree 2 Taylor Series",
"Degree 5 Taylor Series"])
.. figure:: output_single-variable-calculus_47f3dd_75_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute the exponential function
xs = torch.arange(0, 3, 0.01)
ys = torch.exp(xs)
# Compute a few Taylor series approximations
P1 = 1 + xs
P2 = 1 + xs + xs**2 / 2
P5 = 1 + xs + xs**2 / 2 + xs**3 / 6 + xs**4 / 24 + xs**5 / 120
d2l.plot(xs, [ys, P1, P2, P5], 'x', 'f(x)', legend=[
"Exponential", "Degree 1 Taylor Series", "Degree 2 Taylor Series",
"Degree 5 Taylor Series"])
.. figure:: output_single-variable-calculus_47f3dd_78_0.svg
.. raw:: html
.. raw:: html
.. code:: python
# Compute the exponential function
xs = tf.range(0, 3, 0.01)
ys = tf.exp(xs)
# Compute a few Taylor series approximations
P1 = 1 + xs
P2 = 1 + xs + xs**2 / 2
P5 = 1 + xs + xs**2 / 2 + xs**3 / 6 + xs**4 / 24 + xs**5 / 120
d2l.plot(xs, [ys, P1, P2, P5], 'x', 'f(x)', legend=[
"Exponential", "Degree 1 Taylor Series", "Degree 2 Taylor Series",
"Degree 5 Taylor Series"])
.. figure:: output_single-variable-calculus_47f3dd_81_0.svg
.. raw:: html
.. raw:: html
.. raw:: html
`Discussões `__
.. raw:: html
.. raw:: html
`Discussões `__
.. raw:: html
.. raw:: html
`Discussões `__
.. raw:: html
.. raw:: html