Theano (software)


Theano is a Python library and optimizing compiler for manipulating and evaluating mathematical expressions, especially matrix-valued ones.
In Theano, computations are expressed using a NumPy-esque syntax and compiled to run efficiently on either CPU or GPU architectures.
WWW deeplearning.net/software...GitHub Theano

History

Theano is an open source project primarily developed by the Montreal Institute for Learning Algorithms at the Université de Montréal.
The name of the software references the ancient philosopher Theano, long associated with the development of the golden mean.
On 28 September 2017, Pascal Lamblin posted a message from Yoshua Bengio, Head of MILA: major development would cease after the 1.0 release due to competing offerings by strong industrial players. Theano 1.0.0 was then released on 15 November 2017.
On 17 May 2018, Chris Fonnesbeck wrote on behalf of the PyMC development team that the PyMC developers will officially assume control of Theano maintenance once the MILA development team steps down. On 29 January 2021, they started using the name Aesara for their fork of Theano.
On 29 Nov 2022, the PyMC development team announced that the PyMC developers will fork the Aesara project under the name PyTensor.

Sample code

The following code is the original Theano's example. It defines a computational graph with 2 scalars and of type double and an operation between them and then creates a Python function f that does the actual computation.

import theano
from theano import tensor
  1. Declare two symbolic floating-point scalars
a = tensor.dscalar
b = tensor.dscalar
  1. Create a simple expression
c = a + b
  1. Convert the expression into a callable object that takes
  2. values as input and computes a value for c
f = theano.function
  1. Bind 1.5 to 'a', 2.5 to 'b', and evaluate 'c'
assert 4.0 f

Examples

Matrix Multiplication (Dot Product)

The following code demonstrates how to perform matrix multiplication using Theano, which is essential for linear algebra operations in many machine learning tasks.

import theano
from theano import tensor
  1. Declare two symbolic 2D arrays
A = tensor.dmatrix
B = tensor.dmatrix
  1. Define a matrix multiplication operation
C = tensor.dot
  1. Create a function that computes the result of the matrix multiplication
f = theano.function
  1. Sample matrices
A_val = 1, 2],, 7, 8
[with respect to its input. This is useful in training machine learning models.

import theano
from theano import tensor
  1. Define symbolic variables
x = tensor.dscalar # Input scalar
y = tensor.dscalar # Weight scalar
  1. Define a simple function
z = y * x
  1. Compute the gradient of z with respect to x
dz_dx = tensor.grad
  1. Create a function to compute the value of z and dz/dx
f = theano.function
  1. Sample values
x_val = 2.0
y_val = 3.0
  1. Compute z and its gradient
result = f
print # z = y * x = 3 * 2 = 6
print # dz/dx = y = 3

Building a Simple Neural Network

The following code shows how to start building a simple neural network. This is a very basic neural network with one hidden layer.

import theano
from theano import tensor as T
import numpy as np
  1. Define symbolic variables for input and output
X = T.matrix # Input features
y = T.ivector # Target labels
  1. Define the size of the layers
input_size = 2 # Number of input features
hidden_size = 3 # Number of neurons in the hidden layer
output_size = 2 # Number of output classes
  1. Initialize weights for input to hidden layer and hidden to output
W1 = theano.shared
b1 = theano.shared
W2 = theano.shared
b2 = theano.shared
  1. Define the forward pass
hidden_output = T.nnet.sigmoid # Sigmoid activation
output = T.nnet.softmax # Softmax output
  1. Define the cost function
cost = T.nnet.categorical_crossentropy.mean
  1. Compute gradients
grad_W1, grad_b1, grad_W2, grad_b2 = T.grad
  1. Create a function to compute the cost and gradients
train = theano.function
  1. Sample input data and labels
X_val = np.array
y_val = np.array
  1. Train the network for a single step
cost_val, grad_W1_val, grad_b1_val, grad_W2_val, grad_b2_val = train
print
print

Broadcasting in Theano

The following code demonstrates how broadcasting works in Theano. Broadcasting allows operations between arrays of different shapes without needing to explicitly reshape them.

import theano
from theano import tensor as T
import numpy as np
  1. Declare symbolic arrays
A = T.dmatrix
B = T.dvector
  1. Broadcast B to the shape of A, then add them
C = A + B # Broadcasting B to match the shape of A
  1. Create a function to evaluate the operation
f = theano.function
  1. Sample data
A_val = np.array
B_val = np.array
  1. Evaluate the addition with broadcasting
result = f
print