← Back

Teaching · Phys 1379

Machine Learning in the Sciences

Modern machine learning presented from a scientist's point of view, built on linear algebra, statistics and statistical mechanics, with examples drawn from physics, chemistry, biology and climate science.

E. S. Swanson · University of Pittsburgh · 3 credits · 6 November 2026

1About the course

Machine learning is in its third wave, with enormous publicity, funding and investment. Science students need to understand this tool well enough to use it effectively and to judge when it is the right tool. Phys 1379 introduces the core ideas of modern machine learning using concepts science students already know: fitting models to data, statistics, linear algebra and many-body physics.

The course is aimed at senior undergraduates and beginning graduate students. It assumes some linear algebra, some probability and statistics, and prior programming in any language. Multivariable calculus and statistical mechanics help, and the statistical-mechanics material comes with its own primer. Coding is in Python (numpy, matplotlib, scipy, scikit-learn, PyTorch and GPyTorch), introduced as it is needed.

The goal is that by the end students can follow a technical conversation about modern methods and make informed design decisions when applying them to their own data, at the level of directing an AI-assisted implementation rather than hand-coding every method.

2Topics

The course runs for about 45 contact hours in fifteen lecture units (Table 1). Short in-class mini-labs and mini-hackathons apply each unit to a scientific data set.

Table 1. Lecture units of Phys 1379.
UnitTopicMain ideas
1IntroductionWhat machine learning is; the single perceptron; gradient descent; overfitting and regularisation
2Linear algebraVectors, matrices, eigen- and singular-value decompositions
3StatisticsProbability distributions, likelihood, Bayesian inference
4OptimisationGradient descent and its variants, loss landscapes
5Feedforward networksMultilayer networks, backpropagation
6Statistical mechanicsIsing and Hopfield models, energy-based learning
7Unsupervised learningInformation theory, clustering, dimensional reduction, variational autoencoders
8Convolutional networksConvolution, pooling, image data
9TransformersAttention and the transformer architecture
10Large language modelsPre-training, fine-tuning, reinforcement learning from human feedback
11Gaussian processesRegression with uncertainty, Bayesian optimisation
12Generative modelsNormalising flows and diffusion models
13Graph networksGraph neural networks, symbolic regression
14Causal reasoningCausal graphs, robustness
15Wrap-upReview and outlook

Grading uses about eight assignments, a midterm and a final exam. The course grade is the larger of (20% assignments + 30% midterm + 50% final) and the final exam alone.

3A first look: lecture 1

Much of the terminology in the field is unfortunate. "Artificial intelligence" is a broad term for machines that mimic human behaviour, and it has been used as a marketing term since the 1950s. "Neural networks" bear only a vague resemblance to neurons. A working summary of machine learning is fitting a very poor model with a very poor method to a lot of data. That raises the question of why it works at all, which the course answers later.

Machine learning is the branch of AI in which a machine learns a task by finding patterns in large amounts of data. It grew out of computer science, gaming hardware, statistics, information theory and physics. There are three main kinds:

  • unsupervised learning: the machine finds patterns in unlabelled data, usually by clustering or compression;
  • supervised learning: the machine learns from labelled data, with the goal of labelling new data;
  • reinforcement learning: the machine learns by trial and error.

Traditional science builds models with a few parameters and aims at understanding; machine learning uses models with an enormous number of parameters and aims at prediction. The recent advances depend as much on GPU computing and very large data sets (particle physics, sky surveys, protein structures) as on new algorithms.

The single perceptron

The simplest network is a single perceptron: a map from \(I\) inputs to one output,

\[ \begin{equation} y(\vec x; \vec w, b) = \sigma(\vec w\cdot\vec x + b), \tag{1}\label{eq:perceptron} \end{equation} \]

with weights \(\vec w\), a bias \(b\), and an activation function \(\sigma\). Common choices are linear, the logistic sigmoid \(\sigma(a) = 1/(1+e^{-a})\), \(\tanh\), and the rectified linear unit (ReLU). Figure 1 shows the structure.

x₁x₂x₃ w₁w₂w₃ σ(w·x + b)bias b y
Figure 1. A single perceptron with three inputs, Eq. \eqref{eq:perceptron}. Each input is multiplied by a weight, the products and the bias are summed, and the activation function gives the output.

Training as minimisation

As a classifier, the perceptron sorts points into two classes: say five red and five blue points in the plane, with targets \(t_n = 1\) (red) and \(t_n = 0\) (blue) for inputs \(\vec x_n\). Training means choosing the weights and bias to minimise a cost function. A good choice is the relative entropy between the targets and the perceptron outputs,

\[ \begin{equation} \mathcal{C} = -\sum_n \big[ t_n \ln y_n + (1-t_n)\ln(1-y_n) \big], \tag{2}\label{eq:cost} \end{equation} \]

with \(y_n \equiv y(\vec x_n)\). It is smallest when \(y_n = t_n\) for every \(n\). The least sophisticated way to minimise it is gradient descent,

\[ \begin{equation} \vec w \to \vec w - \alpha\,\nabla_w \mathcal{C}, \tag{3}\label{eq:gd} \end{equation} \]

where \(\alpha\) is the learning rate. Updating after every input is on-line learning; updating with the whole data set is batch learning; updating with random subsets is stochastic gradient descent.

Overfitting and regularisation

When the two classes can be separated by a line, the weights grow without limit as training proceeds: the dividing line becomes ever sharper. This is overfitting, a generic feature of data analysis that ruins a model's ability to predict. One remedy is to penalise large weights with a regulator, \(\mathcal{C} \to \mathcal{C} + \tfrac{\beta}{2}\sum_i w_i^2\), so that the update becomes

\[ \begin{equation} w_i \to w_i - \alpha\,\frac{\partial\mathcal{C}}{\partial w_i} - \beta\, w_i. \tag{4}\label{eq:reg} \end{equation} \]

The rule now depends on two hyperparameters, \(\alpha\) and \(\beta\). There is no good general way to set them; experience and experimentation are the usual guides.

Assignment 1: derive the update rule for the single perceptron, then implement the perceptron and gradient descent from scratch (Python or C++) to separate two classes of points.

4Using AI in the course

AI assistants may be used on assignments: students may as well learn to use AI while learning about it. Current AI behaves like an industrious, not very bright and rather obsequious assistant, so its output has to be checked: the code must be free of bugs and must do what was intended. Students document

  • their code design: top-down or bottom-up, how it evolved, and the choice of classes, functions and parameters;
  • their unit tests: what counts as a unit, how each was checked, and the results;
  • the final code, with the requested output.

The working advice: have a plan before starting, and do not let the AI steer. Build the solution one step at a time, testing as you go, and test a method on a simple problem with a known answer before trusting it on a hard one. AI also makes a decent tutor.

5Python demos

Interactive demos built from the course code. More will be added.

6Reading