Intensive Teaching · September 2026

The most important things

Before: Join the Iliad Discord and check you can see #intensive-september. That is where you post any material you need to share with participants. If you can't see it, tell Leonard (@leonardbereska on Discord).
During: At 17:40, make sure everyone fills out the feedback form.
After: Fill out the end-of-day log: rate the day and tell us anything that went wrong.
Fill out Who stood out: name anyone who made a positive impression.

Questions about pay, travel, logistics or materials: see the FAQ.

57 project proposals are open for review. Stars, a sentence, or both. Read them.

Your days

[B.6]Physics for Deep Learning Wed 9 Sep, leading 42 responses · energy 3.8/5

Overall +0.62 better than expected n = 39

Content +0.72 better than expected n = 39

Clarity -0.13 as expected n = 39

Motivation +0.18 as expected n = 39

Exercises +0.15 as expected n = 39

Readings +0.47 better than expected n = 38

Other activities 0.00 as expected n = 26

Pace +0.41 too fast n = 39

Difficulty +0.38 too hard n = 39

Novelty +0.72 mostly new n = 39

Read what they wrote62

Worked well20

Tudor was a very clear and engaging lecturer
I found Tudor to be a great teacher!
doing the lab of lazy vs rich models was very fun! Doing the scaling exercises was nice too.
Exercises, lecture before lunch
The presentation -- LaTeX-rendered is a love letter to Physics Academia. The breaks composition -- very balanced, and having a big break in the middle of the afternoon session -- thank you for that. The fact that lectures weren't longer than 1 hour. The exercises. It was a great day overall, I liked it.
I am very grateful that the lecturer answered questions during the breaks. These explanations really really helped my understanding. I also loved discussing the material with other participants. The reading at the end of the day was great. And the discussion was also thought-provoking. I liked the 3rd exercise sheet most.
Clear synchronisation between lectures and exercises. Solutions provided as well.
We had lenghty periods to reread and ponder the materials
I think that the discussions / exercises in between lectures was very helpful (it was nice to have a bit of a breather between lectures)
The reading and the overall picture of the field.
The content is superb. Super interesting! The exercises are good but sometimes I got a bit lost in the bigger pictures, so even if I could do the maths in the exercises it wasn't useful to do them because I didn't get the utility/what it was getting at.
I liked the teaching style. The topic was very interesting.
The exercises were well-chosen.
Pace and amount of material were adequate and not overloading
I thought Tudor was a great lecturer.
More structured breaks, and lecture lengths. Also the pdf of lecture notes and exercises with solutions is super helpful.
- exercises, extra reading, notes (super well put together)
The long break, the different exercise options. The very clear structure of lecture, exercise, break.
I though the lazy vs rich learning dichotomy was very informative.
it's nice there was more actual in-depth content today, but i was lost super quickly (no physics background)

Would improve26

Perhaps take longer time with some of the more difficult concepts like NTK, I found it difficult to follow (without a physics background)
Occasionally, variables weren't explicitly defined on the slides, so stating what they all are more explicitly would be good, especially when notation changes between topics/slides.
I would divide all this day's wroth of content into 2 or 3 days (or even a whole week!), to really go deep and give us actual tools to use from it, as opposed to just having a vague recollection of what happened.
Nothing was a really interesting day
slides are loaded with information that is sometimes glossed over quickly and could gain from more plots to visualize actual research results.
Make the connection between deep learning and physics clearer before moving onto the physics introduction. The physics introduction was a little derailed by people asking questions attempting to connect the (familiar to most) physics to deep learning.
Post lunch the slides were hard to follow
I think notation in the lectures today could be improved. I was getting lost sometimes. Same goes for exercises. The NTK was quite confusing for me and a few people around. That part should be more visual and geometric (just like LR is explained via N-dimensional vector Y being projected onto d-dimensional space of N-dimensional vectors X (d=number of features)). Also some people were in quite a disbelief that this is useful and asking "So what?" and "It's a toy model, how is it generalizable?". A better detour into Physics for how fundamental knowledge was opening the doors to new scientific directions and technological explosions (Quantum Physics, perhaps GTR), as well as giving us profound understanding of things we could hardly define in the past ("features" is one such term).
The material was quite difficult for me. I think it would be helpful if during the lecture we more often drew these connections between the physics theories and neural networks. For example we talk about Ising model, we have these nodes etc, and nodes correspond to X in neural networks. And Y in physics corresponds to Z in neural networks... etc. So to summarize, I was quite lost during the lectures today. But then during the breaks I asked questions and this helped my understanding a lot!
Today was quite nice!
Add more pictures to the presentation, maybe give more intuitions and fewer equations
Content was good, timing was a little suboptimal (limited time for exercises/readings/questions/discussions), I think it would've been nice to have a few more pictures in the slides. For example, there were a lot of questions when the Bayesian learning model was introduced. Here's a picture that I think would've helped me grasp the motivation: A 2D schematic of the parameter space, loss landscape sketched topographically. Upper panel: Widely-scattered cloud of points representing initialization distribution. Lower left panel "SGD": Cloud of points obtained by running SGD for a short time, more concentrated towards minima. Lower right panel, "Bayesian learning model": Cloud of points obtained by sampling e^{-\beta L}, also more concentrated towards minima, hopefully looks similar.
It would have helped to have a bit more details on the slides especially regarding the definition of the notations used
Another great day of content - only limitation was I felt the methods covered could have been slightly better positioned against the wider literature on end-of-learning and learning dynamics.
Potentially, having some option to make notes directly on the slides or project the presentation onto the board could be great, so that the lecturer could derive and explain some things more clearly without hesitation. This is more of a meta comment though.
I would have liked a more basic and foundational introduction to the field, and maybe some mentions to physics for ML apart from statistical physics.
Quite a bunch of regimes and models were introduced and while they were self-contained and clear in themselves, at the end of the day, I still feel a bit fuzzy as to when and how each is applied. Maybe taking like 5 minutes at the end to recap where each fit and what they do well would be useful.
Extremely interesting content which was relatively well delivered. I think the lectures were an adaptation of the content in the paper he shared ("There Will Be a Scientific Theory of Deep Learning"), and he added the necessary context to understand this well. Comments: - The motivation wasn't extremely clear. The motivation is the paper is much better and I think the lectures would benefit a lot from "copying" a bit what they did. - The prerequisities for this day are not really present, eg. I went on a rabbit hole trying to understand the differential geom part of the lecture (which I loved tbf). This can be completely skipped by doing everything in R^n, OR include in the prerequisites the tools used so that we can revise them a bit before. - Overall, there was a lot of content. This is very interesting so I think it would be valuable to have 2 days instead. Also, the slides are too dense. We have a copy of them so extend them and write things fully!!!
Very interesting day. I wish I had a stronger background in statistical physics to understand the classes better.
The teaching and excercises were very good, I have nothing to suggest improving upon!
I think the section on NTKs, while it felt well-motivated and it felt like the material covered was well-designed, was much denser than the rest of the lecture(s) - I'd probably allot more time to this section than we had. I'd say we probably didn't need as long breaks as we had. Also, I didn't really feel the chosen material covered much about phase transitions, although I understood how the Ising model was relevant in other ways.
I enjoyed there being structured lecture notes and exercises I can return to later. I enjoyed today.
very little suggestions apart from improving iliad-team website with more centralised stuff for material. exercises are good, would want more of that. would want more detailed prereqs for the day though, leadup can be hard for non physicists. the notes and readings are great it was just a little fast.
A bit more bigger picture.
I thought the slides could have had more plots on them. A lot of heavy math but not enough qualitative interpretation of it.
maybe: include some option B material for TCS people who are lost on like slide 2

Changed my mind13

All of this was genuinely new to me and I found it super interesting. Will certainly explore and think about these ideas further.
Im still thinking about this
there are more links between physics and machine learning than I expected, but it stays very complicated.
That it is actually more useful and promising that I thought -- it gives us mental models and predictive power for some things. However, it is also much harder that I thought. Also, the paper "There Will Be a Scientific Theory of Deep Learning" underplayed scaling laws -- we do have models for predicting exponents, but how successful they are is a different question.
I understand not that there may be such a thing as a "universal theory of learning mechanics".
Not really; I was hoping for more discussion of how physics of deep learning work relates to safety goals, I don't think my understanding of that picture really changed.
In the end we skimmed through a paper arguing that there will be at some point a solid theory of learning mechanics which I guess would help to consolidate the different efforts thrown into AI alignment so if this is true I think it is quite good (except if it allows even more rapid development of the AI capabilities comparatively).
Maybe because of a bias or a misconception coming from myself, I thought that predicting features of a model at a large scale had to be too unreasonable. However, to my surprise, a physical-like approach was able to be useful to predict some behaviours/laws. I found this particularly inspiring.
I learned a lot about physics of ML. It helped me understand not only physics of ML, but also the inherent concepts of points of view that physics itself is based on.
Yes- I want to have a look at this in more detail.
I didn't really see how the topics we covered today directly impacted AI safety, although I can see how research on these topics could be valuable to anyone working in AI.
A bit of increased skepticism on if the physics toy models are the solution to the AI Safety problem and unsure about how much part of the AI SAfety pie they should have.
It might be possible to theory-model deep learning.

Other thoughts3

n/a
Great teacher. Leveled with us students a lot, was very curious and into what he was teaching! Liked that he went into details a lot.
N/A

The day

10:00 Lecture starts

12:30 Lunch at LISA

13:30 Afternoon session

17:40 Feedback form

18:00 Day ends. Dinner at LISA

When	Where
Mon 7 Sep	LISA, 25 Holywell Row, London EC2A 4XE
Tue 8 – Mon 14 Sep	23 Curtain Road, London EC2A 3LT
From Tue 15 Sep	69 Wilson Street, Moorgate, London EC2A 2BB

Resources

Elsewhere