Intensive Teaching · November 2026

The most important things

Before: Join the Iliad Discord and check you can see #intensive-november. That is where you post any material you need to share with participants. If you can't see it, tell Leonard (@leonardbereska on Discord).
During: At 17:40, make sure everyone fills out the feedback form.
After: Fill out the end-of-day log: rate the day and tell us anything that went wrong.
Fill out Who stood out: name anyone who made a positive impression.

Questions about pay, travel, logistics or materials: see the FAQ.

Your days

[C.1]Intro to ML Engineering and Transformers Mon 2 Nov, leading 65 responses · energy 4.0/5

Overall +0.08 as expected n = 63

Content +0.30 better than expected n = 63

Clarity +0.06 as expected n = 63

Motivation +0.33 better than expected n = 63

Exercises -0.35 below expected n = 62

Readings +0.14 as expected n = 22

Other activities +0.50 better than expected n = 34

Pace -0.02 about right n = 63

Difficulty -0.21 about right n = 63

Novelty -0.41 mostly review n = 63

Read what they wrote99

Worked well29

Very clean explanation of the contents, and he asks roughly all the question with a interesting answer.
The fact that some of the prerequisites about transformers were covered again! The reinforcement was helpful
The mathematical background was clear
The one-to-ones And I really liked the second part of the lecture and the presentation of the background with some actual code and papers.
It's really all quite new to me, and I only started preparing about a week ago. But I got really interested and where I didn't fully understand, I still got the feeling this is in reach for me to get on top of.
The lecturer was great, energised, knowledgable and friendly.
The lecture material was relevant and easy to follow, well-curated.
The initial discussions to get to know the other participants were particularly effective. Also, the slide presentation was good, simple enough to be understood by everyone in the audience but interesting.
Icebreakers
I think the step-by-step exercises to get accustomed to Claude Code were really well designed. I had not used it before, and now I have a much clearer idea of what agentic coding entails.
The explanation is super clear
I thought the instructor had a great energy! I also thought having time to meet the other participants in the morning was very helpful and inspiring
The lectures were great, given the neccessary intro.
I felt that my answer for the previous question is pretty applicable: So far it's been super in depth and as someone who feels like I haven't had as strong a foundation in machine learning theory, despite having taken courses in it back at Duke, I found it great as both a refresher and a strength builder for my foundational skills in the field.
I really like the exercises. It forced me to use Claude Code in VS Code. I was using VS Code but for simpler type of numerics.
The broad strokes of what attention essentially is trying to accomplish in contrast to MLPs was quite clarifying. Also, I gained a lot from understanding that W_out for MLP determines fixed directions in the residual stream helped.
The initial "get to know each others" activity worked quite well to start discussions with people The speaker was well prepared and available to answer questions
The networking was definitely very productive. The course was good recap on transformers, and the exercises helped with getting some of the software in place.
Working through transformer lens examples live was particularly helpful
the teacher was very clear and well spoken
The "forced" introductions worked a lot better than I would have anticipated.
Good atmosphere. Everyone was very welcoming!
The exercises are well-organised to remove almost all friction to get our hand on it (e.g., the API keys provided, the well-structured code and folders, the use of uv as package manager).
I liked the Collab snipets shown during the class.
The discussions in the morning
The networking activities were fun and useful.
Watching a transformer being live-coded with the theory worked well for me.
I really liked the exercises at the end.
Speaker was very clear, accurate and engaging.

Would improve50

Maybe don't spent so much time on implementation details like se spoke for 20 mins about softmax today which was excessive. Separate mathematical core of the algorithm and implementation/engineering details while explaining attention etc. make it clear that all of this is a bit alchemy like -- there were repeated questions about theory/guarantees etc. for which there is no mathematical foundation as far as i'm aware and getting it out there early and not wasting time rehashing with later questions would be good
I wish I had more time to work through the Claude Code exercises, especially to try and replicate a paper or even simply to understand better how the attention mechanism work. I think that too much time was spent on ML basics in the morning. I understand that not everyone has the same background so covering having a refresher of the basics might be useful. Perhaps it would save time if that material was assigned as a pre-intensive material: setup exercises or basics for one to cover so that on day one we can immediately start to work more deeply on a single topic or sets of exercises.
- during transitions, the people trying to get attention should be louder. They sometimes spoke too quietly or were reluctant to strike the gong enough to make it loud, causing slow transitions - hard to see code, slides, etc. on the screen from more than ~40% back in the room - would have appreciated more time for the exercises, which contain a lot of practical stuff that it is harder for mathematicians to learn rather than theory that is relatively easy for them to learn. I would have gotten more out of the exercises with more time to do them. - when you have a lot of notation, slides result in working memory overload because you can fit so little on them compared to a board - It seemed like the speaker did not have a good model of the audience. A lot of explanations were in the zone "it's too much to take in if you haven't seen it before, but has little new information if you have", which is a bad place to be.
I felt like there were a lot of questions that disrupted the speaker. Consider having question time less often. Maybe a chairperson could manage how many questions get asked during question time.
It was great to know each other. I appreciate there was official time for that.
I think the claude set up was painful, and I didn't like the exercises.
I thought the intro to transformers could have been alot better. luckily I studied the subject before hand, but these are really technical things and it's really difficult to get it across in simple slides. Actually the 3blue1brown videos were probably better than this. I suppose you have to start somewhere. I was VERY jetlagged as well and nearly falling asleep
we're such a large group, it's v hard for one instructor to effectively "teach" the whole group. consider breaking people up into groups of some number (maybe 10?) and have them work together in that group on some exercise or task, and the instructor can float around the groups helping them individually. maybe also an exercise sheet for people to do indivually or in groups? in short, i feel the sessions would greatly benefit from being more interactive.
I think the screen wasn't particularly visible from various locations, but outside of that it was clear the instructor did their best to convey the general vibe
Maybe some clearer system/guidance for audience questions (when to ask/how, microphone access (though I know the sound guy was trying his best)). Maybe also a bit more brief overview ahead of time of what is the motivation behind/objective for the hands-on activity.
The exercise session felt a bit disconnected. I would have liked to spent a bit more time on understanding the implementations of attention in code.
Maybe a bit more places to sit to listen the lecture and see the slides.
I was close enough to see the display, but even in the second row, if I'd been sitting behind someone tall it would have been tough. I prepared as much as I could, but sometimes struggled with the detail. Though I think I still got quite a lot out of it. I put below that I followed parts of it. But that was perhaps quite good considering my background. This is not about the content, but in the breaks or at lunch the noise level is quite load and it can be hard to have a conversation - in that it's not too easy to hear what another person is saying.
Technical terms should be introduced with definitions/explanations - e.g. MLP should be explained when first referred to, not (much, much) later in the day after multiple references to the acronym. I suppose I would prefer to have the separate building blocks explained then fitted together, rather than being shown the high level spec first and then having its components explained.
Main problem was poor visibility of the slides when not sat near the front. The suggestions regarding a higher TV and another in the middle of the room should sort that out.
The presentation and teaching could benefit from a clearer structure. Perhaps discuss less topics at greater depths. Give an overview in the beginning before diving in. If I was not familiar with the topics presented I would have had a hard time following. Also, the temperature in the room was a bit too low (for my liking).
The coding exercises seem completely unrelated to the lecture. It would be useful to add context/motivation for such exercises. It would also be useful to show us how to perform multi-agent tasks with claude code.
Put dimensions of matrices on the slides and be more explicit with these calculations as its very hard to know what is actually going on. It was not clear that in general one actually works with a smaller dimension in each transformer head, which justifies why one should use V and W_O matricies, whose product is low rank. For me, watching the 30 minute video on the arena website gave a lot more information than this and was more clear.
The lecture was good, although a little bit slow. I would improve the visibility of the screen.
The main improvements I would suggest are logistical. It is very hard to read the screen or hear the voices, even from the middle of the room. There was also confusion about how to get Claude Code to work, which took half of the exercise time to resolve. This is unfortunate because that was the most exciting part of the day for me.
I think it's pretty good actually, but I would say we can have more breaks (I wanted to goto bath room a bit =w=, no breaks) and interactive discussions, because I believe the students here actually have a lot of insights to say and answer each others' questions. For the adam thing, I think it's not just a momentum strategy, it actually performs a bit of normalization (think about a quadratic form loss with diagonal matrix). Btw, a lot of my (newly met) friends report today as proceeding so fast (which is fine, because we would always need some preliminaries and this is supposed to be intensive). But I'm a bit worried that we are going to have it running so fast every day owo(whichi is probably true! I'm kind of ready for that) Thanks very much
Nothing
I think it was a little crowded and difficult to find a space with the number of participants! Other than that, it was good!
I hope we could be seated in the extended monitor area when teaching, so we could refer to the monitor when needed.
Found it strange that the exercises were not connected to the lecture topic at all.
The air got quite thick toward the end - is it possible to open the windows during the breaks? I do not see the reason of the excercises if we just ask the Claude to do essentially all of it. Doing it in these groups (or essentially on owr own) did not see much helpful. But the lectures today were great!
I think it could be clearer when more questions are accepted and when we need to move on in the interest of time.
So far it's been super in depth and as someone who feels like I haven't had as strong a foundation in machine learning theory, despite having taken courses in it back at Duke, I found it great as both a refresher and a strength builder for my foundational skills in the field.
I would try to get a bigger screen and a bigger space to fit us all (which also means you have been extremely successful with this call). We were rather tight.
- The size of the group might be a bit to big for the amount of seating available during the lectures.
Describe the motivation of the technique occurring at first and then the principles about the techniques. Every technology or skills occurred because of some shortcomings from former ones.
I feel that given the background does not strictly assume a lot of ML knowledge, day one perhaps should have assumed a little less understanding of jargon and familiarity of concepts from ML. Perhaps a more intuitive initial introduction, or a very clear additional board with the breakdown of jargon that stays 'fixed' throughout the day, might be quite helpful.
I thought it was a great intro - certain aspects were understandably rushed due to the volume of questions, I think if anything I would've appreciated the slides describing MLP and attention sublayers to include the dimensions of the weight matrices, as it would've been easier to parse what feeds into what, etc, particularly with attention. The steg exercises also were a bit orthogonal to much of the content of the slides, although the later exercises replicating the refusal mediated by a single direction paper were much more connected, maybe that could come before the steg exercises?
Name tags for participants would be great, especially for the first conversations but also in the next few days
Despite many questions being interesting, I feel like they took too much time from the speaker. I think crowd participation could be less disruptive while still be encouraged
The coding part seemed a bit orthogonal to the course itself; however it was very interesting. I would say it was very optimistic, as some of us just managed to get through the first 2 exercises out of the five given, so getting to replicating paper figures still seems unattainable in the time given. Perhaps it's because of the overhead of installing all the required software.
Perhaps the explanations of attention could’ve been broken down a bit better, as mathematicians we are more predisposed to thinking of geometric objects and I think some concrete examples of what query and keys do geometrically could’ve been useful (ie Nandas modular arithmetic toy example). I think, personally, I found the order of topics strange. Only dissecting MLPs after discussing the residual stream, even though MLPs are kind of like the fundamental “unit” in ML.
we're so numerous, the number of chairs or screens weren't at scale :D
Perhaps making people join on the subjects they were interested in was kind of pointless since nobody knew what the subject was about.
Overall, the pace was quite good. Even without a lot of background, I was able to follow many parts of the lecture. Only towards the end did it feel a bit rushed, and it would have been good to have a bit more time.
I think the lectures were great; I would have appreciated at the end/beggining of the transformers introduction, a "big diagram" showing the full structure of a transformer, then going over each part in detail and then returning back to the whole picture!
We are too cramped in the space before the screen. If we broadcast the presentation (maybe just the screen, not the sound), then some of us at the back of the room can follow it on the monitor. On the exercises today: quite cool as concept, but I think it can be improved to teach us (or just explain) more on the techniques (the why and why not).
There are big discrepancies between the schedule and the actual time and sequence of events during the day. It would be better to synchronise them. Also, I think today's introductions are too basic for almost everyone.
The class was good! However, the exercises were somewhat unrelated to the content. I understand that we need to learn Claude Code, but maybe we can focus on using it to develop a Transformer or something.
I think when introducing the transformer and other aspects of AI, it's super important to be clear what parts of the architecture and training and so on is principled (i.e. backed up by good theory) vs what is chosen to maximize efficiency in use of hardware (so in particular these decisions don't necessarily make sense from a theory perspective) vs what "just seems to work" and is still mysterious, and so on. I personally think one half-day is not enough to introduce the transformer. It should be considered prerequisite. I think I would have felt deeply lost if I hadn't previously spent at least a week reading and toying with them before arriving.
The Intro to ML engineering on the google doc looked more interesting to me than what we did today.
I would appreciate more detailed slides with all the technical info on transformers. Also, I think the experiments didn't help in understanding the theoretical part, but they were about experimenting with LLMs. The exercises were interesting, but I would have liked more exercises that reinforce what was explained. Also, the exercises were too long if done as intended within the time window given.
I think the coding setup takes some time, and can be very frustrating. It would be nice if there was a way to initialise the setup without the accompanying frustration.
I really liked the transformers lectures but they did not feel super well organised. Given the audience, I would have preferred a more mathematical treatment, and some of the steps of the transformer were not clear to me by the end of the lecture.
It'd be great if you could use a black/purple pen on the board, i couldn't read red because of the reflection. I would have liked a little bit more engagement during the exercises, though I got through fine.

Changed my mind14

I think now I have a better understanding of how transformers works, even better than before, as the explanations were very intuitive.
I really did like the mathematical framework behind various algorithms.
I (somewhat) learned what a transformer is!
Not much
I had a great talk with the instructor, which made me realise how a lot of the mech interp research is pretty shaky, with a lot of contradictions. This made me interested in looking at popular papers from an adversarial point of view. A next project I have in mind is for example look at the "Refusal in Language Models Is Mediated by a Single Direction" and try to find prompts where refusal does not happen in the direction predicted by the paper.
too early to tell
Not really - just an intro day
I improved my understanding of transformers today.
The one thing I had paid less thought to relative to the entire process in the transformer was the tokeniser, Byte-Pair encoding was brand new to me (possibly because it wasn't covered in the 3Blue1brown series).
I am more motivated to understand these topics, especially in the context of alignment. Working back-and-forth with Claude makes coding more interactive (but it is hard to keep track of) so this is definitely something I want to get better at.
Thinking about the fundamentals again helped me further develop intuition for things. I had not explored using transformer lens and today’s session has inspired me to continue exploring it.
This is the first day with general info on transformers, not so much to change your mind on.
Transformers seem simpler than I imagined
N/A

Other thoughts6

I am filling this out a week later so my opinions are in retrospect and perhaps tainted by that. I actually think one of the following would be better: 1. A better technical introduction to what an LLM / a transformer is 2. The day 2 material about Hugging Face and so on
no
It's a bit weird that the discord is called "Mandate of Heaven". I don't really understand why the religious component comes in there
N/A
The idea of the discussion groups at the beginning of the day was very good, but it was very short. I want more of it!
I am just logging this comment here, in case some else faces this issue with VS code on Windows. Although I had VS Code installed, I had to spend a lot of time trying to get uv to run exercise 0 in vs code. Turns out the problem was that my setup uses anaconda, and what I needed to do was setup "uv". Hopefuly, this is now fixed and should pose no futher issues.

The day

10:00 Lecture starts

12:30 Lunch

13:30 Afternoon session

17:40 Feedback form

18:00 Day ends

When	Where
Mon 2 – Fri 6 Nov	An office near LISA, address to follow
From Mon 9 Nov	LISA, 25 Holywell Row, London EC2A 4XE

Resources

Elsewhere