Intensive Teaching · November 2026

The most important things

Before: Join the Iliad Discord and check you can see #intensive-november. That is where you post any material you need to share with participants. If you can't see it, tell Leonard (@leonardbereska on Discord).
During: At 17:40, make sure everyone fills out the feedback form.
After: Fill out the end-of-day log: rate the day and tell us anything that went wrong.
Fill out Who stood out: name anyone who made a positive impression.

Questions about pay, travel, logistics or materials: see the FAQ.

Your days

[D.4]Agent Foundations Thu 5 Nov, leading 53 responses · energy 3.7/5

Overall +0.04 as expected n = 49

Content -0.16 as expected n = 49

Clarity -0.22 as expected n = 49

Motivation -0.12 as expected n = 49

Exercises -0.25 below expected n = 16

Readings -0.22 as expected n = 46

Other activities +0.02 as expected n = 41

Pace -0.27 too slow n = 49

Difficulty +0.20 about right n = 49

Novelty +1.10 mostly new n = 49

Read what they wrote69

Worked well12

I liked the slides, a short text bc the important is the discussion itself.
Ashe was fantastic in, giving answers to very vague questions. He is a very talented teacher.
The discussions, and the breaking up of the day into smaller chunks
Discussion on the many fuzzy points of this was merited.
I really liked the material's philosophical undertones. It made me think about how the prompts are related to the most results in mech interp. I also had a brief discussion with Ashe, from which I got the idea of mathematically understanding the relationship between prompts and features.
The food was tasty!
I liked the discussions in groups
The begining logical induction was nice, could have spent a bit more time implementing that
Again, Ashe was great. I found the course content difficult though.
Discussion breaks were great. Also, frequent breaks did sufficiently help wrt keeping concentration
The logical induction lecture was great.
The instructor was quite engaging but the side discussion occasionally led the lecture astray.

Would improve48

The discussion is a bit disorganized.
Unfortunately, I think the lectures today lacked structure and coherence. Some parts were interesting, but it was a bit disappointing overall.
I think a whole day on this is way too much. It is really important to maintain credibility, and while this may eventually be recognized as a mainstream science, it is still far.
I would prefer if the empirics were interweaved in the theory rather than appearing at the very end when I'm a bit tired! But otherwise very good day. I think again there should be fewer different options with the readings
The course was very interesting, the glossing over many definitions in the morning felt a bit too quick, but compensated by a steadier rhythm in the afternoon.
more math :D but not everyone would agree but usually the math part is the one that i enjoy the most and also (but it's not only for this prsentation in particular) emphasize more on why it matters for alignment
The philosophical discussions today felt hard to apply to AI safety.
Add a little motivation on each reading topic, I read about AIXI (maybe because I couldn't finish all the reading) and I didn't understand the motivation of this for AI safety exactly.
The discussion drifted a lot, and the slides did not have so much detail, so I ended up thinking about other problems sometimes. But it seems the teacher didn't mind so much whatever we were doing. I don't know if this is good or not.
The morning was relatively useful, but the reading I got was pretty useless and the lecture later derailed into a usless philosphy discussion...
I'd give more structure to the presentations. Share the slides (or just an outline) the evening before on discord/email, motivate the talk and have a clear outline of the day.
More discussions I guess still? But this is a general thing.
The schedule was a bit all over the place and good be a bit simpler. Same with the reading group selection.
The clarity of the presentation.
this day was very philosophical and epistemological, and seemed to lack substance that i could engage with. i did like Ashe's slides, which were not overloaded, and conveyed a single point or two; i also liked how he added breaks at certain times and told us the structure of his teaching. but the explaining of logical induction was hard to follow; i can see though that doing that in a lecture setting like we have is probably quite a challenge.
Bit more maths
I struggled a bit with the vagueness. The maths was a bit fast! But interesting overall.
Material was too philosophical for me, would have enjoyed more technical material. Nonetheless I enjoyed the speaker, very intellectual. Also, some participants interrupted presentations with too many questions again.
I think that there's way too many questions, which kind of made the lecture hard to follow sometimes. I don't blame the speaker, but I genuinely think that more than half the time was spent answering a question. Some of them were insightful, but I'd much rather hear the speaker talk
N/A
Be louder at transitions
The logical induction slides could be made much more clear, for example by giving a description/analogy of the situation, putting in context (e.g. with the Ngo reading provided), and then introducing the math. Also, the speaker should cut participants who ask too many (repeating) questions so that everyone can enjoy the lecture.
A bit too vague. Not really clear goal, what we were doing/wanted to do. I never understood what we were doing/what it was about.
A lot of information to absorb, maybe there is no better strategy here, but some directed questions to discuss I think might have been super helpful. We had some moments for discussion but they were too open ended and could have done with prompt questions, the mental fatigue of previous days is building so that I struggled to formulate some deeper questions myself. Given the intensive nature of the programme I might have prefered slightly less content and more guided questions and exercises to think about, based on the content. It would have been easier to engage with, at a point where brain is already quite saturated with new ideas on day 4. But overall, very interesting. Thanks
Satisfied, only need to add some motivations about some concepts
Would be interested to learn more specifics about the kind of research people are currently doing in this area.
It was very hard to hear the things the teacher was saying, and I would improve the sound quality mostly.
The part about the logical induction criterion was a bit dense since we didn't know all the definition clearly, and it was not entirely clear where the discussion was headed.
It would have helped if the speaker repeated the questions from the audience, and maybe held the mic a bit closer. Some parts felt a bit disorganized too - for example the reading groups part.
I would have preferred a bit more formal background in agent foundation
Nothing! It was super interesting today.
I really liked a lot of the concepts today, but I feel like we could have spent more time looking at practical examples of the concepts
Spending more time on fundamental concrete things before jumping to epmpirical experiments
I have no suggestions for improvement; I thought Ashe was a fantastic lecturer and the structure of the day was great. I did find the course content difficult compared to the previous days, but I attribute that more to my lack of philosophical tendencies than the material being unclear. Although, since I am probably not alone, it may actually be beneficial to give a more structured motivation as to why this philosophical viewpoint of agents is useful for AI safety as I was somewhat confused by this.
Some hand on experiences / practical test / exercises /exercises / practical demonstrations that one could have run/tests in their own laptop would have been helpful
I appreciate somehow the general phylosophical framework upon which the lectures were built, but in all honesty I couldn't follow anything out of pure boredom. I would have preferred a more grounded way of teaching, either fully mathematical (like the RL lecture from yesterday) or in coding. Moreover, in the reading tasks, it seems like "dubious" resources were suggested, I would avoid proposing such lesswrong posts as proper scientific reading (with non peer reviewed arxiv on 50 pages of theory without any empirical validation).
Perhaps the slides could give a little more intuition rather than as many definitions (there was plenty of intuition given vocally, but if one tunes out, it would be useful to also get intuition from the slides).
I do not have any suggestions for today
It would’ve been great if we could have delved a bit more into mathematical formalism wrt agentic formalism. I felt like we perhaps diverged away from this as the lectures progressed which I can also appreciate was an artefact of increased difficulty to define objects.
Overall I liked the lectures but at the same time I think that they did not cover the background frameworks (or the problems the proposed approaches try to solve) enough in detail to engage productively with the main readings.
Can we have access to the slides the day before the lecture? Would be nice to check if I have to review some background before the lecture.
Maybe some toy example in the first lectures could have helped
The mathematics in this lecture were not clear to me. I believe this is, in part, due to the presentation.
No improvements
I have no further comments, I think the class was very interesting today and it was well organized
Clearer explanation (/motivation) of how logical induction relates to AI safety.
I think the class got too dragged due to discussions, and many of the questions were addressed or answered some slides after. I think it would be better to organize the lectures so that for 10-15 minutes we are shown some concrete piece of theory and then discussion, limited to that what was shown, is done later.
I think we went overboard with the philosophical musings today. I would have liked to see more of the "logical induction" paper being discussed at length.

Changed my mind7

The question wheter could exist a widely applicable theory of agency was very insightful, I think is an important philosophical question for the future of AI. I'll think more on it.
I got introduced to some new (for me) ideas in agent foundations which may be somehow tractable. So I think I updated a bit towards the tractability of agents foundation research.
RL and embedded agency. I had never learned of how embedding the agent can break the theory so much. Bayesian inference: I didn't know logical inference and found it very interesting. This was the most thought provoking lecture so far for sure.
I learned about Löb's theorem today, which was surprisingly surprising and made me reconsider if mathematical logic might be cool. I am still pretty uncertain of the applicability to AI safety though.
N/A
Again, some of these things are incredibly hard to define without getting very philosophical. But here is where I see the remit of trying to do this using mathematics
Turning english into rigorous math is hard. i've long known this but it always nice to be reminded of it.

Other thoughts2

I can't be here tomorrow morning, as I have to run a tutorial session, but I will come in in the afternoon, around 2pm most likely.
N/a

The day

10:00 Lecture starts

12:30 Lunch

13:30 Afternoon session

17:40 Feedback form

18:00 Day ends

When	Where
Mon 2 – Fri 6 Nov	An office near LISA, address to follow
From Mon 9 Nov	LISA, 25 Holywell Row, London EC2A 4XE

Resources

Elsewhere