Intensive Teaching · September 2026

The most important things

Before: Join the Iliad Discord and check you can see #intensive-september. That is where you post any material you need to share with participants. If you can't see it, tell Leonard (@leonardbereska on Discord).
During: At 17:40, make sure everyone fills out the feedback form.
After: Fill out the end-of-day log: rate the day and tell us anything that went wrong.
Fill out Who stood out: name anyone who made a positive impression.

Questions about pay, travel, logistics or materials: see the FAQ.

57 project proposals are open for review. Stars, a sentence, or both. Read them.

Your days

[D.4]Agent Foundations · Wed 30 Sep Wed 30 Sep, leading 27 responses · energy 3.5/5

Overall -0.20 as expected n = 25

Content -0.12 as expected n = 25

Clarity -0.04 as expected n = 25

Motivation -0.36 below expected n = 25

Exercises -0.25 below expected n = 8

Readings +0.17 as expected n = 24

Other activities -0.05 as expected n = 20

Pace -0.28 too slow n = 25

Difficulty -0.08 about right n = 25

Novelty +0.92 mostly new n = 25

Read what they wrote35

Worked well8

Ashe is very enthusiastic and thoughtful, and engaging. That helped a lot.
I think I had high expectations which weren't fully realised, but the day was still good overall. I particularly liked the 3-person discussion.
I liked the philosophical discussion at the beginning of the day
The readings and the free flowing discussions.
Intuition for updateless decision theory and reflection.
I read about open-source game theory, which was interesting.
The speaker was clear
i liked the reading about logical induction

Would improve20

I liked this approach very much- I appreciate that it has tried to link it back to the content covered during other days.
Maybe it should be more broad and more impartial as in less lesswrong-coded, there are a few research directions that fit "agent foundations" idea well, but don't have a corresponding lesswrong post. As Iliad reaches out to people from academia and other backgrounds, it would be nice to accommodate them as well. Know thy audience.
Maybe a bit more of an overview of agendas would be nice, some broader classification perhaps, with blindspots. As the day seemed to be mostly driven by ourselves and readings. Also, the proof in the end pertaining to Intrapersonal game theory took much more time than was available to it, that needs to be fixed.
I see the value in this day, but it's hard to engage with these topics for three days (built upon earlier agent foundations that were similarly philosophical and qualitative). In particular, it is unsatisfying (as part of a 1-month intense programme) to have this many days at that level of intensity (after all, philosophical discussions are tiring) but not have some measurable progress. On the other hand, there were topics like steganography and data attribution that needed more than one or half a day to get a good handle of the mathematics upon which these fields are built. I would have liked to get a better feel for such topics, while contemplating the broader context of the work they entail in the background (as is common when carrying out any scientific research).
The come-up-with-a-project period was fun, but maybe not optimally structured for getting the most out of it. I spent a lot of time skimming various papers and aimlessly musing, didn't get very deep into anything.
Nothing, it was a great day. Perhaps, a more thorough introduction to the various research agendas would have helped.
As usual, I did not have enough time for the reading parts.
there was so much freedom. it would have been great to have a more directed teaching and program. The topics where sparse
It felt a bit hand wavey at times
I thought the day could've had more structure. The readings were very good but I didn't quite tie together for me.
Maybe would be better to bound more the agenda activity. The goal of the activity EAS not very concrete. Having discussions with other participants was great. It was less intensive than other days.
More time on logical induction trade-offs.
It felt like there was very little teaching today. I didn't find the overview at the top of the day to be very useful. The readings were fine, but reading can be done anywhere at any time.
This day could have had more time, especially given that we had three days for agent foundations, which was far more philosophical rather than quantitative.
In my opinion today needed more teaching and less reading / project development time. Most of us are seeing this stuff for the first time, so I think it's ambitious to expect us to be able to develop theories in new and interesting ways - we first need to learn and understand the material better.
I did not find the final talk was very pedagogical. It needs to be modified to make more pedagogical sense. It was too narrow. The "make your own agenda" is hard with the time we had. Agent foundations days should be back to back. It's the only way to make the "make you own agenda" activity make sense.
I found it too philosophical and not formal enough
It seems the day was little empty and lacked taught material or exercises. Given that the first day on Agent foundations was mostly theoretical, it'd be nice to derive some properties and maybe even see some models / dynamics in computational scenarios that can't be derived with pen and paper. However, maybe accounting for the fact most people are tired is also a good strategy.
actuall following in schedule more strictly and bridging the gap between very general statements and concrete theorems that we can propose and prove about agent foundations - in agent foundations day one we already went through readings with fairly high level of abstraction, so i think more concreteness could help.
I was a bit happy about "make your own agenda" as an exercise. The reason is probably that I am unhappy about the pre-paradigmatic state of AI alignment, where there's no established research directions (with established results and conjectures, senior and junior researchers, and open questions), but instead where everyone cooks his own soup and there's little feedback mechanisms and a lot of re-inventing the wheel. Maybe the exercise could have talked about this? This might also not be super specific to today, even though today's exercise triggered it.

Changed my mind5

Decision theories are much more fundamental to safety than expected.
It's interesting that MAISI is working on open-source game theory.
n/a
I updated my perspective on Open-source game theory, and I'm increasingly convinced it's one of the most important research directions (or maybe some of its extensions or modifications) to pursue. It won't solve alignment, but will definitely help in international agreements, illustrative examples and building the foundations for the agentic economy working at superhuman speed (the current one is kinda broken)
time is an important dimension to think about because not all timescales are equal for all agents

Other thoughts2

Superb continuation of agent foundations.
n/a

The day

10:00 Lecture starts

12:30 Lunch at LISA

13:30 Afternoon session

17:40 Feedback form

18:00 Day ends. Dinner at LISA

When	Where
Mon 7 Sep	LISA, 25 Holywell Row, London EC2A 4XE
Tue 8 – Mon 14 Sep	23 Curtain Road, London EC2A 3LT
From Tue 15 Sep	69 Wilson Street, Moorgate, London EC2A 2BB

Resources

Elsewhere