Iliad Intensive
September 2026
The most important things
- Before
- Join the Iliad Discord and check you can see
#intensive-september. That is where you post any material you need to share with participants. If you can't see it, tell Leonard (@leonardbereskaon Discord). - During
- At 17:40, make sure everyone fills out the feedback form.
- After
- Fill out the end-of-day log: rate the
day and tell us anything that went wrong.
Fill out Who stood out: name anyone who made a positive impression.
- Questions about pay, travel, logistics or materials: see the FAQ.
57 project proposals are open for review. Stars, a sentence, or both. Read them.
Your days
[C.4]Abstractions and Latents Mon 21 Sep, seconding 37 responses · energy 3.5/5
Overall -0.94 below expected n = 31
Content -0.81 below expected n = 31
Clarity -0.58 below expected n = 31
Motivation -1.03 below expected n = 31
Exercises -0.30 below expected n = 30
Readings -0.67 below expected n = 21
Other activities -0.54 below expected n = 13
Pace -0.29 too slow n = 31
Difficulty -0.16 about right n = 31
Novelty +1.13 mostly new n = 31
Read what they wrote47
Worked well9
- Reading through the notes by myself.
- having the time to read and do the exercises at my own pace
- Lecturer communicated clearly.
- - Reading a section and then doing the exercises straight after - The small group discussion
- Speaker was clear.
- Aron (?) seemed far better at explaining and communicating.
- I think the concepts are very interesting and probably something I'd like to think about more in a couple of days when proposing a project!
- Formal distinction between mediation and redundancy.
- The structure: lecture->exercise->lunch->lecture->exercise->discussion
Would improve25
- Improve the teaching and maybe the material selection
- Fewer definitions and theorem proofs, more application of methods and experiment results.
- To me, the connection to Alignment was a bit haphazard, contained in only a few bulletpoints at the end of the notes. I would prefer it to be more center stage, perhaps a presentation + discussion, could be very short. As it was, it felt very vague and swept aside. I found it difficult to focus during the lectures, partially because they were slow, but didn't go much beyond the formulas. I think they would be better if more intuition for what things mean (including questions like "what does this entropy tell us?") was explained.
- There was no motivation at all on why this matters. I did not like the teaching style. Exercises in the first half just for working with notation and really easy.
- The slides seemed completely AI generated and the concepts were not very well motivated and I think there was a shared feeling among participants that we didn't learn anything new.
- Since some point of the intensive, I feel like we are manipulating the same concepts, but using different notations. I feel that there is a big overlap among courses but these frictions make it confusing and the short time that we have to familiarize with each topic everyday make it so that we cannot really delve deeper in the different theories.
- There was very little content today. Most of the content that was present seemed to be information theory and very few new ideas and connections to AI Safety. It would be better if more new content was included. If there isn't much to be discussed as this is a very new field, it should be scrapped/merged with another similar session.
- 1. H needs to be defined with a name on the slide for entropy 2. questions were helpful in the lecture to understand the setup better, 3. in the notes label the conditional mutual information 4. add more motivation and ai safety context, how can these be tested/computed
- I think this literature (which is very niche) could have been positioned better against the wider literature on latent variable models and latent identifiability in ML.
- So many things. This day makes no sense. The research direction is very flimsy, the delivery was not very good and it's just difficult how this fits the alignment and safety direction. Iliad needs to seriously reconsider its curriculum to cover more in-depth established research programs like SLT.
- Although the lecturer presented clearly, I think the material was not well motivated. The basic problem we were trying to solve - how two agents can agree on a set of 'beliefs' - should have been more clearly communicated from the very beginning.
- - A concrete example of a natural latent early on, before the mediation / redundancy conditions - Pointing to the formal versions during the lecture (e.g. the natural latents paper) for going deeper
- I'd like to see a slightly deeper theory + connections to sparse graphical models & linear algebraic algorithms to compute the things the theory cares about. At least for the simple gaussian models to get better intuitions.
- The speaker was clear, but overall I think the material was not well motivated today. We should have begun with a clear description of what we were trying to achieve - i.e. how we could model different agents coming to the same set of 'beliefs' - and properly defined how the associated objects (observations, latents) relate to the problem.
- I found the material very poorly motivated. The lecture exposition would have benefited greatly from a roadmap. The definition theorem format gave little insight into the 'bigger picture'
- This material should be removed from the curriculum. It is absolutely not clear and not motivated why a less wrong blogpost from the last year should be taught for a whole day. Also the teaching needs to be reconsidered.
- A bit more of a link to empirical things would have helped cement the information-theoretic foundations into a more practical context.
- I didn't quite understand why we were learning this and felt a bit blindsided. I enjoyed the work on a technical level but still have little idea why it was useful for AI safety
- I would like to hear about how alignment is done. We're halfway through the intensive and I feel like I've not heard anything about alignment
- I think maybe a little more content on the algebra of information theory - the algebraic manipulations are not always very easy and even though I have some prior experience, I was defeated by the two-witness lemma.
- The material felt very (nomen omen) abstract; I struggled to see how it could be applied to any more practical settings. There was also not much substance in the material, it felt like giving new names to existing concepts. Also, I think the lecturer was quite jet lagged and it negatively affected the flow of the lecture.
- The talks seemed completely unprepared. There was little emphasis and clarity on why this matters. What is the big picture?? How do people use this with deep learning models? And for alignment?
- Everything. This felt like an opaque subject that was not properly motivated in the beginning of the lecture. The exercises in the first part were ok to disentangle the difficult language but exercises in the second part were just "proofs left for the reader" which were not very useful to me. It would have made more sense to study real models where latents are concrete and useful. The lectures should include at least one example showing the different tree vs non-tree examples of latents like was presented in the notes. The notes themselves are better than Eisenstat's paper, but still need to be clarified with notation and language that is coherent with the rest of the Iliad curriculum. Working out an example on the white board would slow the pace down and help everyone understand. I did not follow the second part of the lecture at all. No idea what was happening.
- More visual diagrams for the condensation framework.
- This was too abstract, more examples needed. Not necessarily real life examples (although that is always welcome), but examples to each statement / theorem / computation would be highly beneficial during the lecture. Also, in the very beginning of the lecture we were just submerged into the subject unlubed. Having slides allows one to show the applicability or the empowering example or an impressive result or some gradual real-world examples that help us pick up the thinking so that we can translate it into formulae later. The intro does not need to be long (5-10 mins is completely enough), and I am aware an intro runs a risk of questions arising that drag the intro out into the philosophy (as the Day 1 of SLT has shown). But it has to be there to make the immersion gradual -- some people might have well woken up 15 mins before the lecture, while others, despite traveling there for 1 hour and being for 2 more hours on their feet, do need a gradual context switch. Also, the number of links that was mentioned for exercises was way too big. Like another Iliad website.
Changed my mind7
- n/a
- n/a
- I thought the concept was very interesting and I can almost see how it ties into other topics. I would have appreciated a more holistic view on things though.
- I feel a bit frustrated and want to know more about the purpose and direction of research, and would rather engage with that kind of material. These questions are fun but it feels like a poor use of time. It's hard to choose a research direction when you don't see why you're doing it. I feel this sentiment is shared, If the situation is so urgent, we should be engaging more with actual research and what is the state of the field.
- Understanding condensation does not seem like something useful because we have no proof that LLMs understand concepts that same was humans do. Isn't that the whole point that superintelligent AI could be an existential risk? An example is neuralese!
- Natural abstractions can be mathematically formalized.
- That I probably have no wish to do dry abstract math
Other thoughts6
- Have chats in person with some of us for ideas to improve the intensive. I genuinely think that its a great initiative but there has to be some structural changes to the program.
- n/a
- n/a
- I was sick so I only attended the first half of the day
- Solid foundational theory.
- I will leave after lunch tomorrow because I have an onsite interview. That's why I can't make it to the London tour, sorry -- thank you for organizing btw! Must be very nice for non-locals.
[A.1]AI Alignment Introduction Tue 22 Sep, leading 36 responses · energy 3.8/5
Overall +0.77 better than expected n = 35
Content +0.89 better than expected n = 35
Clarity +0.91 better than expected n = 35
Motivation +1.14 better than expected n = 35
Exercises +0.30 better than expected n = 10
Readings +0.29 better than expected n = 34
Other activities +0.34 better than expected n = 32
Pace +0.11 about right n = 35
Difficulty -0.60 too easy n = 35
Novelty +0.17 an even mix n = 35
Read what they wrote58
Worked well20
- I thought Aden's lecturing was great. I thought his explanations were very clear.
- The suggested readings were engaging, but I would have preferred more time (eg by providing references in advance).
- I enjoyed thinking about broad alignment and safety questions - it was a welcome change bringing alignment and safety front and center of the materials again
- The sense of humor of the lecturer!
- This talked felt very relevant for todays issues in AI safety. It would be a very different talk 6 months ago which is a good thing.
- - The readings mixed classic papers (CEV, corrigibility) with current material (Claude's constitution) - The city walk
- the pace of change between lectures, reading and discussion was quite fast, but I think that ultimately it's a good thing
- It was a good overview of the field of alignment and introduced a lot of new ideas.
- The presenter is good. The technicalities are weak.
- Speaker was very good
- the overall content feels very relavant to the topic of AI alignment and covers argument from both sides and rather than presenting one point of view as superior than others the points of view were all given in a fairly objective manner.
- I enjoyed that the lectures strived to give the big picture instead of focussing on introductory technical details. The overview is instructive
- The level and the discussions
- discussions were informative
- The topic itself was very interested and so was the discussion.
- Best day so far. Was great to finally hear about alignment and to have discussions about perspectives on alignment. Aden was a very engaging speaker.
- Aden is fantastic and his energy is very infectious. Today really helped give a bit more context too the technical stuff we did before and was very refreshing
- Clear taxonomy of alignment agendas and London tour.
- The lecturer is very energetic and did a very good job of motivating the problems/views, which is the main objective of this lecture.
- I think the presentation of the material was great, with a good mixture of seriousness and levity. It was very evident that Aden had thoughth a lot about these things and could give general answers and a variety of perspectives on all of the (sometimes controversial) topics.
Would improve25
- There was not enough time to read the suggested readings.
- I think the timings? Somehow it seemed like we didn't have very much time for readings and discussion - for a few of them, I only managed to read a few pages before it was already time for lectures.
- More time for reading, either by giving less reading or providing it in advance.
- 1. Have this day earlier in the program, 2.have more time for reading
- This was quite good.
- - Felt compressed as a half day, and could come earlier in the course since it frames everything else - More time on critiques of the outer / inner alignment split
- On a philosophically oriented day, I think this was as good as it could be.
- I think it was a great day overall. I loved that the day was discussion focused- having well-formed opinions on things like risks and solutions can only come by discussing and stress-testing these opinions with other people. There should be more of this!
- everything good, that should be moved to an earlier point in the intensive
- Nothing. The day was excellent.
- I thought today was very good and presented material in a motivated fashion.
- Was good enough today
- Most of the staff was really good. Sometimes it was difficult to keep up with each reading session, but I suppose that's related to the short day. We still didn't fully connect it to any of the previous days. Some connections are easy to make, but the most interesting input from the lecturers could cover some non-trivial connections, that aren't immediately obvious.
- Minor quibbles on the slides: I don't think the definition of Inside vs Outside View was the same as the standard usage, the hand-wavy equation used to explain Inductive Bias was perhaps fuzzy enough to be pedagogically anti-helpful. On the big-group discussions: I liked that there was lots of back-and-forth during lecture. However, a good chunk of it seemed to quickly lose sight of the topic at hand. I felt like some fraction of the things people said were either (a) intended part of a different, non-alignment-related conversation or (b) so vaguely phrased that I couldn't understand what was meant or what role the statement was meant to serve in the discussion (Observation? Rebuttal? Thesis?). Perhaps it's possible to find a way to guide, shape, and clarify discussion without stifling it? I don't know of any magic bullet for making this happen though. On small-group discussions: Seemed like people mostly didn't finish the readings with enough spare time for a good discussion.
- I thought this day was run particularly well.
- I think today was nice in general. Maybe a bit more of time for reading on our own.
- Clickable links in the notes to the different articles that are mentioned? Save lecture notes in a different format than the beamer so it's more pleasant to take notes and follow along.
- Nothing. I really liked it today
- Best day so far. Was great to finally hear about alignment and to have discussions about perspectives on alignment. Would've been nice if this day came in week 1
- Could've been a bit more time for the reading
- More time to discuss threat models.
- Having this as the first lesson in the curriculum!
- Maybe a little more time for the reading could be good. I barely managed to get into the texts by the time we were asked to discuss, especially when I asked Aden something after the lecture (and before starting the readings).
- the slides could have been a bit better, but the teaching was great
- move this to week 1 day 1
Changed my mind10
- I do not think an AI agent cannot be goal directed.
- It is surprisingly hard to define alignment.
- n/a
- I need to read up more on how people think about alignment, not just about the concrete technical things being done to understand the capacity of the current models but about the what the future directions would look like, as an extrapolation from observations we've already made about the current models. because the current models will be the weakest the models will ever be..
- Reasons for pessimism at least feel more salient today than they did yesterday, although this may be more a change of mood than of beliefs.
- I gained some vocabularly for some of the more philosophical aspects of alignment.
- Learned a lot about current state of the art and community beliefs about AI alignment.
- I am still unsure if there is anything we can do at the moment, but the point that people in frontier labs are (perhaps)reading alignment research gave a bit of hope for the future
- Sharper distinction between capability vs alignment targets.
- Recently I have been more worried about the misuse aspects of AI, and today reinforced my concern about alignment, mostly because of the discussion around instrumental convergence.
Other thoughts3
- n/a
- I left after lunch because I wasn’t feeling well
- Tour was a great break before proposals.
[D.4]Agent Foundations · Thu 24 Sep Thu 24 Sep, leading 28 responses · energy 3.8/5
Overall +0.50 better than expected n = 26
Content +0.35 better than expected n = 26
Clarity +0.27 better than expected n = 26
Motivation +0.62 better than expected n = 26
Exercises -0.42 below expected n = 26
Readings +0.08 as expected n = 26
Other activities +0.24 as expected n = 21
Pace -0.12 about right n = 26
Difficulty -0.31 too easy n = 26
Novelty +0.96 mostly new n = 26
Read what they wrote37
Worked well12
- I bet the game would've been cool :(
- Emphasis on group discussion and activities.
- Activities and general pace were great! Loved them.
- Aden is a charismatic talker and follows a clear line of reasoning/motivation in his teaching. Very good.
- I think Aden's lecturing was again great.
- - Seeing several of these core ideas/ agendas side by side made it clearer what each one is trying to formalise and how they relate
- The game at the end was fun- something different!
- Aden's explanations of Löb's theorem and embedded agency were fantastic.
- Lecturer was very good.
- Enthusiasm, explaining the different research agendas
- The presentations were very clear and well structured.
- I did like Aden's enthusiasm! Also the freedom to choose what paper to read was very good, as well as exercises for each of the 5 research directions. The game was a fun activity as well.
Would improve15
- Today's exercises used a lot of notation to describe relatively straightforward concepts. The problems themselves also weren't as focused on teasing out the core ideas as I might've hoped. For example, proving that a binary search converges didn't add much to my understanding of why any agent that can respond appropriately to certain changes in the data distribution must've learned an implicit causal model.
- Maybe some preliminary reading would have helped.
- The reading for garrabrant induction was great but the exercise was bad and you should have put in the effort to have one awake and focussed human read it
- Overall, everything was great. It's definitely a bit dense in material, and I'm not sure I got every detail, but as a high level overview of the very broad Agent Foundations agenda, the day appeared to be great. A bit meta, but over the course, I get progressively convinced that people need some external organization and splitting people into separate rooms with whiteboards and a list of topics that are connected to read material would be better than allowing them to self-organize or work alone. At least finding such rooms and encouraging everyone to use them (groups of 5-10(max.) people would be great).
- I think more time spent on the mathematics of agent foundations, especially in the lectures, would have been good.
- I think the exercises were interesting. I would have liked to have time to work on other exercises from the other proposed topics. Today's lectures were refreshing with respect to other days, I think I enjoyed it more than most of other sessions.
- - A lot of ideas in one day! (coherence theorems, Löb's theorem and tiling agents, logical induction, decision theory). Going deeper on two or three would help - A short refresher on the formal logic before Löb's theorem, for people without a logic background
- A bit of a muddled up day. Some parts were good but it was a bit all over the place, it seemed like three lectures in one. Everything was claude generated as well- this is a pity and I actually quite dislike it.
- The exercises were a littel hard. More concrete feedback on the specific feedback form
- Nothing, I enjoy the day throughout
- TV screen for slides is too small to see (23 curtain screen was larger); hot water machine broken (no hot water).
- It was nice in general!
- I thought today was very good.
- I would add a coding demo for one of the topics.
- I don't know if it was just me, but Aden might be too loud at times. But it might've also meant that my sensitivity spiked. When it was the last lecture on cooperation, I think clarifying that we do not consider empirical results or actually mentioning them for completeness (such that in a Prisoner Dilemma tournament tit-for-tat strategy, which is cooperative by default, won) would help so that no one else raises it or points it out. The day felt very abstract, which might be the field's feature rather than the lecturer's. The game was lacking in interpretability a bit. Discussing it in more depth would've been cool. The first lecture started very strong and I was so engaged, but once we started proving a theorem rather than just going through the main statements (proof sketch), I was lost -- for me it was engagement breaker. THe first lecture was also way too long. If I remember correctly, it lasted for 1.5 hours non-stop.
Changed my mind8
- I have less confidence that the entire field is on solid ground.
- In fact, I was more or less interested in game theory for quite a long time, but struggled to find time to dig deeper. Although I think that theories that treat people as perfectly rational agents are confusing, they must work much better for future LLM agents, even not superintelligent once, and having a good theory of safe improvements could allow to handle more problems and negotiate on a wider range of topic successfully. Besides, advancements in game theory have presumably saved lots of lives before, and allowed politicians and stakeholders to build better models for negotiations. I tend to believe that it might be one of the most promising research agendas in the nearby future.
- Ironically, I am more and more skeptical of theoretical frameworks as the intensive progresses. I came in with a lot of enthusiasm for theories (I come from SLT and was expected similar things) but now I am more and more convinced that the way forward in patchy work like mech interp.
- Much clearer perspective on safe self-modification and Vingean reflection.
- Discovered applications of Game theory for agents foundations, so I could use my GT background in this direction
- n/a
- Decision theory is quite nuanced
- That mathematicians and whoever works on all of this are "the drunkards searching for keys under the lightpost cause that's where the light is." It seems like once upon a time, a mathematician relaxed the infinite compute assumption and AIXI immediately manifested and killed them. Why is there so little work on inconsistent preferences and arguments beyond Dutch Books and Money Pumps? It honestly makes me furious -- no one money pumps anyone, and the theoretical possibility does not mean that someone will appear and do that, as that requires trying to find the inconsistency in another agent's preferences and then make sure that the agent would not notice exploitation (and he almost surely will, as he is not stupid despite inconsistent preferences). Logical Induction was the only promising thing out of everything taught today. No one maximizes any utility function, that's a myth, and for that to be happening agents have to have infinite compute / time and that is never the case. No human does that, even if they claim they do, because there is no way they can execute those operations on made-up probabilities and payoffs that do not take a lot of things into account. Aurelio raised nice criticism which was similar in spirit, and it was nice of Aden to respond by two arguments: 1) We study this so we do not reinvent the wheel, as oftentimes people wonder "what if the agents works like this?" and he goes "well, we already thought about this" and 2) So that we can understand the future machine that we will build. But argument 1 is weak because you can just tell people that their assumption is unrealistic instead of launching an entire research program preemptively, while argument 2 precludes us being killed by an agent with inconsistent preferences before we get to build a consistent one. I believe the field should abandon all the convenient math and introduce realistic assumptions (bounded compute, polynomial time etc) and work with them. Otherwise it's waste of math.
Other thoughts2
- Overall great day.
- n/a
[A.2]Alignment in Practice Fri 25 Sep, leading 35 responses · energy 3.5/5
Overall +1.06 better than expected n = 32
Content +1.25 better than expected n = 32
Clarity +0.84 better than expected n = 32
Motivation +1.44 better than expected n = 32
Exercises +0.63 better than expected n = 8
Readings +0.14 as expected n = 22
Other activities +0.11 as expected n = 27
Pace 0.00 about right n = 32
Difficulty -0.13 about right n = 32
Novelty +0.50 mostly new n = 32
Read what they wrote55
Worked well15
- Great teaching!
- I think breaking up the lectures with the game/reading/etc was great, overall just a great day. Aden is an incredibly entertaining and competent lecturer :)
- - Walking through the pipeline phase by phase (pretraining, post-training, deployment) gave a clear structure for where each alignment method fits
- Practical overview of post-training affordances and empirical failure modes.
- Everything landed; best session of the programme
- The lecturer's enthusiasm and a lot of real-life examples + rumours and reports about real companies and state of the world. Aden added a lot to the lecture in this way, by telling about firms etc. Very grounded in reality lecture, I loved it. Resembled Julian days 1-2 in many ways. I also loved the Forbidden Techniques discussions, those were extremely informative. As well as train / val /test sets.
- Great energy from the speaker!
- it was great
- The small game we played was surprisingly instructive. The discussion about the progress of the "practical" alignement field was very important for me to understand the concrete steps being taken at the moment, definitely a core lecture module for the intensive. Kudos to the teacher, who managed to remain engaging and clear through a very long lecture day!
- Aden's lecturing was excellent once again. The content was chosen very well too. Very grateful for the effort Aden must have put into the last few days - thanks so much.
- The content was amazing. Felt super relevant, it was clear why its important and how its useful. It was also amazing context. Wish more days had similar content.
- I think the presentation struck very good balance between technical details and general ideas.
- Motivation with examples.
- Very relevant content and I can tell the teacher is very passionate + knowledgeable about the literature and developments of the field
- The speaker.
Would improve23
- I would need more breaks to follow the entire day
- Maybe some discussions ran a bit off-topic, but it wasn't too bad at all.
- - More time on post-training (RLHF and the methods after it) as a proportion of the day, since that is where most current alignment work happens - Links to the key empirical papers mentioned during the day, to read afterwards
- Readings didn't add a ton beyond what was already explained in lecture, or perhaps it wasn't clear what we were supposed to get out of them/which parts to focus on.
- Nothing.
- More time analyzing concrete lab safety cases.
- Nothing comes to mind
- It was cool, but it was a bit too much talking in one day. It can be split in two
- Add breaks in between 1.5-2h lectures. Also, I'd perhaps decrease the information density delivered vocally. During the lecture today, I was recalling Julian's lecture on Day 1 in MechInterp, which was exemplary in many ways. The slides were very informative and with optimal amount of info -- one figure per paper, few words per slides but many formulae because that's what belongs there. Here, on Day 15, the slides are as many (100+), and they are informative, but we do spend quite a lot of time on each I believe, which destroys pacing. Slides should be changing more often, I think.
- Awesome day.
- The lectures were really great, but I think a couple of more breaks would have been appreciated.
- maybe one more short break after lunch would have been nicer
- The lecture was very good and very well motivated. The framing of part of the content was also somewhat "historical" (i.e. following the evolution of the field). I have found that to be very helpful, and this perspective may be worth to stress further through the material. The lecture also has many short examples of techniques being used and the ideas behind them, this can be a but overwhelming at times, but also very interesting. Maybe cutting 1/4 of the examples and expanding the others would be good (the rest can be given as reading material) Overall the lecture was very successful and I would not change much!
- I think, just in terms of pacing, it was difficult to maintain focus for multiple lectures of this length, despite the fantastic lecturing and content. It being a Friday may have contributed! - maybe this schedule is better suited to earlier in the week? The game today was maybe a bit too simple to feel meaningful. I struggled to follow the section on GRPO a bit.
- Today was the best day so far!! Loved the content. Would've loved for it to be split over two days to maximise retention since the content was so so important.
- This day was awesome but could've easily been to days. We had days with a third of the content here.
- Do breaks after one hour, no 2 hour or 1.45 hour lectures
- I really enjoyed today bcs all topics are super relevant. however for my taste there was too much lecture/ passive engagement and not enough discussion/ active engagement. it might have been nice to distribute this day over 2 or 1.5 days and add more discussions in between.
- It was a very interesting day but way too long. We sat for like 6 hours of lectures and however interesting Aden is that is simply too much. The content is very useful for people (although broadly irrelevant if you've done any applied alignment stuff) so maybe it's worth splitting it.
- more discussions on whether other types of persona could be useful for us + how we expect continual learning to look like / rough predicted timeline on when it would emerge
- I thought today was the best day. Aden was a very good speaker!
- Nothing. The day was fascinating.
- Lectures were a bit long and I found myself not following at different times. Maybe switch with readings a few times during the day?
Changed my mind13
- Learned about some ai safety hackatons at ARC
- More differentiated understanding of risks
- Slightly increased optimism that, given cleverness and perfect implementation, we can find adequate ad-hoc alignment and control strategies for another year or two of scaling.
- Clearer view on how RSP commitments translate to technical evaluations.
- Safety cases feel more tractable than I assumed
- I think it stayed somewhat similar to BlueDot Technical AI Safety, just way more informed. It does tie well with some of the reasoning I was doing regards train / val / test sets as well, was very useful to hear about it again.
- i got convinced that doing safety finetuning directly on internal activations is not a good idea
- I think today's lectures made me more pessimistic about the current state of affairs than I was previously - but it is probably net positive that I'm more accurately calibrated now.
- Frontier models can get out of control very soon
- still not sure how to think about evals
- n/a
- Control techniques are more diversified than I previously thought
- I will not use the forbidden technique
Other thoughts4
- Very engaging and practically relevant.
- I can't find the MechInterp slides, could you please point me to them? Neither on Discord nor on Github nor on the websites
- How are you interpreting these answers? Like, the pace was quite fast for someone who's never seen the material but it's tautologically slow for someone who's already seen everything. How people interpret the questions will vary, so answers should be interpreted conditional on how you think most people will interpret the questions. I think you might get more information overall by having 10mins chat with each of us individually to ask questions. I have a lot of ideas (and concerns!!), and I know the same is true for many other people.
- n/a
The day
10:00 Lecture starts
12:30 Lunch at LISA
13:30 Afternoon session
17:40 Feedback form
18:00 Day ends. Dinner at LISA
| When | Where |
|---|---|
| Mon 7 Sep | LISA, 25 Holywell Row, London EC2A 4XE |
| Tue 8 – Mon 14 Sep | 23 Curtain Road, London EC2A 3LT |
| From Tue 15 Sep | 69 Wilson Street, Moorgate, London EC2A 2BB |
Resources
Elsewhere