people

Hayley Hung: I lead the Human Oriented Machine Intelligence Unit at Delft University of Technology. My research focuses on developing machine perception methods to interpret scenes involving human social behavior, their decision making process and affective experience. A lot of my research involves getting to the bottom of machine perception problems when perfect data may not be available and multiple or more opportunistically available modalities could be leveraged.

I find these problems fundamentally intriguing because the notion of how to obtain, define, or model the ground truth is unclear and becomes part of the research problem, sometimes going hand in hand with a good technical solution. In the age of explainable AI, Large Language Models, and Deep learning, how can we truly say our model “works”? We need our systems to understand increasingly complex, subjective, and nuanced perspectives. Who’s to say what is a right or wrong answer when we go beyond checking factual correctness? However carefully one designs the ground truth, there is always a semantic gap.

To solve these problems, I have realised more and more the dangers of applying the same research process to solve machine perception problems. Soon the process becomes the expected template for all research papers (why not? when reviewers are often overburdened and need simple rules to reject papers quickly). Research practices in ML and applied ML often live or die by a Darwinian process of trial via overburdened reviewer. Just like short cut learning, the system or current way of doing research is perpetuating what I would call shortcut research.

I have spoken to many researchers with similar stories of having concerns about current research practices that are endangering good engineering and scientific endeavours in machine learning. But is that feeling enough? I argue that it is time to stop bemoaning the situation individually and provide a platform to ask: do we have a problem? how big is it? should we be worried? and what options do we have to make things (even) better?

Jan van Gemert: I am the head of the Computer Vision lab at TU Delft, and my main research theme is finding & evaluating powerful yet flexible physical priors for data-efficient visual recognition AI. I aim to do fundamental empirical understanding-based deep learning research, which is why I am interested in metascience for machine learning.

For me, ms4ml is about methodology. I’m not saying one methodology is ‘better’ than another, and strongly believe in that machine learners should do research however they want. Since this includes myself, I have developed over the years an understanding-based style of empirical deep learning research.

I find it important to write down, and share my style. Firstly, to communicate my expectations to students and collaborators, but also because writing them down forces me to reflect on them, where I learn and adapt my methodology according. Although I am not aiming to ‘convert’ people, as I do not think my way is ‘better’, I do believe that sharing my methodology allows others in the field to at least see that there are options in how to do machine learning research, and then still be completely free to do whatever they like with it (which is one of the reasons why I call my methodology ‘guidelines’ and not ‘rules’ ).

I am not a strong believer in stating how things ‘should be’ done, and pointing out problems (complaining?) is not something that I find constructive. Instead, I aim to build, show, and do things to demonstratively make concrete steps towards the type of machine learning research I myself aim to do.

Some examples of my effort on methodology include:

• Research organization, my guidelines on the organization of doing research, the mindset, processes, and the mentality.
• Writing is a collection of common writing issues that I ran into. I find clear communication an inherent part of doing science.
• The storyline, is my main methodological tool for empirical, understanding-based, machine learning research.

I have set up public repositories for ms4ml, which include:

• reproducedPapers.org, a hub for reproduced deep learning papers and their reproductions.
• ControlledExperimentsInMl.org for collecting, and sharing, controlled experiments for machine learning.

I am also teaching a MSc-level ms4ml course called Fundamental Research in Machine and Deep Learning.

More information can be found in my Pitch on the first workshop on ms4ml.

Marco Loog:

While Marco Loog is primarily teaching and researching machine learning, he is also interested in the broader context in which these activities take place. Marco likes to take on challenges in the field, though he also loves dreaming up new ones. Moreover, he enjoys philosophizing about machine learning in the large, but just pointing out a problem can be an occasional delight as well.