- About
- Courses
- Research
- Computational Social Science
- Critical Data Studies
- Data Science
- Economics and Information
- Education Technology
- Ethics, Law and Policy
- Human-Computer Interaction
- Human-Robot Interaction
- Incentives and Computation
- Infrastructure Studies
- Interface Design and Ubiquitous Computing
- Natural Language Processing
- Network Science
- Social Computing and Computer-supported Cooperative Work
- Technology and Equity
- People
- Career
- Undergraduate
- Info Sci Majors
- BA - Information Science (College of Arts & Sciences)
- BS - Information Science (CALS)
- BS - Information Science, Systems, and Technology
- MPS Early Credit Option
- Independent Research
- CPT Procedures
- Student Associations
- Undergraduate Minor in Info Sci
- Our Students and Alumni
- Graduation Info
- Contact Us
- Info Sci Majors
- Masters
- PHD
- Prospective PhD Students
- Admissions
- Degree Requirements and Curriculum
- Grad Student Orgs
- For Current PhDs
- Diversity and Inclusion
- Our Students and Alumni
- Graduation Info
- Program Contacts and Student Advising
Jason Wu is a Research Scientist at Apple in the Human-Centered Machine Intelligence group. Previously, he received a PhD in Human-Computer Interaction from Carnegie Mellon. In his research, Jason applies machine learning to optimize user interfaces for human-computer interaction. His research has resulted in over 30 publications in top venues for human-computer interaction, user interface technology, accessibility, and machine learning, where he has received several best paper awards (CHI 2021, W4A 2021, IUI 2024) and honorable mention awards (CHI 2020, CHI 2023). His work has also been recognized outside of academic conferences by a Fast Company Innovation by Design Student Finalist Award, press coverage in major outlets such as TechCrunch and AppleInsider, and by the FCC Chair Awards for Advancements in Accessibility. Jason is a recipient of the NSF Graduate Research Fellowship and was selected as a Heidelberg Laureate Forum Young Researcher.
Talk: From Agents to Optimization: User Interface Understanding and Generation
Abstract: A grand challenge in human-computer interaction (HCI) is constructing user interfaces (UIs) that make computers useful for all users across all contexts. UIs today are static, manually-constructed artifacts that limit how users and external software can interact with them. In this talk, I describe two types of machine learning approaches that transform interfaces into dynamic, computational objects that can be optimized for users, applications, and contexts. I first discuss my contributions to UI Understanding, a class of approaches that allow machines to reliably understand the semantics (content and functionality) of UIs using the input/output modalities as humans (e.g., visual perception of rendered pixels and mouse input). I show how these capabilities enhance user interaction and unlock new possibilities for systems such as assistive technology, software testing, and UI automation. Next, I discuss my contributions to UI Generation, which enables