Mehrnaz Sabet is a doctoral candidate in information science with a minor in computer science from Shiraz, Iran. She earned her B.Sc. in computer engineering at the University of Tehran and now studies human-autonomy learning under the guidance of Susan Fussell at Cornell.

What is your area of research and why is it important?

I work on autonomous systems, and I study human-autonomy teaming, which is basically looking at how to advance a robot’s ability to perform tasks without external control when they are to conduct those tasks by working alongside humans. More specifically, my research is focused on aerial autonomy, so I design and develop drone systems to understand how we can reliably increase drone autonomy when drones are deployed to work with humans in time-critical applications such as search and rescue operations.

What are the larger implications of this research?

Increasing advancements in robot autonomy call for a shift in the human role from being constantly in control to control by exception. As we push for increased autonomy, specifically in drones, it becomes more and more important to understand how we design and advance autonomy in a way that is not only reliable but is also reflective of the real-world operation scenarios where these drones would need to work alongside other actors in various missions and environments. Without such autonomy, there are increasing risks for failures and increased workload for humans to manually operate the drones instead of focusing on more critical tasks. 

You are currently working to build an industry-academia partnership with the goal of inspiring applied research projects in academia informed by industry challenges with autonomous systems. Can you tell us about the project?

This project is called Shaping Autonomy with the mission to bridge the gap between industry and academia by bringing the research and engineering community together to advance the field of autonomous systems. In my own work, I’ve had the opportunity to talk with many industry experts to learn about the challenges and use those learnings to inform my research path. While doing that, I realized that many other students are working on problems that are not necessarily informed by real-world applications or are struggling to make their contributions meaningful to the industry. Meanwhile, the industry seeks to leverage research through applied projects to address the technical challenges. I believe this gap calls for a new collaborative model to be adapted in this area, and this is what motivates Shaping Autonomy.

What do you hope to accomplish through this industry-academia partnership and with this project?

I’m looking forward to bringing awareness to the challenges that exist in the industry so that we can use those challenges to inspire research projects with real-world applications. I’ve talked with many graduate students in this field who wish they could find the applications of their research and understand a project’s impact on the industry. The outcome that motivates me the most is to enable multi-institutional collaborations for projects and papers so that we can work on proposing solutions for critical problems that go beyond a single lab, or team, which I believe can truly make a difference in advancing the field of autonomous systems and shaping its future.

President Pollack has designated this academic year’s theme as freedom of expression. What does freedom of expression mean to you?

A big part of freedom of expression for me is to pursue new ideas, especially in an academic environment. I think I felt this freedom by heart when starting this project because as soon as this was started, people from both industry and academia reached out to me to talk about how there has always been a need for someone to take the initiative and how much this is needed for the field and I’m happy I did it. It feels like I took a step forward toward this year’s theme to feel free and fearless to embrace new ideas.

What are your hobbies or interests outside of your research or scholarship?

I love art, and it gives me peace; I do graphic design and photography whenever possible. I also love to read, and I often grab a book and spend my time reading when I’m not working. 

Why did you choose Cornell to pursue your degree?

The biggest thing for me is the collaborative environment that we have at Cornell. My field is quite multi-disciplinary, and without collaboration across the fields relating to my research, I wouldn’t have been able to look at a problem from different perspectives. I think we have a unique environment where I personally have been able to work with people from different departments and learn about different things, which has helped me advance in my path. This spirit of collaboration has also helped me to inspire projects like Shaping Autonomy, which is built on bringing different people together to work on a problem.

The Cornell Ann S. Bowers College of Computing and Information Science celebrated the newest cohort of students to declare a major within the college, providing dinner, swag, and introductions to the various clubs, services, and resources now available to them.

More than 160 new majors attended the event, held Oct. 4 in the Statler Hotel Ballroom. About two dozen faculty members, along with student leaders and various support staff from the college's three departments – computer science, information science, and statistics and data science – joined in the celebration.

“The event was a great introduction and welcome to Bowers CIS," said Chris Walkowiak '26, a brand new information science major. "The event really highlighted the college’s commitment to its students and provided great opportunities to meet fellow students and talk with faculty in a more informal setting."

In her opening remarks, Kavita Bala, dean of the college, welcomed the students to their new "intellectual home" within the college.

Undergraduates gain admission to Cornell through one of three admitting colleges: Cornell Engineering, the College of Agriculture and Life Sciences, or the College of Arts and Sciences. But if their academic path leads them to biometry and statistics, information science, computer science, statistical science, or information science, systems, and technology, they become affiliated with Cornell Bowers CIS.

"I tell my students that this is the best time to be in these disciplines," she said. "When you get your degree and go out there, you will be able to have an incredible and lasting real world impact."

Students also heard from leaders from many of the campus organizations affiliated with the college, including Women in Computing at Cornell, Association of Computer Science Undergraduates, Underrepresented Minorities in Computing, and the Information Science Student Association. Additionally, representatives from the college’s Office of Student Services and Office of Diversity, Equity, and Inclusion introduced themselves.

The new majors join a growing body of students interested in the technologies driving the information age. In 2022, more than 2,000 undergraduates obtained a degree offered by the college, representing a sixfold growth in enrollment over the previous ten years.

"I chose Cornell because of Bowers, actually – the information science program here is unique and unlike anything most schools offer," Walkowiak said. "I’d always been interested in technology but more so the intersections between technology and other aspects of society, and IS speaks to that in a way not fully addressed by more theoretical majors."

Is The Witcher immersive? Is The Sims a role-playing game?

Gamers from around the world may have differing opinions, but this diversity of thought makes for better algorithms that help audiences everywhere pick the right games, according to new research from Cornell, Xbox and Microsoft Research.

With the help of more than 5,000 gamers, researchers show that predictive models, fed on massive datasets labeled by gamers from different countries, offer better personalized gaming recommendations than those labeled by gamers from a single country.

The team’s findings and corresponding guidelines have broad application beyond gaming for researchers and practitioners who seek more globally applicable data labeling and, in turn, more accurate predictive artificial intelligence (AI) models.

“We show that, in fact, you can do just as well, if not better, by diversifying the underlying data that goes into predictive models,” said Allison Koenecke, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science.

Koenecke is the senior author of “Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems,” which was presented at the Association for Computing Machinery Fairness, Accountability, and Transparency (ACM FAccT) conference, in June.

Massive datasets inform the predictive models behind recommendation systems. The model’s accuracy depends on its underlying data, especially the proper labeling of each individual piece within that massive trove. Researchers and practitioners are increasingly turning to crowdsourced workers to do this labeling for them, but crowdsourced workforces tend to be homogenous.

During this data-labeling phase, cultural bias can creep in and, ultimately, skew a predictive model intended to serve global audiences, Koenecke said.

“For the datasets used in algorithmic processes, someone still has to come up with either some rules or just some general idea of what it means for a data point to be labeled in some way,” Koenecke said. “That’s where this human aspect comes in, because humans do have to be the decision makers at some point in this process.”

The team surveyed 5,174 Xbox gamers from around the world to help label gaming titles. They were asked to apply labels like “cozy,” “fantasy,” or “pacifist” to games they had played, and to consider different factors, such as whether a title is low or high complexity, or the difficulty of the game controls.

Some game labels – like “zen,” which is used to describe peaceful, calming games – were applied consistently across countries; others, like whether a game is “replayable,” were applied inconsistently. To explain these inconsistencies, the team used computational methods to find that both cultural differences among gamers and translational and linguistic quirks of certain labels contributed to labeling differences across countries.

The researchers then built two models that could predict how gamers from each country would label a certain game – one was fed survey data from globally representative gamers, and the second used survey data from only U.S. gamers. They found that the model trained on labels from diverse global populations improved predictions by 8% for gamers everywhere when compared to the other model trained on labels from just American gamers.

“We see improvement for everyone – even for gamers from the U.S. – when the training data is shifted from being entirely U.S.-centric to being more globally representative,” Koenecke said.

In addition to their findings, researchers crafted a framework to guide fellow researchers and practitioners on ways to audit underlying data labels to check for global inclusivity.

“Companies tend to use homogeneous data labelers to do their data labeling, and if you’re trying to build a global product, you’ll run into issues,” Koenecke said. “With our framework, any academic researcher or practitioner could audit their own underlying data to see if they might be running into issues of representation via their data labels or choices.”

Rock Yuren Pang of the University of Washington is the paper’s lead author; co-authors include Jack Cenatempo of Microsoft Research; and Franklyn Graham, Bridgette Kuehn, Maddy Whisenant, Portia Botchway and Katie Stone Perez, all of Microsoft Corporation, Xbox Division.

This research was partly supported by Microsoft Research and Xbox.

By Louis DiPietro , a writer for the Cornell Ann S. Bowers College of Computing and Information Science.

This story was originally published in the Cornell Chronicle.

“Data Driven: Truckers, Technology, and the New Workplace Surveillance” – penned by Cornell’s Karen Levy – recently received a third award since its publication late last year. 

In August, “Data Driven” received the Best Book Award from the American Sociological Association in the Section on Communications, Information Technology, and Media Sociology. The book had previously received the Best Information Science Book Award, given by the Association for Information Science and Technology, and the 2022 McGannon Book Award, given by Fordham University’s McGannon Center for best book addressing media policy, activism, and social justice.

Levy – an associate professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science and an associate member of the Law School faculty – explores the legal, organizational, social, and ethical aspects of data-intensive technologies.

In “Data Driven,” Levy examines how digital surveillance is changing the trucking industry and raises crucial questions about the role of data collection in broader systems of social control.

Published by Princeton University Press in December 2022, “Data Driven” has been praised by the likes of the Wall Street Journal, the New Yorker, and scholars like danah boyd, who called the book a “must-read for both those who think AI is our salvation and those who see automation as the devil.”

Cisco Research has funded multiple research awards to the Cornell Ann S. Bowers College of Computing and Information science to support projects related to cybersecurity, sustainability, edge computing, and artificial intelligence (AI). 

Five faculty projects and one graduate student will receive funding through this partnership. The resulting research will further the college's leadership in AI and point the way toward innovative solutions to challenges surrounding the use and development of AI models. 

Cisco Research is within Outshift, which serves as Cisco's incubation engine. Outshift is dedicated to pioneering new businesses and new markets in cutting-edge technology domains, including cloud native application security, edge native, quantum, and AI.

"Cisco Research has been pushing the frontiers of technology through innovative, cutting-edge research in areas of emerging technologies such as AI/ML, edge computing, and quantum. We are super excited to partner with several leading researchers in their fields at Cornell who are doing amazing research in these areas,” said Ramana Kompella, head of Cisco Research. "In addition, Cisco Research encourages and promotes a culture of open innovation, and researchers are free to make all the research – publications and software – funded through these awards completely open to benefit everyone, not just Cisco.”

The following Cornell Bowers CIS faculty will receive Cisco Research grants:

Allison Koenecke, assistant professor of information science, is investigating Demographic Biases in Generative AI Hallucinations.  People from different backgrounds and countries have differences in how they speak and write. Those differences are likely to affect the output from generative AI programs that create text or images in response to written prompts. In her project, Koenecke will investigate how a person's demographics affect the accuracy of – or amount of "hallucination" in – text generated by three large language models (LLMs), including ChatGPT. She will also determine whether these models can be fine-tuned to reach more consistent levels of hallucination across demographics.

Ken Birman, professor of computer science, is developing faster data transfer methods so that machine learning (ML) technologies can be applied using edge computing. In his open-source project, Edge Framework for Ultra-Low Latency Computing, Birman will build on a system he previously developed called Derecho, which creates building blocks for fault-tolerant distributed computing. His new project, Cascade, uses Derecho to store ML software, such as for image processing or text generation, and allows the software to run very efficiently on standard high-speed networks. 

Kevin Ellis, assistant professor of computer science, proposes to engineer algorithms for guiding neural networks that generate code. In this project, Nonstandard Generation Strategies for Better LLM Reasoning with Code, Ellis aims to emulate how humans write code – through iteration, trial and error, and divide-and-conquer strategies that break a task into smaller subtasks – instead of having a neural language model write the code all at once. He expects this approach will yield more lightweight models that generate better code.

Rachee Singh, assistant professor of computer science, aims to leverage programmable optical interconnects for making distributed training of machine learning models more efficient. In her work, she develops systems and algorithms for programming photonic interconnects at server and rack-scales such that distributed computation does not get bottlenecked by communication between GPUs.

Volodymyr Kuleshov, assistant professor at the Jacobs Technion-Cornell Institute at Cornell Tech, and Christopher De Sa, assistant professor of computer science, have a vision to enable some of the largest LLMs to operate on consumer computers. They will take a step toward that goal with their project, Scaling Large Language Models to Consumer GPUs via 2-Bit Quantization. They propose to develop a new method called quantization with incoherence processing (QuIP), which will allow LLMs to function using only two bits of memory per parameter. This work will improve the cost and accessibility of generative AI models and bring miniaturized LLMs closer to running on edge devices.

The Cisco partnership will also support Trishita Tiwari, a doctoral student in the field of computer science, working with Edward Suh. Her doctoral research focuses on preventing LLMs from leaking sensitive information contained within their training data. She proposes to probe the weaknesses of LLMs by investigating possible attacks, and to develop solutions for existing security issues by modifying LLM architectures, training and inference.

Academic freedom is closely tied to the First Amendment guarantee of freedom of expression, according to law professor Michael Dorf – and like freedom of speech, it’s not absolute.

“Academic freedom does not mean the liberty to say anything you want in a college or university setting,” said Dorf, the Robert S. Stevens Professor of Law, whose focus is on constitutional law. “It means the freedom to pursue knowledge and truth in good faith, according to the disciplinary standards and the decorum standards and the respect one shows for fellow students and others within the community.”

Dorf and three Cornell Law School colleagues participated in a forum, “The Fundamentals of Freedom of Expression,” held Sept. 7 in Myron Taylor Hall’s Landis Auditorium. The event served as the kickoff for the 2023-24 theme year: “The Indispensable Condition: Freedom of Expression at Cornell.”

The forum focused on foundations of the First Amendment’s protections for speech and assembly; challenges in applying those protections in a democratic and pluralistic society; and how free-speech principles play out in an increasingly digital world.

After a welcome from Jens David Ohlin, the Allan R. Tessler Dean and Professor of Law, President Martha E. Pollack introduced the panel and reminded the audience of the theme year title’s origins: the writings of Benjamin Cardozo, a Supreme Court justice from 1932-38, who called freedom of speech “the matrix, the indispensable condition of nearly every other form of freedom.”

Free speech is a given in this country, she said – “a bedrock assumption on which we’ve all built our lives. The ability to say what we think, ask questions, and listen to others is essential to democratic government, to our right to self-determination, and of course, to our academic enterprise. But over 232 years of American history [since the Bill of Rights was ratified], we’ve concluded that the right to free speech is not absolute.”

The panel was moderated by Gautam Hans, associate clinical professor of law, an expert on First Amendment law and technology policy, and in addition to Professor Dorf included Karen Levy, associate professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science, and associate member of the Law School faculty; and Nelson Tebbe, the Jane M.G. Foster Professor of Law.

Hans asked Tebbe, who researches general constitutional law and freedom of speech and religion, about the courts’ penchant for restricting government actions that impinge upon free-speech rights. Tebbe said the high court’s history with this protection isn’t extremely long – or particularly glorious: From protests against World Wars I and II and the rise of McCarthyism in the 1950s, the failures of the Supreme Court helped shape the evolution of free expression.

“Even though the Supreme Court was not as effective as we would have liked in policing free speech during that time,” he said, “the lessons of those failures, I think, have stuck with the court and with the society in really tenacious and important ways.”

In court cases where internet search engines and platforms like Baidu and YouTube were alleged to have violated free-expression principles of private entities through content moderation, Levy – who studies the intersection of law and technology – said the courts have sided with the websites.

“(These) are cases in which a party is aggrieved because … they don’t feel that a (platform) is listing their results highly enough, or at all, and they bring these First Amendment claims,” Levy said. “Oftentimes the rhetoric that’s leaned upon is that, in some ways, these tech platforms operate as public squares, and I think there is some validity to that argument.

“But that has not translated to the analogous principle that, because a lot of speech happens on these platforms, those platforms constitute state actors,” Levy said. “So, if anything, what courts have found is that those platforms themselves have free-speech rights, and oftentimes they have no choice but to decide what content to prioritize.”

Levy said the regulation of AI, and other new technology, has run into what’s been called the “pacing problem.” “Technology moves unbelievably quickly,” she said, “and this presents a problem when we’re trying to regulate this inherent moving target.”

The panelists also discussed the idea of society as a marketplace of ideas (the “notion that ideas will compete” with one another in a metaphorical “marketplace in the same way that goods and services” compete in the actual marketplace, Dorf said); and how America’s idea of free speech differs from other free societies.

“Many democracies … allow for some regulation of hate speech – some recognition that not just governments but private actors can contribute to the unjust stratification of society,” Tebbe said. “Everyone agrees that freedom of speech is of vital importance. And I think everyone also agrees that people shouldn’t be subordinated on the basis of inherent characteristics in their citizenship status, but instead should stand before one another in the public as equals. … There’s plenty of room for complex negotiation of these competing values, at the court level but also at institutional levels.”

A recording of the 90-minute forum, which included a half-hour Q&A, is available here.

Upcoming theme year-related events include the inaugural Milstein Symposium, with professors Jameel Jaffer of Columbia University and Eugene Volokh of the University of California, Los Angeles, on Sept. 26 in Landis Auditorium.

By Tom Fleischman, Cornell Chronicle

This story was originally published in the Cornell Chronicle.

To Ilan Mandel, a Cornell robotics researcher and builder, the math didn’t add up. How could a new, off-the-shelf hoverboard cost less than the parts that compose it?

“This becomes an ambient frustration as a designer – the incredible cheapness of products that exist in the world, and the incredible expenses for prototyping or building anything from scratch,” said Mandel, a doctoral student in the field of information science, based at Cornell Tech.

While sourcing wheels and motors from old hoverboards to build what would become a fleet of trash robots in New York City, Mandel inadvertently uncovered the subject of his newest research: “Recapturing Product as Material Supply: Hoverboards as Garbatrage,” which received an honorable mention at the Association for Computing Machinery conference on Designing Interactive Systems in July. Wendy Ju, associate professor at the Jacobs Technion-Cornell Institute at Cornell Tech and the Technion, and a member of the Department of Information Science in the Cornell Ann S. Bowers College of Computing and Information Science, co-authored the paper.

“For the large part, we design and manufacture as if we have an infinite supply of perfectly uniform materials and components,” Ju said. “That’s a terrible assumption.”

Building on work in human-computer interaction that aims to incorporate sustainability and reuse into the field, the Cornell pair introduces “garbatrage,” a framework for prototype builders centered around repurposing underused devices. Mandel and Ju use their repurposing of hoverboards – the hands-free, motorized scooters that rolled in and out of popularity around 2016 – as a test case to highlight the economic factors that create opportunities for garbatrage. They also encourage designers to prioritize material reuse, create more circular economies and sustainable supply chains, and, in turn, minimize electronic waste, or e-waste.

The time is ripe for a practice like garbatrage, both for sustainability reasons and considering the global supply shortages and international trade issues of the last few years, the researchers said.

“I think that there’s a real need to appreciate the heterogeneity of hardware that we are surrounded by all the time and look at it as a resource,” Mandel said. “What is often deemed as garbage can be full of value and can be made useful if you are willing to do some bridge work.”

From old desktop computers, smartphones and printers to smart speakers, Internet of Things appliances, and e-vaping devices, most of today’s e-waste has workable components that can be repurposed and used in the prototypes that become tomorrow’s innovations, researchers said.

Instead, these devices – along with their batteries, microcontrollers, accelerometers, motors and LCD displays – become part of the estimated 53 million metric tons of e-waste produced globally each year. Nearly 20% of it is properly recycled, but it’s unclear where the other 80% goes, according to a report from the UN’s Global E-waste Monitor 2020. Some ends up in developing countries, where people burn electronics in open-air pits to salvage any valuable metals, poisoning lands and putting public health at risk.

“Designers are a kind of node of interaction between massive scales of industrialization and end users,” Mandel said. “I think that designers can take that role seriously and use it to leverage e-waste in a way that promotes sustainability, beyond just asking the consumer to reflect more on their own practices.”

By Louis DiPietro, a writer for the Cornell Ann S. Bowers College of Computing and Information Science.

The Cornell Ann S. Bowers College of Computing and Information Science is pleased to announce the recruitment of 11 new faculty this year.

Three of those individuals joined the college for the Fall 2023 semester: Alex Conway, Sainyam Galhotra, and Daniel Susser. The remaining faculty, Saikat Dutta, Kuan Fang, Tanya Goyal, Michael Kim, Wei-Chiu Ma, Kristina Monakhova, Nick Spooner, and Jennifer Sun, will be arriving in coming semesters.  

"I am delighted to have these accomplished and innovative researchers join our faculty," said Kavita Bala, dean of Cornell Bowers CIS. "They are leaders who are building the technologies that drive modern computing and furthering our understanding of the impact of technology on society. Their addition to our faculty will enrich the academic experience for our students and contribute to the continued growth of our institution."

The newest faculty members have expertise in a broad range of fields. Their research will expand the college's leadership in machine learning, computational imaging, robotics, generative artificial intelligence (AI), natural language processing, computer vision, data storage, analytics, software engineering, post-quantum cryptography, and technology governance.

The new additions will also shape the next generation of tech leaders and innovators, while helping to meet the growing demand from students interested in majors offered within the college. Cornell Bowers CIS is now home to more than 2,000 undergraduate majors within its three departments – computer science, information science, and statistics and data science – and 76% of all undergraduate students at Cornell take at least one course within Cornell Bowers CIS. The college aims to increase its faculty number by 35% in the next few years to accommodate the demand.

The Cornell Bowers CIS faculty members joining for the Fall 2023 semester are:

Alex Conway

Assistant professor of computer science at Cornell Tech

Conway's research primarily focuses on randomized data structures and their applications to memory and storage systems. Conway comes to Cornell from the research arm of VMware, a cloud computing and virtualization technology company, where his work spanned every aspect from theory to systems to product. Conway received his Ph.D. in computer science from Rutgers University.

 

Sainyam Galhotra

Assistant professor of computer science 

Galhotra's research program centers around developing data science tools for effective and responsible analytics. He applies techniques from causal inference, data management, theoretical computer science, crowdsourcing, and human-computer interaction to understand various aspects of designing trustworthy systems that are robust, explainable, and fair. Previously, Galhotra was a Computing Innovation Postdoctoral Fellow at the Data Science Institute at the University of Chicago. He received his Ph.D. at the University of Massachusetts, Amherst.

 

Daniel Susser

Associate professor of information science

Susser’s research brings philosophical tools to bear on problems in technology governance, exploring normative issues raised by new and emerging data-driven technologies, and clarifying conceptual issues that stand in the way of addressing them. He has broad interests in technology ethics and policy, philosophy of technology, and science and technology studies, especially critical questions about data, privacy, and the ethics of automation. Previously, Susser was the Haile Family Early Career Assistant Professor of Information Sciences and Technology and a research associate in the Rock Ethics Institute at Penn State University. He received his Ph.D. in philosophy from Stony Brook University. Susser currently serves as a non-resident fellow at the Center for Democracy and Technology.

Additional faculty joining at a later date include:

Saikat Dutta 

Assistant professor of computer science (Fall 2024)

Dutta's research lies at the intersection of software engineering and machine learning, and, at Cornell, he plans to combine these two areas to improve the reliability of traditional and machine learning-based software. He is especially interested in developing novel testing techniques and tools to improve the reliability of machine learning-based systems and leveraging machine learning to address challenging tasks in software engineering. Currently, he is a postdoctoral researcher at the University of Pennsylvania. Saikat completed his Ph.D. in computer science at the University of Illinois Urbana-Champaign, and previously, spent two years as a software engineer at Microsoft, India.  

 

Kuan Fang

Assistant professor of computer science (Fall 2024)

Fang's work aims to enable robots to solve diverse and complex tasks in unstructured environments. Specifically, he is building scalable data-driven methods for perception and control, with a focus on how to automate data collection from tasks performed in simulation and the real world, and acquire general purpose skills for efficiently solving novel, long-horizon tasks, which involve multiple steps. Fang is a postdoctoral scholar at the Berkeley Artificial Intelligence Research lab. He obtained his Ph.D. from Stanford University, and has spent time at Google Brain, Google [x] Robotics, and Microsoft Research Asia.

 

Tanya Goyal

Assistant Professor of computer science (Fall 2024)

Goyal's research interests are in natural language processing, where she works on measuring and improving the generation capabilities of language models. She is interested in building automatic evaluation tools that can be used to reliably compare large-scale AI models. Goyal is currently a postdoctoral scholar at the Language and Intelligence Initiative at Princeton University. She received her Ph.D. in computer science from the University of Texas at Austin in 2023.  

 

Michael P. Kim 

Assistant professor of computer science (Spring 2024)

Kim's research investigates foundational questions about responsible machine learning. Much of his work aims to identify ways in which machine-learned predictors can exhibit problematic behavior (e.g., unfair discrimination) and develop algorithmic tools that mitigate these behaviors. Kim is currently a Miller Postdoctoral Fellow at the University of California, Berkeley. He completed his Ph.D. in computer science at Stanford University. 

 

 

Wei-Chiu Ma 

Assistant professor of computer science (Fall 2024)

Ma's research interests include computer vision, robotics, and machine learning, especially 3D vision and self-driving vehicles. His work involves robot localization, 3D reconstruction, motion estimation, large-scale 3D generation, and closed-loop sensor simulation, in both controlled and noisy, real-world settings. Ma is finishing his Ph.D. at the Massachusetts Institute of Technology and will spend the next year at the Allen Institute for AI. He worked previously at Uber ATG, and is a part-time senior research scientist at Waabi, a self-driving technology company. 

 

Kristina Monakhova 

Assistant professor of computer science (Fall 2024)

Monakhova's work combines ideas from machine learning, signal processing, optics, computer vision, and physics to build better imaging systems (e.g., cameras, microscopes, and telescopes) through the co-design of optics, algorithms, and high-level tasks. Her aim is to design the next generation of smart, computational imagers for scientific discovery, robotics, and medical diagnostics. Monakhova is a postdoctoral fellow at the Massachusetts Institute of Technology and received her Ph.D. from the University of California, Berkeley.

 

Nick Spooner

Assistant professor of computer science (Fall 2024) 

Currently an assistant professor at the University of Warwick and a visiting assistant professor at New York University, Spooner focuses on quantum and post-quantum proof systems. More broadly, he is interested in quantum and post-quantum cryptography, quantum information, coding theory, and computational complexity. Previously, Spooner was a postdoctoral researcher at Boston University. He obtained his Ph.D. from the University of California, Berkeley.

 

Jennifer Sun

Assistant professor of computer science (Fall 2024)

Sun's research focuses on developing AI systems that work together with scientists to accelerate discovery. She is particularly interested in computer vision and machine learning methods that can be integrated into real-world workflows involving expert-in-the-loop interactions, such as the study of behavioral or medical data. Her goal is to optimize expert attention in these workflows by addressing bottlenecks, including data efficiency, model interpretability, generalization, and structure discovery. Sun is finishing her Ph.D. in computing and mathematical sciences at Caltech and is a research scientist at Google.

Sit across from someone wearing augmented reality (AR)  or “smart” glasses and you don’t know what they’re seeing – they could be Googling your face, turning you into a cat, or recording your conversation – and that disparity creates a major power imbalance, Cornell researchers said.

Most work with AR glasses focuses primarily on the experience of the wearer. Researchers from the Cornell Ann S. Bowers College of Computing and Information Science and Brown University teamed up to explore how this technology affects interactions between the wearer and another person. Their explorations showed that, while the device generally made the wearer less anxious, things weren’t so rosy on the other side of the glasses. 

Jenny Fu, a doctoral student in the field of information science, presented the findings in a new study, “Negotiating Dyadic Interactions through the Lens of Augmented Reality Glasses,” at the 2023 Association for Computing Machinery Designing Interactive Systems Conference in July.

AR glasses superimpose virtual objects and text over the field of view to create a mixed-reality world for the user. Some designs are big and bulky, but as AR technology advances, some smart glasses are becoming indistinguishable from regular glasses, raising concerns that a wearer could be secretly recording someone or even generating deepfakes with their likeness.

For the new study, Fu and co-author Malte Jung, associate professor in information science and the Nancy H. ’62 and Philip M. ’62 Young Sesquicentennial Faculty Fellow,  worked with doctoral student Ji Won Chung and Jeff Huang, associate professor of computer science, both at Brown, and Zachary Deocadiz-Smith, an independent extended reality designer. 

They observed five pairs of individuals – a wearer and a non-wearer – as each pair discussed a desert survival activity. The wearer received Spectacles, an AR glasses prototype on loan from Snap Inc., the company behind Snapchat. Spectacles look like avant-garde sunglasses and, in the study, came equipped with a video camera and five custom filters that transformed the non-wearer into a deer, cat, bear, clown or pig-bunny.

Following the activity, the pairs engaged in a participatory design session where they discussed how AR glasses could be improved, both for the wearer, and the non-wearer. The participants were also interviewed and asked to reflect on their experiences.

According to the wearers, the fun filters reduced their anxiety and put them at ease. The non-wearers, however, felt disempowered because they didn’t know what was happening on the other side of the lenses. They were also upset that the filters robbed them of control over their own appearance. The possibility that the wearer could be secretly recording them without consent – especially when they didn’t know what they looked like – also put the non-wearers at a disadvantage.

The non-wearers weren’t completely powerless, however. A few demanded to know what the wearer was seeing, and moved their faces or bodies to evade the filters – giving them some control in negotiating their presence in the invisible mixed-reality world. “I think that’s the biggest takeaway I have from this study: I’m more powerful than I thought I was,” Fu said.

Another issue is that, like many AR glasses, Spectacles have darkened lenses so the wearer can see the projected virtual images. This lack of transparency also degraded the quality of the social interaction.

“There is no direct eye contact, which makes people very confused, because they don’t know where the person is looking,” Fu said. “That makes their experiences of this conversation less pleasant, because the glasses blocked out all these nonverbal interactions.”

To create more positive experiences for people on both sides of the lenses, the study participants proposed that smart glasses designers add a projection display and a recording indicator light, so people nearby will know what the wearer is seeing and recording.

Fu also suggests designers test out their glasses in a social environment, hold a participatory design process like the one in their study, and consider the resulting video interaction as a data source. 

That way, non-wearers can have a voice in the creation of the impending mixed-reality world.

This work received support from the National Science Foundation.

By Patricia Waldron, a writer for the Cornell Ann S. Bowers College of Computing and Information Science.