Mar Gonzalez-Franco

Computer Scientist + Neuroscientist


2022
I am a Senior Research Manager at Google creating a new generation of Immersive tech. I lead the BIRD lab inside the AR org. The BIRD creats blended AI interactions in XR, that perhaps aim to bring more of the real into the virtual and bridge better 2D and 3D. BIRD explores multi-device and multi-modal futures with a focus on AI/ML enabled novel experiences, diffusion models etc. From foundational work to full products. We prototype fast, we demo and we change the outlook of the future.

2016-2022
I was a Principal Researcher in the EPIC (Extended Perception Interaction and Cognition) team at Microsoft Research. Transferred Research to products used in daily basis by millions of people around the world, such as the Avatars and Together mode in Microsoft Teams and Microsoft Soundscape, as well as features on Hololens and Xbox. In my quest to reach even more, I have also released multiple tools and open source libraries such as the Microsoft Rocketbox avatars, or the MoveBox or HeadBox toolkits to democratize avatar animation. The release of Avatars in Teams was awarded Times invention of the year in 2022. And it is to date the largest deployment of avatars in any platform for productivity, available to the 270 million users of MS Teams.
2015 - 2016
I joined the start-up world as a Senior Research Scientist, and built mathematical models to create reputation standards at Traity. A Start-up with $4.7 million series A round and over 4.5M users. Alumni of Silicon Valley 500 Startups and Seedcamp.

2014 - 2015
I created and led the Immersive Technologies lab in the UK to research AR, VR, HMDs and Mixed Reality in the context of aeronautic manufacturing. My prototypes were demoed to several VPs, and the CTO, and later spun into the Holographic Academy of the Design Office in the UK.


2014
Postdoctorate at the Virtual Environments and Computer Graphics group, in the University College London supported by an UK ESRC grant (Reference ESL0031041). Reproduced a Virtual Milgram experiment to explore the banality of evil: a new understanding of conformity and atrocity inside Immersive Virtual Reality (CAVE). Would people obey to authority and electroshock an avatar (Scientific American).

2008 - 2014
PhD, MSc at the Universitat de Barcelona
2010-2014 PhD in Immersive Virtual Reality and Clinical Psychology under the supervision of Prof. Mel Slater, affiliated as a visiting student at the Massachusetts Institute of Technology, MediaLab.

2008-2010 MSc in Biomedical Engineering, focus on signal processing. Master Thesis completed in the Neural Engineering Lab at Tsinghua University. Master Research Student at Institut Guttmann and UPC Creb.

During my PhD I produced anarchic hand experiences into healthy humans (link), as well as stabbed their virtual hands (link). Altogether to explore how our brain perceives our body and search the underlying mechanisms that determine what is real and what is not. Technologically I was using real-time computer graphics, HMDs and body tracking systems.

2004 - 2008
BSc double Major Computer Science and Multimedia Engineering at La Salle - Universitat Ramon Llull. Graduated top class with Best Academic Curricula Award, (GPA 9.22/10, 16 honour mentions).
2007-2008 Digital Image Processing Teaching Assistant at the Department of Communication and Signal Processing.
2006-2008 Undergraduate Research Opportunities Program at the Architecture Representation and Computation group.
bio
Dr. Mar Gonzalez-Franco is a Computer Scientist and Neuroscientist at Google working on a new generation of Immersive technologies and generative AI. With a background in real-time systems in her research she tries to build better interactions for immersive technologies using different disciplines: Virtual Reality, Augmented Reality, Avatars, computer graphics, computer vision, AI, and haptics. All while studying human behavior, perception and neuroscience.

Priorly, she was Principal Researcher in the EPIC (Extended Perception Interaction and Cognition) team at Microsoft Research. Dr. Gonzalez-Franco holds a BSc in Computer Science (URL, Barcelona) and MSc in Biomedical Engineering (Universitat de Barcelona and Tsinghua University). She earned her Ph.D. in Immersive Virtual Reality and Clinical Psychology under the supervision of Prof. Mel Slater at the EVENT-Lab, affiliated as a visiting student at the Massachusetts Institute of Technology, MediaLab. She completed her postdoctoral studies at University College London. Dr. Gonzalez-Franco is an expert advisor to governments (UN, US NSF, Canada NSERC, European Commission), as well as prolific scientist with publications and service as program committee, chair, and reviewed in top venues (IEEE & ACM conferences and transactions, Nature Publishing group, Science Robotics etc). She is often invited to write commentaries on her work for Scientific American and presents keynotes and interviews with international media including Bloomberg or The Royal Society. She was awarded the prestigious IEEE VR New researcher award in 2022. And since 2020, she is the Ethics and Diversity Chair of IEEE VGTC.

Apart from her prolific scientific output and academic recognitions, Dr. Gonzalez-Franco's work has also transferred to products used on a daily basis by many around the world, such as the Avatars and Together mode in Microsoft Teams and Microsoft Soundscape, as well as features on Hololens and Xbox. The release of Avatars in Teams was awarded Times invention of the year in 2022. And it is to date the largest deployment of avatars in any platform for productivity, available to the 270 million users of MS Teams.
research interests
Over the years my work has evolved in three ways> (1) to continue exploring the wonders of VR and spatial computing; (2) to create new devices and explore what these technologies do to our perceptual systems, most prominent is my work exploring new haptic devices and spatial audio; and (3) to advance Immersive Tech by creating and adopting AI for scene/activity understanding and content generation.

My interest on impacting larger audiences made me pivot from academia into industrial laboratories and after having been at the MIT, Tsinghua, UB and UCL; I created and led an immersive tech lab at Airbus Group; joined the Startup Traity working on the creation of reputation standards for a user base of 4.5M. From 2016-2022 I was with Microsoft Research. Since 2022 I am a Research Scientist and Manager at Google.

lines of research
We are moving from having the content inside the screen to having the user inside the content. This paradigm shift together with the increasing number of connected and digital content and AI, opens new avenues of interaction: new needs for devices, better auditory affordances, new forms of representation (in occasions via avatars), and new ways to move around the digital content.


Avatars
In Virtual Reality and spatial computing simulations, humans are often represented by avatars. I have explored how users interact with them in and out VR, how they can be animated, and their consequences. I have also created first person substitutions inside VR, by which avatars are embodied by participants. To advance this field I have proposed new theories (such as the Self-Avatar Follower Effect or the theory of Illusions in Virtual Reality), metrics and questionnaires.

In an effort to democratize the access and use of avatars I have also released as open source various libraries and tools such as the Microsoft Rocketbox library of avatars and the Movebox a toolbox to animate the Microsoft Rocketbox avatars.

Featured work:
Frontiers in Virtual Reality
Mar Gonzalez-Franco, Eyal Ofek, Ye Pan, Angus Antley, Anthony Steed, Bernhard Spanlang, Antonella Maselli, Domna Banakou, Nuria Pelechano, Sergio Orts-Escolano, Veronica Orvalho, Laura Trutoiu, Markus Wojcik, Maria V. Sanchez-Vives, Jeremy Bailenson, Mel Slater, Jaron Lanier
IEEE AIVR 2020
Mar Gonzalez-Franco, Zelia Egan, Matthew Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Derek F Reilly, Tabitha Peck, Andrea Stevenson Won, Cheng Yao Wang, Anthony Steed, Eyal Ofek
Frontiers in Virtual Reality
Tabitha C. Peck, Mar Gonzalez-Franco
Haptic Devices
Haptics are the ultimate frontier of perception for Virtual Objects, together with my amazing team of collaborators (Eyal Ofek, Mike Sinclair and our visitors and summer interns) I have explored both the perceptual boundaries as well as tried to propose new prototypes that could solve some of the needs and tasks involving haptic feedback. In 2021 I wrote a blog post about our vision for Shape Displays on Nature Device and Materials Engineering.

Featured work:
ACM UIST 2021
Eric J Gonzalez, Eyal Ofek, Mar Gonzalez-Franco, Mike Sinclair
Nature Communications
Anthony Steed, Eyal Ofek, Mike Sinclair Mar Gonzalez-Franco
ACM UIST 2020
Robert Kovacs, Eyal Ofek, Alexa Fay Siu, Sebastian Marwecki, Christian Holz, Mike Sinclair, Mar Gonzalez-Franco
ACM UIST 2019
Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco, Christian Holz
ACM CHI 2019
Jaeyeon Lee, Mike Sinclair, Mar Gonzalez-Franco, Eyal Ofek, Christian Holz
Sonification
Once users are inside the content, sound can be spatialized. The implications of this sonification shift will revolutionize many existing experiences from videoconferencing to GPS navigation. In that scope I have explored spatial sonification across many aspects including HRTFs, using Beacons and concurrent talking. In that context I have transferred many of my learnings to Microsoft Soundscape. You can read more about how this new form of GPS based on audio Beacons is more ecologically adapted to humans than turn-by-turn in our Scientific American piece.

Featured work:
Scientific Reports 2021
Gregory Dane Clemenson, Antonella Maselli, Alex Fiannaca, Amos Miller, Mar Gonzalez-Franco
Frontiers in Neuroscience
Christopher Berger, Mar Gonzalez-Franco, Ana Tajadura-Jimenez, Dinei Florencio, Zhengyou Zhang
Scientific Reports 2017
Mar Gonzalez-Franco, Antonella Maselli, Dinei Florencio, Nikolai Smolyanskiy, Zhengyou Zhang
patents
U.S. Patent 10,838,486, issued November 17, 2020.
Mar Gonzalez-Franco, Eyal Ofek, Michael J Sinclair, Christian Holz, Jaeyeon Lee
US Patent 16/439,596, issued December 17, 2020
Michael J Sinclair, Mar Gonzalez-Franco, Christian Holz, Eyal Ofek
US Patent 11,003,247, issued May 11, 2021
Michael J Sinclair, R Kovacs, Eyal Ofek, Mar Gonzalez-Franco
US Patent 11,055,891, issued July 6, 2021
Eyal Ofek, Mar Gonzalez-Franco, Andrew D Wilson, Karan Ahuja, Christian Holz
US Patent 11,086,398, issued August 10, 2021
Mar Gonzalez-Franco, Eyal Ofek, Michael J Sinclair, Anthony James Steed
US Patent 17,333,737, issued December 1, 2021
Ken P Hinckley, Michel Paud, Mar Gonzalez-Franco, Edward Sean Lloyd Rintel, Eyal Ofek, Jaron Zepel Lanier, Molly Jane Nicholas, Payod Panda
US Patent 11,226,685, issued March 31, 2022
Michael J Sinclair, Mar Gonzalez-Franco, Christian Holz, Eyal Ofek
US Patent 17,549,704, issued March 31, 2022
Michael J Sinclair, Mar Gonzalez-Franco, Christian Holz, Eyal Ofek
US Patent 17,333,582, issued December 1, 2022
Ken P Hinckley, Michel Paud, Mar Gonzalez-Franco, Edward Sean Lloyd Rintel, Eyal Ofek, Jaron Zepel Lanier, Molly Jane Nicholas, Payod Panda
US Patent 17,356,513, issued December 29, 2022
Mar Gonzalez-Franco, Eyal Ofek, Michael J Sinclair, Ryo Suzuki
US Patent 11,556,168, issued January 17, 2023
Mar Gonzalez-Franco,Michael J Sinclair, Eyal Ofek, Eric J Gonzalez
US Patent 18,080,991, issued April 13, 2023
Mar Gonzalez-Franco,Michael J Sinclair, Eyal Ofek, Eric J Gonzalez
6 More patents filed.
full list of publications
interaction
perception
avatars
faces
AR
audio
accessibility
haptics
devices
review
locomotion
motor control
theory
latency
EEG
open source
ML/AI




2023
arXiv 2023
Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, Mar Gonzalez-Franco


arXiv 2023
Tiffany Do, Steve Zelenty, Mar Gonzalez-Franco, Ryan P McMahan


IEEE ISMAR 2023
Vrushank Phadnis, Kristin Moore, Mar Gonzalez-Franco


ACM DIS 2023
Payod Panda, Molly Jane Nicholas, David Nguyen, Eyal Ofek, Michel Pahud, Sean Rintel, Mar Gonzalez-Franco, Ken Hinckley, Jaron Lanier


ACM CHI 2023
Yujie tao, Cheng Yao Wang, Andy D Wilson, Eyal Ofek, Mar Gonzalez-Franco


ACM CHI 2023
Jeremy Gordon, Alex Fiannaca, Melanie Kneisel, Ed Cutrell, Amos Miller, Mar Gonzalez-Franco


Philosophical Transactions of the Royal Society B
Antonella Maselli, Eyal Ofek, Brian Cohn, Ken Hinckley, Mar Gonzalez-Franco


2022
IEEE ICIR 2022
Cheng Yao Wang, Oron Nir, Sai Vemprala, Ashish Kapoor, Eyal Ofek, Mar Gonzalez-Franco


TechRxiv 2022
Mar Gonzalez-Franco, Eyal Ofek, Christian Holz, Antony Steed, Jaron Lanier, Bill Buxton, Ken Hinckley, Mike Sinclair


CHIWORK 2022
Payod Panda, Molly Jane Nicholas, Mar Gonzalez-Franco, Kori Inkpen, Eyal Ofek, Ross Cutler, Ken Hinckley, Jaron Lanier


ACM CHI 2022
Tiffany Liu, Javier Hernandez,Mar Gonzalez-Franco, Antonella Maselli, Melanie Kneisel, Adam Glass, Jarnail Chudge, Amos Miller


IEEE VR 2022
Matias Volonte, Eyal Ofek, Ken Jakubzak, Shawn Bruner, Mar Gonzalez-Franco


Microsoft Report
Jaime Teevan, Nancy Baym, Jenna Butler, Brent Hecht, Sonia Jaffe, Kate Nowak, Abigail Sellen, Longqi Yang, Marcus Ash, Kagonya Awori, Mia Bruch, Piali Choudhury, Adam Coleman, Scott Counts, Shiraz Cupala, Mary Czerwinski, Ed Doran, Elizabeth Fetterolf, Mar Gonzalez-Franco,Kunal Gupta, Aaron L Halfaker, Constance Hadley, Brian Houck, Kori Inkpen, Shamsi Iqbal, Eric Knudsen, Stacey Levine, Siân Lindley, Jennifer Neville, Jacki O’Neill, Rick Pollak, Victor Poznanski, Sean Rintel, Neha Parikh Shah, Siddharth Suri, Adam D. Troy, Mengting Wan


2021
ACM UIST 2021
Eric J Gonzalez, Eyal Ofek,Mar Gonzalez-Franco, Mike Sinclair


ACM UIST 2021
Ryo Suzuki, Eyal Ofek, Mike Sinclair, Daniel Leithinger,Mar Gonzalez-Franco


Nature Communications
Anthony steed, Eyal Ofek, Mike Sinclair, Mar Gonzalez-Franco


Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Karan Ahuja, Eyal Ofek,Mar Gonzalez-Franco, Christian Holz, Andy Wilson


Scientific Reports 2021
Gregory Dane Clemenson, Antonella Maselli, Alex Fiannaca, Amos Miller, Mar Gonzalez-Franco


ACM CHI 2021
Massimiliano di Luca, Hasti Seifi, Simon Egan, Mar Gonzalez-Franco


IEEE VR 2021
Abbey Singh, Ramanpreet Kaur, Peter Haltner, Matthew Peachey, Mar Gonzalez-Franco, Joseph Malloch, Derek Reilly


Frontiers in Virtual Reality 2021
Tabitha C. Peck, Mar Gonzalez-Franco


2020
Frontiers in Virtual Reality 2020
Mar Gonzalez-Franco, Eyal Ofek, Ye Pan, Angus Antley, Anthony Steed, Bernhard Spanlang, Antonella Maselli, Domna Banakou, Nuria Pelechano, Sergio Orts-Escolano, Veronica Orvalho, Laura Trutoiu, Markus Wojcik, Maria V. Sanchez-Vives, Jeremy Bailenson, Mel Slater, Jaron Lanier


IEEE AIVR 2020
Mar Gonzalez-Franco, Zelia Egan, Matthew Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Derek F Reilly, Tabitha Peck, Andrea Stevenson Won, Cheng Yao Wang, Anthony Steed, Eyal Ofek


IEEE AIVR 2020
Brian Cohn, Antonella Maselli, Eyal Ofek, Mar Gonzalez-Franco


ACM Symposium on Applied Perception 2020 (SAP '20)
Mar Gonzalez-Franco, Mike Sinclair, Eyal Ofek


IEEE Transactions on Visualization and Computer Graphics 2020
Mar Gonzalez-Franco, Anthony Steed, Steve Hoogendyk, Eyal Ofek


IEEE VR 2020
Robert Gruen, Eyal Ofek, Anthony Steed, Ran Gal, Mike Sinclair, Mar Gonzalez-Franco


IEEE VR 2020
Mar Gonzalez-Franco, Brian Cohn, Dalila Burin, Eyal Ofek, Antonella Maselli


ACM UIST 2020
Robert Kovacs, Eyal Ofek, Alexa Fay Siu, Sebastian Marwecki, Christian Holz, Mike Sinclair, Mar Gonzalez-Franco


2019
ACM UIST 2019
Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco, Christian Holz


ACM CHI 2019
Jaeyeon Lee, Mike Sinclair, Mar Gonzalez-Franco, Eyal Ofek, Christian Holz


UIST 2019
Sebastian Marwecki, Andrew D. Wilson, Eyal Ofek, Mar Gonzalez-Franco, Christian Holz


ISMAR 2019
Martez Mott, Ed Cutrell, Mar Gonzalez-Franco, Christian Holz, Eyal Ofek, Richard Stoakley, Meredith Ringel Morris


IEEE Transactions on Haptics 2019
Mar Gonzalez-Franco, Christopher C Berger


ACM CHI 2019
Parastoo Abtahi, Mar Gonzalez-Franco, Eyal Ofek, Anthony Steed


IEEE VR 2019
Mar Gonzalez-Franco, Parastoo Abtahi, Anthony Steed


2018
PLoS ONE 2018
Mar Gonzalez-Franco, Mel Slater, Megan E. Birney, David Swapp, S. Alexander Haslam, Stephen D. Reicher


ACM Symposium on Applied Perception (SAP) 2018
Mar Gonzalez-Franco, Christopher Berger


Frontiers in Robotics and AI 2018
Mar Gonzalez-Franco, Tabitha C Peck


Science Robotics 2018
Mar Gonzalez-Franco, Christopher C Berger, Eyal Ofek, Ken Hinckley


Frontiers in Neuroscience 2018
Christopher Berger, Mar Gonzalez-Franco, Ana Tajadura-Jimenez, Dinei Florencio, Zhengyou Zhang


2017
Scientific Reports 2017
Mar Gonzalez-Franco, Antonella Maselli, Dinei Florencio, Nikolai Smolyanskiy, Zhengyou Zhang


ACM Symposium on Virtual Reality Software and Technology (VRST) 2017
Daniel McDuff, Chrisophe Hurter, Mar Gonzalez-Franco


Frontiers in Psychology 2017
Mar Gonzalez-Franco, Jaron Lanier


Frontiers in Robotics and AI 2017
Nikolai Smolyanskyi, Mar Gonzalez-Franco


Frontiers in Robotics and AI 2017
Mar Gonzalez-Franco, Rodrigo Pizarro, Julio Cermeron, Katie Li, Jacob Thorn, Windo Hutabarat, Ashutosh Tiwari, Pablo Bermell


ACM on Multimedia Conference 2017
Jacob Thorn, Rodrigo Pizarro, Bernhard Spanlang, Pablo Bermell-Garcia, Mar Gonzalez-Franco


2016
Frontiers in Human Neuroscience 2016
Mar Gonzalez-Franco, Ana Bellido Rivas, Kristopher J Blom, Mel Slater, Antoni Rodriguez-Fornells


NeuroImage 2016
Gonçalo Padrao ,Mar Gonzalez-Franco, Maria V Sanchez-Vives, Mel Slater, Antoni Rodriguez-Fornells


2015
International Conference on Interactive Tabletops and Surfaces (ITS) 2015
Rodrigo Pizarro, Mark Hall, Pablo Bermell-Garcia,Mar Gonzalez-Franco


3DUI 2015
Mar Gonzalez-Franco, Mark Hall, Devon Hansen, Karl Jones, Paul Hannah, Pablo Bermell-Garcia


2014
Frontiers in Robotics and AI 2014
Bernhard Spanlang , Jean-Marie Normand , David Borland , Konstantina Kilteni , Elias Giannopoulos, Ausiàs Pomés, Mar Gonzalez-Franco, Daniel Pérez Marcos, Jorge Arroyo-Palacios, Xavi Navaro Muncunill , Mel Slater


MIT PRESENCE: Teleoperators and Virtual Environments 2014
Sameer Kishore, Mar Gonzalez-Franco, , Christoph Hintemüller, Christoph Kapeller, Christoph Guger, Mel Slater, Kristopher J Blom


Quality of Multimedia Experience (QoMEX) 2014
Mar Gonzalez-Franco, Philip A Chou


Experimental Brain Research 2014
Mar Gonzalez-Franco, Tabitha C. Peck , Antoni Rodríguez-Fornells, Mel Slater


2013
Experimental Brain Research 2013
Joan Llobera,Mar Gonzalez-Franco, Daniel Perez-Marcos, Josep Valls-Sole, Mel Slater, Maria V Sanchez-Vives


2011
IEEE Engineering in Medicine and Biology Society (EMBS) 2011
Mar Gonzalez-Franco, Peng Yuan, Dan Zhang, Bo Hong, Shangkai Gao


2010
IEEE VR 2010
Mar Gonzalez-Franco, Daniel Perez-Marcos, Bernhard Spanlang, Mel Slater



Also on Google Scholar
awards
2022 Avatars in Teams awarded Times invention of the year.
2022 IEEE VGTC Virtual Reality New Significant Researcher award
2021 ACM Senior Member
2021 Generation Next Technology and Science top 10. Mujer Hoy
2020 IEEE Senior Member
2019 MAS Technology award
2019 Business Insider ES – Tech Expert under 35
media