11/22/2019 Jamie Hendrickson for Illinois Physics
University of Illinois at Urbana-Champaign physics graduate student Colin Lualdi quickly realized he was venturing into uncharted territory when he arrived at Illinois Physics at the start of Fall 2017. Deaf since birth and a native speaker of American Sign Language (ASL), Lualdi was now among a very small group worldwide of Deaf individuals working in physics. The exhilaration of performing cutting-edge research was accompanied by a sobering discovery: the lack of a common language model for effective scientific discourse in ASL was going to be a far greater challenge than he’d anticipated. Lualdi has embraced his own accessibility challenges as an opportunity to address a pressing need in the broader Deaf community. He has teamed up with colleagues at other research institutions to develop a robust and shared framework for scientific discourse in ASL. Specifically, Colin has been working with ASL Clear and ASLCORE, two national scientific sign language initiatives that are making good progress.
Written by Jamie Hendrickson for Illinois Physics
Working toward a new paradigm for communication, access, and inclusivity in physics
University of Illinois at Urbana-Champaign physics graduate student Colin Lualdi quickly realized he was venturing into uncharted territory when he arrived at Illinois Physics at the start of Fall 2017. Deaf since birth and a native speaker of American Sign Language (ASL), Lualdi was now among a very small group worldwide of Deaf individuals working in physics. The exhilaration of performing cutting-edge research was accompanied by a sobering discovery: the lack of a common language model for effective scientific discourse in ASL was going to be a far greater challenge than he’d anticipated.
“As we were conversing about the details of his work, it suddenly struck me that we were using many of the signs that we had developed over the past week and from other sessions in recent months. These signs, being natural ASL signs, could be used in proper ASL sentences rather than clumsy pidgin sign-language constructions using English-based loan signs. They flowed together so beautifully that I found myself marveling at how effortlessly David and I were able to engage in very technical discourse.”
“Scientific concepts can be communicated in ASL,” Lualdi notes, “but many ASL users are unaware of how to do this efficiently, fully utilizing all that ASL has to offer as a language.”
The communication and accessibility complications Lualdi faces aren’t unique.
“Deaf scientists across various STEM fields are currently working to address this communication deficiency by introducing new language models that support effective STEM-field discourse, complete with specialized ASL vocabulary,” Lualdi adds.
Lualdi has embraced his own accessibility challenges as an opportunity to address a pressing need in the broader Deaf community. He has teamed up with colleagues at other research institutions to develop a robust and shared framework for scientific discourse in ASL. Specifically, Colin has been working with ASL Clear and ASLCORE, two national scientific sign language initiatives that are making good progress.
Such initiatives stand to give Deaf ASL users easier mastery of scientific concepts, the ready ability to collaborate with colleagues, and greater access to the high-level academic training required for some STEM-field careers—whether in the private sector, at national labs, or in academia. And equally important, these initiatives will give those STEM fields greater access to the Deaf community’s talent pool.
Lualdi was recruited into the Illinois Physics scientific community
Lualdi is a member of Illinois Physics Professor Paul Kwiat's research group, working on fundamental research in quantum information science, a field poised to revolutionize computing and communications technologies.
Illinois Physics Professor Paul Kwiat talks about Deaf graduate student Colin Lualdi’s promising research in quantum information science.
For his research I suggested Colin work with my top postdoc on a very difficult project—using time-multiplexing to greatly improve the performance of a single-photon source. This project is challenging because good performance is only achieved if everything is aligned essentially perfectly, and all the losses are made as low as possible. To go from difficult to nearly impossible, that postdoc left very soon after Colin started, which meant Colin had to come up to speed in a few months on a project that had taken several years to set up. I’ve been continually impressed with his thoroughness and desire to deeply understand all aspects of the system.
I brought Colin last May to his first physics conference as a member of my group, and his submission was selected for a presentation. It’s normal, I suppose, for an advisor to be a little nervous at their student’s first talk, and this was no different. As usual, I had pen and paper ready to note any comments I had on the talk, both in terms of the scientific description, but also the wider range of presentation skills, e.g., pacing, clarity, ability to answer questions—I then can give feedback to the student afterward, to improve future presentations. After Colin’s talk, my paper had only a single word: “Perfect!" This rarely happens for any talk, but I’ve NEVER seen it with a student’s first talk. However, I’ve since come to expect such excellence from Colin.
Kwiat shares, “I first met Colin at a major quantum cryptography conference in Washington, D.C. He was giving a poster that I found very interesting, and I was super impressed with the depth of his knowledge, such as I could evaluate in the 15 minutes we spent talking via his interpreter. I knew immediately that I wanted him to be part of my quantum information group and made sure to get his contact information so I could issue a more formal invitation.
“I’m super grateful that he accepted—Colin is extremely bright, while completely lacking any of the arrogance that sometimes unfortunately accompanies that. To the contrary, I believe he is universally loved and valued in my group.”
As Lualdi got busy working in the lab and taking graduate-level courses, he ran headlong into communication barriers. The field of physics has a tremendous lexicon unto itself, developed over time by physicists to more efficiently and precisely communicate advanced and often novel concepts. The very small number of Deaf professionals working in physics means that there was no language model in place from which Lualdi could learn to effectively express the nuances of physics in ASL. This lack of language models has also resulted in a dearth of interpreters trained in physics subject matter.
ASL is Lualdi’s primary means of communication, and he uses professional interpreters—provided by the university’s Disability Resources and Educational Services (DRES)—to facilitate face-to-face conversations with his peers and colleagues who may not know ASL. Until working with Lualdi, the small number of ASL-English interpreters working with DRES had minimal physics exposure and limited experience working at the graduate level.
“The presence of non-physicists in the translation process naturally introduced gaps in information as a lack of contextual familiarity made it challenging for the interpreters to accurately facilitate communication,” Lualdi describes. “It was as though I was trying to reassemble a broken piece of china with multiple fragments missing, while my hearing peers were handed unbroken pieces. The same applies to the reverse process—whenever I contributed to a physics discussion, my words would end up as broken china in my peers’ hands.”
At Illinois Physics, Lualdi is known among his colleagues not only for his keen intellect, but also for his cheery, good-natured disposition. He tackled the communication barrier with a we-can-make-this-happen approach. Initially, he collaborated closely with his team of interpreters, setting up regular trainings to equip them with physics literacy as well as the necessary language tools.
However, by the next academic year, a second Deaf graduate student arrived on campus, to pursue a master of fine arts in acting at Illinois Theatre. The doubled demand for ASL interpreting services on campus significantly reduced interpreter availability, limiting Lualdi’s ability to train his team of interpreters and frequently forcing him to forgo interpreting services, at the expense of effective communication for himself and his colleagues. At the start of the 2019/20 academic year, DRES had hired a second ASL interpreter, but Lualdi’s primary interpreter departed the university. Interpreter scheduling challenges persisted, and Lualdi had to devote significant time to training the new interpreter and the short-term freelance interpreters enlisted by the university.
These impediments limited the progress Lualdi could make, particularly in the laboratory, and made the many Illinois Physics graduate program enrichment opportunities less enriching for him than for the department’s hearing students.
Lualdi has talked about these impediments with Illinois Physics Professor and Associate Head for Graduate Programs Lance Cooper. The two agree, for Lualdi’s progress toward his doctoral degree to continue uninhibited, he really needs a dedicated interpreter.
“We are deeply committed to providing the same access to education for all of our students,” remarks Cooper. “That includes professional development and networking opportunities necessary to our graduate students’ future successes. Like all of our PhD students, Colin should be able to attend and communicate effectively at conferences, research group activities, departmental colloquia and seminars, etc.
“The best way to accomplish this would be to assign Colin a designated interpreter who can develop familiarity with the language of physics, signed and spoken, through continued exposure, and on whom he can rely for interpreting at frequent and irregularly scheduled meetings, events, and research activities.”
Designated interpreters are commonly used by Deaf professionals working in specialized fields, including graduate students, as a cost-effective means to achieve high-quality communication access.
In the meantime, the department has done what it can to create a supportive environment for Lualdi by leveraging technology and alternative communication modes. One example of this is the live captioning Lualdi now receives during departmental colloquia, an effort implemented by teaching laboratory specialist Erika Smith in collaboration with Lualdi and with funding from DRES. Additionally, the Graduate Office contributed to Lualdi’s purchase of a pair of smart glasses, to improve the usability of live-captioning services by streaming the captions in the field of vision as opposed to on a laptop display. Lualdi is currently working with Professor Lawrence Angrave in the Department of Computer Science to further refine this technology for use by other Deaf and hard-of-hearing students at the U of I.
Lualdi notes, “While these solutions are immensely helpful, it is important to recognize the inherent limitations of such technologies, particularly as captioning enables only one-way communication: without skilled interpreters, it is difficult for me to engage in the conversations and discussions that make Illinois Physics so vibrant.”
The need for a common language model
How is a new ASL sign for physics terminology created? Lualdi describes the process.
1) We start with the concept (the English word) that we want to translate into ASL. We distill it down to its core meaning. We ask questions like, “What aspects of the concept does the ASL sign need to capture?” and, “What is its usage?”
2) Each ASL sign is a function of five parameters: handshape, location, movement, palm orientation, and non-manual signals. This is analogous to how in spoken languages individual phonemes are combined to create words. In creating a sign, we need to give careful consideration to each of these five parameters, as well as how the sign itself integrates into ASL syntax. For instance, as a photon is a massless object with an affinity for high speeds, we need to choose a handshape with the appropriate connotations. Also, as it is frequently used, the sign needs to be easily produced, or it would be akin to saying “zitterbewegung” every five words—a one-handed sign is the way to go.
3) For concepts that belong to a particular category, such as various types of quantum states, we also try to develop a family of signs that share a common root (i.e., one part of the sign remains the same for all of them). This helps link related signs to each other.
4) We then test the sign by using it in sentences in natural dialogue, to see how well it flows in regular use. We also evaluate the clarity of the sign by collecting feedback from native ASL speakers. Is it recognizable? Is it easy to form? We revise the sign accordingly.
5) Finally, we document the sign itself. This often involves capturing it on video in a professional studio along with a usage example and/or definition. We then put these signs to use. Sometimes they work great, but sometimes we discover difficulties we had overlooked. We revisit the signs when the development team reconvenes.
It was in his sophomore year as a physics major at Princeton University when Lualdi first experienced a serious communication gap. While taking advanced physics courses, he encountered higher-level physics concepts that were unfamiliar to both him and his interpreters. No shared conventions existed for expressing in ASL the physics concepts being explained in class, and Lualdi’s interpreters sometimes struggled to convey what was being taught.
“I first encountered the term ‘degenerate’ as an undergraduate,” Lualdi recalls. “My interpreters at that time, like me, were not familiar with its meaning in a physics context and so assumed the vernacular definition of degenerate, using a sign usually associated with ‘decline.’
“At that time, I didn’t know any better, and it was plausible to assume that ‘decline’ or ‘decay’ was appropriate in the context of degenerate energy levels—I thought perhaps these are the energy levels that atoms decay to, and so didn’t have any reason to look up the proper physics definition.
“As a result, I was stuck with a very hazy understanding of degeneracy for several years, until I finally figured out the source of confusion.”
This is just one of many examples of how using unsuitable ASL constructions can have a detrimental impact on Deaf students’ ability to engage with specialized STEM content.
To address these challenges without the ready availability of language models to emulate, Lualdi and his interpreters at Princeton worked to develop a number of in-house provisional signs specific to the concepts that would be encountered most often in Lualdi’s physics courses. And this effort went a long way toward improving Lualdi’s undergraduate physics education.
However, as Lualdi discovered at the start of his graduate training at Illinois Physics, the language challenges at this higher level of study were much more profound than he had originally thought.
“It became obvious that the temporary fixes that had worked in undergrad were not going to cut it anymore in graduate school,” Lualdi stresses. “The physics concepts are significantly more advanced and technical. And in doing research full time, I found myself needing to communicate physics in ASL far more frequently, to be able to discuss my ideas with fellow group members. As an undergrad, I could read textbooks on my own to fill in information gaps, but in doing original research, there is no textbook. And collaboration is vital to success.”
In response to the new communication challenges graduate school presented, Lualdi and his interpreters at Illinois continually developed new signs using a much more sophisticated process compared to that of Lualdi’s undergraduate years.
“Right from the beginning, we would meet for an afternoon every week during each semester,” Lualdi describes, “to go through the list of English words without clear corresponding signs that we had encountered during that week. We analyzed the significance and context of each word and then came up with a few candidate signs that we felt sufficiently captured the meaning of that word while conforming to ASL linguistic requirements. These new signs would then be tested in real-world scenarios the following week and refined as needed. Using this method, we eventually amassed a list of hundreds of signs.”
This tremendous investment paid off in the short term. However, due to interpreter availability and turnover issues beyond Lualdi’s control, he and the deparment are still searching for a more permanent communication access solution.
Formalizing scientific signs for the broader Deaf community
Lualdi is now working to share the signs developed at Illinois with the broader ASL community, both to gain critical feedback and to allow for a broader establishment of language models for ASL STEM discourse. In Spring 2019, Lualdi started working with two national efforts to broaden the ASL lexicon—the ASL Clear and the ASLCORE projects.
The ASL Clear project began in 2005 as a collaboration between The Center for Research and Training (CRT) at The Learning Center for the Deaf in Framingham, MA; the Massachusetts Department of Elementary and Secondary Education, and the Center for the Study of Communication and Deaf People at Boston University. The project has the goal of developing a free online resource for ASL STEM vocabulary that provides definitions, examples, and micro-lectures, to enhance scientific discourse in ASL in academic settings. The ASL Clear project is currently working with Harvard University’s Center for Integrated Quantum Materials (CIQM) to develop quantum-science ASL vocabulary, to be made available for Deaf interns in CIQM programs. Lualdi and David Spiecker, a Deaf graduate student pursuing a PhD in Optics with the Institute of Optics at University of Rochester, are currently assisting with the development of ASL vocabulary that is relevant for students in CIQM programs.
In working with ASL Clear, Lualdi has come full circle: he is a 2013 graduate of the Marie Philip School at the Learning Center for the Deaf.
The ASLCORE project is operated by the National Technical Institute for the Deaf (NTID) at Rochester Institute of Technology (RIT). The project was initiated in 2014 by RIT staff interpreters Sarah Cannon, Miriam Lerner, and Sarah Schneckenburger with the goal of providing a workshopping space for Deaf professionals and translators to create and disseminate ASL signs for non-vernacular and subject-specific concepts from a variety of fields, including physics. The ASLCORE project currently has over 1,800 signs in their free online database, including over 200 physics terms, names, and concepts. Lualdi, Spiecker, and Giordon Stark, a Deaf experimental particle physicist and current postdoc at the University of California, Santa Cruz, are currently working with ASLCORE to significantly expand its number of lexical signs for physics terminology, with many of their newly-developed signs recently added to ASLCORE.
Check out Lualdi's bio and video on the ASLCORE website.
Lualdi explains the teams’ process when they come together and brainstorm, “We meet in person every several months for a week or so, in addition to working and prepping remotely, and there we put our heads together to develop signs, test them out, and add them to the record by filming signed demonstrations as well as usage examples— which are lecture videos incorporating these signs.”
The future of ASL in STEM
After less than a year since he started working with ASL Clear and ASLCORE, Lualdi says he is already delighted at the results and progress that he has seen in his communication with friends, colleagues, and interpreters.
“Near the end of a recent ASLCORE workweek in Rochester, I spoke with my friend and fellow sign developer David [Spiecker] about his recent summer research project,” Lualdi reflected. “As we were conversing about the details of his work, it suddenly struck me that we were using many of the signs that we had developed over the past week and from other sessions in recent months. These signs, being natural ASL signs, could be used in proper ASL sentences rather than clumsy pidgin sign-language constructions using English-based loan signs. They flowed together so beautifully that I found myself marveling at how effortlessly David and I were able to engage in very technical discourse.
“With there being so few of us Deaf physicists fluent in ASL and the long-time lack of an efficient way to talk physics in ASL, it was my first time engaging in a natural face-to-face verbal conversation with someone else without any communication barriers. It was a truly remarkable experience, and I’m thrilled that this work has made this possible. It is my hope that with time these signs, and any yet undeveloped signs, will enter the mainstream such that the experience that I had with David would no longer be an outlier, but the norm.”
Lualdi has many such goals for the signing Deaf physics students of the future. Among them, here are his top three. He hopes over time to (1) increase the number of ASL STEM language models and thereby give Deaf students, ranging from middle school to grad school, the language tools that they need to articulate the STEM, and specifically physics, concepts they encounter; (2) provide ASL interpreters with the same set of tools so that they can facilitate clear communication when working with Deaf clients in physics contexts; and (3) Encourage the next generation of Deaf students to take up the study of physics by improving communication access in physics for ASL users, thereby expanding the number of signing Deaf physicists beyond the handful working in the field today.