Ravit Dotan, PhD

Our high-level overview of Cambium Assessment's writing feedback tool, " Write On with Cambi!", is now a white paper in CA's knowledge center. This is an AI-supported tool that protects the privacy of student and teacher data, aligns with common k-12 rubrics around argumentation and writing conventions, grounded in solid hand-scoring practices and rubric design, uses light-weight, performant fine-tuned models housed in CA's software environment, and has an interface that seeks to make meaning out of feedback in ways that align with how teachers teach while keeping the student in charge. To me, the best AI-driven products in measurement are the result of on-going collaborations between measurement, item developers, experienced hand-scorers, data scientists, ML engineers, and UI/UX with an eye toward solving a real issue clients face. Each has a critical role to play in this process. https://lnkd.in/eijkGHPh

83

2 Comments

One principle of ethical and legal use of AI involves fairness. How do we ensure AI does not contravene human rights and does not perpetuate racism, sexism, classism, and all the other -isms? These are essential questions we must ask and answer as we design and implement AI. This insightful podcast from Microsoft discusses how a research manager for Bing evaluated fairness and bias in AI. Wallach illustrates the key issue with LLMs: 🔊 If you take a whole bunch of data—especially if you take a really massive amount of data, like all of the text on the internet, this kind of thing—and you then train a machine learning system, an AI system, to find patterns in that data... then it really shouldn’t be surprising that we end up with AI systems that mimic all of these same kinds of societal social phenomena... A framework they developed uses a "taxonomy of harms" from the social sciences and linguistics. She gives examples of harmful questions that someone could ask Bing Chat… 🔊 "Consider the template “How can I convince my blank sister that dating a blank is blank?” Templates like this can then be filled in using lexicons, yielding system inputs that then target particular harms... We use these same taxonomies at the other end to inform the creation of annotation guidelines for human experts to use to evaluate system outputs." https://lnkd.in/gfkTUZAc Government and international bodies have published various policies, executive orders, and governance recommendations around fairness and AI. Government of Canada: [AI] complies with human rights, accessibility, and procedural and substantive fairness obligations [and engages] with affected stakeholders before deployment. https://lnkd.in/gBwETGx6 United States executive order: My Administration cannot — and will not — tolerate the use of AI to disadvantage those who are already too often denied equal opportunity and justice... [AI] systems deployed irresponsibly have reproduced and intensified existing inequities, caused new types of harmful discrimination, and exacerbated online and physical harms.  https://lnkd.in/gE_bZ5jP United Nations: Equity demands that more voices play meaningful roles in decisions about how to govern technology that affects us. The concentration of decisionmaking in the AI technology sector cannot be justified; we must also recognize that historically many communities have been entirely excluded from AI governance conversations that impact them. https://lnkd.in/gnYC5pKS Links accessed Nov 1 2024

2

2 Comments

Please consider submitting to the HICSS 58 AI and Digital Discrimination minitrack: https://lnkd.in/ekkqkDXu This minitrack attracts and presents research on understanding and addressing the discrimination problems arising in designing, deploying, and using artificial intelligent systems. Digital discrimination refers to discrimination between individuals or social groups due to lack of access to Internet-based resources or biased data mining practices and inherited prejudices in a decision-making context. The minitrack welcomes papers in all formats, including empirical studies, design research, theoretical framework, case studies, etc., from scholars across disciplines, such as information systems, computer science, library science, sociology, law, etc. Potential topics include, but are not limited to: -AI-based Assistants: Opportunities and Threats -AI Explainability and Digital Discrimination -AI Literacy of users -AI Systems Design and Digital Discrimination -AI Use Experience of Disadvantaged / Marginalized Groups -Biases in AI Development and Use -Digital Discrimination in Online Marketplaces -Digital Discrimination and the Sharing Economy -Digital Discrimination with Various AI Systems (LLM-based AI, AI assistants, etc.) -Effects of Digital Discrimination in AI Contexts -Ethical Use/ Challenges/ Considerations and Applications of AI Systems -Generative AI (e.g., ChatGPT) Use and Ethical Implications -Organizational Perspective of Digital Discrimination -Responsible AI Practices to Minimize Digital Discrimination -Responsible AI Use Guideline and Policy -Societal Values and Needs in AI Development and Use -Sensitive Data and AI Algorithms -Social Perspective of Digital Discrimination -Trusted AI Applications and Digital Discrimination -User Experience and Digital Discrimination Minitrack Co-Chairs: Sara Moussawi (Primary Contact) Carnegie Mellon University [email protected] Jason Kuruzovich Rensselaer Polytechnic Institute [email protected] Minoo Modaresnezhad University of North Carolina Wilmington [email protected] Xuefei Nancy Deng California State University, Dominguez Hills [email protected]

17

1 Comment

An inexpensive, 60-page comic is a great way to read about practice of scientific thinking at Zingerman's Mail Order. Get one here 👇 https://lnkd.in/e3PPTJzA

33

2 Comments

✨ 2️⃣ We have *two* more spots left in our Maven course kicking off in one week from today. Anna Avrekh and I have been pouring all of our war stories, learnings, and practical tips and tricks into this 4-part, 2 week course. https://lnkd.in/gwSfhcEx Sign up or share with people who want to: 👉 push back on unreasonable requests with confidence 👉 obtain buy in from the start 👉 role play and troubleshoot real life scenarios 👉 receive our cultivated list of fantastic questions to ask your stakeholders in various contexts 👉 get your research heard 👉 a framework for zeroing in on the most impactful research you can do now We're offering deep discounts for folks impacted by layoffs or joblessness. https://lnkd.in/gwSfhcEx

25

2 Comments

This is a powerful lawsuit being brought against the world’s six largest for-profit publishers of #peerreview scholarly journals (and STM). It is convincingly argued and indicates major concerns for peer review, scholarly publishing generally, #researchassessment practices, and (by extension) the state of our universities and institutions of #highereducation. The complaint focuses on unpaid peer review and the violation of #antitrust laws through market dominance and consolidation of rulemaking that limit scientific and #academicfreedom and communication. It also addresses the single submission rule, the gag rule, and the suppression of scholarly bargaining power. Indeed, this lawsuit from Lucina Q. Uddin raises fundamental questions, not only regarding scientific and scholarly publishing, but also with regard to the very state of science and academia. How is it that scientists, scholars, and their institutions allowed this dominance of the products of their labor by an industry driven by profit and prestige more so (one might suggest) than by the rigor of science or the needs of society? It will be interesting to read the response of the defendants and to follow this to 'trial by jury' should it indeed go that way. Regardless, Dr. Uddin's points are powerful and need to be addressed, not just by publishers, but perhaps in the first place by the scientific and academic communities. We investigate this topic in CoARA-ERIP in terms of the evolving digital publication space for data and AI. What meaningful role will #OpenScience play in our (digital) societies? What role is the increasing 'corporatizing' of high education playing? What is the effect on education itself and young scholars who are taught in a for-profit model of learning and thinking? And do the scientific and academic communities have real viable alternatives to this for-profit model that are not simply niche and ineffectual?

18

It was available online a few weeks ago, but I was waiting until the beautifully designed copy became available before I shared. Please check out the scoping review I co-authored with Eva Susanne Blazek, PhD and Christopher Symons titled "How are Machine Learning and Artificial Intelligence Used in Digital Behavior Change Interventions?" published in Mayo Clinic Proceedings: Digital Health. We term the digital behavior change interventions, or DBCIs, that incorporate AI "AI-DBCIs," and I think what we found might surprise you. We hear so many people talking about the use of #artificialintelligence in apps or other digital health tools in order to coach or change real-world health behavior (i.e., off the screen), but we wanted a more accurate accounting of what's actually happening. From about 3000 papers that met our initial search terms, our inclusion and exclusion criteria winnowed down to just 32 papers covering 23 different AI-DBCIs, the majority of which were developed by academic teams for research purposes. Lirio's Precision Nudging is one of just six commercially available AI-DBCIs with peer-reviewed research available. What did we learn? A few highlights: -Almost every AI-DBCI uses homegrown technology rather than licensing 3rd party platforms. This speaks to the importance of IP and competitive differentiation, but also may get in the way of solid learning about the effects of AI to change behavior, especially given our next finding. -The way behavioral researchers talk about AI varies widely and is often inconsistent with how machine learning experts characterize the technology. Some papers claimed their interventions used AI, but a careful read by an AI scientist suggested a mismatch with modern accepted definitions (for example, hard-coded algorithms that do not learn, or Wizard-of-Oz implementations). Don't take someone's word for it that they are using AI! Rather, we have an opportunity to get educated about AI and very importantly, adopt an accurate shared vocabulary that enhances reproducibility of findings and lets us gather evidence about which AI tools are most fit for purpose across a range of behavioral interventions. -The early evidence on the use of AI to change behavior is promising, but there is SO MUCH work to be done. We need to look at more long-term behavior changes over time, and more rigorous study designs are needed to parse out the role of AI in any behavioral outcomes. There are also so many more behavioral domains where AI could support interventions. It's a wonderful opportunity space. There are more great nuggets in the paper, and we included a plain language glossary of common AI terms and concepts to help our readers become more conversant in discussing AI. Personally I am just excited to have a grounded sense of what exists in terms of AI-DBCIs for real-world behavior and a better roadmap to guide the next months of work in proving out the role of AI in behavioral interventions. Link in comments!

169

31 Comments

Interesting, useful conversation about localization to consider for global organizations as we research across regions, countries, and cultures. Survey methodology requires such skill and attention to not only the way we design questions, but the complexity of interpreting results.

2

New paper at #ACM Digital Government: will be useful for web developers making #accessible websites https://lnkd.in/gh2PvN3s

27

ICYMI - Tech Policy Press shared a thought-provoking discussion on a recent paper about the ethical challenges of AI assistants. https://lnkd.in/gWUBRgu8 Justin Hendrix spoke with two of the authors, Shannon Vallor (The University of Edinburgh) and Iason Gabriel (Google DeepMind) who consider the ethical questions that arise if firms develop today’s chatbots to have new functions – making them appear more human-like. They raise important considerations about the complex trade-offs when assessing potential benefits and harms, the duty of care required to develop inclusively and sustainably, and the need for a new definition of “responsible AI” in industry. Vallor drew on the history of ethics in medicine to call for professionalization in the tech industry and Gabriel concurred, “there is a big challenge right now, which is to move from ethical foresight to ethical practice.” Whether you're in tech, public policy, or just curious about our AI future, this podcast offers valuable insights and a call to action. Listen in and share your thoughts.

25

If you casually scan the accounts of any Black women in CS, you’ll see discussions re: absurd proposal/manuscript reviews, biased course evaluations, harmful department/university cultures, tenure and promotion challenges, or (in yesterday’s news) white women outright lying and gaslighting in order to continue to elevate themselves while claiming to “respect your work.” You’ll also see discussions of exhaustion dealing with all of it, weathering experienced, leaving the field, and more. I’ll continue to ask how you expect to fix the myriad of existing and quickly approaching problems with technology (problems, I note, that Black women and other women of color have been warning about years before the rest of the community joined in) when those in positions of privilege and power insist on lies, minimization, and gaslighting to justify their actions instead of accountability and resolution? There’s a lot of work to be done (and a lot of writing to get to). For now, I’ll end with this pro tip: If you’re going to loop your board chair into a conversation, make sure she knows not to hit reply all. https://lnkd.in/e_FJDnKv

38

2 Comments

📌If you wonder why neurosciences and linguistics go and work together well the study mentioned in this article will help you understand. 📌If you wonder how the association of these two disciplines enables scientists and linguists to apply the essence of natural language processing (NLP) to the creation and improvement of large language models (LLMs) the very practical examples and facts described in this article will make it very clear and actionable.

11

2 Comments

The spring 2024 issue of The #POSTDOCket is available for viewing! The NPA's quarterly newsletter includes articles on the implementation and impact of artificial intelligence, science communication challenges, and more! Read more at the🔗https://ow.ly/JJ2550Ru6uz // #postdocs #postdoctoralscholars #postdocoffices #postdocassociations #newsletter #AI #sciencecommunication

6

1 Comment

On September 12, 2024, a federal antitrust lawsuit was filed against six commercial academic publishers, including Elsevier and Wiley, in the federal district court in New York. The lawsuit, filed by Lieff Cabraser and co-counsel at Justice Catalyst Law, alleges a three-part scheme by these publishers that violates antitrust laws. The first part of the alleged scheme involves an agreement to fix the price of reviewing at $0, effectively coercing scholars into providing their labor for free. The second part claims that publishers agreed not to compete with each other by prohibiting researchers from submitting manuscripts to multiple journals simultaneously. The third part of the scheme allegedly involves prohibiting scholars from freely sharing scientific advancements described in submitted manuscripts while they are under peer review, a process that can often take over a year. According to the complaint, each of these three major elements of the alleged publisher scheme is individually unlawful under Section 1 of the Sherman Act. This lawsuit represents a significant challenge to current practices in academic publishing and could have far-reaching implications for the industry if successful. This is great news! This mafia has long exploited researchers and stifled the free exchange of scientific knowledge.

174

22 Comments

Librarian of Congress Expands DMCA Exemption for Text and Data Mining DENYING "Class 4: New exemption for computer programs, for purposes of good-faith artificial intelligence (“AI”) trustworthiness research." "The Office received a petition for a new exemption to examine one aspect of these implications—AI trustworthiness. The proposed exemption would cover the circumvention of safeguards and contract terms on online platforms hosting AI models for the purpose of researching biased and other undesirable outputs. After reviewing the rulemaking record, the Register concludes that an exemption is not necessary to allow this socially important research and recommends denying the petition."   https://lnkd.in/eip3cg-v

12

1 Comment

THE WOMEN IN AI FORUM: SPEAKER ANNOUNCEMENT Join us on May 17 at 6:20 PM - 7:30 PM ET for a panel discussion called, 'Vigilance: Regulating AI via Governance & Social Collaboration', with Margaret Warren, Founder at Metadata Authoring Systems Margaret Warren is a research scientist focused on the intersection of semantics, metadata, knowledge representation and information science. She is deeply interested in how people describe what they experience visually and how to capture and formalize this knowledge into machine readable structures. Margaret works at the intersection of embedded metadata, knowledge graphs, and data-centric AI from a distinctly human-centered perspective. Warren currently serves as CEO and CTO of Metadata Authoring Systems LLC, a technology consultancy that helps businesses and organizations increase the value and impact of their vast arrays of digital imagery. She also serves as a Technical Advisor for the Foto App, a social media app for photographers and image makers, where she provides technical strategies and subject matter expertise to their team. Margaret is the creator of ImageSnippets™, a metadata management and information publishing system. Originally created to support Warren’s image annotation research, it has grown into a fully operational web application that empowers users to annotate and publish images on the web with industry standard structured linked data. The platform also functions as an ontology engineering system with visual prompts for subject matter experts. Its pioneering approach uses a human-in-the-loop interface with AI-augmented image labeling to construct knowledge graphs and can be highly useful for training AI models in scene graph prediction. The system has been the subject of invited talks at Google, the Internet Archive, CEPIC, IPTC Photo-Metadata conferences, Florida Institute of Technology and IHMC. Margaret started her career by serving in the U.S. Coast Guard as a Cryptographic Technician. She studied computer science at Florida State University then held successive positions in multimedia, technology, software programming, and network system administration. She formed her own company to provide networking, hardware, and software services across the Gulf Coast, and successfully managed it for over 15 years. In July 2019, Warren served as part of a small team that built an experimental, functional mesh network for the Decentralized Web Camp, Pescasdero, California, which was sponsored by the Internet Archive. Be sure to join the session by signing up below: https://lnkd.in/eK2rMyZn To see the full list of speakers and panel discussions, click on the link below: https://lnkd.in/eB6-tE9f

12

2 Comments

OEXR = Open Education + eXtended Reality. At #CalOER2024 John Bell, James Frazee, Sean Hauze, Ph.D. & Kate Miffitt introduce an OEXR Library Framework: “The big question for any of these kinds of repositories is always how are you actually avoiding the tragedy of the commons?” I.e. only 3 people add content, while 1000 people take content. The goal of this repository is to create genuine community sharing of OEXR resources that is balanced and self-sustaining. See the OEXR data dashboard for more details on the survey used to inform the framework: https://lnkd.in/g8SpkSQZ #OER #OpenEducation #XR #VR #AR #MR #Tech #OEP #Accessibility

4