Research lntegrity in the Age of AI by Prof. Joseph Ali (Seminar Note)

Posted on In

Below are my own raw note. The images belong to the speaker or the organizer. Content for reference only. The speaker and organizer hold the ultimate authority.

Prof. Joseph Ali, JD

started from 20th century on AI in research integrity
(from perspective of public health and digital health)

Situation

acceptability of AI (“Andersen GenAI in the research process”):

  • AI use is not one category, language assistance was accepted generally but peer review/exp design contested

capacity:

  • lower practical barriers (help coding) / extend analytic (explore alter)
  • participate in discovery itself (ginkgo bioworks autonomus lab)

what does it mean to produce trustworthy science in the AI era?

Definitions

trustworthyness matters (principles is consistent, while method to achieve changes over time)

  1. make decision
  2. cumulative: build on relationship
  3. error cascades can make harm at large scale
  4. unreliable research wastes resources / make harm
  5. trustworthyness more likely to produce real public benefit instead of causing harm

Research integrity definition and way to distinguish them

how AI complicated evaluation of trustworthyness:

  1. honesty:
    1. “Fabrication and errors in bibliographic citations gen by ChatGPT” (confident hallucination)
    2. enhance integrity checks (“SILA” system) -> AI can direct human attention to material that deserves closer review by a human eye
  2. Rigor
    1. same task -> different outputs. introduces new sources of methodological variability
    2. provide reproducibility (“Dobbins LLM model based agents for autonmated research reproducibility”) -> can recreate anaysis (science + meta science)
  3. Transparency
    1. norms evolve: underdeveloped norms about which AI uses materially affect the research and its evaluation (disclosure may become a beauracracy burden). vs tradition: prospective registration of clinical trials + data sharing expectation(consent for usage controversy, a risk like privacy, how to operationally operate it? e.g. my data is used 10 years and analyzed many times)
    2. what to disclose? tool, task, where AI materially entered the workflow; important input/output, limitation, what human reviewed or changed
    3. authorship problem: AI can materially contribute, this contribution needs to be visible (current guidance reserve authorship for ppl who can take responsibility for the work)
    4. detection of undisclosed AI use reliability? existing system not reliable esepecially for none native speaker (become discriminatory)
  4. Accountability
    1. delegation: rely on software, instruments, … not new. but AI can operate well-beyond the researcher’s expertise; what a researcher must understand, verify, or supervise before responsibily relying on AI-medated result. (engage with experts for domains not familiar; boundary matters)
    2. stewardship of science: narrower collective research focus and more parallel less interconnected follow-on research -> benefit individual AI-active researchers at cost of diversity and cross-talk in science? (AI has potential for cross. do not move alone)
    3. bottleneck: produce more but less time to think critically; responsibility does not shift, but the space shrinks (morally distress)

Potential Solution

Research integrity is becoming a systems problem

  • does research ecosystem support trustworthy AI-mediated science? address from individual/institution/societal level

institution level: praised HKU guidance for its close alignment with ICMJE, european commission guidance, etc. ; but leaves open questions (adaptive governance)

incentives: intitution have an obligation to counterbalance pressure from rewarding “output”, but reward rigorous methods, transparency, mentoring, data stewardship, and contribution to trustworthy research -> reduce gaming for decoration, commertialized science; prioritize social value, align systems of reward and accountability with this end

summary

QA

mindset for student(thinking useless)

  • early social media brings worry on brainrot -> acknoledge the risk ; losing capability to recall knoweldge; be aware of the use and stay connected with people

self evolving AI era -> what is value of human?

  • be transformative thinker: we can change perspectives and incoprate new perspectives; our own development has symbiotic relationship with AI’s development; ask hard questions and engage in different ways; use creativity to tackle with imposible to make it possible; be critical and careful, ask good question, evaluate, and judge; know what you can or need to check (meta science)
  • back and forth to engage, and understand better

how can we enhance the “cross-talk”?

  • build a platform where researchers engage. Because everyone is experimenters now.