Bootcamp / Session 1 / 1.4

Session 1.4 · Closing · Alper Kucukural

Flight rules: from pilot to practice.

Eric showed why the plane can fly confidently in the wrong direction. Tommy showed how a good pilot flies it anyway. This last part is about the rules for flying it here, with our data, on our cluster, and what that looks like on an ordinary working day.

Recordings

This part was taught in two pieces: the rules and the Foundry Connect setup before Tommy's talk, then verification and safety afterwards.

Foundry Connect and responsible AI

Alper Kucukural · 5 minutes · Download (36 MB)

Verification and safety

Alper Kucukural · 22 minutes · Download (61 MB)

A. One table for the whole afternoon

Eric ended with a table of what you will see, what causes it, and the habit that catches it. Here it is again with two more columns: what Tommy showed about each failure, and the rule or tool we use for it.

What you seeWhy (Eric, 1.2)The pilot's habit (Tommy, 1.3)Our rule or tool (1.4)
It makes something upPrediction without lookup. Nothing in the mechanism checks truth.Run generated code line by line. “Hallucination is even more dangerous” than irreproducibility.Check every number against the raw report and every citation against the paper. A second, independent reviewer model.
Same prompt, different answerSampling: a weighted dice roll at every token.Traceability and reproducibility are different questions. Commit to GitHub.Log model, version, date, settings, and prompts. Keep the agent transcript and the git history.
Out-of-date answersTraining stops at a cutoff date.Start with /plan and give it the context: PDFs, data, links, your old scripts.Ground it in current documentation and your own files. A project CLAUDE.md that says what is true now.
Repeats the field's old mistakesIt copies the patterns in its training text, flaws included.Foundations matter. Your experience spots the batch effect in the PCA.Your judgement is the final verifier. Ask a domain expert before you believe a surprise.
Echoes sensitive textNothing in the mechanism keeps secrets.Covered by the rules in section B.Patient and unpublished data only in approved tools. Data stays on the cluster; the connector shows only what you allow.
Does more than you askedAgents add tool use on top of prediction.--permission-mode auto is fast, and it trades away the pause before each action.Permission prompts, confirm before any write, deny rules, and hooks that enforce the rules no matter what the model decides.

B. The flight rules

Patient data, checking outputs, disclosure, reproducibility, bias, and when to leave AI out. One rule for each, with the reason.

Protect the data before you protect your time

Rule: patient data (PHI) and unpublished results go only into tools the institution has approved for that data, with training on your inputs turned off. Never into a consumer chat account. If you are not sure a tool is approved for this data, treat it as not approved and ask.

Why: In 2023, Samsung engineers pasted confidential source code into a consumer chatbot, and the company responded by restricting the tools for everyone. Eric's FAQ answered “does it learn from my chat?” with no. Whether the provider keeps and can read what you sent is a separate question, and for patient data it is the one that matters.

Identifiers are wider than names. Dates of service, ZIP codes, record numbers, and full-face photos all count under HIPAA.

You are the author, and you are accountable

Rule: AI is never an author. Major journals, including the Nature and Science families, do not accept AI tools as authors, because a tool cannot take responsibility for the work. Every claim in your paper is yours.

Disclose what you used, for what, and how you checked it

Rule: name the tool and version, the task it did, and the verification you ran. Follow the venue's policy. Session 2 builds a reusable disclosure template.

We follow this rule too: see the AI use disclosure for these materials.

Log it so someone can trace it

Rule: record the model and version, the date, settings such as temperature when you control them, and the prompts. With a coding agent, keep the session transcript and commit the code it wrote. As Tommy put it, you may not get the same output twice, but you should always be able to show how you got this one.

Expect the bias you would expect from the literature

Rule: read AI output the way you read a field's older papers. Training text over-represents some populations, tissues, and model organisms, and the model repeats that skew fluently.

Hand AI the labour, keep the judgement

Rule: AI may do recoverable labour: formatting, first drafts of code, polishing language, summarizing a source you will check. It must not replace the judgement that defines research: framing the question, calling an outlier, interpreting a surprise, answering a reviewer, or anything clinically consequential without expert review. Leave AI out entirely when you cannot check its answer.

C. How I actually work

Tommy's demo ran an agent on a laptop against a local analysis. My day is the next step up: an agent connected to the systems this bootcamp runs on, Foundry Connect and the UMass Chan HPC cluster. In class this is a live demo; the steps below walk through it.

Start from context, not from a prompt

Every project has a CLAUDE.md: what the project is, how it is laid out, and the rules the agent follows there. I also keep a global one with rules for every project. Before any change, the agent writes a plan, and I read it before it acts.

Same idea as Tommy's “always start with a /plan”, made permanent so I do not have to repeat myself each session.

Skills are our lab's standard operating procedures

A skill is a markdown file with instructions, the SKILL.md from Tommy's slides. We write them for the procedures we do not want improvised:

  • Submitting a job to the HPC cluster: which LSF queue, how to request memory and time, how to run inside an Apptainer container, and how to generate a job script safely.
  • Launching a Foundry Connect pipeline run: build a dataset from the FASTQ files on disk, duplicate a known-good run, change only the inputs, then launch.

A skill is a protocol written once, reviewed by a person, and followed the same way every time. That is what makes an agent's work traceable.

Connect it to Foundry Connect

Through the Foundry Connect connector (the fourth green light from 1.0), I ask in plain language:

  • “Which of my runs failed this week, and why?” It reads run status and logs.
  • “Pull the QC report for this RNA-seq run and summarize anything unusual.” It reads result files.
  • “Rerun it with the corrected sample sheet.” It prepares the change and stops to ask for confirmation before launching anything.

The data and the compute stay on the cluster. The assistant sees what the connector exposes, and every write waits for a person.

Guardrails: instructions are requests, hooks are enforcement

A chat assistant sees only what a connector shows it. A coding agent on your own machine can do anything your terminal can do. That is why permission modes matter, and why auto mode belongs in a throwaway project, not next to credentials or patient data.

A true story from this summer. While debugging a database connection, an agent helpfully printed a live access token and a database password into its session transcript. Nothing malicious happened, and nothing was misused. Both credentials still had to be revoked and replaced, because anything printed into a transcript is kept.

What changed afterwards:

  • A written rule in my global CLAUDE.md: never print a credential, never type a password into a command.
  • Deny rules so the agent cannot open credential files at all.
  • A hook: a small script that runs before every command the agent issues and blocks anything that would print a secret. It has no bypass. It occasionally blocks something harmless, and that is the right trade.

The model can forget or misread a rule in its instructions. A hook runs every time, whatever the model decides.

Verification is a procedure, not a feeling

  • Independent review: a fresh subagent, or a model from a different family, critiques the analysis without seeing the author's reasoning. The same model cannot be both author and critic.
  • Check the numbers at the source: sample counts, gene counts, and p-values in the summary must match the pipeline's own report.
  • Watch for agreement: if the agent enthusiastically agrees with your hypothesis, treat that as a prompt to check, not as support.
  • Git is the flight recorder: every change the agent makes is a commit a person can read.

Keep it sharp: small context, fresh sessions

Long sessions get slower, more expensive, and less accurate. I send exploration to subagents and keep only their conclusions, and I start a fresh session once a task ships. This is Tommy's /compact and subagents slide, applied as a habit.

D. Six AI-literacy skills, and where you saw them today

Adapted from Michael Zyphur's Responsible AI in Research and Research Training.

SkillThe one-line ruleWhere you saw it
Citation verificationCheck every AI-produced citation against the primary source.Eric's fake-reference quiz (1.2)
Model and parameter reportingReport model, version, temperature, seed, and prompt.Eric's temperature slider (1.2)
Prompt disciplineA prompt is an analytic choice. Test whether a small rewording changes the answer.Session 2 and Session 3
Adversarial reviewCritique with a different model than the one that did the work.Verification step (1.4 C)
Sycophancy detectionAgreement is not evidence.Verification step (1.4 C)
Failure-mode reportingSay what you checked, what failed, and what risk remains.The table in section A

E. Activity: spot the leak

Eric's quiz asked which references were invented. This one asks which prompts should never be sent to a consumer chat tool. Decide for each one before you open the card. All records below are fictional.

Prompt 1 A results table from a public study
Here is the DESeq2 output for liver Jnk1/Jnk2 knockout versus wild type from GEO (public accession). Which of the top 50 genes are in the PPARα pathway?
Fine to send. The data is already public and has no personal information. Still verify the pathway membership against a database, not the model's memory.
Prompt 2 Help writing up a biopsy note
Turn this into a sample description: MRN 0000-000, 52-year-old woman, vitiligo, lesional biopsy taken 03/14 at the dermatology clinic, follow-up scheduled.
Leak. A record number, an exact date of service, age, and diagnosis together identify a patient. This is PHI whether or not a name appears.
Prompt 3 A sample sheet with “no names”
Check this sample sheet for errors: sample_id, donor_age, sex, biopsy_date, home_zip, condition, batch
Leak. No names, but biopsy dates and ZIP codes are HIPAA identifiers, and together with age and sex they can re-identify people. Remove those columns, or use an approved tool.
Prompt 4 Polishing an abstract
Make this abstract more compelling for Nature: [our unpublished preliminary result, not yet submitted]
Leak in a consumer tool. Unpublished findings are confidential to your lab. Use the approved instance, and disclose the editing help (flight rule 3).

F. What happens next

1. Tell us how it went

Two minutes: whether you made it, what worked, and what to change. We ask this after every session.

Session 1 feedback

2. Finish setup before Session 2

Get your four green lights using the setup checklist. Everything you saw in section C depends on them, and so do Sessions 4 to 6.

3. Homework: your lab's five-line AI ground rules

Use the nine-question checklist from 1.1 and the flight rules above. Write five lines your lab could actually adopt, and bring them to Session 2. An example to react to, not to copy:

  1. No patient or unpublished data in any AI tool that is not approved for it.
  2. Every AI-produced number or citation is checked at the source before it goes in a figure or a draft.
  3. Agent-written code is committed and reviewed like a colleague's code.
  4. Methods sections name the tool, version, task, and how it was checked.
  5. Interpretation stays with people: AI can draft, it does not decide.

Will you be the pilot or the passenger? Pilots know how the plane works, follow the flight rules, and check the instruments against the ground.